Adding Unit Tests
Unit testing is a fundamental part of programming, and essential component of the TMT quality assurance policy. TMT CSW extensively uses unit testing for both Scala and Java code, using ScalaTest for the former, and primarily JUnit for the latter (TestNG is used in one instance for the Event Service). While this guide will not attempt to educate the reader on how to use these popular packages, it will serve to show some examples of how tests can be created for component code and demonstrate some tools provided by CSW to simplify and enable integration of TMT components and other software with CSW and its services.
CSW Test Kit
CSW provides a set of tools for use by developers called the CSW Test Kit. This allows the developer to start CSW services within the testing environment, so that they can be accessed by the components and/or applications being tested, as well as the testing fixtures themselves. It also provides simple methods to start components or a set of components within a container, as well as an ActorContext to be used if other Actors are needed to be created in the tests.
More information about testing with CSW and the CSW Test Kit can be found here.
Tutorial: Writing unit tests for our components
In this part of the tutorial, we will write a few unit tests for our components. These tests are in no way meant to be comprehensive, but hopefully, they show enough to get you started.
The giter8 template provides the required directory structure, and skeletons for tests of the HCD and Assembly in both Java and Scala. It also provides some Component Configuration (ComponentInfo) files for running each of the HCD and Assembly in standalone mode for both languages. They are there for convenience, but may not be required depending your deployment and testing strategy. We will be using them in our tutorial.
We will first look at the tests for the Assembly. As described on the Testing Manual page, the Scala version overrides the CSW-provided superclass csw.testkit.scaladsl.ScalaTestFrameworkTestKit
to get access to the services it needs. By passing in the needed services in the constructor, those services are started in the superclass’s beforeAll
method. In the Java version, we must create an instance of csw.testkit.javadsl.FrameworkTestKitJunitResource
to get access to and start our services, with the services we want to start passed into the constructor of this object.
- Scala
-
source
class SampleTest extends ScalaTestFrameworkTestKit(AlarmServer, EventServer) with AnyFunSuiteLike { import frameworkTestKit._
- Java
-
source
public class JSampleTest { @ClassRule public static final FrameworkTestKitJunitResource testKit = new FrameworkTestKitJunitResource(Arrays.asList(JCSWService.AlarmServer, JCSWService.EventServer));
For our tests, we will want to run the Assembly first. We will do this in the beforeAll
method in Scala, and in a method with a @BeforeClass
annotation in Java, so that it is run only once, before any of the tests are run. The Component Configuration files use are the one provided by the giter8 template. Note that for Scala, we must call the superclass’s beforeAll
method to ensure the services are run.
This code has been provided as part of the giter8 template.
- Scala
-
source
override def beforeAll(): Unit = { super.beforeAll() spawnStandalone(com.typesafe.config.ConfigFactory.load("SampleStandalone.conf")) }
- Java
-
source
@BeforeClass public static void setup() { testKit.spawnStandalone(com.typesafe.config.ConfigFactory.load("JSampleStandalone.conf")); }
Next, let’s add a test. We will add a simple test that uses the Location Service to make sure the Assembly is running and resolve the registration information for it.
This test has been provided as part of the giter8 template as an example.
- Scala
-
source
import scala.concurrent.duration._ test("Assembly should be locatable using Location Service") { val connection = AkkaConnection(ComponentId(Prefix(CSW, "sample"), ComponentType.Assembly)) val akkaLocation = Await.result(locationService.resolve(connection, 10.seconds), 10.seconds).get akkaLocation.connection shouldBe connection }
- Java
-
source
@Test public void testAssemblyShouldBeLocatableUsingLocationService() throws ExecutionException, InterruptedException { AkkaConnection connection = new AkkaConnection(new ComponentId(Prefix.apply(JSubsystem.CSW, "sample"), JComponentType.Assembly)); ILocationService locationService = testKit.jLocationService(); AkkaLocation location = locationService.resolve(connection, Duration.ofSeconds(10)).get().orElseThrow(); Assert.assertEquals(location.connection(), connection); }
You can try running the test either using sbt (sbt test
from the project root directory) or directly in the IDE. If you are using IntelliJ, you can run the test by right-clicking on the file in the project explorer and clicking on Run 'SampleTest'
or Run 'JSampleTest'
. You can also right-click in the class body or the specific test body, if you want to run a single test.
The Assembly we have written does not have much of a public API, so we’ll turn to the HCD now, which has a few additional things we can test, including the publishing of Events and the handling of commands.
First, we will set up the test fixtures similarly as we did for the Assembly, and add a similar test to show the component registers itself with the Location Service on startup.
This also has been provided in the giter8 template.
- Scala
-
source
class SampleHcdTest extends ScalaTestFrameworkTestKit(AlarmServer, EventServer) with AnyFunSuiteLike with BeforeAndAfterEach { import frameworkTestKit._ override def beforeAll(): Unit = { super.beforeAll() spawnStandalone(com.typesafe.config.ConfigFactory.load("SampleHcdStandalone.conf")) } import scala.concurrent.duration._ test("HCD should be locatable using Location Service") { val connection = AkkaConnection(ComponentId(Prefix("CSW.samplehcd"), ComponentType.HCD)) val akkaLocation = Await.result(locationService.resolve(connection, 10.seconds), 10.seconds).get akkaLocation.connection shouldBe connection }
- Java
-
source
public class JSampleHcdTest { @ClassRule public static final FrameworkTestKitJunitResource testKit = new FrameworkTestKitJunitResource(Arrays.asList(JCSWService.AlarmServer, JCSWService.EventServer)); @BeforeClass public static void setup() { testKit.spawnStandalone(com.typesafe.config.ConfigFactory.load("JSampleHcdStandalone.conf")); } @Test public void testHCDShouldBeLocatableUsingLocationService() throws ExecutionException, InterruptedException { AkkaConnection connection = new AkkaConnection(new ComponentId(Prefix.apply(JSubsystem.CSW, "samplehcd"), JComponentType.HCD)); ILocationService locationService = testKit.jLocationService(); AkkaLocation location = locationService.resolve(connection, Duration.ofSeconds(10)).get().orElseThrow(); Assert.assertEquals(connection, location.connection()); }
Now let’s add a test to verify our component is publishing. We will set up a test subscriber to the counterEvent
Events published by the HCD. Since we cannot guarantee the order in which the tests are run, we cannot be certain how long the component has been running when this specific test is run. Therefore, checking the contents of the Events received is tricky. We will wait a bit at the start of the test to ensure we don’t get a InvalidEvent, which would be returned if we start our subscription before the HCD publishes any Events. Then, after setting up the subscription, we wait 5 seconds to allow the HCD to publish two additional Events plus the one we receive when the subscription starts. We will look at the counter value of the first counterEvent
to determine what the set of counter values we expect to get in our subscription.
- Scala
-
source
test("should be able to subscribe to HCD events") { val counterEventKey = EventKey(Prefix("CSW.samplehcd"), EventName("HcdCounter")) val hcdCounterKey = KeyType.IntKey.make("counter") val subscriber = eventService.defaultSubscriber // wait for a bit to ensure HCD has started and published an event Thread.sleep(2000) val subscriptionEventList = mutable.ListBuffer[Event]() subscriber.subscribeCallback(Set(counterEventKey), { ev => subscriptionEventList.append(ev) }) // Sleep for 5 seconds, to allow HCD to publish events Thread.sleep(5000) // Q. Why expected count is either 3 or 4? // A. Total sleep = 7 seconds (2 + 5), subscriber listens for all the events between 2-7 seconds // 1) If HCD publish starts at 1st second // then events published at 1, 3, 5, 7, 9 etc. seconds // In this case, subscriber will receive events at 2(initial), 3, 5, 7, i.e. total 4 events // 2) If HCD publish starts at 1.5th seconds // then events published at 1.5, 3.5, 5.5, 7.5, 9.5 etc. seconds // In this case, subscriber will receive events at 2(initial), 3.5, 5.5, i.e. total 3 events val recEventsCount = subscriptionEventList.toList.size recEventsCount should (be(3) or be(4)) // extract counter values to a List for comparison val counterList = subscriptionEventList.toList.map { case sysEv: SystemEvent if sysEv.contains(hcdCounterKey) => sysEv(hcdCounterKey).head case _ => -1 } // we don't know exactly how long HCD has been running when this test runs, // so we don't know what the first value will be, // but we know we should get three consecutive numbers val expectedCounterList = (0 to 2).toList.map(_ + counterList.head) counterList shouldBe expectedCounterList }
- Java
-
source
@Test public void testShouldBeAbleToSubscribeToHCDEvents() throws InterruptedException, ExecutionException { EventKey counterEventKey = new EventKey(Prefix.apply("csw.samplehcd"), new EventName("HcdCounter")); Key<Integer> hcdCounterKey = JKeyType.IntKey().make("counter", JUnits.NoUnits); IEventService eventService = testKit.jEventService(); IEventSubscriber subscriber = eventService.defaultSubscriber(); // wait for a bit to ensure HCD has started and published an event Thread.sleep(3000); ArrayList<Event> subscriptionEventList = new ArrayList<>(); IEventSubscription subscription = subscriber.subscribeCallback(Set.of(counterEventKey), event -> { // discard invalid event if (!event.isInvalid()) { subscriptionEventList.add(event); } }); subscription.ready().get(); // Sleep for 5 seconds, to allow HCD to publish events Thread.sleep(5000); // Q. Why expected count is either 3 or 4? // A. Total sleep = 7 seconds (2 + 5), subscriber listens for all the events between 2-7 seconds // 1) If HCD publish starts at 1st second // then events published at 1, 3, 5, 7, 9 etc. seconds // In this case, subscriber will receive events at 2(initial), 3, 5, 7, i.e. total 4 events // 2) If HCD publish starts at 1.5th seconds // then events published at 1.5, 3.5, 5.5, 7.5, 9.5 etc. seconds // In this case, subscriber will receive events at 2(initial), 3.5, 5.5, i.e. total 3 events int recEventsCount = subscriptionEventList.size(); Assert.assertTrue("expected:<3> or <4> but was:<" + recEventsCount + ">", recEventsCount == 3 || recEventsCount == 4); // extract counter values to a List for comparison List<Integer> counterList = subscriptionEventList.stream() .map(ev -> { SystemEvent sysEv = ((SystemEvent) ev); if (sysEv.contains(hcdCounterKey)) { return sysEv.parameter(hcdCounterKey).head(); } else { return -1; } }) .collect(Collectors.toList()); // we don't know exactly how long HCD has been running when this test runs, // so we don't know what the first value will be, // but we know we should get three consecutive numbers int counter0 = counterList.get(0); List<Integer> expectedCounterList = Arrays.asList(counter0, counter0 + 1, counter0 + 2); Assert.assertEquals(expectedCounterList, counterList); }
Next, we’ll add a test for command handling in the HCD. The HCD supports a “sleep” command, which sleeps some amount of seconds as specified in the command payload, and then returns a CommandResponse.Completed
. We will specify a sleep of 5 seconds, and then check that we get the expected response. Note that the obtaining a CommandService
reference in Java requires an Akka Typed Actor System, so our code will create one using the Actor System provided by the Test Kit. In Scala, import frameworkTestkit._
brings in implicit Actor System in scope, hence you do not need to create one in your test.
- Scala
-
source
test("should be able to send sleep command to HCD") { import scala.concurrent.duration._ implicit val sleepCommandTimeout: Timeout = Timeout(10000.millis) // Construct Setup command val sleepTimeKey: Key[Long] = KeyType.LongKey.make("SleepTime") val sleepTimeParam: Parameter[Long] = sleepTimeKey.set(5000).withUnits(Units.millisecond) val setupCommand = Setup(Prefix("csw.move"), CommandName("sleep"), Some(ObsId("2020A-001-123"))).add(sleepTimeParam) val connection = AkkaConnection(ComponentId(Prefix(CSW, "samplehcd"), ComponentType.HCD)) val akkaLocation = Await.result(locationService.resolve(connection, 10.seconds), 10.seconds).get val hcd = CommandServiceFactory.make(akkaLocation) // submit command and handle response val responseF = hcd.submitAndWait(setupCommand) Await.result(responseF, 10000.millis) shouldBe a[Completed] }
- Java
-
source
private final ActorSystem<SpawnProtocol.Command> typedActorSystem = testKit.actorSystem(); // DEOPSCSW-39: examples of Location Service @Test public void testShouldBeAbleToSendSleepCommandToHCD() throws ExecutionException, InterruptedException, TimeoutException { // Construct Setup command Key<Long> sleepTimeKey = JKeyType.LongKey().make("SleepTime", JUnits.millisecond); Parameter<Long> sleepTimeParam = sleepTimeKey.set(1000L); Setup setupCommand = new Setup(Prefix.apply(JSubsystem.CSW, "move"), new CommandName(("sleep")), Optional.of(ObsId.apply("2020A-001-123"))).add(sleepTimeParam); Timeout commandResponseTimeout = new Timeout(5, TimeUnit.SECONDS); AkkaConnection connection = new AkkaConnection(new ComponentId(Prefix.apply(JSubsystem.CSW, "samplehcd"), JComponentType.HCD)); ILocationService locationService = testKit.jLocationService(); AkkaLocation location = locationService.resolve(connection, Duration.ofSeconds(5)).get().orElseThrow(); ICommandService hcd = CommandServiceFactory.jMake(location, typedActorSystem); CommandResponse.SubmitResponse result = hcd.submitAndWait(setupCommand, commandResponseTimeout).get(5, TimeUnit.SECONDS); Assert.assertTrue(result instanceof CommandResponse.Completed); }
Finally, we will show an example of tests that check that exceptions are thrown when expected. We will do this by using the “sleep” command, but failing to wait long enough for the sleep to complete. This causes a TimeoutException
in Scala, and an ExecutionException
in Java, and our tests check to see that these types are in fact thrown.
- Scala
-
source
test("should get timeout exception if submit timeout is too small") { import scala.concurrent.duration._ implicit val sleepCommandTimeout: Timeout = Timeout(1000.millis) // Construct Setup command val sleepTimeKey: Key[Long] = KeyType.LongKey.make("SleepTime") val sleepTimeParam: Parameter[Long] = sleepTimeKey.set(5000).withUnits(Units.millisecond) val setupCommand = Setup(Prefix("csw.move"), CommandName("sleep"), Some(ObsId("2020A-001-123"))).add(sleepTimeParam) val connection = AkkaConnection(ComponentId(Prefix(CSW, "samplehcd"), ComponentType.HCD)) val akkaLocation = Await.result(locationService.resolve(connection, 10.seconds), 10.seconds).get val hcd = CommandServiceFactory.make(akkaLocation) // submit command and handle response intercept[java.util.concurrent.TimeoutException] { val responseF = hcd.submitAndWait(setupCommand) Await.result(responseF, 10000.millis) shouldBe a[Completed] } }
- Java
-
source
@Test public void testShouldGetExecutionExceptionIfSubmitTimeoutIsTooSmall() throws ExecutionException, InterruptedException { // Construct Setup command Key<Long> sleepTimeKey = JKeyType.LongKey().make("SleepTime", JUnits.millisecond); Parameter<Long> sleepTimeParam = sleepTimeKey.set(5000L); Setup setupCommand = new Setup(Prefix.apply(JSubsystem.CSW, "move"), new CommandName("sleep"), Optional.of(ObsId.apply("2020A-001-123"))).add(sleepTimeParam); Timeout commandResponseTimeout = new Timeout(1, TimeUnit.SECONDS); AkkaConnection connection = new AkkaConnection(new ComponentId(Prefix.apply(JSubsystem.CSW, "samplehcd"), JComponentType.HCD)); ILocationService locationService = testKit.jLocationService(); AkkaLocation location = locationService.resolve(connection, Duration.ofSeconds(10)).get().orElseThrow(); ICommandService hcd = CommandServiceFactory.jMake(location, typedActorSystem); Assert.assertThrows(ExecutionException.class, () -> hcd.submitAndWait(setupCommand, commandResponseTimeout).get()); }
Other ways to spawn Assembly and HCD in the standalone mode using CSW Test Kit
Spawning a HCD
- Scala
-
source
spawnHCD(Prefix("TCS.sampleHcd"), (ctx, cswCtx) => new SampleHcdHandlers(ctx, cswCtx))
Spawning an Assembly
- Scala
-
source
spawnAssembly(Prefix("TCS.sampleAssembly"), (ctx, cswCtx) => new SampleHandlers(ctx, cswCtx))
Spawning a Component using DefaultComponentHandlers
- Scala
-
source
spawnAssembly( Prefix("TCS.defaultAssembly"), (ctx, cswCtx) => new DefaultComponentHandlers(ctx, cswCtx) { override def onSubmit(runId: Id, controlCommand: ControlCommand): CommandResponse.SubmitResponse = { controlCommand.commandName.name match { case "move" => cswCtx.timeServiceScheduler.scheduleOnce(UTCTime(UTCTime.now().value.plusSeconds(5))) { cswCtx.commandResponseManager.updateCommand(Completed(runId)) } Started(runId) case _ => Completed(runId) } } } )
Full source at GitHub