Adding Unit Tests

Unit testing is a fundamental part of programming, and essential component of the TMT quality assurance policy. TMT CSW extensively uses unit testing for both Scala and Java code, using ScalaTest for the former, and primarily JUnit for the latter (TestNG is used in one instance for the Event Service). While this guide will not attempt to educate the reader on how to use these popular packages, it will serve to show some examples of how tests can be created for component code and demonstrate some tools provided by CSW to simplify and enable integration of TMT components and other software with CSW and its services.

CSW Test Kit

CSW provides a set of tools for use by developers called the CSW Test Kit. This allows the developer to start CSW services within the testing environment, so that they can be accessed by the components and/or applications being tested, as well as the testing fixtures themselves. It also provides simple methods to start components or a set of components within a container, as well as an ActorContext to be used if other Actors are needed to be created in the tests.

More information about testing with CSW and the CSW Test Kit can be found here.

Tutorial: Writing unit tests for our components

In this part of the tutorial, we will write a few unit tests for our components. These tests are in no way meant to be comprehensive, but hopefully, they show enough to get you started.

The giter8 template provides the required directory structure, and skeletons for tests of the HCD and Assembly in both Java and Scala. It also provides some Component Configuration (ComponentInfo) files for running each of the HCD and Assembly in standalone mode for both languages. They are there for convenience, but may not be required depending your deployment and testing strategy. We will be using them in our tutorial.

We will first look at the tests for the Assembly. As described on the Testing Manual page, the Scala version overrides the CSW-provided superclass csw.testkit.scaladsl.ScalaTestFrameworkTestKit to get access to the services it needs. By passing in the needed services in the constructor, those services are started in the superclass’s beforeAll method. In the Java version, we must create an instance of csw.testkit.javadsl.FrameworkTestKitJunitResource to get access to and start our services, with the services we want to start passed into the constructor of this object.

Scala

class SampleTest extends ScalaTestFrameworkTestKit(AlarmServer, EventServer) with AnyFunSuiteLike {
  import frameworkTestKit.frameworkWiring._

Java

public class JSampleTest extends JUnitSuite {
    @ClassRule
    public static final FrameworkTestKitJunitResource testKit =
            new FrameworkTestKitJunitResource(Arrays.asList(JCSWService.AlarmServer, JCSWService.EventServer));

For our tests, we will want to run the Assembly first. We will do this in the beforeAll method in Scala, and in a method with a @BeforeClass annotation in Java, so that it is run only once, before any of the tests are run. The Component Configuration files use are the one provided by the giter8 template. Note that for Scala, we must call the superclass’s beforeAll method to ensure the services are run.

Scala

override def beforeAll(): Unit = {
  super.beforeAll()
  spawnStandalone(com.typesafe.config.ConfigFactory.load("SampleStandalone.conf"))
}

Java

@BeforeClass
public static void setup() {
    testKit.spawnStandalone(com.typesafe.config.ConfigFactory.load("JSampleStandalone.conf"));
}

Next, let’s add a test. We will add a simple test that uses the Location Service to make sure the Assembly is running and resolve the registration information for it.

Scala

import scala.concurrent.duration._
test("Assembly should be locatable using Location Service") {
  val connection   = AkkaConnection(ComponentId(Prefix(CSW, "sample"), ComponentType.Assembly))
  val akkaLocation = Await.result(locationService.resolve(connection, 10.seconds), 10.seconds).get

  akkaLocation.connection shouldBe connection
}

Java

@Test
public void testAssemblyShouldBeLocatableUsingLocationService() throws ExecutionException, InterruptedException {
    AkkaConnection connection = new AkkaConnection(new ComponentId(Prefix.apply(JSubsystem.CSW, "sample"), JComponentType.Assembly));
    ILocationService locationService = testKit.jLocationService();
    AkkaLocation location = locationService.resolve(connection, Duration.ofSeconds(10)).get().orElseThrow();

    Assert.assertEquals(location.connection(), connection);
}

You can try running the test either using sbt (sbt test from the project root directory) or directly in the IDE. If you are using IntelliJ, you can run the test by right-clicking on the file in the project explorer and clicking on Run 'SampleTest' or Run 'JSampleTest'. You can also right-click in the class body or the specific test body, if you want to run a single test.

The Assembly we have written does not have much of a public API, so we’ll turn to the HCD now, which has a few additional things we can test, including the publishing of Events and the handling of commands.

First, we will set up the test fixtures similarly as we did for the Assembly, and add a similar test to show the component registers itself with the Location Service on startup.

Scala

class SampleHcdTest extends ScalaTestFrameworkTestKit(AlarmServer, EventServer) with AnyFunSuiteLike with BeforeAndAfterEach {
  import frameworkTestKit.frameworkWiring._

  override def beforeAll(): Unit = {
    super.beforeAll()
    spawnStandalone(com.typesafe.config.ConfigFactory.load("SampleHcdStandalone.conf"))
  }

  import scala.concurrent.duration._
  test("HCD should be locatable using Location Service") {
    val connection   = AkkaConnection(ComponentId(Prefix("CSW.samplehcd"), ComponentType.HCD))
    val akkaLocation = Await.result(locationService.resolve(connection, 10.seconds), 10.seconds).get

    akkaLocation.connection shouldBe connection
  }

Java

public class JSampleHcdTest extends JUnitSuite {

    @ClassRule
    public static final FrameworkTestKitJunitResource testKit =
            new FrameworkTestKitJunitResource(Arrays.asList(JCSWService.AlarmServer, JCSWService.EventServer));


    @BeforeClass
    public static void setup() {
        testKit.spawnStandalone(com.typesafe.config.ConfigFactory.load("JSampleHcdStandalone.conf"));
    }

    @Test
    public void testHCDShouldBeLocatableUsingLocationService() throws ExecutionException, InterruptedException {
        AkkaConnection connection = new AkkaConnection(new ComponentId(Prefix.apply(JSubsystem.CSW, "samplehcd"), JComponentType.HCD));
        ILocationService locationService = testKit.jLocationService();
        AkkaLocation location = locationService.resolve(connection, Duration.ofSeconds(10)).get().orElseThrow();

        Assert.assertEquals(connection, location.connection());
    }

Now let’s add a test to verify our component is publishing. We will set up a test subscriber to the counterEvent Events published by the HCD. Since we cannot guarantee the order in which the tests are run, we cannot be certain how long the component has been running when this specific test is run. Therefore, checking the contents of the Events received is tricky. We will wait a bit at the start of the test to ensure we don’t get a InvalidEvent, which would be returned if we start our subscription before the HCD publishes any Events. Then, after setting up the subscription, we wait 5 seconds to allow the HCD to publish two additional Events plus the one we receive when the subscription starts. We will look at the counter value of the first counterEvent to determine what the set of counter values we expect to get in our subscription.

Scala

test("should be able to subscribe to HCD events") {
  val counterEventKey = EventKey(Prefix("CSW.samplehcd"), EventName("HcdCounter"))
  val hcdCounterKey   = KeyType.IntKey.make("counter")

  val eventService = eventServiceFactory.make(locationService)(actorSystem)
  val subscriber   = eventService.defaultSubscriber

  // wait for a bit to ensure HCD has started and published an event
  Thread.sleep(2000)

  val subscriptionEventList = mutable.ListBuffer[Event]()
  subscriber.subscribeCallback(Set(counterEventKey), { ev => subscriptionEventList.append(ev) })

  // Sleep for 5 seconds, to allow HCD to publish events
  Thread.sleep(5000)

  // Event publishing period is 2 seconds.
  // Expecting 3 events: first event on subscription
  // and two more events 2 and 4 seconds later.
  subscriptionEventList.toList.size shouldBe 3

  // extract counter values to a List for comparison
  val counterList = subscriptionEventList.toList.map {
    case sysEv: SystemEvent if sysEv.contains(hcdCounterKey) => sysEv(hcdCounterKey).head
    case _                                                   => -1
  }

  // we don't know exactly how long HCD has been running when this test runs,
  // so we don't know what the first value will be,
  // but we know we should get three consecutive numbers
  val expectedCounterList = (0 to 2).toList.map(_ + counterList.head)

  counterList shouldBe expectedCounterList
}

Java

@Test
public void testShouldBeAbleToSubscribeToHCDEvents() throws InterruptedException {
    EventKey counterEventKey = new EventKey(Prefix.apply(JSubsystem.CSW, "samplehcd"), new EventName("HcdCounter"));
    Key<Integer> hcdCounterKey = JKeyType.IntKey().make("counter");

    IEventService eventService = testKit.jEventService();
    IEventSubscriber subscriber = eventService.defaultSubscriber();

    // wait for a bit to ensure HCD has started and published an event
    Thread.sleep(5000);

    ArrayList<Event> subscriptionEventList = new ArrayList<>();
    subscriber.subscribeCallback(Set.of(counterEventKey), subscriptionEventList::add);

    // Sleep for 4 seconds, to allow HCD to publish events
    Thread.sleep(4000);

    // Event publishing period is 2 seconds.
    // Expecting 3 events: first event on subscription
    // and two more events 2 and 4 seconds later.
    Assert.assertEquals(3, subscriptionEventList.size());

    // extract counter values to a List for comparison
    List<Integer> counterList = subscriptionEventList.stream()
            .map(ev -> {
                SystemEvent sysEv = ((SystemEvent) ev);
                if (sysEv.contains(hcdCounterKey)) {
                    return sysEv.parameter(hcdCounterKey).head();
                } else {
                    return -1;
                }
            })
            .collect(Collectors.toList());

    // we don't know exactly how long HCD has been running when this test runs,
    // so we don't know what the first value will be,
    // but we know we should get three consecutive numbers
    int counter0 = counterList.get(0);
    List<Integer> expectedCounterList = Arrays.asList(counter0, counter0 + 1, counter0 + 2);

    Assert.assertEquals(expectedCounterList, counterList);
}

Next, we’ll add a test for command handling in the HCD. The HCD supports a “sleep” command, which sleeps some amount of seconds as specified in the command payload, and then returns a CommandResponse.Completed. We will specify a sleep of 5 seconds, and then check that we get the expected response. Note that the obtaining a CommandService reference requires an Akka Typed Actor System, so our code will create one using the Actor System provided by the Test Kit.

Scala

implicit val typedActorSystem: ActorSystem[_] = actorSystem
test("basic: should be able to send sleep command to HCD") {
  import scala.concurrent.duration._
  implicit val sleepCommandTimeout: Timeout = Timeout(10000.millis)

  // Construct Setup command
  val testPrefix: Prefix = Prefix("CSW.test")

  // Helper to get units set
  def setSleepTime(setup: Setup, milli: Long): Setup = setup.add(sleepTimeKey.set(milli).withUnits(Units.millisecond))

  val setupCommand = setSleepTime(Setup(testPrefix, hcdSleep, Some(ObsId("2018A-001"))), 5000)

  val connection = AkkaConnection(ComponentId(Prefix(Subsystem.CSW, "samplehcd"), ComponentType.HCD))

  val akkaLocation = Await.result(locationService.resolve(connection, 10.seconds), 10.seconds).get

  val hcd = CommandServiceFactory.make(akkaLocation)
  // submit command and handle response
  val responseF = hcd.submitAndWait(setupCommand)

  Await.result(responseF, 10000.millis) shouldBe a[CommandResponse.Completed]
}

Java

private ActorSystem<SpawnProtocol.Command> typedActorSystem = testKit.actorSystem();

// DEOPSCSW-39: examples of Location Service
@Test
public void testShouldBeAbleToSendSleepCommandToHCD() throws ExecutionException, InterruptedException, TimeoutException {

    // Construct Setup command
    Parameter<Long> sleepTimeParam = sleepTimeKey.set(5000L).withUnits(JUnits.millisecond);

    Setup setupCommand = new Setup(Prefix.apply(JSubsystem.CSW, "move"), hcdSleep, Optional.of(new ObsId("2018A-001"))).add(sleepTimeParam);

    Timeout commandResponseTimeout = new Timeout(10, TimeUnit.SECONDS);

    AkkaConnection connection = new AkkaConnection(new ComponentId(Prefix.apply(JSubsystem.CSW, "samplehcd"), JComponentType.HCD));
    ILocationService locationService = testKit.jLocationService();
    AkkaLocation location = locationService.resolve(connection, Duration.ofSeconds(10)).get().orElseThrow();

    ICommandService hcd = CommandServiceFactory.jMake(location, typedActorSystem);

    CommandResponse.SubmitResponse result = hcd.submitAndWait(setupCommand, commandResponseTimeout).get(10, TimeUnit.SECONDS);
    Assert.assertTrue(result instanceof CommandResponse.Completed);
}

Finally, we will show an example of tests that check that exceptions are thrown when expected. We will do this by using the “sleep” command, but failing to wait long enough for the sleep to complete. This causes a TimeoutException in Scala, and an ExecutionException in Java, and our tests check to see that these types are in fact thrown.

Scala

test("should get timeout exception if submit timeout is too small") {
  import scala.concurrent.duration._
  implicit val sleepCommandTimeout: Timeout = Timeout(1000.millis)

  // Construct Setup command
  val testPrefix: Prefix    = Prefix("CSW.test")
  val hcdSleep: CommandName = CommandName("hcdSleep")
  // Helper to get units set
  def setSleepTime(milli: Long): Parameter[Long] = sleepTimeKey.set(milli).withUnits(Units.millisecond)

  val setupCommand = Setup(testPrefix, hcdSleep, Some(ObsId("2018A-001"))).add(setSleepTime(5000))

  val connection = AkkaConnection(ComponentId(Prefix(Subsystem.CSW, "samplehcd"), ComponentType.HCD))

  val akkaLocation = Await.result(locationService.resolve(connection, 10.seconds), 10.seconds).get

  val hcd = CommandServiceFactory.make(akkaLocation)

  // submit command and handle response
  intercept[java.util.concurrent.TimeoutException] {
    val responseF = hcd.submitAndWait(setupCommand)
    Await.result(responseF, 10000.millis) shouldBe a[CommandResponse.Completed]
  }
}

Java

@Rule
public ExpectedException thrown = ExpectedException.none();

@Test
public void testShouldGetExecutionExceptionIfSubmitTimeoutIsTooSmall() throws ExecutionException, InterruptedException {

    // Construct Setup command
    Key<Long> sleepTimeKey = JKeyType.LongKey().make("SleepTime");
    Parameter<Long> sleepTimeParam = sleepTimeKey.set(5000L).withUnits(JUnits.millisecond);

    Setup setupCommand = new Setup(Prefix.apply(JSubsystem.CSW, "move"), hcdSleep, Optional.of(new ObsId("2018A-001"))).add(sleepTimeParam);

    Timeout commandResponseTimeout = new Timeout(1, TimeUnit.SECONDS);

    AkkaConnection connection = new AkkaConnection(new ComponentId(Prefix.apply(JSubsystem.CSW, "samplehcd"), JComponentType.HCD));
    ILocationService locationService = testKit.jLocationService();
    AkkaLocation location = locationService.resolve(connection, Duration.ofSeconds(10)).get().orElseThrow();

    ICommandService hcd = CommandServiceFactory.jMake(location, typedActorSystem);

    thrown.expect(ExecutionException.class);
    hcd.submitAndWait(setupCommand, commandResponseTimeout).get();
}