Events

Events are the most basic type of asynchronous notification in TMT when an activity occurs somewhere in the TMT system and other components need to be notified. Each type of event has a unique purpose and unique information, but they all share same structural features. All events have EventInfo and ParameterSet.

EventTime

It captures the instance of a time in UTC format. To create current instance of time use default constructor. For other utility functions, see below examples:

Scala

//default constructor will return current time in UTC
val now: EventTime = EventTime()

//using constructor
val anHourAgo: EventTime = EventTime(Instant.now().minusSeconds(3600))

//current event time using utility function
val currentTime: EventTime = EventTime()

//some past time using utility function
val aDayAgo = EventTime(Instant.now.minusSeconds(86400))

Java

//apply returns current time in UTC
EventTime now = EventTime.apply();

//using constructor
EventTime anHourAgo = new EventTime(Instant.now().minusSeconds(3600));

//return current time in UTC
EventTime currentTime = EventTime.apply();

//some past time using utility function
EventTime aDayAgo = EventTime.apply(Instant.now().minusSeconds(86400));

System Event

SystemEvent is used to describe a demand or other algorithm input from one component to the other. It is also used to publish internal state or status values of a component that may be of interest to other components in the system.

Scala

//keys
val k1: Key[Int]    = KeyType.IntKey.make("encoder")
val k2: Key[Int]    = KeyType.IntKey.make("speed")
val k3: Key[String] = KeyType.StringKey.make("filter")
val k4: Key[Int]    = KeyType.IntKey.make("notUsed")

//prefixes
val ck1   = Prefix("wfos.red.filter")
val name1 = EventName("filterWheel")
val ck3   = Prefix("iris.imager.filter")
val name3 = EventName("status")

//parameters
val p1: Parameter[Int]    = k1.set(22)
val p2: Parameter[Int]    = k2.set(44)
val p3: Parameter[String] = k3.set("A", "B", "C", "D")

//Create SystemEvent using madd
val se1: SystemEvent = SystemEvent(ck1, name1).madd(p1, p2)
//Create SystemEvent using apply
val se2: SystemEvent = SystemEvent(ck3, name3, Set(p1, p2))
//Create SystemEvent and use add
val se3: SystemEvent = SystemEvent(ck3, name3).add(p1).add(p2).add(p3)

//access keys
val k1Exists: Boolean = se1.exists(k1) //true

//access Parameters
val p4: Option[Parameter[Int]] = se1.get(k1)

//access values
val v1: Array[Int] = se1(k1).values
val v2: Array[Int] = se2.parameter(k2).values
//k4 is missing
val missingKeys: Set[String] = se3.missingKeys(k1, k2, k3, k4)

//remove keys
val se4: SystemEvent = se3.remove(k3)

//add more than one parameters, using madd
val se5: SystemEvent = se4.madd(k3.set("X", "Y", "Z").withUnits(Units.day), k4.set(99, 100))
val paramSize: Int   = se5.size

//update existing key with set
val se6: SystemEvent = se5.add(k2.set(5, 6, 7, 8))

Java

//keys
Key<Integer> k1 = JKeyType.IntKey().make("encoder");
Key<Integer> k2 = JKeyType.IntKey().make("speed");
Key<String> k3 = JKeyType.StringKey().make("filter");
Key<Integer> k4 = JKeyType.IntKey().make("notUsed");

//prefixes
Prefix prefix1 = new Prefix("wfos.red.filter");
EventName name1 = new EventName("filterWheel");
Prefix prefix2 = new Prefix("iris.imager.filter");
EventName name2 = new EventName("status");

//parameters
Parameter<Integer> p1 = k1.set(22);
Parameter<Integer> p2 = k2.set(44);
Parameter<String> p3 = k3.set("A", "B", "C", "D");

//Create SystemEvent using madd
SystemEvent se1 = new SystemEvent(prefix1, name1).madd(p1, p2);
//Create SystemEvent using add
SystemEvent se2 = new SystemEvent(prefix2, name2).add(p1).add(p2);
//Create SystemEvent and use add
SystemEvent se3 = new SystemEvent(prefix2, name2).add(p1).add(p2).add(p3);

//access keys
Boolean k1Exists = se1.exists(k1); //true

//access Parameters
Optional<Parameter<Integer>> p4 = se1.jGet(k1);

//access values
List<Integer> v1 = se1.jGet(k1).get().jValues();
List<Integer> v2 = se2.parameter(k2).jValues();
//k4 is missing
Set<String> missingKeys = se3.jMissingKeys(k1, k2, k3, k4);

//remove keys
SystemEvent se4 = se3.remove(k3);

Observe Event

ObserveEvent is used to describe an event within a standardized data acquisition process. Published only by Science Detector Assemblies, who emit ObserveEvents during their exposures to signal the occurrence of specific activities/actions during the acquisition of data. Observe Events are published by the detector system using the Event Service.

Scala

//keys
val k1: Key[Int]    = KeyType.IntKey.make("readoutsCompleted")
val k2: Key[Int]    = KeyType.IntKey.make("coaddsCompleted")
val k3: Key[String] = KeyType.StringKey.make("fileID")
val k4: Key[Int]    = KeyType.IntKey.make("notUsed")

//prefixes
val ck1   = Prefix("iris.ifu.detectorAssembly")
val name1 = EventName("readoutEnd")
val ck3   = Prefix("wfos.red.detector")
val name3 = EventName("exposureStarted")

//parameters
val p1: Parameter[Int]    = k1.set(4)
val p2: Parameter[Int]    = k2.set(2)
val p3: Parameter[String] = k3.set("WFOS-RED-0001")

//Create ObserveEvent using madd
val se1: ObserveEvent = ObserveEvent(ck1, name1).madd(p1, p2)
//Create ObserveEvent using apply
val se2: ObserveEvent = ObserveEvent(ck3, name3, Set(p1, p2))
//Create ObserveEvent and use add
val se3: ObserveEvent = ObserveEvent(ck3, name3).add(p1).add(p2).add(p3)

//access keys
val k1Exists: Boolean = se1.exists(k1) //true

//access Parameters
val p4: Option[Parameter[Int]] = se1.get(k1)

//access values
val v1: Array[Int] = se1(k1).values
val v2: Array[Int] = se2.parameter(k2).values
//k4 is missing
val missingKeys: Set[String] = se3.missingKeys(k1, k2, k3, k4)

//remove keys
val se4: ObserveEvent = se3.remove(k3)

Java

//keys
Key<Integer> k1 = JKeyType.IntKey().make("readoutsCompleted");
Key<Integer> k2 = JKeyType.IntKey().make("coaddsCompleted");
Key<String> k3 = JKeyType.StringKey().make("fileID");
Key<Integer> k4 = JKeyType.IntKey().make("notUsed");

//prefixes
Prefix prefix1 = new Prefix("iris.ifu.detectorAssembly");
EventName name1 = new EventName("readoutEnd");
Prefix prefix2 = new Prefix("wfos.red.detector");
EventName name2 = new EventName("exposureStarted");

//parameters
Parameter<Integer> p1 = k1.set(4);
Parameter<Integer> p2 = k2.set(2);
Parameter<String> p3 = k3.set("WFOS-RED-0001");

//Create ObserveEvent using madd
ObserveEvent oc1 = new ObserveEvent(prefix1, name1).madd(p1, p2);
//Create ObserveEvent using add
ObserveEvent oc2 = new ObserveEvent(prefix2, name2).add(p1).add(p2);
//Create ObserveEvent and use add
ObserveEvent oc3 = new ObserveEvent(prefix2, name2).add(p1).add(p2).add(p3);

//access keys
Boolean k1Exists = oc1.exists(k1); //true

//access Parameters
Optional<Parameter<Integer>> p4 = oc1.jGet(k1);

//access values
List<Integer> v1 = oc1.jGet(k1).get().jValues();
List<Integer> v2 = oc2.parameter(k2).jValues();
//k4 is missing
Set<String> missingKeys = oc3.jMissingKeys(k1, k2, k3, k4);

//remove keys
ObserveEvent oc4 = oc3.remove(k3);

JSON Serialization

Events can be serialized to JSON. The library has provided JsonSupport helper class and methods to serialize Status, Observe and System events.

Scala

import play.api.libs.json.{JsValue, Json}

//key
val k1: Key[MatrixData[Double]] = DoubleMatrixKey.make("myMatrix")

val name1  = EventName("correctionInfo")
val prefix = Prefix("aoesw.rpg")

//values
val m1: MatrixData[Double] = MatrixData.fromArrays(
  Array(1.0, 2.0, 3.0),
  Array(4.1, 5.1, 6.1),
  Array(7.2, 8.2, 9.2)
)
//parameter
val i1: Parameter[MatrixData[Double]] = k1.set(m1)
//events
val observeEvent: ObserveEvent = ObserveEvent(prefix, name1).add(i1)
val systemEvent: SystemEvent   = SystemEvent(prefix, name1).add(i1)

//json support - write
val observeJson: JsValue = JsonSupport.writeEvent(observeEvent)
val systemJson: JsValue  = JsonSupport.writeEvent(systemEvent)

//optionally prettify
val str: String = Json.prettyPrint(systemJson)

//construct command from string
val systemEventFromPrettyStr: SystemEvent = JsonSupport.readEvent[SystemEvent](Json.parse(str))

//json support - read
val observeEvent1: ObserveEvent = JsonSupport.readEvent[ObserveEvent](observeJson)
val systemEvent1: SystemEvent   = JsonSupport.readEvent[SystemEvent](systemJson)

Java

//key
Key<MatrixData<Double>> k1 = JKeyType.DoubleMatrixKey().make("myMatrix");

//prefixes
Prefix prefix1 = new Prefix("aoesw.rpg");
EventName name1 = new EventName("correctionInfo");

//values
Double[][] doubles = {{1.0, 2.0, 3.0}, {4.1, 5.1, 6.1}, {7.2, 8.2, 9.2}};
MatrixData<Double> m1 = MatrixData.fromJavaArrays(Double.class, doubles);

//parameter
Parameter<MatrixData<Double>> i1 = k1.set(m1);


//events
ObserveEvent observeEvent = new ObserveEvent(prefix1, name1).add(i1);
SystemEvent systemEvent = new SystemEvent(prefix1, name1).add(i1);

//json support - write
JsValue observeJson = JavaJsonSupport.writeEvent(observeEvent);
JsValue systemJson = JavaJsonSupport.writeEvent(systemEvent);

//optionally prettify
String str = Json.prettyPrint(systemJson);

//construct DemandState from string
SystemEvent statusFromPrettyStr = JavaJsonSupport.readEvent(Json.parse(str));

//json support - read
ObserveEvent observeEvent1 = JavaJsonSupport.readEvent(observeJson);
SystemEvent systemEvent1 = JavaJsonSupport.readEvent(systemJson);

Unique Key Constraint

By choice, a ParameterSet in either ObserveEvent or SystemEvent event will be optimized to store only unique keys. When using add or madd methods on events to add new parameters, if the parameter being added has a key which is already present in the paramSet, the already stored parameter will be replaced by the given parameter.

Here are some examples that illustrate this point:

Scala

//keys
val encoderKey: Key[Int] = KeyType.IntKey.make("encoder")
val filterKey: Key[Int]  = KeyType.IntKey.make("filter")
val miscKey: Key[Int]    = KeyType.IntKey.make("misc")

//prefix
val prefix = Prefix("wfos.blue.filter")

val name1 = EventName("filterWheel")

//params
val encParam1 = encoderKey.set(1)
val encParam2 = encoderKey.set(2)

val encParam3    = encoderKey.set(3)
val filterParam1 = filterKey.set(1)
val filterParam2 = filterKey.set(2)

val filterParam3 = filterKey.set(3)

val miscParam1 = miscKey.set(100)
//StatusEvent with duplicate key via constructor
val systemEvent =
  SystemEvent(prefix, name1, Set(encParam1, encParam2, encParam3, filterParam1, filterParam2, filterParam3))
//four duplicate keys are removed; now contains one Encoder and one Filter key
val uniqueKeys1 = systemEvent.paramSet.toList.map(_.keyName)

//try adding duplicate keys via add + madd
val changedStatusEvent = systemEvent
  .add(encParam3)
  .madd(
    filterParam1,
    filterParam2,
    filterParam3
  )
//duplicate keys will not be added. Should contain one Encoder and one Filter key
val uniqueKeys2 = changedStatusEvent.paramSet.toList.map(_.keyName)

//miscKey(unique) will be added; encoderKey(duplicate) will not be added
val finalStatusEvent = systemEvent.madd(Set(miscParam1, encParam1))
//now contains encoderKey, filterKey, miscKey
val uniqueKeys3 = finalStatusEvent.paramSet.toList.map(_.keyName)

Java

//keys
Key<Integer> encoderKey = JKeyType.IntKey().make("encoder");
Key<Integer> filterKey = JKeyType.IntKey().make("filter");
Key<Integer> miscKey = JKeyType.IntKey().make("misc");

//prefix
Prefix prefix1 = new Prefix("wfos.blue.filter");
EventName name1 = new EventName("filterWheel");

//params
Parameter<Integer> encParam1 = encoderKey.set(1);
Parameter<Integer> encParam2 = encoderKey.set(2);
Parameter<Integer> encParam3 = encoderKey.set(3);

Parameter<Integer> filterParam1 = filterKey.set(1);
Parameter<Integer> filterParam2 = filterKey.set(2);
Parameter<Integer> filterParam3 = filterKey.set(3);

Parameter<Integer> miscParam1 = miscKey.set(100);

//StatusEvent with duplicate key via madd
SystemEvent event = new SystemEvent(prefix1, name1).madd(
        encParam1,
        encParam2,
        encParam3,
        filterParam1,
        filterParam2,
        filterParam3);
//four duplicate keys are removed; now contains one Encoder and one Filter key
List<String> uniqueKeys1 = event.jParamSet().stream().map(Parameter::keyName).collect(Collectors.toList());

//try adding duplicate keys via add + madd
SystemEvent changedEvent = event.add(encParam3).madd(filterParam1, filterParam2, filterParam3);
//duplicate keys will not be added. Should contain one Encoder and one Filter key
List<String> uniqueKeys2 = changedEvent.jParamSet().stream().map(Parameter::keyName).collect(Collectors.toList());

//miscKey(unique) will be added; encoderKey(duplicate) will not be added
SystemEvent finalEvent = changedEvent.madd(miscParam1, encParam1);
//now contains encoderKey, filterKey, miscKey
List<String> uniqueKeys3 = finalEvent.jParamSet().stream().map(Parameter::keyName).collect(Collectors.toList());

Protobuf

Protobuf aka Protocol buffers, are a language-neutral, platform-neutral extensible mechanism for serializing structured data. For more, visit Protobuf home page

In TMT observatory, subsystems and components could be running on JVM(Java virtual machine) and Non-JVM platform. This leads to solving a non-trivial problem of a Non-JVM component wanting to consume an Event produced by a JVM component. Amongst the available options for data over the wire, Protobuf was chosen for its performance, data compression and official/unofficial support many mainstream languages.

csw-messages library enhances the Protobuf support, by providing out of the box helper methods, to convert events from/to protobuf binary data.

Here are some examples:

Scala

//Key
val raDecKey = RaDecKey.make("raDecKey")

//values
val raDec1 = RaDec(10.20, 40.20)
val raDec2 = RaDec(11.20, 50.20)

//parameters
val param = raDecKey.set(raDec1, raDec2).withUnits(arcmin)

val prefix = Prefix("tcs.pk")
val name   = EventName("targetCoords")
//events
val observeEvent: ObserveEvent = ObserveEvent(prefix, name).add(param)
val systemEvent1: SystemEvent  = SystemEvent(prefix, name).add(param)
val systemEvent2: SystemEvent =
  SystemEvent(prefix, name).add(param)

//convert events to protobuf bytestring
val byteArray2: PbEvent = PbConverter.toPbEvent(observeEvent)
val byteArray3: PbEvent = PbConverter.toPbEvent(systemEvent1)
val byteArray4: PbEvent = PbConverter.toPbEvent(systemEvent2)

//convert protobuf bytestring to events
val pbObserveEvent: ObserveEvent = PbConverter.fromPbEvent(byteArray2)
val pbSystemEvent1: SystemEvent  = PbConverter.fromPbEvent(byteArray3)
val pbSystemEvent2: SystemEvent  = PbConverter.fromPbEvent(byteArray4)

Java

//prefixes
Prefix prefix1 = new Prefix("tcs.pk");
EventName name1 = new EventName("targetCoords");
Prefix prefix2 = new Prefix("tcs.cm");
EventName name2 = new EventName("guiderCoords");

//Key
Key<RaDec> raDecKey = JKeyType.RaDecKey().make("raDecKey");

//values
RaDec raDec1 = new RaDec(10.20, 40.20);
RaDec raDec2 = new RaDec(11.20, 50.20);

//parameters
Parameter<RaDec> param = raDecKey.set(raDec1, raDec2).withUnits(JUnits.arcmin);

//events
ObserveEvent observeEvent = new ObserveEvent(prefix1, name1).add(param);
SystemEvent systemEvent1 = new SystemEvent(prefix1, name1).add(param);
SystemEvent systemEvent2 = new SystemEvent(prefix2, name2).add(param);

//convert events to protobuf bytestring
PbEvent byteArray2 = PbConverter.toPbEvent(observeEvent);
PbEvent byteArray3 = PbConverter.toPbEvent(systemEvent1);
PbEvent byteArray4 = PbConverter.toPbEvent(systemEvent2);

//convert protobuf bytestring to events
ObserveEvent pbObserveEvent = PbConverter.fromPbEvent(byteArray2);
SystemEvent pbSystemEvent1 =  PbConverter.fromPbEvent(byteArray3);
SystemEvent pbSystemEvent2 =  PbConverter.fromPbEvent(byteArray4);

Source Code for Examples

• Scala Example
• Java Example