Packages

package root

Definition Classes: root

package csw

Definition Classes: root

package framework

This project contains the framework for creating components, such as HCDs and Assemblies.

Framework

This project contains the framework for creating components, such as HCDs and Assemblies.

A component is implemented by extending the csw.framework.scaladsl.ComponentHandlers base class. These handlers are executed under an Supervisor Actor: csw.framework.internal.supervisor.SupervisorBehavior and TLA Actor (Top Level Actor): csw.framework.internal.component.ComponentBehavior defined in the framework which handles the lifecycle and supervision of this component.

Components are controlled by a csw.framework.internal.supervisor.SupervisorBehavior actor that intercepts common messages (ex. Shutdown, Restart) or lifecycle messages (ex. GoOffline or GoOnline) sent from external entities to determine the state of the component.

Components that are not in the csw.messages.framework.SupervisorLifecycleState.Running state, do not receive csw.messages.CommandMessage.

When Component is created using this framework, it guarantees that component (HCD/Assembly) is registered with LocationService only when Component moves to csw.messages.framework.SupervisorLifecycleState.Running state. That means, one component can resolve other component only when its in Running state and ready to accept messages.CommandMessage

Important Actors in Framework

- csw.framework.internal.supervisor.SupervisorBehavior : Each component created by this framework is supervised by this actor.

Default strategy of supervisor is to stop child actor but depending on nature of the failure, the supervisor has a choice of the following two options:

csw.framework.exceptions.FailureRestart : Restart the child actor i.e. kill the current child actor that failed and create a new one in its place
csw.framework.exceptions.FailureStop : Stop the child actor permanently

Supervisor changes its csw.messages.framework.SupervisorLifecycleState based on the messages it receives. Decision to handle external messages or not is taken by the supervisor based on its current csw.messages.framework.SupervisorLifecycleState. For complete list of supported messages per csw.messages.framework.SupervisorLifecycleState, see csw-messages project.

- csw.framework.internal.component.ComponentBehavior : Like Supervisor, evey component is associate with this actor which is known as TLA (Top Level Actor) And it also maintains its own state csw.framework.internal.component.ComponentLifecycleState based on messages it receives.

Main purpose of this actor is to invoke component specific code written in their corresponding handlers. This is where framework code meets Component specific code.

- csw.framework.internal.pubsub.PubSubBehavior : This actor is created by framework which is wrapped into csw.framework.CurrentStatePublisher for easy interaction with this actor and then passed to component handlers so that component can publish their csw.messages.params.states.CurrentState.

If one component (ex. Assembly) is interested in csw.messages.params.states.CurrentState published by other component (ex. HCD) then Assembly can subscribe to HCD's current state.

PubSub actor maintains the list of subscribers and keeps publishing csw.messages.params.states.CurrentState to all subscribers.

- csw.framework.internal.container.ContainerBehavior : When multiple components needs to be started in container, then this actor is created. Job of this actor is just to logically group multiple components and support csw.messages.SupervisorContainerCommonMessages. It receives csw.messages.SupervisorContainerCommonMessages.Shutdown or csw.messages.SupervisorContainerCommonMessages.Restart message and forwards it to all the components residing in this container.

deploy package

- csw.framework.deploy.containercmd.ContainerCmd : ContainerCmd is a helper utility provided by framework to start multiple components in container mode or single component in Standalone mode.

Example of Container Config File

Here is an example of a config file for creating a container with the name IRIS_Container that contains one assembly (named Filter) and depends on the services of two HCDs (Instrument_Filter and Disperser). Factory class for assembly is csw.common.components.framework.SampleComponentBehaviorFactory. A Supervisor actor will be created to manage the assembly, which includes registering it with the location service, using the given name and prefix. The prefix can be used to distribute parts of the configurations to different HCDs. HCDs register themselves with the Location Service and specify a unique prefix that can be used for this purpose.

name = "IRIS_Container"
components: [
  {
    name = "Filter"
    componentType = assembly
    behaviorFactoryClassName = csw.common.components.framework.SampleComponentBehaviorFactory
    prefix = tcs.mobie.blue.filter
    locationServiceUsage = RegisterOnly
    connections = [
      {
        name: Instrument_Filter
        componentType: hcd
        connectionType: akka
      },
      {
        name: Disperser
        componentType: hcd
        connectionType: akka
      }
    ]
  },
  {
    name = "Instrument_Filter"
    componentType = hcd
    behaviorFactoryClassName = csw.common.components.framework.SampleComponentBehaviorFactory
    prefix = tcs.mobie.blue.filter
    locationServiceUsage = RegisterOnly
  },
  {
    name = "Disperser"
    componentType: hcd
    behaviorFactoryClassName: csw.common.components.framework.SampleComponentBehaviorFactory
    prefix: tcs.mobie.blue.disperser
    locationServiceUsage = RegisterOnly
  }
]

Example of Standalone Config File

name = "IFS_Detector"
componentType = hcd
behaviorFactoryClassName = csw.common.components.framework.SampleComponentBehaviorFactory
prefix = iris.ifs
locationServiceUsage = RegisterOnly
connections = []

- csw.framework.deploy.hostconfig.HostConfig : This is just a helper to create host configuration application. This support starting multiple containers on a given host machine and each container will have single/multiple components.

Example of Host Config file

# This is a host configuration file which contains list of containers to be spawned by host configuration app
containers: [
    {
      # mode can be one of Container or Standalone
      mode: "Container"
      # path of individual container configuration file
      configFilePath: "/resources/assemblyContainer.conf"
      # provide 'Remote' if file needs to fetched from config service else
      # provide 'Local' to fetch file from local machine
      configFileLocation: "Local"
    },
    {
      mode: "Standalone"
      configFilePath: "/resources/hcdStandalone.conf"
      configFileLocation: "Local"
    }
]

Definition Classes: csw

package messages

This project is intended to hold reusable models and messages used throughout the csw source code.

Messages

This project is intended to hold reusable models and messages used throughout the csw source code.

This also provides out of the box support to cater to the diverse communication requirements. Consumer of this library will be able to create Commands, Events, States to store ParameterSets.

Imp Packages

Commands and Events

This packages contains classes, traits and models used to create *commands* and *events*. These are all based on type-safe keys and items (a set of values with optional units). Each key has a specific type and the key's values must be of that type.

Two types of csw.messages.commands.Command are supported:

csw.messages.commands.SequenceCommand
- This commands are targeted to Sequencer. Subtypes of this are: Setup, Observe and Wait.
csw.messages.commands.ControlCommand
- This commands are targeted to Assemblies and HCD's. Subtypes of this are: Setup and Observe.

Following are the concrete commands supported by csw:

Two types of csw.messages.events.Event are supported:

Another important feature provided by *commands* package is csw.messages.commands.matchers.Matcher One of the use case for using matcher is when Assembly sends csw.messages.CommandMessage.Oneway command to HCD and in response to this command HCD keeps publishing its current state. Then Assembly can use Matcher with the matching definition as provided by csw.messages.commands.matchers.StateMatcher to match against the current states published by HCD.

Location and Framework

These packages contain reusable classes, traits and models. We are keeping all the models which are getting transferred over the wire and requires serialization and deserialization in csw-messages project. All the models are marked with csw.messages.TMTSerializable. csw.messages.TMTSerializable is a marker trait which extends scala.Serializable. This is configured to use kryo serialization. Also these models are being shared between multiple projects. csw-location, csw-framework and csw-logging depends on csw-messages project which uses these models.

Location Service uses csw.messages.location.Connection model to register component/container of type:

When you resolve/find a csw.messages.location.Connection, you get csw.messages.location.Location in return which can be one of below type:

Framework package contains following actor messages:

Messages of type csw.messages.framework.PubSub are supported by PubSubActor
Below Lifecycle messages can be sent to component when component is in csw.messages.framework.SupervisorLifecycleState.Running state, note that these messages should be wrapped inside csw.messages.RunningMessage.Lifecycle before sending it to Supervisor actor.
- csw.messages.framework.ToComponentLifecycleMessages.GoOnline
- csw.messages.framework.ToComponentLifecycleMessages.GoOffline

Params

This package supports serialization and deserialization of commands, events and state variables in following formats:

JSON : csw.messages.params.formats.JsonSupport
Kryo : csw.messages.params.generics.ParamSetSerializer
Protobuf : package pb contains utility and directory protobuf contains proto schema files.

Scala and Java APIs

All the param and event classes are immutable. The set methods return a new instance of the object with a new item added and the get methods return an Option, in case the Key is not found. There are also value methods that return a value directly, throwing an exception if the key or value is not found.

Key Types

A set of standard key types and matching items are defined. Each key accepts one or more values of the given type.

Following csw.messages.params.generics.KeyType are supported by csw:

+--------------+-------------------------+---------------------------+
|  Primitive   |      Scala KeyType      |       Java KeyType        |
+--------------+-------------------------+---------------------------+
| Boolean      | KeyType.BooleanKey      | JKeyTypes.BooleanKey      |
| Character    | KeyType.CharKey         | JKeyTypes.JCharKey        |
| Byte         | KeyType.ByteKey         | JKeyTypes.ByteKey         |
| Short        | KeyType.ShortKey        | JKeyTypes.ShortKey        |
| Long         | KeyType.LongKey         | JKeyTypes.LongKey         |
| Int          | KeyType.IntKey          | JKeyTypes.IntKey          |
| Float        | KeyType.FloatKey        | JKeyTypes.FloatKey        |
| Double       | KeyType.DoubleKey       | JKeyTypes.DoubleKey       |
| String       | KeyType.StringKey       | JKeyTypes.StringKey       |
| Timestamp    | KeyType.TimestampKey    | JKeyTypes.TimestampKey    |
| ----------   | ----------              | ----------                |
| ByteArray    | KeyType.ByteArrayKey    | JKeyTypes.ByteArrayKey    |
| ShortArray   | KeyType.ShortArrayKey   | JKeyTypes.ShortArrayKey   |
| LongArray    | KeyType.LongArrayKey    | JKeyTypes.LongArrayKey    |
| IntArray     | KeyType.IntArrayKey     | JKeyTypes.IntArrayKey     |
| FloatArray   | KeyType.FloatArrayKey   | JKeyTypes.FloatArrayKey   |
| DoubleArray  | KeyType.DoubleArrayKey  | JKeyTypes.DoubleArrayKey  |
| ----------   | ----------              | ----------                |
| ByteMatrix   | KeyType.ByteMatrixKey   | JKeyTypes.ByteMatrixKey   |
| ShortMatrix  | KeyType.ShortMatrixKey  | JKeyTypes.ShortMatrixKey  |
| LongMatrix   | KeyType.LongMatrixKey   | JKeyTypes.LongMatrixKey   |
| IntMatrix    | KeyType.IntMatrixKey    | JKeyTypes.IntMatrixKey    |
| FloatMatrix  | KeyType.FloatMatrixKey  | JKeyTypes.FloatMatrixKey  |
| DoubleMatrix | KeyType.DoubleMatrixKey | JKeyTypes.DoubleMatrixKey |
| ----------   | ----------              | ----------                |
| Choice       | KeyType.ChoiceKey       | JKeyTypes.ChoiceKey       |
| RaDec        | KeyType.RaDecKey        | JKeyTypes.RaDecKey        |
| Struct       | KeyType.StructKey       | JKeyTypes.StructKey       |
+--------------+-------------------------+---------------------------+

Detailed information about creating Keys and Parameters can be found here: https://tmtsoftware.github.io/csw-prod/services/messages/keys-parameters.html

Detailed information about creating commands can be found here: https://tmtsoftware.github.io/csw-prod/services/messages/commands.html

Detailed information about creating events can be found here: https://tmtsoftware.github.io/csw-prod/services/messages/events.html

Definition Classes: csw

package services

Definition Classes: csw

package command

This project defines the basic classes and traits for the Command Service.

Command Service

This project defines the basic classes and traits for the Command Service.

Related projects are: - csw-messages:

This defines the types of command (Oneway/Submit etc.) and types of configurations (Setup/Observe/Wait etc.)
Complete usage of Messages is available at: https://tmtsoftware.github.io/csw-prod/services/messages.html

- framework:

This defines the Hcd and Assembly handlers, lifecycle manager and supervisor for components.
Framework allows component writer to override onValidation, onSubmit and onOneway handlers. (Note it allows overriding other handlers as well.)
On every command received by component, onValidation handler gets invoked where received command gets validated and validation response is returned.
Based on validation response and command type, onSubmit/onOneway hooks gets invoked where command gets processed.
Complete details of handling commands can be found here : https://tmtsoftware.github.io/csw-prod/framework/handling-lifecycle.html#handling-commands

Important classes in this project are:

- csw.services.command.CommandResponseManager

This class wraps CommandResponseManagerActor and provides helpers to interact with actor which is responsible for adding/updating/querying command result. Component writers will get handle to CommandResponseManager in their handlers.

- csw.services.command.internal.CommandResponseManagerBehavior maintains two states:

csw.messages.commands.CommandResponseManagerState: It maintains csw.messages.params.models.Id of Commands and their corresponding csw.messages.commands.CommandState.
csw.messages.commands.CommandCorrelation : It maintains commands csw.messages.params.models.Id correlation between parent to child and child to parent.

- csw.services.command.scaladsl.CommandService

This class wraps the csw.messages.location.AkkaLocation and provides helpers to send commands to component actor extracted from provided location. Normal component writers workflow would be to first resolve component using location service and then create CommandService instance using resolved location.

Using this instance, you can Submit Command/Commands to other component or query for command result or subscribe for long running command result.

- csw.services.command.scaladsl.CommandDistributor

When you have multiple commands targeted to multiple components then you can use ConfigDistributor. Using CommandDistributor utility you can send all these commands in one go and get aggregated response.

Complete guide of usage of different API's provided by CommandService is available at: https://tmtsoftware.github.io/csw-prod/command.html

package config

package event

This module implements an Event Service responsible for publishing an csw.messages.events.Event or subscribing to an csw.messages.events.Event.

Event Service

This module implements an Event Service responsible for publishing an csw.messages.events.Event or subscribing to an csw.messages.events.Event. An event is published on a csw.messages.events.EventKey. The event key is composed of a csw.messages.params.models.Prefix depicting the source and an csw.messages.events.EventName. The subscriber can subscribe to the csw.messages.events.EventKey to receive all events published on the Key.

Example: Event Service

val eventServiceFactory          = new EventServiceFactory()
val eventService: EventService   = eventServiceFactory.make(locationService)

Using above code, you can create instance of csw.services.event.api.scaladsl.EventService. EventService is the factory to create publishers and subscribers.

You can choose to use defaultPublisher in case you want to share same connection for publishing different events.

Example: Event Publisher

Event Publisher provides asynchronous APIs to publish one or more Events

val event = SystemEvent(prefix, EventName("filter_wheel"))

val publisher    = eventService.defaultPublisher
publisher.publish(event)

Example: Event Subscriber (subscribe and get API)

Event Subscriber provides asynchronous APIs to subscribe to one or more Event Keys. It also provides APIs to subscribe using a pattern. In addition, there are APIs to get latest events for one ore more Event Keys without making a subscription

The EventSubscriber provides various API's to subscribe to csw.messages.events.EventKey. One of such a subscribe method takes an ActorRef of an arbitrary actor or a callback function to be called when an event matching the given csw.messages.events.EventKey is received. In the example below, we only provide the callback argument value. You could also provide an ActorRef of some actor that should receive the Event message.

 def callback(ev: Event): Unit = {
   // ...
 }

 async {
   val subscriber    = eventService.defaultSubscriber
   subscriber.subscribeCallback(Set(EventKey(prefix, EventName("filter_wheel"))), callback)

   val event = await(subscriber.get(event.eventKey))

   event match {
     case e: ObserveEvent => assert(e.prefix == expectedPrefix)
     case _               => fail("Expected ObserveEvent")
   }
}

csw-prod provides two implementations for Event Service which can be accessed through csw-event-client : 1. csw.services.event.internal.redis.RedisEventService - Redis as event store and publisher/subscriber functionality 2. csw.services.event.internal.kafka.KafkaEventService - Kafka as event store and publisher/subscriber functionality

Complete guide of usage of different API's provided by EventService is available at: https://tmtsoftware.github.io/csw-prod/services/event.html

package location

The Location Service implemented in this project is based on CRDT (Conflict Free Replicated Data).

Location Service

The Location Service implemented in this project is based on CRDT (Conflict Free Replicated Data). The Location Service helps you resolve the hostname and port number for a service which can be used for further communication. In case of Akka connection, It helps you to resolve component reference with which you can send messages as well as other information for example, logAdminActorRef which is used to dynamically change the log level of component.

Location Service supports three types of services as follows: Akka, Tcp and HTTP based services.

Before calling LocationServiceFactory.make() to get an handle of LocationService, it is recommended to have below environment variables set to System global environments or system properties:

interfaceName (The network interface where akka cluster is formed.) Ex. interfaceName=eth0 if not set, first inet4 interface by priority will be picked
clusterSeeds (The host address and port of the seedNodes of the cluster) Ex. clusterSeeds="192.168.121.10:3552, 192.168.121.11:3552"
clusterPort (Specify port on which to start this service) Ex. clusterPort=3552 if this property is not set, service will start on random port.

Location service make use of classes defined in csw-messages package, for example: - csw.messages.location.Location : When you resolve component location using csw.services.location.scaladsl.LocationService, you get back this location. There are three types of Locations:

csw.messages.location.AkkaLocation: holds hostname, port and actorRef which is ready to receive csw.messages.commands.Command from outside.
csw.messages.location.TcpLocation: holds hostname and port which can be used for further communication.
csw.messages.location.HttpLocation: holds hostname and port which can be used for further communication.

- csw.messages.location.Connection : Connection holds the identity of component which includes component name and csw.messages.location.ComponentType. Every component needs to register with location service using unique connection. There are three types of Connections:

Another important actor from LocationService is DeathwatchActor csw.services.location.internal.DeathwatchActor which gets created with LocationService initialization. Job of this actor is to watch health of every component and on termination, unregister terminated component from LocationService.

Complete guide of usage of different API's provided by LocationService is available at: https://tmtsoftware.github.io/csw-prod/services/location.html

package logging

This is a Actor based logging library which supports writing logs to File and StdOut Console.

Logging Service

This is a Actor based logging library which supports writing logs to File and StdOut Console.

Features

Supports component specific log levels, ex. HCD1 can choose to log at info level and HCD2 can choose to log at debug level
Supports dynamically changing component log levels
Asynchronous thread safe logging
Structured logging
Supports overriding default logging properties per component viz
Intercepting logs from akka/slf4j
Supports JSON logging

LogActor

LogActor is the heart of logging library. It receives messages from following classes:

Slf4jAppender: Intercepts Slf4j logs and forwards it to LogActor via MessageHandler
AkkaLogger: Intercepts Akka logs and forwards it to LogActor via MessageHandler.
LoggerImpl: Provides csw logging API for component writer to log messages which gets forwarded to LogActor via MessageHandler

Logging Appenders

This library supports two types of csw.services.logging.appenders.LogAppender:

csw.services.logging.appenders.FileAppender: Common log messages are written in Pretty Json form. This appender is useful during development. It is recommended to disable it in production.
csw.services.logging.appenders.StdOutAppender: Log messages are written as Json, one per line. Ordinary log messages are written to the common log files. Each log file includes a day as part of its name. Each day a new file is created.

You can specify the appender in application.conf file as shown below:

csw-logging {
   appenders = ["csw.services.logging.appenders.StdOutAppender$", "csw.services.logging.appenders.FileAppender$"]
 }

Component Specific Log Levels

For each component, ComponentLoggingState instance gets created which maintains log levels which are enabled and disabled for this particular component. Every time message gets logged by component, LoggerImpl checks in corresponding componentLoggingState whether current log level enabled or not. If enabled, then only log message gets forwarded to LogActor via MessageHandler.

You can specify the component specific log levels in application.conf file as shown below:

component-log-levels {
   tromboneHcd = debug
   tromboneAssembly = error
}

̄ Detailed documentation of Logging Service is available at: https://tmtsoftware.github.io/csw-prod/services/logging.html

BuildInfo

csw

services

package services

Ordering

Alphabetic

Visibility

Public
All

Value Members

object BuildInfo extends Product with Serializable
This object was generated by sbt-buildinfo.

Packages

Framework

Important Actors in Framework

deploy package

Example of Container Config File

Example of Standalone Config File

Example of Host Config file

Messages

Imp Packages

Commands and Events

Location and Framework

Params

Scala and Java APIs

Key Types

Command Service

Event Service

Example: Event Service

Example: Event Publisher

Example: Event Subscriber (subscribe and get API)

Location Service

Logging Service

Features

LogActor

Logging Appenders

Component Specific Log Levels

services 

package services

Value Members

Ungrouped

services