Giter Club home page Giter Club logo

jaws-effective-processor's Introduction

jaws-effective-processor CI Docker

A set of connected Kafka Streams apps for JAWS that process alarm registration and notification data and compute effective state.

Read more about Overrides and Effective State and Software Design.



Quick Start with Compose

  1. Grab project
git clone https://github.com/JeffersonLab/jaws-effective-processor
cd jaws-effective-processor
  1. Launch Compose
docker compose up
  1. Monitor for expiration tombstone message
docker exec -it jaws list_overrides --monitor 
  1. Shelve an alarm for 5 seconds
docker exec jaws set_override --override Shelved alarm1 --reason Other --expirationseconds 5

See: More Usage Examples

Install

This application requires a Java 11+ JVM and standard library to run.

Download from Releases or build the distribution yourself.

Launch with:

UNIX:

bin/jaws-effective-processor

Windows:

bin/jaws-effective-processor.bat

Configure

Environment Variables

Name Description
BOOTSTRAP_SERVERS Comma-separated list of host and port pairs pointing to a Kafka server to bootstrap the client connection to a Kafka Cluser; example: kafka:9092
SCHEMA_REGISTRY URL to Confluent Schema Registry; example: http://registry:8081
STATE_DIR Directory where local Kafka Streams state is stored [1], [2]. Defaults to java.io.tmp system property value with an appended subdir named kafka-streams.

Build

This project is built with Java 17 (compiled to Java 11 bytecode), and uses the Gradle 7 build tool to automatically download dependencies and build the project from source:

git clone https://github.com/JeffersonLab/jaws-effective-processor
cd jaws-effective-processor
gradlew build

Note: If you do not already have Gradle installed, it will be installed automatically by the wrapper script included in the source

Note for JLab On-Site Users: Jefferson Lab has an intercepting proxy

See: Docker Development Quick Reference

Develop

In order to iterate rapidly when making changes it's often useful to run the app directly on the local workstation, perhaps leveraging an IDE. In this scenario run the service dependencies with:

docker compose -f deps.yaml up

Then run the app with:

gradlew run

Note: The STATE_DIR config is set to the gradle build dir such that running a clean task will clear the local state. You may need to reset the Kafka server state after running a clean task by restarting Kafka from scratch else using the Reset Tool.

Note: Javadocs can be generated with the command:

gradlew javadoc

Release

  1. Bump the version number in the VERSION file and commit and push to GitHub (using Semantic Versioning).
  2. The CD GitHub Action should run automatically invoking:
    • The Create release GitHub Action to tag the source and create release notes summarizing any pull requests. Edit the release notes to add any missing details. A zip file artifact is attached to the release.
    • The Publish docker image GitHub Action to create a new demo Docker image.

See Also

jaws-effective-processor's People

Contributors

slominskir avatar

Watchers

James Cloos avatar Stephen Wood avatar  avatar Nathan Baltzell avatar Wesley Moore avatar Vardan Gyurjyan avatar  avatar Theo avatar  avatar

jaws-effective-processor's Issues

Revisit double message strategy

As discussed here the jaws-effective-processor writes two messages for every state change: first the current state as it is observed by the processor and then again with the updated Overrides. Doubling the number of messages may not be so great after all. Maybe revisit this strategy - when transitioning perhaps drop the intermediate messages.

Filter by area not implemented yet

Currently each individual alarm can be filtered, but we'd like to have an intermediary topic and Kafka Streams app that allows operators to mask an entire area of the machine with a single command - the intermediary would be responsible for mapping between these high-level area filters to individual alarm filters. Since an alarm can only have one filter at a time special care may be needed.

Generalize this project for all automated overrides

We need to rename this project alarm-auto-override-processor and expand the scope to include all automated overrides, not just continuous shelving expiration. The missing overrides include:

  • one-shot shelving
  • on-delays
  • off-delays
  • latch
  • masking

Make transitioning behavior configurable

We initially passed a message that resulted in a new override message through and then switched to dropping it. It isn't clear which approach is best and it probably depends on workload. Maybe this should be a configurable option.

See: #7

EPICS "NO ALARM" activations vs overrides

The EPICS "NO ALARM" activation should not be considered an activation when computing overrides and effective activations. For example, a "NO ALARM" activation should NOT result in the LATCHED state. However, it probably is reasonable for "INVALID" EPICS activations and Channel Error activations to latch.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.