Implement exchange interfaces for ComponentRegistry and ExchangeComponents.

Create a cluster component registry that is shard by all member/module components. Each entry or update to a member/module registry should be forwarded to the cluster registry. This registry will be backed by proven's IMDG.

Updates:

• Payara Micro - 5.201
• Jet - 3.2.2
• IMDG - 3.12.6

Refactor StreamManager to use its created map to store MessageStream components it's managing.

Create a Maintenance Operation for PipelineManager to identify a missing/failed Jet job and resume the missing job from it's current location.

Performing the translation on the routing service creates the potential for bottlenecks. The raw gridappsd. message will be bundled as a part of the knowledge message where it is translated into the general form and then ingested into the Influx store.

Remove ComponentType enumenration and replace with Class value at construction time. This will facilitate extending ManagedComponents in new modules w/o havin to update an enumerated type.

Implement managed component status operations for PipelineJob and PipelineManager.

Add missing group label and creation time to ComponentEvent serializer.

Provide implementation for ProvenModule activation. This includes async creation of ManagerComponents specified by ActiveManagers annotation and Producer methods for ManagerComponents created via injection.

Member performance scaling is a maintenance operation performed by a ProvenModules.

Modules are responsible for identifying and creating/removing A selected module per member is selected and responsible for identifying and deploying/removing new modules to/from members. Note: ProvenModules should be annotated as Scalable to support this change.

Upgrade Jet component to 3.2.2 from 3.0
The new Jet IMDG version is 3.12.6

Scale requests should utilize components that can be re-used versus creation of a new component. For example, "Offline" components can be re-activated, if exists, instead of creating a new component. Current scale operation should be refactored to include recycling.

Remove component from observer registry if it is out of service.

Data is transferred to a selected component of same type:

• Available components in same module given preference
• Intra-member transfer for non-partitioned data
• Any member for partitioned data

Only applies to components having distributed objects

Create a maintenance operation performed by modules to destroy components created by them that are out of service, can't be used to trigger a scale operation, and has no distributed objects that require transfer or recovery. This is a cleanup operation to save memory and decrease registry sizes. This will include adding a decorated destroy method to Creator interface.

Server currently supports only a single disclosure endpoint, with a pre-defined name. This is sufficient for a single node, but needs to be changed to support a cluster environment that may include multiple disclosure endpoints with dynamic naming. A discovery mechanism will be built to allow discovery of endpoints and their current status.

Here is the work I think still needs completing regarding the pipeline. I thought I had checked in work, but couldn't find it and ended up determining my changes that need to be incorporated in the latest develop branch.
Finalize message formats at the message travels through the streams on the platform
Finalize message path through the streams
Finalize how responses are reported and managed
Finalize on the message lifecycle (FIFO etc)
Finalize TSStream and TSPipeline
Migrate code Hybrid code to platform
Document how to test a pipeline outside Proven (based on Todd's example).

Modify hazelcast xml configuration for memeber registry (ISet) and the Cluster registry (IMap).

Records ManagedComponents at the module/application level.

These properties are necessary in the event of a lost member (payara/hazelcast), in order to perform scaleRecovery status operations.

Message stream proxy creation and/or retrieval from StreamManager should not be allowed if underlying MessageStream component does not have a ManagedStatus of Online. Calls to StreamManager to retrieve a proxy will need to be refactored to account for this case.

Create registry entries for storage in component registries. Should include component location, distributed object identification, as well as properties specific to the entries managed component it represents. The properties provided support registry processing.

Add schedule timer method to check for overdue registry notifications in ModuleComponentRegistry. If overdue, mark as "Unknown", so component will not be a candidate for operation selection.

Check if component still exists (resolveObserverMethods for a Fail operation event message). If not exists then remove from registries. If exists, send fail operation event message to component if "Unknown" status is maintained for a specified number of overdue checks.

Implement scaling support at the ProvenModule level. New managed components are created to replace busy or failed components of the same type. Task schedule should be extended to support scale requests.

Identify and implement v2.0 MO's. For OnlineError severity errors or higher, send a registry notification to ComponentRegistry at point of occurrence, so the component's status may be updated as soon as possible and will no longer be considered as a candidate for selection.

This feature would define the assignment of a variable number of ProvenModule(s) (PM) components to monitor a member for scaling (out/in) events. Scaling would attempt to recycle any suspended modules before deploying new.

Change entry identifiers to be in reverse domain format by default. This format is used to name distributed objects and is used in Hazelcast declarative configuration.

Current implementation does not allow for recovery of SSE sessions due a failed HZ member, potentially causing a client to not receive registered message events.

The master branch converts JSON formatted files to Java classes, making the messages bulky. The purpose of this task is the leverage the design on the Proven Measurement / Proven Metrics classes while reducing the size of the message in more efficient forms.

Records status and component information for a member's components.
Forwards it's module information to cluster registry.
Maintains current list of external facing disclosure endpoints, use by clients to identify disclosure locations.

Add initial support for Jet pipeline monitoring. This should include simple recovery (i.e. RESUME) if pipelines are stalled. Monitoring solution should be integrated in both hybrid and disclosure modules to support the disclosure processing pipeline.

Review event messages and revert String identifiers back to their original UUID type.

Components failing because of a lost member AND managing distributed object(s) must be recovered on another member. These components should be annotated with Scalable annotation. If non-partitioned recovery, new component should be on same member as new data member (i.e. previously the backup member). If data is partitioned then any member may be selected.

Recovery scaling is always done even if capacity is violated at time of scale event.

Recovery scaling is a maintenance operation performed by managers. Each manager type is responsible for identifying and recovering component types created under them.

Remove code duplication between status operation observer methods.