ibmstreams / streamsx.health Goto Github PK

Want to be able to detect different artifacts in waveform data.
Need to specify what we need here for ECG, EEG?

As we are now following the Hubflow process for contributing code, we should consider adding a version number to the platform. This will allow us to generate releases with appropriate tags and keep track of what features were added to which release.

It needs to point to streamsx.health/sample/...

Roadmap draft here:
https://github.com/IBMStreams/streamsx.health/wiki/Streams-Healthcare-Analytics-Platform-Roadmap

• Testing of services - James
• Documentation - Samantha

@cancilla
Please mark complete and give thumbs up if we are ready to go.

Description Resource Path Location Type
CDISP0053E An unknown identifier was referenced in the SPL program: PublishJson_T. com.ibm.streamsx.health.emr Line 23, column 11 SPL Build
CDISP0053E An unknown identifier was referenced in the SPL program: PublishJson_T. com.ibm.streamsx.health.emr Line 27, column 18 SPL Build
CDISP0392E The identifier that extends a tuple must itself be a tuple type or a stream that that is used as a tuple type. com.ibm.streamsx.health.emr Line 23, column 11 SPL Build
CDISP0449E Errors were generated during the instantiation of the com.ibm.streamsx.health.ingest.connector::IngestClinicalNoteSubscribe composite operator.
com.ibm.streamsx.health.emr/ClincialNotesRead.spl:9:3: The operator was expanded from the IngestClinicalNoteSubscribe_1 composite operator. com.ibm.streamsx.health.emr Line 18, column 1 SPL Build
CDISP0460E The actual parameter for the $streamType formal parameter with the type expression mode is not valid. com.ibm.streamsx.health.emr Line 27, column 18 SPL Build

If we were to begin spinning "releases" of the platform, what would that look like? Would we pre-compile all of the JARs and include dependencies? Are we building SABs and if so, for which platform? What else should be provided in a release?

The RPeakDetectService compiles the SAB to output/com.ibm.streamsx.health.analytics.ecg.service.RPeakDetectService.BuildConfig. Should compile to output/com.ibm.streamsx.health.analytics.ecg.service.RPeakDetectService/BuildConfig

The resp rate always seems to be zero. The EWS service does not work as a result.

Need to investigate what the "value" and "UOM" attributes should contain

For each component / service, we need to clean up the readme and make sure that we have enough instructions to get people started.

Want to be able to use a camera to monitor patient's movement. If movement is detected, annotate the signal, so we can better handle bad signals.

In the README.md in the repository root, there is a section entitled "Real-time ECG Monitoring Sample". The sample actually demonstrates monitoring of more than just ECG waveforms. I recommend that all references to "ECG Monitoring" be replaced with "Healthcare Monitoring".

Investigate parsing of ADT messages.
Do we want to parse the entire set of ADT? If not, what ADT information is useful in a Streams application?

In a testcase where the ORU01 message has a PID segment, the parser is unable to parse out the OBR and OBX segment.

The problem is in the ObxToSplMapper code, where we expect a flat structure in the message.

In the case where PID is present, the OBR and OBX segments are nested into a Patient object.

ViNES(R) from True Process is a scalable, vendor-neutral platform for acquiring, sharing, and storing high-resolution data from biomedical devices.

Add ViNES to the ingest service framework as described here.

Some services are launched via a shell script (i.e. Vines) and some services are launched using Gradle (i.e. Watson Explorer services).

Need to migrate all services such that they are launched via Gradle rather than a separate script. Using Gradle allows for the dependencies to easily be added to the classpath when launching, rather than having to copy the dependencies into a shell script.

I have a question about the simulator.

What data is being simulated? and what is the data format of the simulator?
Can the simulator simulate one or more patients?

The ECGBeaconService should be replaced with a service that is capable of generating different vitals and waveforms.

This project should have its own directory under the Analyze component

Need to revisit and clean up

• Clean up schema type definition
• Removed hard-coded patient name and resp
• Clean up export data

Need to set up build for project so it's easier to get started

Trying to detect upward or downward trends from vitals

• version number for microservices
• clean up readme / build instructions
• test to make sure all components are stable
• deliver back to master

Trace statements does not work properly in HapiServer code when running in distributed mode.

Hello, I get the following error message when I try to run the 2nd cell on the HealthcareDemo-AnalyticsService demo:

/home/streamsadmin/anaconda3/lib/python3.5/json/decoder.py in raw_decode(self, s, idx)
356 except StopIteration as err:
--> 357 raise JSONDecodeError("Expecting value", s, err.value) from None
358 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

FileNotFoundError Traceback (most recent call last)
/home/streamsadmin/.local/lib/python3.5/site-packages/streamsx/rest.py in get_vcap_services(config)
183 try:
--> 184 with open(vs) as vcap_json_data:
185 vs = json.load(vcap_json_data)

FileNotFoundError: [Errno 2] No such file or directory: 'vcap_services.json'

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last)
in ()
51
52 # Retrieve view for a patient
---> 53 patient_view = getPatientView(3)
54
55 print('DONE')

in getPatientView(patient_id)
47 patient_view = patient_data.view()
48
---> 49 submit('ANALYTICS_SERVICE', topo, config)
50 return patient_view
51

/home/streamsadmin/.local/lib/python3.5/site-packages/streamsx/topology/context.py in submit(ctxtype, graph, config, username, password, log_level)
75 """
76 logger.setLevel(log_level)
---> 77 context_submitter = _SubmitContextFactory(graph, config, username, password).get_submit_context(ctxtype)
78 try:
79 return context_submitter.submit()

/home/streamsadmin/.local/lib/python3.5/site-packages/streamsx/topology/context.py in get_submit_context(self, ctxtype)
378 if not (sys.version_info.major == 3 and sys.version_info.minor == 5):
379 raise RuntimeError("The ANALYTICS_SERVICE context only supports Python version 3.5")
--> 380 return _RemoteBuildSubmitter(ctxtype, self.config, self.app_topology)
381 else:
382 logger.debug("Using the BaseSubmitter, and passing the context type through to java.")

/home/streamsadmin/.local/lib/python3.5/site-packages/streamsx/topology/context.py in init(self, ctxtype, config, app_topology)
271 super(_RemoteBuildSubmitter, self).init(ctxtype, config, app_topology)
272
--> 273 self._set_vcap()
274 self._set_credentials()
275

/home/streamsadmin/.local/lib/python3.5/site-packages/streamsx/topology/context.py in _set_vcap(self)
302 def _set_vcap(self):
303 "Set self.vcap to the VCAP services, from env var or the config"
--> 304 self._vcap = VcapUtils.get_vcap_services(self._config())
305
306 def _set_credentials(self):

/home/streamsadmin/.local/lib/python3.5/site-packages/streamsx/rest.py in get_vcap_services(config)
185 vs = json.load(vcap_json_data)
186 except:
--> 187 raise ValueError("VCAP_SERVICES information is not JSON or a file containing JSON:", vs)
188 return vs
189

ValueError: ('VCAP_SERVICES information is not JSON or a file containing JSON:', 'vcap_services.json')

Please advise.

Many thanks, Sarah

Add analytics to check that vital signs are within normal range

Fields such as Reading.readingType and Reading.uom will generally be populated with the same subset of values.

Take for example he Reading.readingType attribute, which describes the type of reading that was observed. One of the challenges we have is that different ingest services are free to different spellings to specify the same reading type. For example, you may get "ECG Lead I", "ecg_lead_I", "lead I", etc.

We should adopt a standard nomenclature for the Streams Healthcare Analytic Platform to be used when specifying the values for these types. Analytic applications will likely rely on the value set by this attribute to determine the type of analysis to run. Likewise, visualization applications may use this value to generate the graphs. In short, this will help to promote interoperability between services and applications.

We can start by looking at ISO 11073 and PCD Profile Rosetta Terminology Mapping for guidance on how to define this nomenclature.

https://en.wikipedia.org/wiki/ISO/IEEE_11073
http://wiki.ihe.net/index.php/PCD_Profile_Rosetta_Terminology_Mapping

An empty trend service project has leaked in. You may want to clean that up.

Includes:

• Symptom Event Service
• Medication Event Service

Need data type and connectors for clinical notes.
Need Kafka connector for clinical notes.

https://github.com/IBMStreams/streamsx.health/wiki/Ingest-Service-Framework

Import streamsx.health.ingest as existing project in Streams Studio.
It reported the following compile error:

• required source java/impl/test is not found

Unsure if it's intentional that we add this to the build path.

Updated schema proposal can be found here: Standardized Glossary Design Doc

Implement this proposal: Standardized Glossary Design Doc

This involves updates to the following:

• Vines ingest service
• Physionet ingest service
• Vitals EWS
• HealthBeacon simulator
• PatientsMonitoringDemo

Want to find all the peaks and valleys in waveform data

To make it easier to find the HL7 ingest service, I refactoring ingest/hapi folder to ingest/hl7. Users may not immediately know that the HL7 ingest service is available under the 'hapi' folder.

We need to extract the PV1 PL (3), MSH 3, and MSH 4 in to the output tuple.

We have recently published a roadmap for the Streams Healthcare Analytics Platform. You can find the roadmap here:

Roadmap

The platform is currently designed to use a micrservice architecture, which can contain multiple toolkits applications that can be assembled together to form an application.

The repository needs to be reorganized to better facilitate the development of this platform.

From a Streams application, it will be helpful to be able to export / store data in physionet format. This will make it easier for researchers to develop algorithms in Matlab or R.