Giter Club home page Giter Club logo

jmisb's Introduction

jMISB

Build Status codecov CodeQL Maven Central Gitter

About

jMISB is an open source Java library implementing various MISB standards. It leverages the excellent work by bytedeco on bringing video support to Java. Stay tuned here for updates, and please join us on gitter if you need help or would like to participate!

Why jMISB?

The Motion Imagery Standards Board, or MISB's mission is to develop and maintain standards for interoperability between motion imagery systems in use within the Department of Defense (DoD) and Intelligence Community (IC).

The goal of the jMISB project is to provide an open implementation of these standards and allow government and industry to leverage them more easily and effectively. The jMISB project is not affiliated with, nor endorsed by MISB in any way.

Scope

The MISB has been quite prolific in creation of new standards since its inception in 2000. As of April 2021, over four dozen standards are listed on its web site. While the scope of the jMISB project is to support as many of these standards as possible, the initial focus will be on those in most widespread use.

The table below lists the status of currently-supported standards:

Identifier Name Implementation Status Known Issues
ST 0102 Security Metadata Universal and Local Sets for Digital Motion Imagery Implemented as of ST 0102.12. There is read-only support for some tags (not UMID) that were removed in ST 0102.12.
EG 0104 Predator UAV Basic Universal Metadata Set Read only support for EG 0104.5. Writing is not planned, since this metadata set is deprecated by MISB.
ST 0601 UAS Datalink Local Set Mostly implemented as of ST 0601.17. #140
ST 0603 MISP Time System and Timestamps Partly implemented as of ST 0603.5. #97
ST 0604 Timestamps for Class 1 / Class 2 Motion Imagery Partly implemented as of ST 0604.6. #102
ST 0805 KLV to Cursor-on-Target (CoT) Conversions Implemented as of ST 0805.1. Interoperability testing with FalconView and CoT Debug Tool.
ST 0806 Remote Video Terminal Metadata Set Implemented as of ST 0806.5. Unit tests only, no interoperability testing.
ST 0808 Ancillary Text Metadata Sets Implemented as of ST 0808.2. Local Set support only, no universal set support. Deprecated by MISB.
ST 0903 Video Moving Target Indicator and Track Metadata VMTI and VTrack Local Sets implemented as of ST 0903.5. We also support pre-ST0903.4 files.
ST 1108 Motion Imagery Interpretability and Quality Metadata Implemented as of ST 1108.3. ST 1108.2 and earlier is also supported. No interoperability testing.
ST 1201 Floating Point to Integer Mapping Fully implemented per ST 1201.5.
ST 1204 Motion Imagery Identification System (MIIS) Core Identifier Implemented as of ST 1204.3.
ST 1206 Synthetic Aperture Radar (SAR) Motion Imagery Metadata Implemented as of ST 1206.1. Unit tests only, no interoperability testing.
ST 1303 Multi-Dimensional Array Pack (MDAP) Partly implemented as of ST 1303.2. Only formats and dimensions known to be used are available. Limited testing. #198
ST 1402 MPEG-2 Transport Stream for Class 1/Class 2 Motion Imagery, Audio, and Metadata Mostly implemented, support for Sync and Asynchronous multiplexing.
ST 1902 Motion Imagery Metadata (MIMD): Model-to-KLV Transmutation Instructions Implemented as of ST 1902.1. No interoperability testing.
ST 1903 Motion Imagery Metadata (MIMD): Model Implemented as of ST 1903.1. No interoperability testing.
ST 1904 Motion Imagery Metadata (MIMD): Base Attributes Implemented as of ST 1904.1. No interoperability testing.
ST 1905 Motion Imagery Metadata (MIMD): Platform Implemented as of ST 1905.1. No interoperability testing.
ST 1906 Motion Imagery Metadata (MIMD): Staging System Implemented as of ST 1906.1. No interoperability testing.
ST 1907 Motion Imagery Metadata (MIMD): Payload Implemented as of ST 1907.1. No interoperability testing.
ST 1908 Motion Imagery Metadata (MIMD): Imager System Implemented as of ST 1908.1. No interoperability testing.
ST 1909 Metadata Overlay for Visualization Mostly implemented as of ST 1909.1. No support for Frame Time, next zoom or the reticle. #97

jMISB aims to be cross-platform to run on any modern operating system. However, since efficient video coding tends to leverage natively-compiled binaries, currently platform support is limited to Linux, Windows, and MacOS. Android is next on our roadmap (see #253).

Including in Your Project

If you are using a dependency management tool such as Maven with access to the Central Repository, you can configure it to use jMISB as a dependency. For Maven, add the following to your pom.xml:

    <dependency>
        <groupId>org.jmisb</groupId>
        <artifactId>jmisb-api</artifactId>
        <version>1.12.0</version>
    </dependency>

For Gradle, include the following:

dependencies {
    implementation 'org.jmisb:jmisb-api:1.12.0'
}

API Usage

A primary objective of jMISB is to provide an easy-to-use API allowing non-domain experts to create applications leveraging MISB standards.

The primary API for reading/writing video and metadata is in the org.jmisb.api package. See the javadocs for an extensive API reference.

Below is a simple example of reading a network stream containing video and (optionally) metadata.

        try (IVideoStreamInput stream = new VideoStreamInput())
        {
            stream.open("udp://127.0.0.1:35800");
            stream.addFrameListener(new ExampleProcessor());
            stream.addMetadataListener(new ExampleProcessor());
            while (stream.isOpen()) {
                Thread.sleep(1000);
            }
        }
        catch (IOException e) {
            System.out.println("Could not open the stream");
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

The ExampleProcessor class simply needs to implement the IVideoListener and IMetadataListener interfaces to receive video and metadata asynchronously as the data arrives.

class ExampleProcessor implements IVideoListener, IMetadataListener
{
    @Override
    public void onVideoReceived(VideoFrame frame)
    {
        BufferedImage image = frame.getImage();
        System.out.println("Center pixel RGB: " +
            image.getRGB(image.getWidth()/2, image.getHeight()/2));
    }

    @Override
    public void onMetadataReceived(MetadataFrame frame)
    {
        IMisbMessage metadata = frame.getMisbMessage();
        if (metadata instanceof UasDatalinkMessage)
        {
            UasDatalinkMessage msg = (UasDatalinkMessage)metadata;
            System.out.println("Sensor position: " +
                msg.getField(UasDatalinkTag.SensorLatitude).getDisplayableValue() +
                ", " +
                msg.getField(UasDatalinkTag.SensorLongitude).getDisplayableValue());
        }
        else if (metadata instanceof SecurityMetadataMessage)
        {
            // ...
        }
    }
}

The result of msg.getField(UasDatalinkTag.SensorLatitude) will be an instance of the SensorLatitude class (implementing IUasDatalinkValue).

For more complete examples of usage, see the examples directory, as well as viewer, a Java Swing-based tool for displaying video and metadata.

Elevation

While not a core focus, jMISB provides some elevation / terrain related support. This includes:

  • EGM 96 conversion of altitude between ellipsoid (aka HAE) and geoidal (aka MSL).

Building

To build the library from the command line, simply run the Maven command:

mvn install

This will compile the source code, run unit tests, and install the JARs to your local Maven repository.

To get started, you may want to run jmisb-viewer and experiment with some test data. This is a sample application intended mainly to aid in development. To run it from the command line, issue:

cd viewer
mvn exec:exec

Versioning

jMISB adheres to semantic versioning to communicate to client developers about the scope of changes in any new release. Version numbers are formatted as major.minor.patch, where:

  1. The major number is incremented to indicate incompatible API changes.
  2. The minor number is incremented to indicate new functionality has been added, but in a backward-compatible manner.
  3. The patch number is incremented to indicate a backwards-compatible bug fix.

In other words, users of the library should feel comfortable updating to use a new version unless the major number has changed. In general, users should keep up to date with the latest patch release for a given major.minor release branch.

Use of -SNAPSHOT within the version number indicates that the version is for internal development only, i.e., the artifact is not to be used in a production environment.

jmisb's People

Contributors

bradh avatar dependabot[bot] avatar jackhart avatar lannybroo avatar wlfgang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

jmisb's Issues

Upgrade to ST0601.16

What is the use case for your proposed feature? Please describe.
The MISB 2020.1 cycle just pushed ST0601.16 (along with a stack of other versions and new docs).
https://www.gwg.nga.mil/misb/docs/standards/ST0601.16.pdf

Describe the solution you'd like
Review ST0601.16 for changes and update the implementation.

Describe alternatives you've considered
Instead of updating the implementation, just raise tickets for each required change.
Possibly just work off the list of changes - that could be part of it, but risks missing something important. Unfortunately there are no change bars on the document. The list of changes is:
 Updated Items 26-33 to form a proper bounding box around the frame center lat/lon
 Updated Items 43 and 44 - set the resolution to 2 pixels
 Updated Item 61 corrected length value in the example
 Updated Item 62 to use a valid example value
 Updated Item 63 to use a valid example value
 Added examples for Item 80 and 93
 Updated Items 82-89 so they form a proper bounding box around the frame center lat/lon
 Updated Item 94 corrected length value in the example
 Updated Item 124, set min value to one
 Updated Item 129 - KLV_uint to KLV_val (in KLV-To-Software) and Item 129’s example to encode an alpha numeric string
 Added examples for Items 130, 138, 140, and 141
 Updated Item 141 - Figure 71 showed a leading bit equal to 1 which is incorrect; changed to 0
 Added “Continuous Zoom” to Table 4

Support ST1204 (Core Identifier)

What is the use case for your proposed feature? Please describe.
From ST1204:

Many different sensors produce Motion Imagery Data distributed across many different networks and received by many different users and systems. Assigning a consistent name, label, or identity to each Motion Imagery source helps to coordinate analysis, manage assets, and avoid confusion.
The Motion Imagery Identification System (MIIS)-Core Identifier addresses this issue. Specifically, this standard defines: (1) required identifiers for a Motion Imagery stream or file; (2) points to insert identifiers as the Motion Imagery Data flows from source to user; and (3) methods and formats for inserting identifiers into a Motion Imagery stream or file. This standard also provides guidance on extracting Universal Unique Identifiers (UUID) for use as enterprise identifiers of Motion Imagery systems.

This is a failing check in CMITT 1.3.0 for files we produce.

Describe the solution you'd like
Read and write core identifier.

Describe alternatives you've considered
Write only, but seems incomplete and would make testing harder.

Additional context
N/A

CMITT 1.3.0 ST0604.3-7 non compliance

Describe the bug
On some self-generated code that attempts to encode the image frame, CMITT flags a non-compliance: "ST 0604.3-07: All video frames must contain a timestamp. "

To Reproduce
Steps to reproduce the behavior:

  1. I'm using output.addVideoFrame(new VideoFrame(image, pts)); where image is a BufferedImage, and pts is the same timestamp I'm passing to the KLV build.
  2. Build video
  3. Test in CMITT 1.3.0
  4. See flagged issue.

Expected behavior
Green tick in CMITT.

Screenshots
N/A

Configuration (please complete the following information):
N/A.

Additional context
api/src/main/java/org/jmisb/api/video/VideoOutput.java has a couple of methods that are probably related.

CMITT 1.3.0 says we have the wrong encoding for ST0102 Object Country Codes

Describe the bug
During testing using CMITT 1.3.0, I hit a non-compliance related to ST0102 Object Country Codes.

The problem is that the specified encoding is RFC2781, which is UTF-16.

To Reproduce
Steps to reproduce the behavior:

  1. Generate a file that contains ST0102 Object Country Codes (Tag 13)
  2. Run it in CMITT 1.3.0
  3. Note the non-compliance.

(In the current viewer application, this shows up in how we display the FLI data - "A U S" type representation)

Expected behavior
We should use the RFC2781 encoding.

Screenshots
N/A.

Configuration (please complete the following information):

  • OS: Generated on Linux, ran CMITT on Windows.
  • JDK: Not important
  • jMISB version or commit: current as of 2020-04-14.

Additional context
We probably have a few other places where we aren't using an explicit encoding. Possibly some kind of automated checker can be used to find those (e.g. SpotBugs DM_DEFAULT_ENCODING rule).

README example uses `SensorLatitude` which is confusing

I'm copying the README.md example for metadata, and it has a line that looks like:

System.out.println("Sensor position: " + msg.getField(SensorLatitude) + ", " + msg.getField(SensorLongitude));

My IDE (Netbeans) adds the SensorLatitude and SensorLongitude class imports. However it looks like it should actually import UasDatalinkTag and then look like:

System.out.println("Sensor position: " + msg.getField(UasDatalinkTag.SensorLatitude).getDisplayableValue() + ", " + msg.getField(UasDatalinkTag.SensorLongitude).getDisplayableValue());

Happy to propose a simple PR against the docs if I've got the concept right.

Add check for Laser PRF code

What is the use case for your proposed feature? Please describe.
This is follow-up for #57

I would like to add a check in the constructor to ensure no zeros/nines, but we can probably add that later.

Describe the solution you'd like
Verify construction from value doesn't contain invalid content.

Describe alternatives you've considered
N/A

Additional context
N/A

Step through an input file

API should provide the ability to synchronously step though an input file and get decoded frames/metadata. Initially implement forward direction only.

Support Java 11

One of the tests currently fails when building on JDK 11, below is the output. Code should be updated so it builds/runs on Java 11.

`[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
[ERROR] Tests run: 165, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.528 s <<< FAILURE! - in TestSuite
[ERROR] testNow(org.jmisb.api.klv.st0601.PrecisionTimeStampTest) Time elapsed: 0.015 s <<< FAILURE!
java.lang.AssertionError: expected [127108000] but found [127000000]
at org.jmisb.api.klv.st0601.PrecisionTimeStampTest.testNow(PrecisionTimeStampTest.java:49)

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] PrecisionTimeStampTest.testNow:49 expected [127108000] but found [127000000]
[INFO]
[ERROR] Tests run: 165, Failures: 1, Errors: 0, Skipped: 0`

Tests have logging output

Describe the bug
During a build, some of the tests output (expected) logging messages. That is undesirable because it hides potential real problems (unexpected outputs).

I made this worse during the ST0903 activity. At the time it was useful to check the tests were doing what I expected, but that is no longer relevant - now its just annoying.

To Reproduce
Steps to reproduce the behavior:

  1. Run mvn install
  2. Note output, mostly related to unknown tags.

Expected behavior
Tests check for expected logger outputs, verify they occur when they should, and not have them appear in the test output.

Configuration (please complete the following information):
N/A

Additional context
I'll work this soon.

Check ST0601.16a Tag 72 (Event Start Time UTC) example

Task Background
The test value in ST0601.16a looks to be an hour out.

Completion criteria
Check this with MISB before implementing.

Implementation Notes and Ideas

$ date --utc --date='@798036294'
Sun 16 Apr 12:44:54 UTC 1995

Add Windows build to PR / Push

Task Background
I had good success with adding MacOS build with github actions. We should do that for Windows too.

Completion criteria
Its running on each push and PR with windows-latest.

Implementation Notes and Ideas
N/A.

Add support for RTSP to viewer application

What is the use case for your proposed feature? Please describe.
It would be handy if the viewer application could read from an RTSP URL, to support testing of that kind of source.

Describe the solution you'd like
From a UI perspective, I'd be happy if it had another entry under the File... menu, or just worked as another URL.

Describe alternatives you've considered
Not really assessed.

Additional context
Possibly bytedeco/javacpp-presets#395 is relevant.

Upgrade to ST1204.3

What is the use case for your proposed feature? Please describe.
MISB has a draft ST1204.3. We should implement that in a branch to provide useful feedback. At this stage, the key change is likely to be a method or class that implements the UUID combining.

Describe the solution you'd like
(This will be worked out during the update)

Describe alternatives you've considered
(We could not bother, and just wait for it to be released).

Additional context
There are also changes to the Appendix B (check digit) text, but I think we already have those based on some emails from Paul Hed.

CMITT complains about length not encoded in fewest possible bytes

Describe the bug
Running CMITT 1.3.0 on a file with a VMTI LS of length 157 bytes [0x9d] or 148 bytes [0x95] results in a compliance finding for requirement ST0601.8-07, with "Length not encoded in the shortest possible manner".

To Reproduce
If you uncomment the debug statement in the ST0102 LocalSet build test (https://github.com/WestRidgeSystems/jmisb/blob/develop/api/src/test/java/org/jmisb/api/klv/st0102/localset/SecurityMetadataLocalSetTest.java#L147) you'll see it output a length that has a 0x00 second byte.

Expected behavior
Pass CMITT.

Screenshots
N/A

Configuration (please complete the following information):

  • OS: Linux
  • java version "1.8.0_201", Java(TM) SE Runtime Environment (build 1.8.0_201-b09), Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
  • trunk (with changes).

Additional context
Add any other context about the problem here.

Don't throw on unrecognised keys

Describe the bug
We currently throw if an unrecognised key is hit. That is inconsistent with requirement ST 0107.3-04 (as an example):

Applications which decode MISB KLV Local Sets shall skip unknown Local Set values so as to not impact the decoding of known Local Set items within the same Local Set instance.

To Reproduce
Steps to reproduce the behavior:

  1. Open a file containing an unknown tag in viewer
  2. Note no metadata shown.

Expected behavior
Unhandled metadata is ignored, other metadata is parsed.

Some files show checksum errors (no metadata display)

Describe the bug
With some of the MISB FLI data files (e.g. HD_H264_06011_TS_SYN_V1_001.mpg) I see checksum errors (which propagate to KlvParser exceptions, leading to no metadata shown in the viewer application).

On debugging, it looks like both checksum bytes are out by 2 from what is expected.

This doesn't happen with all files - some are fine, some have errors.

To Reproduce
Steps to reproduce the behavior:

  1. Open HD_H264_06011_TS_SYN_V1_001.mpg in viewer application
  2. Note absence of metadata.

Expected behavior
Should show metadata, no checksum errors.

Screenshots
If applicable, add screenshots to help explain your problem.

Configuration (please complete the following information):

  • OS: Linux
  • JDK: javac 1.8.0_201
  • jMISB version or commit: master

Additional context
"2" is probably significant because its the last byte we're adding (length of the checksum field). Additional debugging shows that it only happens on files with odd numbers of bytes in the UasMessage. I think we need to left shift the final byte (0x02) before adding it, which would be consistent with the algorithm shown on page 10 of ST0601.15.

Option to encode h.265

Currently the video encoder is hard wired to use h.264. Add an option to allow the client to choose between h.264/265.

Slow motion and fast-forward

Allow playback speed to be changed by specifying a multiplier. E.g., 0.5 would result in playback at half speed, 4.0 would be 4x fast-forward.

Separate unit & integration testing

Robust integration testing is an important requirement to ensure stability. Integration tests will, e.g.:

  • Generate synthetic data
  • Stand up live streams and read from them
  • Read/write to the filesystem
  • [Future] Pull down external sample data for testing

All of these are beyond the scope of unit tests and should not be run as part of the unit test suite. Need to determine how best to manage the integration test suite within the maven project and document when & how they should be run.

ST 0601 Tag 133 "On-board MI Storage Capacity" doesn't handle 3 byte case

Describe the bug
The subject tag is variable length (1-4 bytes). We don't handle the three byte case.

To Reproduce
Inspect code.

Expected behavior
Handle three byte case.

Screenshots
N/A

Configuration (please complete the following information):
N/A.

Additional context*
I'll work this. Plan is to modify the PrimitiveConverter to have a "fixed four byte" and a "variable up to four byte" utility methods, since there are other tags that need this.

ST0903 VTrackerLS DetectionStatus should have enumerated values

What is the use case for your proposed feature? Please describe.
The DetectionStatus is a bit hard to use, because it requires a byte value argument for the value constructor:

Map<VTrackerMetadataKey, IVmtiMetadataValue> map = new TreeMap<>();
map.put(VTrackerMetadataKey.detectionStatus, new DetectionStatus((byte)0x01));

That would be better if it just had an enumeration values.

Describe the solution you'd like
Add the four enums to DetectionStatus (see ST0903.5 Table 10).

Describe alternatives you've considered
We could just have static DetectionStatus values for each case. Or both.

Additional context
Done:
[ ] Enum values added
[ ] Docs / example added
[ ] Full unit tests.

Add an event for end-of-file streaming

What is the use case for your proposed feature? Please describe.
As a developer, I'd like to know when my file has finished streaming. Then my IMetadataListener implementation can signal "done".

Describe the solution you'd like
Probably a new interface with a onFileEvent(FileEvent event) which could signal a new file, closing an existing file, possibly some other events.

Describe alternatives you've considered
None at this stage.

Additional context
Not sure if this is exposed in ffmpeg bindings.

Metadata file / stream output mixes sync and async conventions

Describe the bug
When we output metadata, we can do sync or async. We're mixing the stream ID and stream types for those.

From ST1402.1, on sync:

ISO/IEC 13818-1 Table-34 defines a stream_type = 0x15 for “Metadata carried in PES packets,”
and Table 2-22 defines a stream_id = 0xFC for “metadata stream.”

From ST1402.1, on async:

In ISO/IEC 13818-1, Table-34 defines a stream_type = 0x06 for “PES packets containing private
data,” and Table 2-22 defines a stream_id = 0xBD for “private_stream_1.”

We are doing 0x06 with 0xFC.

To Reproduce
Steps to reproduce the behavior:

  1. Run the VideoFileInputOutputIT::testWithData() integration test
  2. Inspect the testWithData.ts file (I used tsanalyze from tsduck)
  3. Note:
|=============================================================================|
|  PID: 0x0101 (257)                                 MPEG-2 PES private data  |
|  PES stream id: 0xFC (MPEG-7 metadata stream)                               |
|  Service: 0x0001 (1) Service01                                              |
|-----------------------------------------------------------------------------|
|  Single Service PID        Transport:                Discontinuities:       |
|  Bitrate: .... 43,057 b/s  Packets: ........... 240  Expected: ......... 0  |
|  Access: .......... Clear  Adapt.F.: .......... 120  Unexpect: ......... 0  |
|                            Duplicated: .......... 0  PES:                   |
|                            PCR: ................. 0  Packets: ........ 120  |
|                                                      Inv.Start: ........ 0  |
|=============================================================================|

That should say:
MPEG-2 PES private data and PES stream id: 0xBD
or
MetaData in PES packets and PES stream id: 0xFC

This is a non-conformance in CMITT 1.3.0, and is a likely interoperability problem.

Expected behavior
Conformance with ST1204. Ideally we'd be able to select either, but at least one would be enough to solve this ticket (with the other being a future feature).

N/A

Configuration (please complete the following information):

  • OS: Linux
  • JDK: java version "1.8.0_201"
  • jMISB version or commit: f081146

Additional context
I tried setting the metadata stream identifier in VideoOutput (i.e. metadataStream.stream_identifier(0xBD);) but that didn't have any effect.

@wlfgang any ideas?

Consider building ST0601Version from short (rather than byte) value

Describe the bug
We have:

    /**
     * Create from value
     * @param version The version number
     */
    public ST0601Version(byte version)

That might be better as short, to support the valid range (0...255).

We could potentially deprecate the existing constructor and add a new one, removing the deprecated one later (next major version).

Add proper support for ST0603 / MISP Time System

Describe the bug
MISP is big on time, but we're being a bit loose around some of the concepts. In particular, we need to be more careful about conversion to and from UTC.

To Reproduce
Steps to reproduce the behavior:

  1. Review MISB Handbook and ST0603
  2. Note the part about not including leap seconds.

Expected behavior
Probably move the existing code (plus code we should have) in ST0601 for parsing out timestamps, correction factors and leap seconds into some shared code. Make sure that the types (LocalDateTime, ZonedDateTime) we're using underneath have the right leap second semantics.
Rework the ST0903 times to use that.

Maybe add support for the nanosecond timestamp.

Screenshots
N/A

Configuration (please complete the following information):
N/A

Additional context
I'll try to get this done, but it looks like it could be more complex that I originally thought, so there is no schedule.

Definition of Done (subject to change):

[X] Migrate shared code to new st0603 module (covered by #114 )
[X] Add time status implementation (covered by #114)
[ ] Update API / docs / implementation to reflect MISP vs UTC references
[ ] Add conversion routines that handle different time bases (basically the 12 cases in Motion Imagery Handbook 2020.1 Table 6-4.)
[ ] Use conversion routines in ST1909 to show FT and MT in UTC.

out of scope:

  • Nanosecond time stamp conversion. This can be a separate ticket if we need it.

Upgrade log4j

Describe the bug
We currently use 1.2.17. We need a newer version.

To Reproduce
OWASP checks.

Expected behavior
Upgrade or avoid this logger.

Screenshots
N/A.

Configuration (please complete the following information):
N/A.

Additional context
N/A

Selectively disable decoding

When reading an input file or stream, add an option to allow the client to disable video or metadata decoding, e.g., if it only wants one of the streams.

SpotBugs issues for core

Describe the bug
Running spotbugs on core produces:

[INFO] BugInstance size is 4
[INFO] Error size is 0
[INFO] Total bugs: 4
[ERROR] Found reliance on default encoding in org.jmisb.core.video.FfmpegUtils.formatError(int): new String(byte[]) [org.jmisb.core.video.FfmpegUtils] At FfmpegUtils.java:[line 39] DM_DEFAULT_ENCODING
[ERROR] Found reliance on default encoding in org.jmisb.core.video.FfmpegUtils.fourCcToTag(String): String.getBytes() [org.jmisb.core.video.FfmpegUtils] At FfmpegUtils.java:[line 92] DM_DEFAULT_ENCODING
[ERROR] Found reliance on default encoding in org.jmisb.core.video.FfmpegUtils.tagToFourCc(int): new String(byte[]) [org.jmisb.core.video.FfmpegUtils] At FfmpegUtils.java:[line 76] DM_DEFAULT_ENCODING
[ERROR] Integral division result cast to double or float in org.jmisb.core.video.FfmpegUtils.getDuration(AVFormatContext) [org.jmisb.core.video.FfmpegUtils] At FfmpegUtils.java:[line 103] ICAST_IDIV_CAST_TO_DOUBLE

To Reproduce
I have added the maven spotbugs plugin.

Expected behavior
No errors.

Screenshots
If applicable, add screenshots to help explain your problem.

Configuration (please complete the following information):
Current develop.

Additional context
Two of the string conversions are related to FourCC, which appears to use US ASCII. The other string conversion should probably be UTF-8.
The division is probably easy to fix by doing the cast first.

BER-OID decoding

What is the use case for your proposed feature? Please describe.
Newer MISB 0601 standards are using Tag values greater than 127, which triggers the BER-OID encoding.

Describe the solution you'd like
Support BER-OID decoding, which the below works.

        int value = 0;
        int read;
        int size = 0;
        do {
            read = is.read();
            int highbits = (value << 7);
            int lowbits = (read & 0x7F);
            value = highbits + lowbits;
            size++;
        } while ((read & 0x80) == 0x80);
        return new BerOidField(size, value);

CMITT 1.3.0 complains about insufficient PAT and PMT rate

Describe the bug
CMITT complains "ST 1402-02: PAT and PMT shall refresh 4 times per second "

To Reproduce
Build file, run through CMITT.

Expected behavior
Green tick in CMITT

Screenshots
N/A

Configuration (please complete the following information):
N/A

Additional context
Looks like we need to set some encoding / header option in VideoFileOutput and VideoStreamOutput (av_dict_set(opts, "pat_period", "250000", 0); ?)

Evaluate adding getDisplayName to IUasDatalinkValue

What is the use case for your proposed feature? Please describe.
During #32 evaluation, @wlfgang suggested moving the getDisplayName() abstract method up to IUasDatalinkValue.

This task / feature request is to evaluate the consequences of that, and implement if appropriate.

Describe the solution you'd like
Pull getDisplayName() up from UasEnumeration to the IUasDatalinkValue interface.

Describe alternatives you've considered
Only including it in classes where its useful (perhaps just UasEnumeration). Subject to further thought.

ControlCommand can repeat in a single ST0601 LS.

Describe the bug
ST0601 allows a small number of tags to repeat (see MUL column in Table 1). We don't currently implement any of those though, except for ControlCommand (Tag 115). That will get overwritten in the current parsing approach.

To Reproduce
Steps to reproduce the behavior:
This would require special test data to reproduce. I don't yet have any.

Expected behavior
We'd need to think harder about the change. Possibly a multimap would help.

Screenshots
N/A

Configuration (please complete the following information):
N/A.

Additional context
None at this stage.

Implement special cases encoding / decoding for ST1201

What is the use case for your proposed feature? Please describe.
ST1201 describes some special cases, which we implement a subset of. As a developer, I'd like to know they're available and being handled if we ever hit them on decode.

Describe the solution you'd like
The current implementation describes an encodeSpecial() that doesn't appear to be implemented. We probably need that, and a set of corresponding checks in the decode() path.

Describe alternatives you've considered
None at this time.

Additional context
This is a follow-on to #72 work. That should probably go in first.

SpotBugs errors for api

Describe the bug
Running spotbugs on api:

[INFO] BugInstance size is 22
[INFO] Error size is 0
[INFO] Total bugs: 22
[ERROR] new org.jmisb.api.klv.LdsField(int, byte[]) may expose internal representation by storing an externally mutable object into LdsField.data [org.jmisb.api.klv.LdsField] At LdsField.java:[line 21] EI_EXPOSE_REP2
[ERROR] org.jmisb.api.klv.LdsParser.parseFields(byte[], int, int) concatenates strings using + in a loop [org.jmisb.api.klv.LdsParser] At LdsParser.java:[line 61] SBSC_USE_STRINGBUFFER_CONCATENATION
[ERROR] new org.jmisb.api.klv.RawMisbMessage(UniversalLabel, byte[]) may expose internal representation by storing an externally mutable object into RawMisbMessage.bytes [org.jmisb.api.klv.RawMisbMessage] At RawMisbMessage.java:[line 14] EI_EXPOSE_REP2
[ERROR] new org.jmisb.api.klv.UdsField(UniversalLabel, byte[]) may expose internal representation by storing an externally mutable object into UdsField.value [org.jmisb.api.klv.UdsField] At UdsField.java:[line 15] EI_EXPOSE_REP2
[ERROR] new org.jmisb.api.klv.UniversalLabel(byte[]) may expose internal representation by storing an externally mutable object into UniversalLabel.bytes [org.jmisb.api.klv.UniversalLabel] At UniversalLabel.java:[line 27] EI_EXPOSE_REP2
[ERROR] Found reliance on default encoding in new org.jmisb.api.klv.st0102.CcmDate(byte[]): new String(byte[]) [org.jmisb.api.klv.st0102.CcmDate] At CcmDate.java:[line 28] DM_DEFAULT_ENCODING
[ERROR] Found reliance on default encoding in new org.jmisb.api.klv.st0102.DeclassificationDate(byte[]): new String(byte[]) [org.jmisb.api.klv.st0102.DeclassificationDate] At DeclassificationDate.java:[line 28] DM_DEFAULT_ENCODING
[ERROR] new org.jmisb.api.klv.st0102.ItemDesignatorId(byte[]) may expose internal representation by storing an externally mutable object into ItemDesignatorId.itemDesignatorId [org.jmisb.api.klv.st0102.ItemDesignatorId] At ItemDesignatorId.java:[line 22] EI_EXPOSE_REP2
[ERROR] Found reliance on default encoding in new org.jmisb.api.klv.st0102.OcmDate(byte[]): new String(byte[]) [org.jmisb.api.klv.st0102.OcmDate] At OcmDate.java:[line 28] DM_DEFAULT_ENCODING
[ERROR] org.jmisb.api.klv.UniversalLabel is incompatible with expected argument type Integer in org.jmisb.api.klv.st0102.SecurityMetadataKey.getKey(UniversalLabel) [org.jmisb.api.klv.st0102.SecurityMetadataKey] At SecurityMetadataKey.java:[line 71] GC_UNRELATED_TYPES
[ERROR] Found reliance on default encoding in new org.jmisb.api.klv.st0102.SecurityMetadataString(byte[]): new String(byte[]) [org.jmisb.api.klv.st0102.SecurityMetadataString] At SecurityMetadataString.java:[line 27] DM_DEFAULT_ENCODING
[ERROR] Found reliance on default encoding in new org.jmisb.api.klv.st0102.universalset.ClassificationUniversal(byte[]): new String(byte[]) [org.jmisb.api.klv.st0102.universalset.ClassificationUniversal] At ClassificationUniversal.java:[line 30] DM_DEFAULT_ENCODING
[ERROR] Found reliance on default encoding in org.jmisb.api.klv.st0102.universalset.ClassificationUniversal.getDisplayableValue(): new String(byte[]) [org.jmisb.api.klv.st0102.universalset.ClassificationUniversal] At ClassificationUniversal.java:[line 90] DM_DEFAULT_ENCODING
[ERROR] new org.jmisb.api.klv.st0601.OpaqueValue(byte[]) may expose internal representation by storing an externally mutable object into OpaqueValue.bytes [org.jmisb.api.klv.st0601.OpaqueValue] At OpaqueValue.java:[line 16] EI_EXPOSE_REP2
[ERROR] int value cast to float and then passed to Math.round in org.jmisb.api.klv.st0601.TargetTrackGateSize.getBytes() [org.jmisb.api.klv.st0601.TargetTrackGateSize] At TargetTrackGateSize.java:[line 60] ICAST_INT_CAST_TO_FLOAT_PASSED_TO_ROUND
[ERROR] Found reliance on default encoding in new org.jmisb.api.klv.st0601.UasDatalinkString(String, byte[]): new String(byte[]) [org.jmisb.api.klv.st0601.UasDatalinkString] At UasDatalinkString.java:[line 45] DM_DEFAULT_ENCODING
[ERROR] Found reliance on default encoding in new org.jmisb.api.klv.st0601.WavelengthsList(byte[]): new String(byte[], int, int) [org.jmisb.api.klv.st0601.WavelengthsList] At WavelengthsList.java:[line 94] DM_DEFAULT_ENCODING
[ERROR] Found reliance on default encoding in org.jmisb.api.klv.st0601.WavelengthsList.getBytes(): String.getBytes() [org.jmisb.api.klv.st0601.WavelengthsList] At WavelengthsList.java:[line 125] DM_DEFAULT_ENCODING
[ERROR] Possible null pointer dereference of platformDesignation in org.jmisb.api.klv.st0805.KlvToCot.getPlatformUid(UasDatalinkMessage) [org.jmisb.api.klv.st0805.KlvToCot, org.jmisb.api.klv.st0805.KlvToCot] Dereferenced at KlvToCot.java:[line 175]Known null at KlvToCot.java:[line 173] NP_NULL_ON_SOME_PATH
[ERROR] Found reliance on default encoding in new org.jmisb.api.klv.st0903.shared.VmtiUtf8(String, byte[]): new String(byte[]) [org.jmisb.api.klv.st0903.shared.VmtiUtf8] At VmtiUtf8.java:[line 56] DM_DEFAULT_ENCODING
[ERROR] Wait not in loop in org.jmisb.api.video.ProcessingThread.pauseOrResume() [org.jmisb.api.video.ProcessingThread] At ProcessingThread.java:[line 31] WA_NOT_IN_LOOP
[ERROR] Possible null pointer dereference of VideoDecodeThread.bgrFrame in org.jmisb.api.video.VideoDecodeThread.allocateImages(int, int) [org.jmisb.api.video.VideoDecodeThread, org.jmisb.api.video.VideoDecodeThread, org.jmisb.api.video.VideoDecodeThread] Dereferenced at VideoDecodeThread.java:[line 246]Null value at VideoDecodeThread.java:[line 240]Known null at VideoDecodeThread.java:[line 242] NP_NULL_ON_SOME_PATH

(Note there are additional warnings - I'm filtering those).

To Reproduce
Run spotbugs on api

Expected behavior
Error free output.

Screenshots
If applicable, add screenshots to help explain your problem.

Configuration (please complete the following information):
Current develop.

Additional context

Primitive conversions can change "existing" values

Describe the bug
When building up a local set, its common to use a value to bytes conversion routine "under the hood" several times (e.g. latitude, then longitude, both of which will probably use PrimitiveConverter.int32ToBytes()). That can cause unexpected behaviour, where the byte array for latitude gets changed when you calculate the longitude.

To Reproduce

    @Test
    public void testSignedInt32ToBytesReuse()
    {
        byte[] bytes1 = PrimitiveConverter.int32ToBytes(1);
        Assert.assertEquals(bytes1, 
                   new byte[]{(byte)0x00, (byte)0x00, (byte)0x00, (byte)0x01});

        byte[] bytes2 = PrimitiveConverter.int32ToBytes(2);
        Assert.assertEquals(bytes2, 
                   new byte[]{(byte)0x00, (byte)0x00, (byte)0x00, (byte)0x02});

        // Verify the original array is still OK
        Assert.assertEquals(bytes1, 
                   new byte[]{(byte)0x00, (byte)0x00, (byte)0x00, (byte)0x01});
    }

That last assert will fail (with bytes1[3] now being set to 2).

Expected behavior
We probably should return a independent byte array each time.

Screenshots
N/A.

Configuration (please complete the following information):

  • OS: Linux
  • JDK: javac 1.8.0_201
  • jMISB version or commit: f5ceb9c

Additional context
This is the documented behaviour for ByteBuffer (see https://docs.oracle.com/javase/8/docs/api/java/nio/ByteBuffer.html#array--), which are shared (within a thread) in PrimitiveConverter. I think we need to return a copy of the array (or possibly instantiate a new ByteBuffer on each call, but that seems like it would be worse).

@wlfgang: Do we change the behaviour of the old methods, or deprecate and add new methods?

ST0601 Tag 93 might have wrong encoding

Describe the bug
ST0601.16 added an example for this Tag (previous versions didn't provide it). That doesn't work with the current implementation. There is ambiguity in the spec about whether the range is -180/180 or -90/90. We currently do the latter, but the example does the former.
The text and table are pretty much split evenly on which applies. Perhaps some cut-n-paste issues in the spec as well as our current implementation.

I've asked the MISB secretariat. There is a meeting next week.

I have the example implemented, but it isn't clear about what should actually happen, so this is just to track the issue for now.

Add test coverage support to build

What is the use case for your proposed feature? Please describe.
Not a feature, but a nice-to-have for development: jacoco or similar unit test coverage checks.

I have a PR in work.

Add formatter plugin to build

What is the use case for your proposed feature? Please describe.
As a developer, I want consistently formatted code to make it easier to understand.

Describe the solution you'd like
I'd like the formatting to be applied automatically prior to compilation (e.g. using https://code.revelc.net/formatter-maven-plugin/)

Describe alternatives you've considered
Configuration / plugins for at least netbeans would also be nice.

Something like fmt-maven-plugin would also be OK, although that uses Google Java format, which I have no objection to, but looks quite different to the current code format.

Possibly checkstyle would be an alternative - I haven't tried that.

Any alternative to manually formatting would be an improvement on what I have now.

Additional context
Add any other context or screenshots about the feature request here.

Create (output) network streams

Add the ability to generate MPEG-TS streams over UDP. The interface should allow queueing video and metadata frames to be multiplexed into the output.

Stream recorder

Add the option to record incoming streams. Data should be saved to chunked files, which can later be re-combined or played back directly.

ST1201 implementation is incomplete

Describe the bug
ST1201 doesn't limit lengths - it can be anything. Actual usage of this is 1-8 bytes. We support 1, 2, 3, 4 and 8. We need to support 5, 6 and 7.

To Reproduce
This is from code inspection and unit testing during #60 work. I only implemented length = 3 for that.

Expected behavior
We should support 5-7 as well. Also, we should be careful about signed / unsigned at length = 8.

Screenshots

Configuration (please complete the following information):
N/A

Additional context
I'm yet to see any "expected results" for those lengths. We may need to generate them.

Implement ST 1909

What is the use case for your proposed feature? Please describe.
ST 1909 is used to display metadata overlays on video. It may come in handy for developers during testing, as well as for end users when viewing a feed without "burned-in" metadata.

Describe the solution you'd like
API should allow clients to receive raw video frames, then annotate them with overlays using SecurityMetadataMessage/UasDatalinkMessages. Example usage should be added to the viewer application, with an option to toggle overlays on/off.

Describe alternatives you've considered
API could return video frames already annotated to simplify the interface, but some clients would like access to both raw and annotated frames.

Additional context
N/A

Add Android build to CI

Task Background
I had good success with adding MacOS build with github actions. We should do that for Android too.

Completion criteria
Its running on each push and PR with windows-latest.

Implementation Notes and Ideas
Have look at how bytedeco does it for javacpp-presets.

Add check for unique display names

What is the use case for your proposed feature? Please describe.
See #73 for the background. This is not directly user visible, it supports development.

Describe the solution you'd like
Ensure that each entry in the results of the parse has a distinct display name.

Describe alternatives you've considered
It might be possible to instantiate every entry and check the results. That could be messy in that you'd need to know what the right byte length is. Possibly it could be brute forced by trying 1,2,4 and 8 bytes.

Additional context
N/A.

Need a github template for non-user visible "Task"

What is the use case for your proposed feature? Please describe.
At this stage we only have "Bug" and "Feature". Some issues are just future tasks (e.g. for test coverage, or cleanup). The descriptions don't really support that.

Describe the solution you'd like
Add a new ISSUE type for general task.

Describe alternatives you've considered
It could all be considered Features (like this one), but I think its worth distinguishing between them.

Additional context
N/A

Viewer application Metadata Panel could use a scrollbar

What is the use case for your proposed feature? Please describe.
There is sometimes so many metadata entries that it won't all fit in the metadata panel (even when I go full screen).

Describe the solution you'd like
Scrollbars come on when needed.

Describe alternatives you've considered
A second panel would be an alternative (swapping vertical to horizontal). It would look a bit weird, but would have the advantage of showing all the data at the same time.

Another alternative would be to provide some kind of filtering to show less data. That seems a bit excessive though.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.