eyevinn / chaos-stream-proxy Goto Github PK

The Relative Sequence index config, rsq, is great for setting up an HLS corruption quickly. Skipping the step of having to look up what the stream's current media-sequence number is.

However, it would be even more effective if the users could configure to target the corruption onto a segment calculated from the bottom of the playlist manifest, instead of the top, as this can be a cumbersome task to calculate depending on the size of the playlist manifest.

Corrupting the segments at/near the bottom of the manifest is a good way to simulate errors from players who get too close to the live edge in a stream, trying to access a segment file that is yet to be downloadable.

This improvement would streamline the process of simulating real-world scenarios and enhance the product's capability to replicate edge-case conditions encountered by HLS players.

Proposed Solution

I propose a solution for rsq to also be able to accept negative integers in which it will step from the bottom of the segments list rather than the top.

ie.

The Corruption query ?url=https://mock.mock.com/stream/hls/master.m3u8&statusCode=[{rsq:-1,code:404}]&delay=[{rsq:-2,ms:2000}]
Results in the following playlist manifest on the first load:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:10045
#EXT-X-PLAYLIST-TYPE:LIVE
#EXTINF:10.0000,
https://mock.mock.com/stream/hls/manifest_1_010045.ts
#EXTINF:10.0000,
https://mock.mock.com/stream/hls/manifest_1_010046.ts
#EXTINF:10.0000,
https://mock.mock.com/stream/hls/manifest_1_010047.ts
#EXTINF:10.0000,
https://mock.mock.com/stream/hls/manifest_1_010048.ts
#EXTINF:10.0000,
https://mock.mock.com/stream/hls/manifest_1_010049.ts
#EXTINF:10.0000,
../../segments/proxy-segment?url=https%3A%2F%2Fmock.mock.com%2Fstream%2Fhls%2Fmanifest_1_010050.ts&delay=%7Bms%3A2000%7D
#EXTINF:10.0000,
../../segments/proxy-segment?url=https%3A%2F%2Fmock.mock.com%2Fstream%2Fhls%2Fmanifest_1_010051.ts&statusCode=%7Bcode%3A404%7D

If you try to play streams with $Time segments it fails eg. http://localhost:8000/api/v2/manifests/dash/proxy-master.mpd?url=https://livesim.dashif.org/livesim2/segtimeline_1/testpic4_8s/Manifest.mpd

Proxying DASH stream results in 500 Internal Server Error:
{"reason":"Cannot read properties of undefined (reading 'match')"}

For source URL and other specifics - please contact me directly.

JWT token endpoint protection

If the service is started with the environment variable JWT_SECRET it should require a JWT token as query-parameter to the endpoint requests (health endpoint excluded).

The JWT-token should include the following payload:

{
  "company": <COMPANY>,
  "email": <EMAIL-TO-PERSON-AT-COMPANY>
}

The JWT-token is signed by the secret specified in the env variable JWT_SECRET and validated on requests. If not valid a "Not Authorized" status code should be returned.

Additional logging

All requests should be logged as a JSON-structure including the information provided in the JWT and endpoint accessed

Currently only HLS is supported for rsq corruptions. The same feature should be implemented also for DASH

Support for returning code 404 for a specific segment with LIVE MPEG-DASH that uses SegmentTemplate timeline:

<SegmentTemplate
        timescale="48000"
        initialization="tnse_nickelodeon_se_fi_live-$RepresentationID$.dash"
        media="tnse_nickelodeon_se_fi_live-$RepresentationID$-$Time$.dash">
  <SegmentTimeline>
    <S t="80727778699407" d="184320" r="78" />
  </SegmentTimeline>
</SegmentTemplate>

Insert in all responses a custom header X-Version indicating the version of chaos-stream proxy and ensure it is allowed and exposed in Access-Control headers [1], e.g.

Access-Control-Allow-Headers: X-Version
Access-Control-Expose-Headers: X-Version
X-Version: 0.3.0 

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Headers

A way to make it possible to throttle transfer of specific segments. For example you want to simulate that the network conditions changed and check how the player's ABR algorithm handles it.
The difference to the delay corruption is that this would respond immediately but emulate that the network bandwidth/througput is reduced

Currently it can be tricky to figure out why something isn't working with the CSP, would be good to look over the responses being sent when something goes wrong.

Currently, when configuring corruption on a bitrate level in Chaos Stream Proxy, each bitrate requires a separate configuration item. For example:

https://<chaos-proxy>/api/v2/manifests/hls/proxy-master.m3u8?url=https://maitv-vod.lab.eyevinn.technology/VINN.mp4/master.m3u8&delay=[{i:5,ms:1500}]&statusCode=[{i:5,code:400,br:1212000}, {i:5,code:400,br:3131000}]

It makes sense to require multiple corruption items when you want to target multiple segments. But if I want to target the same segment, creating multiple items only to change the br value seems a bit tedious.
I would like to request an improvement that allows specifying multiple bitrates within a single configuration item if you want it to apply to the same target segment. This would enhance the user experience and simplify the configuration process. The desired format would be:

https://<chaos-proxy>/api/v2/manifests/hls/proxy-master.m3u8?url=https://maitv-vod.lab.eyevinn.technology/VINN.mp4/master.m3u8&delay=[{i:5,ms:1500}]&statusCode=[{i:5,code:400,br:[1212000,3131000]}]

By enabling the ability to provide a list of bitrates for a single configuration item, users can target multiple bitrates more efficiently and with cleaner configuration syntax. This enhancement would streamline the configuration process and improve the usability of Chaos Stream Proxy for users managing multiple bitrates simultaneously.

So the new type would be

    br?: number | number[] | "*", // apply only to specific bitrate

Issue happens when parsing a HLS manifest, specifically when parsing a #EXT-X-MEDIA with TYPE=AUDIO and without an URI.

URI is only a required property with TYPE=SUBTITLES

Currently the chaos stream proxy does not add the master manifest location as a prefix to the MP4 init segment URLs which makes playback of CMAF VOD fail.

Example CMAF VOD source file:
https://vod.streaming.a2d.tv/a72e697d-63ea-44d8-821c-1d3be9fa4259/142d4370-56c9-11ee-a816-2da19aea6ed1_20571919.ism/.m3u8

Example chaos proxy URL:
https://tv4-stateful.eyevinn-chaos-stream-proxy.auto.prod.osaas.io/api/v2/manifests/hls/proxy-master.m3u8?url=https://vod.streaming.a2d.tv/a72e697d-63ea-44d8-821c-1d3be9fa4259/142d4370-56c9-11ee-a816-2da19aea6ed1_20571919.ism/.m3u8&delay=[{i:4,ms:3000}]

The proxy will add the full URL master manifest location to the media segments, e.g.:
https://vod.streaming.a2d.tv/a72e697d-63ea-44d8-821c-1d3be9fa4259/142d4370-56c9-11ee-a816-2da19aea6ed1_20571919.ism/hls/142d4370-56c9-11ee-a816-2da19aea6ed1_20571919-audio=128000-1.m4s

However, for the EXT-X-MAP init segment this is not done. This will cause the player to ad the base of the proxy URL master manifest to the relative path of the init segment. This causes the client to fetch a path on the proxy that does not exist (404 and playback fails. Example of 404 URL that is not rewritten with the actual VOD base URL:
https://tv4-stateful.eyevinn-chaos-stream-proxy.auto.prod.osaas.io/api/v2/manifests/hls/hls/142d4370-56c9-11ee-a816-2da19aea6ed1_20571919-video=300000.m4s

There are some typing issues remaining that are currently disabled from the eslint checks eslint-disable-* that should be addressed and fixed.

Currently we string replace $Number$ with the segment index, but there is also for example a zero-pad syntax $Number%0[width]d$ where you should zero-pad with the number specified as the width. I have fix for this coming up, but there seems to be more to it. For example if you look at dash.js issue tracker for PR 32 and 196, they have further details.

https://ottverse.com/structure-of-an-mpeg-dash-mpd/#Segment_Templates

#53 fixes the known one, but the dash.js pr 196 supports a lot more variants, and it would be good to also add tests.

For LIVE streams add support for offset based corruptions. For example it should be possible to specify that the n:th segment from first load should be corrupted.

When baseURL is defined on Period level (and not global) the resulting proxied manifest is incorrect.

For example streams, contact @ABLindberg

To be able to simulate an abr stream whose variants are not perfectly in sync, we would need to add the ability to corrupt segments targeted not only by segment index but also by variant.

I suggest exploring 2 approaches, but please share any of you own:
A) Set an optional br target variant parameter in the query corruption item
B) Add new query param, for some type of filter value. Reminiscent to the one in channel-engine