scramjetorg / framework-js Goto Github PK

Feature description

Introduce DataStream.all() method similar to v4:

Processes a number of functions in parallel, returns a stream of arrays of results.

This method is to allow running multiple asynchronous operations and receive all the results at one, just like Promise.all behaves.

If you are interested in this issue or this type of transforms fits your use case, please upvote with 👍.

Feature description

Introduce DataStream.flatten() method similar to v4:

A shorthand for streams of arrays or iterables to flatten them.

More efficient equivalent of: .flatmap(i => i); Works on streams of async iterables too.

If you are interested in this issue or this type of transforms fits your use case, please upvote with 👍.

Feature description

Introduce WindowStream similar to WindowStream from v4:

In essence it's a stream of Array's containing a list of items - a window.

It would be a stream for moving window calculation with some simple methods.

Use case

Any data calculations which requires moving window (e.g. calculating moving averages).

If you are interested in this issue or this stream type fits your use case, please upvote with 👍.

Feature description

Introduce BinaryStream similar to BufferStream from v4.

By default, each chunk flowing through the stream would be a single byte. The stream should allow manipulating binary data (grouping, splitting, parsing, shifting, etc). Also it should be possible to convert such stream into other types (most notably StringStream and DataStream).

Use case

Various file types transformations, parsing binary formats, binary data manipulation (images, videos, etc).

If you are interested in this issue or this stream type fits your use case, please upvote with 👍.

Feature description

Introduce DataStream.empty() method similar to v4:

Called only before the stream ends without passing any items.

If you are interested in this issue or this type of transforms fits your use case, please upvote with 👍.

Feature description

Introduce MultiStream/MuxStream similar to MultiStrean from v4.

The idea behind a MultiStream is being able to mux and demux streams when needed. MultiStream is an object consisting of multiple streams than can be refined or muxed.

Use case

Merging streams from various sources so all data can be processed by the same pipeline. Distributing data into multiple streams.

If you are interested in this issue or this stream type fits your use case, please upvote with 👍.

I have noticed recently that CI is being unstable and some tests are failing from time to time. It seems to be the case of pipe tests mostly:

  streams › data › native-interface › Piped DataStream can be unpiped via '.unpipe(instance)' #2

  build/test/unit/streams/data/native-interface.spec.js:109

   108:             await stream.end();                                
   109:             t.deepEqual(await stream.toArray(), ["fo", "o\n"]);
   110:             resolve();                                         

  Difference:

    [
      'fo',
  +   `o␊
  +   `,
    ]

  › ReadStream.<anonymous> (build/test/unit/streams/data/native-interface.spec.js:109:15)



  streams › data › native-interface › Piped DataStream can be unpiped via '.unpipe()' #2

  build/test/unit/streams/data/native-interface.spec.js:145

   144:             await stream.end();                                
   145:             t.deepEqual(await stream.toArray(), ["fo", "o\n"]);
   146:             resolve();                                         

  Difference:

    [
      'fo',
  +   `o␊
  +   `,
    ]

  › ReadStream.<anonymous> (build/test/unit/streams/data/native-interface.spec.js:145:15)

  ─

  2 tests failed
  5 tests skipped

This may mean there is some issue with how it works internally.

Feature description

Introduce FileStream similar to FSStream proposal from v4.

FileStream items would point to filesystem items like files, directories, etc and should provides methods like stat, contents.

The concept is similar to Vinyl objects so the stream should allow filesystem items manipulations (similar to how it is done in Gulp, Grunt, etc).

Use case

Manipulating files in batches, listing filesystem, files pipelines.

If you are interested in this issue or this stream type fits your use case, please upvote with 👍.

Feature description

In v4 .batch() is a simple transform which groups chunks by a given number:

// input: [1,2,3,4,5,6,7,8,9,10]
// batch(amount)
.batch(2)  -> [1,2], [3,4], [5,6], ...

and in pre-v5 it accepts a callback but we plan to introduce batching by amount the same as v4 too.

Now, it could be extended by adding optional step param which will allow to change how chunks are grouped:

// input: [1,2,3,4,5,6,7,8,9,10]
// batch(amount, step = amount)
.batch(2)    -> [1,2], [3,4], [5,6], ...
.batch(2, 2) -> [1,2], [3,4], [5,6], ...
.batch(2, 1) -> [1,2], [2,3], [3,4], ...
.batch(3, 2) -> [1, 2, 3], [3, 4, 5], [5, 6, 7], ...

Use case

The use case is - creating a WindowStream out of any other stream with ability to decide how each window frame content is grouped. And in the context of the above proposal it could be:

anyStream.batch(3, 2).as(WindowStream)

Feature description

Introduce DataStream.assign() method similar to v4:

Transforms stream objects by assigning the properties from the returned data along with data from original ones.

The original objects are unaltered.

If you are interested in this issue or this type of transforms fits your use case, please upvote with 👍.