Peeking ahead at new values about dissolve HOT 12 CLOSED

Ah, so there's no terminating bytes or anything?

Edit: what I mean to ask is: is there any way to know before hitting the next header that there's going to be another header coming? Or that you're at the end of a track? Maybe you have a count of events in that track?

from dissolve.

@deoxxa - No just the start of the header for the next track :(

from dissolve.

@deoxxa - There is a chunkSize for all of the events in the track. I would be able to do a simple if startOffset+chunkSize = currentOffset end(true) but the offset isn't exposed currently

from dissolve.

How do you tell the difference between a header and a track?

from dissolve.

@deoxxa - A track is the track header and the events for the track. The track header is always "MTrk" followed by the uint32be chunkSize (size of all events in bits). I think if the offset variable was exposed it would be enough to solve the problem.

from dissolve.

Hrmmm... I see what's going on now. That's definitely tricky! There's a problem though, in that Dissolve doesn't keep track of any kind of permanent offset. There is an offset variable, but it only lives for as long as a single _transform call. Let me see if I can dig up an example from some real-world code, because I've done this kind of thing a couple of times now with other formats.

from dissolve.

@deoxxa - Perfect! Thanks for the help. Do you think an offset variable that gets incremented at the end of each _transform would be reasonable?

from dissolve.

#!/usr/bin/env node

var Dissolve = require("dissolve");

var parser = new Dissolve();

parser.uint8("totalChunkSize").tap(function() {
  // use this guy to keep track of how much more we have to parse
  this._remaining = this.vars.totalChunkSize;
}).loop("chunks", function(end) {
  this.uint8("chunkSize").string("chunk", "chunkSize").tap(function() {
    // decrement the remaining bytes needed
    this._remaining -= this.vars.chunkSize;

    // oh! we're done!
    if (this._remaining === 0) {
      return end();
    }
  });
}).tap(function() {
  //     much happy
  // very data
  //              wow
  console.log(this.vars);
});

parser.write(Buffer([
  "0a", // total payload length is 10 bytes
  "05", "68656c6c6f", // "hello"
  "05", "776f726c64", // "world"
].join(""), "hex"));

➜  scratch  ./derp.js
{ totalChunkSize: 10,
  chunks:
   [ { chunkSize: 5, chunk: 'hello' },
     { chunkSize: 5, chunk: 'world' } ] }

from dissolve.

The problem with a per-_transform variable would be that it wouldn't play nice with jobs that span multiple calls... Streaming parsers are actually pretty hard to get right :<

from dissolve.

@deoxxa Reporting back to the loop from the child parser how many bits it read (like your sample) solved it for me. It was a bit tricky since sometimes I don't necessarily know how much I read (readVarInt ugh) but after hooking some lower level functions I think I have it now. The cascading variable scope made it pretty easy so good job on that!

Hypothetically if I didn't know the chunkSize for all of the events what would you do?

from dissolve.

In that case, I would do (and have done something like) this:

#!/usr/bin/env node

var Dissolve = require("dissolve");

var parser = new Dissolve();

parser.loop(function(end) {
  this.buffer("firstByte", 1).tap(function() {
    var buffer = this.vars.firstByte;

    if (this.vars.firstByte[0] === 0x00) {
      console.log("keep-alive");
    } else {
      this.loop(function(end) {
        this.buffer("anotherByte", 1).tap(function() {
          if (this.vars.anotherByte[0] === 0x00) {
            console.log("data", buffer);

            return end();
          }

          buffer = Buffer.concat([buffer, this.vars.anotherByte]);
        });
      });
    }
  });
});

parser.write(Buffer("well hi there"));
parser.write(Buffer([0x00]));
parser.write(Buffer([0x00]));
parser.write(Buffer([0x00]));
parser.write(Buffer([0x00]));
parser.write(Buffer([0x00]));
parser.write(Buffer("hello again"));
parser.write(Buffer([0x00]));
parser.write(Buffer([0x00]));

➜  scratch  ./herp.js
data <Buffer 77 65 6c 6c 20 68 69 20 74 68 65 72 65>
keep-alive
keep-alive
keep-alive
keep-alive
data <Buffer 68 65 6c 6c 6f 20 61 67 61 69 6e>
keep-alive

An annoyingly common case of this is that of the null-terminated string. Bane of my damned existence, those things are.

from dissolve.

I'm gonna go ahead and close this now, but let me know if you have any other problems.

from dissolve.