Comments (12)
Ah, so there's no terminating bytes or anything?
Edit: what I mean to ask is: is there any way to know before hitting the next header that there's going to be another header coming? Or that you're at the end of a track? Maybe you have a count of events in that track?
from dissolve.
@deoxxa - No just the start of the header for the next track :(
from dissolve.
@deoxxa - There is a chunkSize for all of the events in the track. I would be able to do a simple if startOffset+chunkSize = currentOffset end(true)
but the offset isn't exposed currently
from dissolve.
How do you tell the difference between a header and a track?
from dissolve.
@deoxxa - A track is the track header and the events for the track. The track header is always "MTrk" followed by the uint32be chunkSize (size of all events in bits). I think if the offset variable was exposed it would be enough to solve the problem.
from dissolve.
Hrmmm... I see what's going on now. That's definitely tricky! There's a problem though, in that Dissolve doesn't keep track of any kind of permanent offset. There is an offset variable, but it only lives for as long as a single _transform
call. Let me see if I can dig up an example from some real-world code, because I've done this kind of thing a couple of times now with other formats.
from dissolve.
@deoxxa - Perfect! Thanks for the help. Do you think an offset variable that gets incremented at the end of each _transform would be reasonable?
from dissolve.
#!/usr/bin/env node
var Dissolve = require("dissolve");
var parser = new Dissolve();
parser.uint8("totalChunkSize").tap(function() {
// use this guy to keep track of how much more we have to parse
this._remaining = this.vars.totalChunkSize;
}).loop("chunks", function(end) {
this.uint8("chunkSize").string("chunk", "chunkSize").tap(function() {
// decrement the remaining bytes needed
this._remaining -= this.vars.chunkSize;
// oh! we're done!
if (this._remaining === 0) {
return end();
}
});
}).tap(function() {
// much happy
// very data
// wow
console.log(this.vars);
});
parser.write(Buffer([
"0a", // total payload length is 10 bytes
"05", "68656c6c6f", // "hello"
"05", "776f726c64", // "world"
].join(""), "hex"));
➜ scratch ./derp.js
{ totalChunkSize: 10,
chunks:
[ { chunkSize: 5, chunk: 'hello' },
{ chunkSize: 5, chunk: 'world' } ] }
from dissolve.
The problem with a per-_transform
variable would be that it wouldn't play nice with jobs that span multiple calls... Streaming parsers are actually pretty hard to get right :<
from dissolve.
@deoxxa Reporting back to the loop from the child parser how many bits it read (like your sample) solved it for me. It was a bit tricky since sometimes I don't necessarily know how much I read (readVarInt ugh) but after hooking some lower level functions I think I have it now. The cascading variable scope made it pretty easy so good job on that!
Hypothetically if I didn't know the chunkSize for all of the events what would you do?
from dissolve.
In that case, I would do (and have done something like) this:
#!/usr/bin/env node
var Dissolve = require("dissolve");
var parser = new Dissolve();
parser.loop(function(end) {
this.buffer("firstByte", 1).tap(function() {
var buffer = this.vars.firstByte;
if (this.vars.firstByte[0] === 0x00) {
console.log("keep-alive");
} else {
this.loop(function(end) {
this.buffer("anotherByte", 1).tap(function() {
if (this.vars.anotherByte[0] === 0x00) {
console.log("data", buffer);
return end();
}
buffer = Buffer.concat([buffer, this.vars.anotherByte]);
});
});
}
});
});
parser.write(Buffer("well hi there"));
parser.write(Buffer([0x00]));
parser.write(Buffer([0x00]));
parser.write(Buffer([0x00]));
parser.write(Buffer([0x00]));
parser.write(Buffer([0x00]));
parser.write(Buffer("hello again"));
parser.write(Buffer([0x00]));
parser.write(Buffer([0x00]));
➜ scratch ./herp.js
data <Buffer 77 65 6c 6c 20 68 69 20 74 68 65 72 65>
keep-alive
keep-alive
keep-alive
keep-alive
data <Buffer 68 65 6c 6c 6f 20 61 67 61 69 6e>
keep-alive
An annoyingly common case of this is that of the null-terminated string. Bane of my damned existence, those things are.
from dissolve.
I'm gonna go ahead and close this now, but let me know if you have any other problems.
from dissolve.
Related Issues (13)
- Bump bl to 1.0.0
- Update npm package HOT 3
- example-complex.js Question with mcstring16 HOT 1
- .scan to search for a buffer HOT 1
- How does 'readable' event really work?
- Why does the _job_up function set __proto__ = null.
- Install with NPM does not include skip method.
- Nested loops HOT 5
- Asynchronous .tap()
- The _transform function needs to be broken up a bit HOT 3
- Loop not terminated by EOF HOT 2
- skip() ? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dissolve.