Comments (11)
What do you mean by this? You want to push audio data directly to the hardware? Aurora is about taking encoded data, decoding it, and then playing it, not about generating audio. For that, check out sink.js. Let me know if that does what you're looking for.
from aurora.js.
I'm not sure I explained myself enough. I have chunks of a media file (could be mp3, flac, etc.) and want to be able to decode that media stream and play with aurora. The data stream is not decoded, it is just pieces of a file. Is there any possibility that I could push the binary stream to aurora to decode and play? (by binary stream I mean the mp3, etc. binary data)
from aurora.js.
In the current form, the data is type String (but can be converted to any type defined in js). Edit: Can also easily be base64
from aurora.js.
Oh I see. Yes, definitely possible, albeit a bit less elegant than you were probably looking for. I assume this will be raw audio data and you know what codec it is, rather than being embedded in a container format. Is this correct, or is it a buffer of a whole audio file including the container? Also, why is it a string? That's probably not a good way to represent binary data... :)
from aurora.js.
Here's an example of doing this manually with the existing Aurora API. The problem is that you need to know a lot about the raw audio data that is usually given by the container format. You could use the decoders and AudioDevice
directly, something like this maybe (example uses a FLAC decoder). YOUR_BUFFER
must be a Uint8Array
.
// create a buffer
var buf = new Buffer(YOUR_BUFFER);
// the known format (usually embedded in the container format)
var format = {
formatID: 'flac',
sampleRate: 44100,
channelsPerFrame: 2,
bitsPerChannel: 16
};
// the cookie used by the decoder (usually embedded in the container)
var cookie = {
maxBlockSize: 4096,
maxFrameSize: 14323,
minBlockSize: 4096,
minFrameSize: 1751
};
// create a fake source event emitter
var source = new EventEmitter;
// create the decoder
var decoder = new FLACDecoder(source, format);
// emit the cookie
source.emit('cookie', cookie);
// emit the buffer of audio data to the decoder
source.emit('data', buf);
// create a queue to hold decoded chunks
var queue = new Queue(decoder);
// create an AudioDevice for playback
var device = AudioDevice.create(format.sampleRate, format.channelsPerFrame);
// read the first frame
var frame = queue.read();
var frameOffset = 0;
var div = decoder.floatingPoint ? 1 : Math.pow(2, format.bitsPerChannel - 1);
// start listening for the device's refill event
device.on('refill', function(buffer) {
var bufferOffset = 0;
// write the decoded frames into the buffer
while (frame && bufferOffset < buffer.length) {
var max = Math.min(frame.length - frameOffset, buffer.length - bufferOffset);
for (var i = 0; i < max; i++) {
buffer[bufferOffset++] = frame[frameOffset++] / div;
}
if (frameOffset === frame.length) {
frame = queue.read();
frameOffset = 0;
}
}
// if we ran out of data, stop the device
if (!frame) {
device.off('refill', arguments.callee);
}
});
If I was wrong and you actually have your data in a container format, this will be a bit easier. Let me know.
from aurora.js.
Ok I understand much better now how this works, but I have one more question. In the example above, you create the device and add to the buffer, but how do you keep adding to the buffer while the device continues to read/decode from the buffer and play/pause depending on if there is still data left in the buffer?
Also, is this api: https://github.com/ofmlabs/aurora.js/wiki/AudioDevice-Class-Reference still up to date on how to interact with the AudioDevice?
from aurora.js.
Just keep emitting data
events from the source
object :) The code I gave will automatically stop the device when there is no more data. Yes, the API reference is correct, although I was using the Device
interface directly.
Here's another example if you don't just have raw audio data but actually have it as part of a container as I suspect you meant after a rereading of your original question (you said "chunks of a file"). As long as you send it the chunks in the order of the original file, you can do this with a normal Player
object and a custom Source. You have to implement a few methods and then emit data
events and an end
event when you're done. Then just create the Asset
and Player
instances manually. See the example below:
// Implement a source (see https://github.com/ofmlabs/aurora.js/wiki/Source-Interface-Reference)
var MySource = EventEmitter.extend({
start: function() {
// emit some data
this.emit('data', YOUR_BUFFER);
this.emit('end');
},
pause: function() {
// stop
},
reset: function() {}
});
// create a source, asset and player
var source = new MySource();
var asset = new Asset(source);
var player = new Player(asset);
player.play();
I hope that works better!
from aurora.js.
Sorry, I seem to be having a bit of trouble with this. So I am trying to get this to work with socket.io and I am sending base64 data from the server. This is a snippet of the code on the client:
var BASE64_MARKER = ';base64,';
function convertDataURIToBinary(dataURI) {
var base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length;
var base64 = dataURI.substring(base64Index);
var raw = window.atob(base64);
var rawLength = raw.length;
var array = new Uint8Array(new ArrayBuffer(rawLength));
for(i = 0; i < rawLength; i++) {
array[i] = raw.charCodeAt(i);
}
return array;
}
var socket = io.connect('/');
var num = 0;
var source;
var asset;
var player;
socket.on('message', function(data) {
var abData = new Buffer(convertDataURIToBinary(data));
if (num == 0) {
// Implement a source (see https://github.com/ofmlabs/aurora.js/wiki/Source-Interface-Reference)
var MySource = EventEmitter.extend({
start: function() {
// emit some data
this.emit('data', abData);
this.emit('end');
},
pause: function() {
// stop
},
reset: function() {}
});
// create a source, asset and player
source = new MySource();
asset = new Asset(source);
player = new Player(asset);
console.log(source);
player.play();
console.log(player);
} else {
source.emit('data', abData);
this.emit('end');
}
num++;
});
I am not quite sure where I went wrong but it doesn't seem to error, but never plays anything. The player object seems to be to be getting data correctly, so if you have the time to help I would greatly appreciate it. Thanks!
from aurora.js.
The data conversion from string to Uint8Array
is a bit off, I think. Try this:
var array = new Uint8Array(rawLength);
for(var i = 0; i < rawLength; i++) {
array[i] = raw.charCodeAt(i) & 0xff;
}
Also, you're emitting an end
event after every message. There should be only one end
event sent after all of the chunks have been received.
Also, I'd flip this code upside down and put the socket.io logic inside the source. Put the socket.io message
event handler inside the source's start
method. What you have here should work (with the changes above), but that might be a little cleaner. Here's a possibility:
var MySource = EventEmitter.extend({
start: function() {
var source = this
source.socket = io.connect('/');
source.socket.on('message', function(data) {
var abData = new Buffer(convertDataURIToBinary(data));
source.emit('data', abData);
if (LAST_CHUNK) {
source.emit('end');
}
});
},
pause: function() {
this.socket.disconnect();
delete this.socket;
},
reset: function() {}
});
// create a source, asset and player
var source = new MySource();
var asset = new Asset(source);
var player = new Player(asset);
player.play();
from aurora.js.
Thank you so much for all your help! I have everything working perfectly now and just need to optimize performance so that the decoding doesn't block the music playback. Thanks again and I love using aurora. Hopefully it keeps on getting better.
from aurora.js.
Well, I tried to copy the above - but with no success. Here's what I have so far:
<script type="text/javascript" src="exjs/aurora/aurora.js"></script>
<script type="text/javascript" src="exjs/aurora/mp3.js"></script>
var aurora_source=null;
var player=null;
.. get a chunk of data ... than call makeNoise()
makeNoise(chunk)
{
var uarr = makeUint8Array(chunk);
var abData = new AV.Buffer(uarr);
if (!aurora_source) {
var MySource = AV.EventEmitter.extend ({
start: function() {
console.log("start data");
this.emit('data',abData);
},
pause: function() {
// nope
},
reset: function() {
// nope
}
});
aurora_source = new MySource();
asset = new AV.Asset(aurora_source);
player = new AV.Player(asset);
player.play();
} else {
console.log("emit data");
aurora_source.emit('data',abData);
}
Result: No noise and an error on aurora_source.emit in "else" tree: Failed to load script: blob:09782643-635a-489c-a958-3a2c0a917d02
from aurora.js.
Related Issues (20)
- How do I decode to raw, resample to 44,1khz and split stereo into dual mono buffers?
- "Something is wrong in HTTPSource.loop" while using AV.Asset.fromURL(remoteUrl)
- [Solved]switch backend from Speaker to PortAudio(naudiodon)
- Aurora not playing certain flac files with no error
- Live decoding on Android
- Node example HOT 1
- No default jsDelivr CDN file set
- Problems with 24-bit WAV files
- Questions about memory manage.
- Is decoding a specific part of a gargantuan file supported efficiently? HOT 3
- HTTP requests still sending to server even if HTTP error is returned
- Unit tests fail on master branch
- Chrome browser change for same site cookies
- gyp build fail HOT 1
- build for browser
- m4a demuxer fails when processing a file with co64 atom
- Window Not Defined
- decode fail
- How to install additional codecs? HOT 1
- Uncaught SyntaxError: Unexpected identifier 'key'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aurora.js.