Giter Club home page Giter Club logo

Comments (11)

devongovett avatar devongovett commented on July 21, 2024

What do you mean by this? You want to push audio data directly to the hardware? Aurora is about taking encoded data, decoding it, and then playing it, not about generating audio. For that, check out sink.js. Let me know if that does what you're looking for.

from aurora.js.

fireph avatar fireph commented on July 21, 2024

I'm not sure I explained myself enough. I have chunks of a media file (could be mp3, flac, etc.) and want to be able to decode that media stream and play with aurora. The data stream is not decoded, it is just pieces of a file. Is there any possibility that I could push the binary stream to aurora to decode and play? (by binary stream I mean the mp3, etc. binary data)

from aurora.js.

fireph avatar fireph commented on July 21, 2024

In the current form, the data is type String (but can be converted to any type defined in js). Edit: Can also easily be base64

from aurora.js.

devongovett avatar devongovett commented on July 21, 2024

Oh I see. Yes, definitely possible, albeit a bit less elegant than you were probably looking for. I assume this will be raw audio data and you know what codec it is, rather than being embedded in a container format. Is this correct, or is it a buffer of a whole audio file including the container? Also, why is it a string? That's probably not a good way to represent binary data... :)

from aurora.js.

devongovett avatar devongovett commented on July 21, 2024

Here's an example of doing this manually with the existing Aurora API. The problem is that you need to know a lot about the raw audio data that is usually given by the container format. You could use the decoders and AudioDevice directly, something like this maybe (example uses a FLAC decoder). YOUR_BUFFER must be a Uint8Array.

// create a buffer
var buf = new Buffer(YOUR_BUFFER);

// the known format (usually embedded in the container format)
var format = {
    formatID: 'flac',
    sampleRate: 44100,
    channelsPerFrame: 2,
    bitsPerChannel: 16
};

// the cookie used by the decoder (usually embedded in the container)
var cookie = {
    maxBlockSize: 4096,
    maxFrameSize: 14323,
    minBlockSize: 4096,
    minFrameSize: 1751
};

// create a fake source event emitter
var source = new EventEmitter;

// create the decoder
var decoder = new FLACDecoder(source, format);

// emit the cookie
source.emit('cookie', cookie);

// emit the buffer of audio data to the decoder
source.emit('data', buf);

// create a queue to hold decoded chunks
var queue = new Queue(decoder);

// create an AudioDevice for playback
var device = AudioDevice.create(format.sampleRate, format.channelsPerFrame);

// read the first frame
var frame = queue.read();
var frameOffset = 0;
var div = decoder.floatingPoint ? 1 : Math.pow(2, format.bitsPerChannel - 1);

// start listening for the device's refill event
device.on('refill', function(buffer) {
    var bufferOffset = 0;

    // write the decoded frames into the buffer
    while (frame && bufferOffset < buffer.length) {
        var max = Math.min(frame.length - frameOffset, buffer.length - bufferOffset);
        for (var i = 0; i < max; i++) {
            buffer[bufferOffset++] = frame[frameOffset++] / div;
        }

        if (frameOffset === frame.length) {
            frame = queue.read();
            frameOffset = 0;
        }
    }

    // if we ran out of data, stop the device
    if (!frame) {
        device.off('refill', arguments.callee);
    }
});

If I was wrong and you actually have your data in a container format, this will be a bit easier. Let me know.

from aurora.js.

fireph avatar fireph commented on July 21, 2024

Ok I understand much better now how this works, but I have one more question. In the example above, you create the device and add to the buffer, but how do you keep adding to the buffer while the device continues to read/decode from the buffer and play/pause depending on if there is still data left in the buffer?

Also, is this api: https://github.com/ofmlabs/aurora.js/wiki/AudioDevice-Class-Reference still up to date on how to interact with the AudioDevice?

from aurora.js.

devongovett avatar devongovett commented on July 21, 2024

Just keep emitting data events from the source object :) The code I gave will automatically stop the device when there is no more data. Yes, the API reference is correct, although I was using the Device interface directly.

Here's another example if you don't just have raw audio data but actually have it as part of a container as I suspect you meant after a rereading of your original question (you said "chunks of a file"). As long as you send it the chunks in the order of the original file, you can do this with a normal Player object and a custom Source. You have to implement a few methods and then emit data events and an end event when you're done. Then just create the Asset and Player instances manually. See the example below:

// Implement a source (see https://github.com/ofmlabs/aurora.js/wiki/Source-Interface-Reference)
var MySource = EventEmitter.extend({
    start: function() {
        // emit some data
        this.emit('data', YOUR_BUFFER);
        this.emit('end');
    },

    pause: function() {
        // stop
    },

    reset: function() {}
});

// create a source, asset and player
var source = new MySource();
var asset = new Asset(source);
var player = new Player(asset);

player.play();

I hope that works better!

from aurora.js.

fireph avatar fireph commented on July 21, 2024

Sorry, I seem to be having a bit of trouble with this. So I am trying to get this to work with socket.io and I am sending base64 data from the server. This is a snippet of the code on the client:

var BASE64_MARKER = ';base64,';

function convertDataURIToBinary(dataURI) {
  var base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length;
  var base64 = dataURI.substring(base64Index);
  var raw = window.atob(base64);
  var rawLength = raw.length;
  var array = new Uint8Array(new ArrayBuffer(rawLength));

  for(i = 0; i < rawLength; i++) {
    array[i] = raw.charCodeAt(i);
  }
  return array;
}

var socket = io.connect('/');
var num = 0;
var source;
var asset;
var player;
socket.on('message', function(data) {
    var abData = new Buffer(convertDataURIToBinary(data));
    if (num == 0) {
        // Implement a source (see https://github.com/ofmlabs/aurora.js/wiki/Source-Interface-Reference)
        var MySource = EventEmitter.extend({
            start: function() {
                // emit some data
                this.emit('data', abData);
                this.emit('end');
            },

            pause: function() {
                // stop
            },

            reset: function() {}
        });

        // create a source, asset and player
        source = new MySource();
        asset = new Asset(source);
        player = new Player(asset);
        console.log(source);
        player.play();
        console.log(player);
    } else {
        source.emit('data', abData);
        this.emit('end');
    }
    num++;
});

I am not quite sure where I went wrong but it doesn't seem to error, but never plays anything. The player object seems to be to be getting data correctly, so if you have the time to help I would greatly appreciate it. Thanks!

from aurora.js.

devongovett avatar devongovett commented on July 21, 2024

The data conversion from string to Uint8Array is a bit off, I think. Try this:

var array = new Uint8Array(rawLength);
for(var i = 0; i < rawLength; i++) {
    array[i] = raw.charCodeAt(i) & 0xff;
}

Also, you're emitting an end event after every message. There should be only one end event sent after all of the chunks have been received.

Also, I'd flip this code upside down and put the socket.io logic inside the source. Put the socket.io message event handler inside the source's start method. What you have here should work (with the changes above), but that might be a little cleaner. Here's a possibility:

var MySource = EventEmitter.extend({
    start: function() {
        var source = this
        source.socket = io.connect('/');
        source.socket.on('message', function(data) {
            var abData = new Buffer(convertDataURIToBinary(data));
            source.emit('data', abData);

            if (LAST_CHUNK) {
                source.emit('end');
            }
        });
    },

    pause: function() {
        this.socket.disconnect();
        delete this.socket;
    },

    reset: function() {}
});

// create a source, asset and player
var source = new MySource();
var asset = new Asset(source);
var player = new Player(asset);

player.play();

from aurora.js.

fireph avatar fireph commented on July 21, 2024

Thank you so much for all your help! I have everything working perfectly now and just need to optimize performance so that the decoding doesn't block the music playback. Thanks again and I love using aurora. Hopefully it keeps on getting better.

from aurora.js.

secumundo avatar secumundo commented on July 21, 2024

Well, I tried to copy the above - but with no success. Here's what I have so far:

<script type="text/javascript" src="exjs/aurora/aurora.js"></script>
<script type="text/javascript" src="exjs/aurora/mp3.js"></script>

var aurora_source=null;
var player=null;

.. get a chunk of data ... than call makeNoise()

makeNoise(chunk)
{
       var uarr = makeUint8Array(chunk);
       var abData = new AV.Buffer(uarr);
        if (!aurora_source)     {
                var MySource = AV.EventEmitter.extend ({
                        start: function() {
                                console.log("start data");
                                this.emit('data',abData);
                                },
                        pause: function() {
                                // nope
                                },
                        reset: function() {
                                // nope
                                }
                        });
                aurora_source = new MySource();
                asset = new AV.Asset(aurora_source);
                player = new AV.Player(asset);
                player.play();
} else {
                console.log("emit data");
                aurora_source.emit('data',abData);
                }

Result: No noise and an error on aurora_source.emit in "else" tree: Failed to load script: blob:09782643-635a-489c-a958-3a2c0a917d02

from aurora.js.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.