jojobond / 3las Goto Github PK
View Code? Open in Web Editor NEWLow Latency Live Audio Streaming
License: GNU General Public License v2.0
Low Latency Live Audio Streaming
License: GNU General Public License v2.0
Hi, thanks again for this great work!
Sometimes a client would be listening to an mp3 stream over WS perfectly fine (nice audio quality) and all of a sudden it starts playing with some background "static" and quality seems to drop. The source is fine, as I am listening to it on another client on a nother machine, where everything is fine. If I simply reload the player on the browser where I hear the bad quality.static, then it all goes back to nice quality.
This happens also when for example in Safari when we call "FormatReader.PurgeData();" after that call, static is heard on the audio playback. I can replicate this by simply adding a click handler to an HTML button on the playback page that calls "FormatReader.PurgeData();" from the click handler..
Hey Jojo or anyone else.
I have been looking for a solution like this for low latency audio streaming locally, and happened to find this project! It sounds like it's exactly what I've been looking for, but I'm not a web developer haha. I'm a fairly low level server admin I'm struggling to figure out how to set this up. Would anyone be willing to help me out and teach me a bit more about this?
Thanks!
Steps:
Run the server
Start Playing,
Refresh the page
The server then prints an error message to the console:
events.js
throw er; Unhandled 'error' event
Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:200:27)....
Essentially it is failing when the websocket disconnects after it is played. So I need to rerun the server after it disconnects. Ideally it would keep running. I would try and fix it myself but unsure how.
From what I've seen from other websocket uses in nodeJS...
const Websocket = require('ws');
const wss = new WebSocket.Server({ port: 9999});
wss.on('connection, function(ws){
ws.on('close', console.log('disconnected');)
});
whereas in 3las.stdinstreamer.js:
this.Server.on('connection', this.OnServerConnection.bind(this));
I think something like this might handle the error when dealing with a specific client? Currently there is no code to deal with any disconnections. I don't know how the "this" would carry into the function.
Sorry if this doesn't make any sense!!
Very interesting project, I wonder if it would be possible to have it use a single port (over one port for each stream) and like a 'mountpoint' where clients can stream from like /mount1, /mount2 etc ...
Maybe with nginx ? Any idea on how to achieve this ?
Thanks !
Hello,
I am fairly new to micro controllers but I have a project where I would like to live stream the audio signal of a microphone using a Raspberry Pico connected to the internet. For each connected microphone I can expect to have many thousand clients that need to hear the audio with latency below 1 second.
Do you think that it is achievable using 3LAS and does it work on Raspberry Pico for example ?
Hello,
It's a pita to get working.
I installed ws and dependency :
3LAS/
node_modules/safe-buffer/
node_modules/ultron/
node_modules/ws/
Node version :
(root|/var/www) node -v
v8.4.0
(root|/var/www) npm -v
5.3.0
I succefuly configured the ffmpeg cmdline to my hardware (you must move the "-ac 1" argument to begin of the ffmpeg cmdline.)
ffmpeg -y -f alsa -ac 1 -i hw:2,0 -rtbufsize 64 -probesize 64 \
-acodec libmp3lame -ab 320k -reservoir 0 -f mp3 \
-fflags +nobuffer - \
| nodejs stdinstreamer.js -port 9601 -type mp3 -burstsize 1
Error log :
(root|/var/www/3LAS/server) ./robot.sh
ffmpeg version N-81539-gd2e7431 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.9.2 (Debian 4.9.2-10)
configuration: --enable-gpl --enable-postproc --enable-swscale --enable-avfilter --enable-libmp3lame --enable-libvorbis --enable-libtheora --enable-libx264 --enable-libspeex --enable-shared --enable-pthreads --enable-libopenjpeg --enable-libfaac --enable-nonfree --extra-libs=-lasound
libavutil 55. 29.100 / 55. 29.100
libavcodec 57. 54.102 / 57. 54.102
libavformat 57. 48.102 / 57. 48.102
libavdevice 57. 0.102 / 57. 0.102
libavfilter 6. 59.100 / 6. 59.100
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 1.100 / 2. 1.100
libpostproc 54. 0.100 / 54. 0.100
Guessed Channel Layout for Input Stream #0.0 : mono
Input #0, alsa, from 'hw:2,0':
Duration: N/A, start: 1503762482.264030, bitrate: 768 kb/s
Stream #0:0: Audio: pcm_s16le, 48000 Hz, 1 channels, s16, 768 kb/s
[mp3 @ 0x2351c00] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, mp3, to 'pipe:':
Metadata:
TSSE : Lavf57.48.102
Stream #0:0: Audio: mp3 (libmp3lame), 48000 Hz, mono, s16p, 320 kb/s
Metadata:
encoder : Lavc57.54.102 libmp3lame
Stream mapping:
Stream #0:0 -> #0:0 (pcm_s16le (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
/var/www/node_modules/ws/index.js:9
const WebSocket = require('./lib/WebSocket');
^^^^^
SyntaxError: Use of const in strict mode.
at Module._compile (module.js:439:25)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object.<anonymous> (/var/www/3LAS/server/stdinstreamer.js:7:6)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
If I remove use strict from all scripts, I get the illegal use of reserved word "class" strange error:(
Hi again,
I am still trying to get this to work for my needs, I have two questions.
Firstly, how do I reduce the bitrate of the stream to accommodate for more users? I have tested it with about 30 clients and my poor little i5 6500 in this machine can't handle it. It chokes and maxes out at 100% usage, will kick clients off the stream when choking.
Next question is, I would like to be able to use a smartphone as the audio input, but so far I haven't found how to do that. I can install audio relay and I can get audio from a smartphone into the server, but it comes in as virtual audio device. I do not see any way to change what audio device 3las uses. I tried this on a virtual machine with no hardware audio devices, the only audio device it had was the virtual input from audio relay, but I could not get 3las to output the audio.
As always, thank you so much for Jojo.
hi. JoJoBond.
I useful 3LAS.
really thank you very much.
I have a few questions.
I know it is an absurd question. But I believe you will help.
I am trying to set this up on a local network and am therefore leaving the RtcConfig setting as null. I get an error when launching test.sh because of this it seems :
node:internal/validators:163
throw new ERR_INVALID_ARG_TYPE(name, 'string', value);
^
TypeError [ERR_INVALID_ARG_TYPE]: The "file" argument must be of type string. Received undefined
at new NodeError (node:internal/errors:399:5)
at validateString (node:internal/validators:163:11)
at normalizeSpawnArguments (node:child_process:545:3)
at spawn (node:child_process:750:13)
at new AFallbackProvider (/Users/reedz/websites/srts_streaming/3LAS/example/server/3las.server.js:316:50)
at new FallbackProviderMp3 (/Users/reedz/websites/srts_streaming/3LAS/example/server/3las.server.js:333:9)
at AFallbackProvider.Create (/Users/reedz/websites/srts_streaming/3LAS/example/server/3las.server.js:324:20)
at new StreamServer (/Users/reedz/websites/srts_streaming/3LAS/example/server/3las.server.js:212:62)
at StreamServer.Create (/Users/reedz/websites/srts_streaming/3LAS/example/server/3las.server.js:310:16)
at Object. (/Users/reedz/websites/srts_streaming/3LAS/example/server/3las.server.js:421:29) {
code: 'ERR_INVALID_ARG_TYPE'
}
Any ideas ? Many thanks
You have done an Awesome job! I was trying to think of how i was going to implement something like this. thanks a million. Now my question is, How would i send the ffmpeg output to 3LAS running on a remote server? you see i want to let the users use their own internet connection and not Wi-Fi to connect to the stream. but the stream should be sent from the recording computer to a remote server running 3LAS.
At the moment i'm using butt (broadcast using this tool) to record and stream the feed to an icecast server. the lag is ok for non-realtime application but for use with translation it is unacceptable. Your tool works great for this. but i am not sure how to pipe the output of ffmpeg to my google cloud vm instance. thanks again and any help on this matter would be greatly appreciated. Cheers.
Hi there!
First of all: Great tool, this is really high value. Many thanks for this!
Seems like I am doing something wrong:
sudo ffmpeg -y -f alsa -i hw:1 -rtbufsize 64 -probesize 64 -acodec pcm_s16le -ar 16000 -ac 1 -f s16le -fflags +nobuffer -packetsize 384 -flush_packets 1 - | node ./3las.stdinstreamer.js -port 9603 -type pcm -chunksize 384
Show/Hide event log We recommend viewing this page with Google Chrome. Download Google Chrome Ignore and continue
No audio playing, no player buttons shown. The console window does not give me any hint except for these messages for all the js files:
The script from “http://[IP]:8888/script/3las.helpers.js” was loaded even though its MIME type (“application/octet-stream”) is not a valid JavaScript MIME type.
Not sure what I could be doing wrong. Is there any more information in some log files?
Many thanks!
Hi there,
1st of all great project! works great! except if there are brief network packet losses for a client. Where would I look to add some sort of small buffer (1 second for example) to reduce choppy audio??
I've followed the direction and I'm getting the following error:
node: symbol lookup error: node_modules/wrtc/build/Release/wrtc.node: undefined symbol: napi_add_env_cleanup_hook
What am I missing?
Hi,
How many latency do you get ? In mp3 encoding, I got something very "real time" with an iOS (iPhone 6).
With an HTC One, Galaxy S7 or cheap Android, I always have, something like 400/500ms latency.
Do you have any idea to decrease this latency ?
Thanks you a lot for your hard and good work,
Marc
Hello,
It's possible to use socket.IO instead of ws ?
It can solve two new issues (I opened here) with the client and the server part of 3LAS...
I want to add your project inside mine, a Low latency web mobile robot, For all I use socket.io but not for audio where I use HTML5 and ffserver to stream but I get 1 to 5 second latency... I need to stream the audio inside websocket tunel and add a WebAudio API player.
I have 3LAS running behind a nginx reverse proxy; most of the time it works superbly well... but at some point i start to get a few "kipped buffer because it was too old;" errors and then it all the sudden fills up with this errror and stops playing.
Reloading the stream manually works; but i was wondering if we could 'somehow' restart the stream automatically after a number of 'too old' buffers; i've tried adding " this.NextScheduleTime = 0.0;" after the Logger.Log but has no effect.
Any proper way of doing it ? Thanks !
I'm trying to find a low latency audio system to send audio from a shortwave radio through raspberry PI (With external sound card) to a webbrowser (Chrome), and came across 3LAS. It looks interesting and Im about to test it. But I see that there has not been done any work on this project for a long time.
Is it a dead end to use this one? Maybe someone else have similar javascript based solution?
Concerning the audio link, only low bitrate is needed in my case. (for SSB on short wave, a max of approximately 3kb audio bandwith should be enough, thus ~8kbit sampling rate should be good.
Br
Hello,
It seems that I have found another issue, this time on Android.
Seems like that after my Android phone's screen goes off, the WebRTC stream seems to stop after a few minutes. Tested on Android 12, and Android 11 (Nothing Phone 1 and OnePlus 6T).
Also, there's no MediaControls anywhere on the lockscreen/notification-bar, like on iOS.
I suspect that a MediaSession is missing or something?
I think this is something that needs to be added to the webpage, in order to keep the audio stream alive?
Been listening to webradio on my OnePlus phone for an hour now, still works.
How could we fix this?
Just to add some more details, seems like that battery plays a factor here, when on the charger, it seems to not suspend.
Client tested on Firefox version 55.0.3 and 53.0.3 (32 bits)
Log :
[17:44:26] Detected: Windows, Firefox
[17:44:26] Using MIME: audio/mpeg on port: 9601
[17:44:26] Init of HTMLPlayerControls succeeded
[17:44:26] Init of PCMAudioPlayer succeeded
[17:44:26] Init of AudioFormatReader succeeded
[17:44:27] Init of WebSocketClient failed: SecurityError: The operation is insecure.
Hey there,
first of thanks for providing such a cool tool. Have been playing around with it a lot and it works amazingly well, except on iOS 11 (i'm using 11.02 on an iPhone 7). I hit the play button on my dev iPhone but i'm not able to hear any audio. The event log seems fine, tested in both Chrome and Safari.
[21:51:27] Detected: iOS, Chrome
[21:51:27] Using MIME: audio/mpeg on port: 9601
[21:51:27] Init of HTMLPlayerControls succeeded
[21:51:27] Init of PCMAudioPlayer succeeded
[21:51:27] Init of AudioFormatReader succeeded
[21:51:29] Init of WebSocketClient succeeded
[21:51:29] Trying to connect to server.
[21:51:29] Established connection with server.
Any clue how to solve this issue? Any help is very much appreciated!
Hi,
is there a chance to avoid standby on iOS devices after a certain period independently from the users iOS settings? The stream always pauses when the screen goes dark.
I was thinking of providing the user with an additional audio stream that he/she can open directly in a native audio app when required such as VLC player. This way, the screen can go dark but audio will continue to play.
Maybe you can do it even with the HTML/JS-player but I could not find a way. DO you have an idea how to do this or recycle the server command for a suitable stream on a separate port?
#MP3
ffmpeg -y -f alsa -i hw:1 -rtbufsize 64 -probesize 64 \
-acodec libmp3lame -b:a 320k -ac 1 -reservoir 0 \
-f mp3 -write_xing 0 -id3v2_version 0 -fflags +nobuffer -flush_packets 1 - \
| node /home/3LAS/example/server/3las.stdinstreamer.js -port 9601 -type mpeg &
Hi,
Somebody uses this library to stream audio for live fm tuner, in this case fallback mp3 is used, but sound in all browsers has regularly sound drops milliseconds interval like missing samples.
Do you have any idea to improve this issue increase buffer sizes or something? Probably it occurs at client side but I'm not sure.
I don't know why but my previous issue seems to have been deleted :(
Anyway posting back. I have noticed that when using mp3, I clearly hear a kind of very light noise (similar to hearing an old vinyl). It is very clear on PC but on Mac I don't hear it.
Can this be something related to the audio decoding on Windows ?
Hi JoJoBond.
I am always grateful.
I don't understand you source file in 3las.formatreader.js
` AudioFormatReader_WAV.prototype.ExtractIntSamples = function () {
// Extract sample data from buffer
var intSampleArray = new Uint8Array(this.DataBuffer.buffer.slice(0, this.TotalBatchByteSize));
// Remove samples from buffer
this.DataBuffer = new Uint8Array(this.DataBuffer.buffer.slice(this.BatchBytes));
return intSampleArray;
};
`
Why are the cut lengths different?
I think... 0, this.TotalBatchByteSize and.. this.BataBuffer.buffer.slice(this.TotalBatchByteSize)
I wait for your answer.
I've set up the solution on a local network that may be disconnected from the internet at times.
When offline, the "iceConnectionState" never becomes "connected" but remains at the "new" stage.
When online, it works fine and the audio stream plays without issues.
I am wondering why WebRTC seems to require an internet connection when both the client and server are running locally.
Any ideas or workaround to avoid this?
Thank you
Hi @JoJoBond ,
It's me again on another issue...
I have tested my implementation of 3LAS with about ten simultaneous clients playing the audio stream and saw a huge surge of my laptop CPU (node using 40% CPU on a recent macbook pro).
The plan is to handle about 300 simultaneous clients in-house (on a local network) for a conference in a few months, but it seems it will be difficult.
Are there ways to mitigate this issue at all ?
Thanks again for your work and dedication
Hello,
Seems like after loading a audio stream (PCM) from my Raspberry Pi,
that the audio plays flawlessly, and with minimal latency, which I like.
The only problem now, is that my iOS device is producing an occasional click, does not manifest on the other WebRTC clients such as on Linux and Android.
I tested iOS's Safari, Chrome and Firefox apps, which it seems to manifest. Sounds something iOS related?
It will click once or twice every now and then.
Do you have an idea on why that could happen?
I tested with a Bluetooth audio device as well, seems to be present there, too.
Besides this, seems like the volume control is not working on my iOS device. It works fine on the other platforms.
Any idea how I can fix this?
hi JoJoBond.
I'm use WAV Stream player and mpeg player.
so....
What mean is TotalBatchSampleSize, TotalBatchByteSize?
Hello,
As I'm trying to solve performance issues on my network, I captured the packets and found that when the audio is playing, there are very frequent STUN binding requests sent back en forth, as you can see on this screenshot.
These occur every second or every two seconds, does this show an issue ?
This was recorded when playing the stream from Chrom v115 / Mac OS X.
Thank you.
Android works without any problems.
The current situation is as follows.
1.You are calling.
2.Start streaming using the button on the screen.
Current time get Web Audio Context
3.call end.
4. 3las play streaming sound.
I expect it to be streaming. But there is no sound.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.