Live streaming audio with WebRTC and WebAudio (WebSockets)

I am trying to set up a live streaming system where the client will stream audio from their microphone (accessed with getUserMedia) to one or more peers. To do this, chunks of the audio stream are sent over the WebSocket to the server, which then broadcasts this information to all peers connected to the WebSocket.

My main problem has to do with how to reproduce the chunks of data received by peers on the website.

First, how do I post chunks of audio data on my client by streaming a JS script:

var context = new AudioContext();
var audioStream = context.createMediaStreamSource(stream);
// Create a processor node of buffer size, with one input channel, and one output channel
var node = context.createScriptProcessor(2048, 1, 1);
// listen to the audio data, and record into the buffer
node.onaudioprocess = function(e){
        var inputData = e.inputBuffer.getChannelData(0);
        ws.send(JSON.stringify({sound: _arrayBufferToBase64(convertoFloat32ToInt16(inputData))}));
}
audioStream.connect(node);
node.connect(context.destination);

      

arrayBufferToBase64 and convertoFloat32ToInt16 are the methods I use to send a base64 stream respectively, and to convert inputData to Int16 instead of this fancy Float32 representation (I used methods found on SO that should work).

Then, after the data has gone through the WebSocket, I collect the data in another script to be executed on each partner's website:

var audioCtx = new AudioContext();
var arrayBuffer = _base64ToArrayBuffer(mediaJSON.sound);
audioCtx.decodeAudioData(arrayBuffer, function(buffer) {
     playSound(buffer);
});

      

I also need to convert the base64 data received into an ArrayBuffer, which will then be decodedAudioData to create an AudioBuffer audio buffer. The playSound function is simple:

function playSound(arrBuff) {
    var src = audioCtx.createBufferSource();
    src.buffer = arrBuff;
    src.looping = false;
    src.connect(audioCtx.destination);
    src.start();
}

      

But for some reason I cannot hear the sound on this script. I'm sure the broadcast script is correct, but not the script "listener". Can anyone help me with this?

Thank!

+3


source to share





All Articles