Play incoming binary ArrayBuffer data at the same time from binaryjs server

Good day! I am streaming video today and I am facing the problem of an incoming ArrayBuffer that contains binary audio data.

Here is the code I found to play binary audio data (Uint8Array):

function playByteArray(byteArray) {
    var arrayBuffer = new ArrayBuffer(byteArray.length);
    var bufferView = new Uint8Array(arrayBuffer);
    for (i = 0; i < byteArray.length; i++) {
        bufferView[i] = byteArray[i];
    }
    context.decodeAudioData(arrayBuffer, function(buffer) {
        buf = buffer;
        play();
    });
}

// Play the loaded file
function play() {
    // Create a source node from the buffer
    var source = context.createBufferSource();
    source.buffer = buf;
    // Connect to the final output node (the speakers)
    source.connect(context.destination);
    // Play immediately
    source.start(0);
}

      

Now, below, I have used the MediaStreamRecorder from https://github.com/streamproc/MediaStreamRecorder to record the stream from getUserMedia. This code will continuously send the written binary data to the server.

if (navigator.getUserMedia) {
        navigator.getUserMedia({audio: true, video: true}, function(stream) {
            video.src = (window.URL || window.webkitURL).createObjectURL(stream); //get this for video strewam url
          video.muted = true;
            multiStreamRecorder = new MultiStreamRecorder(stream);
            multiStreamRecorder.canvas = {
              width: video.width,
              height: video.height
            };
            multiStreamRecorder.video = video;

          multiStreamRecorder.ondataavailable = function(blobs) {
            var audioReader = new FileReader();
              audioReader.addEventListener("loadend", function() {
                var arrBuf = audioReader.result;
                var binary = new Uint8Array(arrBuf);
                streamToServ.write(binary);
                // streamToServ is the binaryjs client
              });
              audioReader.readAsArrayBuffer(blobs.audio);
            };
          multiStreamRecorder.start(1);
        }, onVideoFail);
    } else {
        alert ('failed');
    }

      

Convert the resulting blobs (audio and video) to binaries and send them to binaryjs to be played on another client with this:

client.on('stream', function (stream, meta) {
    stream.on('data', function(data) {
        playByteArray(new Uint8Array(data));
    });
});

      

I had no problem transferring binary data, but the problem is that there is a hiccup sound in playback in every binary being played. Is there something wrong with the way I am playing with the incoming ArrayBuffers? I am also thinking about asking streamproc about this.

Thanks in advance!

Greetings.

+3


source to share


2 answers


I found a solution to this problem by making a sound buffer queue. Most of the code is from here:

Choppy / unintelligible playback using audio in channel via Web Audio API



Thank.

+1


source


Not sure if this is the problem, but perhaps instead of source.start (0), you should use source.start (time), where the time starts when you want to start the source. source.start (0) will start playing immediately. If your byte array is faster than real-time, the sources may overlap because you are running them ASAP.



0


source







All Articles