Speex encode / decode causing hiss noise (Objective-c)
When I bypass the speex encoding / decoding steps, the original audio output is correct. I want all the buffer captured from my callback to be encoded, decoded, and sent back to the play loop. A few things I'm not sure about are the following:
- What size is allocated for enc_buffer and dec_buffer
- What length to specify in speex_bits_read_from (SpeexBits * bits, char * bytes, int len)
- What is the maximum size specified in int speex_bits_write (SpeexBits * bits, char * bytes, int max_len)
Here is my speex codec initialization:
#define SAMPLE_RATE 8000
#define MAX_FRAMES 100
#define FRAME_SIZE 160
enc_state = speex_encoder_init(&speex_nb_mode);
dec_state = speex_decoder_init(&speex_nb_mode);
spx_int32_t tmp;
tmp=5;
speex_encoder_ctl(enc_state, SPEEX_SET_QUALITY, &tmp);
tmp=1;
speex_encoder_ctl(enc_state, SPEEX_SET_COMPLEXITY, &tmp);
speex_encoder_ctl(enc_state, SPEEX_GET_FRAME_SIZE, &enc_frame_size );
speex_decoder_ctl(dec_state, SPEEX_GET_FRAME_SIZE, &dec_frame_size );
tmp = SAMPLE_RATE;
speex_encoder_ctl(enc_state, SPEEX_SET_SAMPLING_RATE, &tmp);
speex_decoder_ctl(dec_state, SPEEX_SET_SAMPLING_RATE, &tmp);
speex_bits_init(&enc_bits);
speex_bits_init(&dec_bits);
//Unsure of this allocation size
enc_buffer = (char*)malloc(sizeof(char)*enc_frame_size*MAX_FRAMES);
dec_buffer = (spx_int16_t*)malloc(sizeof(spx_int16_t)*dec_frame_size*MAX_FRAMES);
My encoding / decoding methods:
-(char*)encodeAudioBuffer:(spx_int16_t*)audioBuffer withByteSize:(int)numberOfFrames andWriteSizeTo:(int*)inSize{
speex_bits_reset(&enc_bits);
speex_encode_int(enc_state, audioBuffer, &enc_bits);
//Unsure of this third argument. 'numberOfFrames' is the stored number of input frames from my recording callback.
*inSize = speex_bits_write(&enc_bits, enc_buffer, numberOfFrames*enc_frame_size);
return enc_buffer;
}
-(spx_int16_t*)decodeSpeexBits:(char*)encodedAudio withEncodedSize:(int)encodedSize andDecodedSize:(int)decodedSize{
//Unsure of this third argument. 'encodedSize' is the number written to *inSize in the encode method
speex_bits_read_from(&dec_bits, encodedAudio, encodedSize*dec_frame_size);
speex_decode_int(dec_state, &dec_bits, dec_buffer);
return dec_buffer;
}
And they are called like this:
- (void)encodeBufferList:(AudioBufferList*)bufferList withNumberOfFrames:(int)numberOfFrames{
AudioBuffer sourceBuffer = bufferList->mBuffers[0];
int speexSize = 0;
char* encodedAudio = [speexCodec encodeAudioBuffer:(spx_int16_t*)sourceBuffer.mData withByteSize:numberOfFrames andWriteSizeTo:&speexSize];
spx_int16_t* decodedAudio = [speexCodec decodeSpeexBits:encodedAudio withEncodedSize:speexSize andDecodedSize:sourceBuffer.mDataByteSize];
memcpy(audioBuffer.mData, sourceBuffer.mData, numberOfFrames * sizeof(SInt32));
}
where "bufferList" is the one returned from my record / play callbacks. Can anyone verify that I am filling my buffer correctly? I saw a similar problem reported here , but couldn't see where in my code I could get it wrong:
static OSStatus recordingCallback(void *inRefCon, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inBusNumber, UInt32 inNumberFrames, AudioBufferList *ioData)
{
AudioBuffer buffer;
OSStatus status;
AudioStreamer *input = (__bridge AudioStreamer*) inRefCon;
buffer.mDataByteSize = inNumberFrames * sizeof(SInt16);
buffer.mNumberChannels = 1;
buffer.mData = malloc( inNumberFrames * sizeof(SInt16));
AudioBufferList bufferList;
bufferList.mNumberBuffers = 1;
bufferList.mBuffers[0] = buffer;
status = AudioUnitRender([input rioAUInstance], ioActionFlags, inTimeStamp, inBusNumber, inNumberFrames, &bufferList);
[input encodeBufferList:&bufferList withNumberOfFrames:inNumberFrames];
return noErr;
}
static OSStatus playbackCallback(void *inRefCon, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inBusNumber, UInt32 inNumberFrames, AudioBufferList *ioData)
{
AudioStreamer* input = (__bridge AudioStreamer*)inRefCon;
UInt32 size = MIN(ioData->mBuffers[0].mDataByteSize, [input audioBuffer].mDataByteSize);
memcpy(ioData->mBuffers[0].mData, input.audioBuffer.mData, size);
return noErr;
}
The noise generated by encoding / decoding is a grainy static hiss, but this is not entirely random information - when I blow into the microphone, I hear it behind the noise.
Any help putting this issue to bed would be greatly appreciated. I will probably end up blogging about this eventually, once I figure it out, it looks like a lot of people are running into various trivial problems asking this codec.
source to share
So it was a problem in the encoding / decoding functions, I needed to call speex_encode_int on multiple frames as it only processes one frame at a time and then writes them to the encoding buffer like this:
-(char*)encodeAudioBuffer:(spx_int16_t*)audioBuffer withNumberOfFrames:(int)numberOfFrames andWriteSizeTo:(int*)inSize{
speex_bits_reset(&enc_bits);
for(int i = 0; i < numberOfFrames; ++i){
speex_encode_int(enc_state, audioBuffer+i, &enc_bits);
}
*inSize = speex_bits_write(&enc_bits, enc_buffer, numberOfFrames);
return enc_buffer;
}
And similarly to decode speex_bits_read_from of the encoded buffer and then iterate through the dec_bits for each frame, write to the decoded buffer
-(spx_int16_t*)decodeSpeexBits:(char*)encodedAudio withEncodedSize:(int)encodedSize andNumberOfFrames:(int)numberOfFrames{
speex_bits_read_from(&dec_bits, encodedAudio, encodedSize);
for(int i = 0; i < numberOfFrames; ++i){
speex_decode_int(dec_state, &dec_bits, dec_buffer+i);
}
return dec_buffer;
}
This is still pretty slow for me . Even after setting up the speex library to use fixed point computation instead of floating point computation, it still runs slower than my audio loop (causing a new kind of discontinuity). Any hints on how to get this to work faster?
source to share
on both of your loops, you are passing in a sound buffer, but not considering the frame size:
for(int i = 0; i < numberOfFrames; ++i){
speex_encode_int(enc_state, audioBuffer+i, &enc_bits);
}
and it should be:
for(int i = 0; i < numberOfFrames; ++i){
speex_encode_int(enc_state, audioBuffer + (i * enc_frame_size), &enc_bits);
}
hope this helps.
source to share