NLayer MpegFile to SFML.Net SoundStream

[s.baus]

Hello everybody,

I'm currently working on SFML.Net to expand with mp3 support. Therefore I wrote a Stream class which uses NLayer MpegFile to decode the mp3.

   

public class Mp3StreamSFML : SoundStream
    {
        private MpegFile mp3file;
        private int currentBufferSize;
        private short[] currentBuffer;

        public Mp3StreamSFML(String _filename)
        {
            mp3file = new MpegFile(_filename);
            Initialize((uint)mp3file.Channels, (uint)mp3file.SampleRate);
            currentBufferSize = 0;
            currentBuffer = new short[currentBufferSize];
        }

        #region implemented abstract members of SoundStream

        protected override bool OnGetData(out short[] samples)
        {
            if (currentBufferSize <= mp3file.Position)
            {
                byte[] buffer = new byte[512];
                if (mp3file.ReadSamples(buffer, 0, buffer.Length) > 0)
                {
                    Array.Resize(ref currentBuffer, currentBuffer.Length + (buffer.Length / 2));
                    Buffer.BlockCopy(buffer, 0, currentBuffer, currentBufferSize, buffer.Length);
                    currentBufferSize = currentBuffer.Length;
                }
                samples = currentBuffer;
                return true;
            }
            else
            {
                samples = currentBuffer;
                return false;
            }
        }

        protected override void OnSeek(TimeSpan timeOffset)
        {
            mp3file.Position = (long)timeOffset.TotalSeconds;
        }

        #endregion
    }

I use it this way:

                   

try
                    {
                        stream = new Mp3StreamSFML(this.objProgram.getObjCuesheet().getAudiofilePath(true));
                        stream.Play();
                        log.debug("samplerate = " + stream.SampleRate);
                    }
                    catch(Exception ex)
                    {
                        log.fatal(ex.ToString());
                    }

Unfortunately, there is not the correct sound played, its just "juttering" and sound really weird. What I'm doing wrong? Seems like a problem between the 2 Frameworks.

NLayer:https://nlayer.codeplex.com/

Thanks for your help .
Sven

[s.baus]

No one any ideas?

[zsbzsb]

Without ever using the library in question... You appear to be using it improperly. You assume as long as the MpegFile.ReadSamples returns > 0 that your entire buffer (512 bytes) will be filled. However from taking a glance at the source code the function actually returns the length of what was filled in your buffer.

Also, I do not believe simply converting the array as you are doing is the proper way to convert audio data from a byte array to a short array (PCM) that SFML requires.

[s.baus]

Thanks for your help, I changed the Method this way:

protected override bool OnGetData(out short[] samples)
        {
            if (currentBufferSize <= mp3file.Position)
            {
                byte[] buffer = new byte[512];
                int readSamples = mp3file.ReadSamples(buffer, 0, buffer.Length);
                if (readSamples > 0)
                {
                    Array.Resize(ref currentBuffer, currentBuffer.Length + (readSamples / 2));
                    Buffer.BlockCopy(buffer, 0, currentBuffer, currentBufferSize, readSamples);
                    currentBufferSize = currentBuffer.Length;
                }
                samples = currentBuffer;
                return true;
            }
            else
            {
                samples = currentBuffer;
                return false;
            }
        }

Also, I do not believe simply converting the array as you are doing is the proper way to convert audio data from a byte array to a short array (PCM) that SFML requires.

I have read in the internet, that this would be the safest way of converting byte[] to short[]. How should it be done?

[Laurent]

This is not just a type conversion. You first have to find out what sample format NLayer is giving you. It's a byte array, but what does it really contain? Apparently the returned array contains 32-bits float samples. So the conversion would be a little more complicated.

[s.baus]

Ok, thanks again for your information. I'm not familiar with all this audio stuff, so I'm asking a bit dumb, you might think.
You said, this is a 32-bits-float sample, so I would say 4 elements of the read byte array represent 1 sample, correct? This 1 sample would take 2 elements in the short array passed to sfml,correct?

[s.baus]

I tried now this (from http://www.codeproject.com/Articles/501521/How-to-convert-between-most-audio-formats-in-NET), but this also doesn't work:

        protected override bool OnGetData(out short[] samples)
        {
            if (currentBufferSize <= mp3file.Position)
            {
                byte[] buffer = new byte[512];
                int readSamples = mp3file.ReadSamples(buffer, 0, buffer.Length);
                if (readSamples > 0)
                {
                    Array.Resize(ref currentBuffer, currentBuffer.Length + buffer.Length);
                    //Buffer.BlockCopy(buffer, 0, currentBuffer, currentBufferSize, readSamples);
                    int destOffset = currentBufferSize;
                    for (int sample = 0; sample < buffer.Length; sample++)
                    {
                        // adjust volume
                        float sample32 = buffer[sample];
                        // clip
                        if (sample32 > 1.0f)
                            sample32 = 1.0f;
                        if (sample32 < -1.0f)
                            sample32 = -1.0f;
                        currentBuffer[destOffset++] = (short)(sample32 * 32767);
                    }
                    currentBufferSize = currentBuffer.Length;
                }
                samples = currentBuffer;
                return true;
            }
            else
            {
                samples = currentBuffer;
                return false;
            }
        }

[zsbzsb]

If the returned byte array really is floats in the form of bytes you can use BitConverter.ToSingle(...) to convert 4 bytes to a float.

[s.baus]

Working with this code, I get only some "clicking" sound:

protected override bool OnGetData(out short[] samples)
        {
            if (currentBufferSize <= mp3file.Position)
            {
                byte[] buffer = new byte[512];
                int readSamples = mp3file.ReadSamples(buffer, 0, buffer.Length);
                if (readSamples > 0)
                {
                    Array.Resize(ref currentBuffer, currentBuffer.Length + buffer.Length);
                    //Buffer.BlockCopy(buffer, 0, currentBuffer, currentBufferSize, readSamples);
                    for (int i = 0; i < (buffer.Length / 4); i++)
                    {
                        currentBuffer[currentBufferSize + (i/4)] = (short)(BitConverter.ToSingle(buffer, i * 4) * 32767);
                    }
                }
                samples = currentBuffer;
                return true;
            }
            else
            {
                samples = currentBuffer;
                return false;
            }
        }

How can I check, what soundformat is read by nlayer?

[Laurent]

How can I check, what soundformat is read by nlayer?

Yeah, before trying to integrate to SFML, make sure you read the samples correctly with NLayer. Which means studying its documentation, and asking on their forum or whatever they have, if the documentation is not enough (I couldn't find one on their website).

[s.baus]

Sorry for answering so late, but I had to get the information from NLayer developer. Here is the answer:

It returns 32-bit little-endian floats. You can copy it to a float[] buffer (use Buffer.BlockCopy), then do your processing from there.

The samples are started in channel-interleaved format (just like WAV files).

Do you have a hint for me, how to do the work from float buffer to short buffer? Thanks in advance .

[zsbzsb]

Ok, so it returns a float array in the form of a byte array (why it does I have no clue). Now you need to know what the range of the floats are. Is it the normal range of wav files [-32768 ... +32768] or normalized [-1.0 ... +1.0] or maybe something else?

[s.baus]

Thank you again, here the answer :

It's nominally [-1.0 ... +1.0], but you should expect some out of range values with "loud" sounds. Deal with them like you would any other out-of-range samples (usually clipping, but an audio compressor will also do the trick if set up correctly).

Do we need more information?

[zsbzsb]

Something like this should work.

var normalizedaudiodata = new float[] { 1.0f, -1.0f, 0.5f, -0.5f, 2.0f, -2.0f }; // normalized data from decoder
var pcmaudiodata = new short[normalizedaudiodata.Length]; // converted data
for (int i = 0; i < normalizedaudiodata.Length; i++)
{
    // clip the data
    if (normalizedaudiodata[i] > 1.0f) normalizedaudiodata[i] = 1.0f;
    else if (normalizedaudiodata[i] < -1.0f) normalizedaudiodata[i] = -1.0f;

    // convert to pcm data
    pcmaudiodata[i] = (short)(normalizedaudiodata[i] * short.MaxValue);
}
// do whatever you want with the converted data

[s.baus]

Thank you for your great help. I use this code, but now I hear nothing (turned the volume up to 100 ).:

protected override bool OnGetData(out short[] samples)
        {
            if (currentBufferSize <= mp3file.Position)
            {
                byte[] buffer = new byte[512];
                int readSamples = mp3file.ReadSamples(buffer, 0, buffer.Length);
                if (readSamples > 0)
                {
                    float[] normalizedaudiodata = new float[readSamples / 4];// { 1.0f, -1.0f, 0.5f, -0.5f, 2.0f, -2.0f }; // normalized data from decoder
                    Buffer.BlockCopy(buffer, 0, normalizedaudiodata, 0, buffer.Length);
                    short[] pcmaudiodata = new short[normalizedaudiodata.Length]; // converted data
                    for (int i = 0; i < normalizedaudiodata.Length; i++)
                    {
                        // clip the data
                        if (normalizedaudiodata [i] > 1.0f)
                        {
                            normalizedaudiodata [i] = 1.0f;
                        }
                        else
                        {
                            if (normalizedaudiodata [i] < -1.0f)
                            {
                                normalizedaudiodata [i] = -1.0f;
                            }
                        }
                        // convert to pcm data
                        pcmaudiodata[i] = (short)(normalizedaudiodata[i] * short.MaxValue);
                    }
                    // do whatever you want with the converted data
                    int startIndex = currentBuffer.Length;
                    Array.Resize(ref currentBuffer, currentBuffer.Length + pcmaudiodata.Length);
                    for (int i = startIndex; i < currentBuffer.Length; i++)
                    {
                        currentBuffer [i] = pcmaudiodata [i - startIndex];
                    }

                    //Array.Resize(ref currentBuffer, currentBuffer.Length + buffer.Length);
                    //Buffer.BlockCopy(buffer, 0, currentBuffer, currentBufferSize, readSamples);
//                    for (int i = 0; i < (buffer.Length / 4); i++)
//                    {
//                        currentBuffer[currentBufferSize + (i/4)] = (short)(BitConverter.ToSingle(buffer, i * 4) * 32767);
//                    }
                }
                samples = currentBuffer;
                return true;
            }
            else
            {
                samples = currentBuffer;
                return false;
            }
        }

What I am doing wrong? How can I find an error?