Combining Soundbuffers

[Foaly]

Hello everybody!

I have a little problem and I am not quiet sure how to solve it. Hopefully someone can point me the right direction.
So i would like to "combine" two soundbuffers. Meaning i have two different soundbuffers and i want to combine them into one. My idea was that I use GetSamples() to get the samples of each buffer and then add them together into one array and then call the LoadFromMemory() member of a third buffer to finish up. So the real question is how do I add the two array together? Or is there a better method to solve the whole problem?
I hope someone can help me. Thanks in advance.
Foaly

[Laurent]

The algorithm to add two sounds is very simple: basically it consists of adding audio samples of both sounds two by two. The most difficult part is to normalize the result (to avoid overflows) without messing it up.

Found this on Google:
http://stackoverflow.com/questions/376036/algorithm-to-mix-sound

... which links to this:
http://www.vttoth.com/CMS/index.php/technical-notes/68

Don't forget that your range is [-32767, 32767] since audio samples are signed 16-bit integers in SFML.

[Foaly]

Thank you very much for the quick reply! The article is very interesting and helpful. So I assume that I need to use a for-loop to iterate through the two buffers to process them and "add" them together, right?
I am rather inexperienced with processing sounds, so how exactly do I access the sounddata-array ? I assume it is something like this:

Code: [Select]

const sf::Int16* Samples = Buffer.GetSamples();
n = Buffer.GetSampleRate() * Buffer.GetSamplesCount() * Buffer.GetChannelsCount();

for(int i = 0; i < n; i++)
{
   Sample[i]; //process data
}

Is that how I access every integer in the Samplearray? Could someone please correct me if I'm wrong?

[Laurent]

You're right, except that the total number of samples is just Buffer.GetSamplesCount().

[Foaly]

So n would be simply calculated like this:

Code: [Select]

n = Buffer.GetSamplesCount();

[Laurent]

Yes.

[Foaly]

Ok I started writing some code, but i ran into another problem. I read and understood the text, but I am not quiet sure, which formulas to translate to code. Obviously I have to take the ones on the bottom of the page, because i have signed values. But which one? Would it be like this:

Code: [Select]

   const sf::Int16* SamplesOne = BufferOne.GetSamples();
    int n = BufferOne.GetSamplesCount();
   
    const sf::Int16* SamplesTwo = BufferTwo.GetSamples();
   
    if(n < BufferTwo.GetSamplesCount())
    {
        n = BufferTwo.GetSamplesCount();
    }
   
    sf::Int16* FinalSamples;

    for(int i = 0; i < n; i++)
    {
        if(SamplesOne[i] < 0 && SamplesTwo[i] < 0)
        {
            FinalSamples[i] = 2 * SamplesOne[i] * SamplesTwo[i];
        }
        else
        {
            FinalSamples[i] = 2 * (SamplesOne[i] + SamplesTwo[i]) - 2 * SamplesOne[i] * SamplesTwo[i] - 1;
        }
    }

Or like this:

Code: [Select]

   for(int i = 0; i < n; i++)
    {
        if(SamplesOne[i] < 0 && SamplesTwo[i] < 0)
        {
            FinalSamples[i] = (SamplesOne[i] * SamplesTwo[i]) / 128;
        }
        else
        {
            FinalSamples[i] = 2 * (SamplesOne[i] + SamplesTwo[i]) - (SamplesOne[i] * SamplesTwo[i]) / 128 - 256;
        }
    }

Or is it something completely different, because it says that the range is between 0 and 1 for the first and 0 and 255 for the second, but my range is actually between -32767 and 32767. But I am not sure about how to do it, because of the negative numbers.
Thanks again.
Foaly

[Laurent]

If you don't know how to adapt the formula to 16-bit signed integers, adapt the 16-bit signed integers to the range used in the formula

Code: [Select]

double SampleOne = (SamplesOne[i] + 32768.) / 65535.;
double SampleTwo = (SamplesTwo[i] + 32768.) / 65535.;

...

FinalSamples[i] = static_cast<sf::Int16>(result * 65535. - 32768.);
... and then apply the formula for numbers between 0 and 1.

This is not the most efficient solution though, so if you really need maximum performances you will have to find how to adapt the formula anyway.

[Foaly]

Thanks a lot! I will give it a try!
But I'd like to have the formula for 16-bit signed integers. Do you know how I could get it? I searched on google, but I couldn't find anything. Or do you know how to convert the existing formula to 16-bit signed integer? I would be really thankful if someone could help me, because i cant figure it out. Thanks in advance

Also I have another quick question (maybe this one is stupid...): How do i allocate the array for the FinalSamples correctly? If i simply put:

Code: [Select]

sf::Int16* FinalSamples;
I get a Segmentation fault. So how do I do this correctly? thanks again.
Foaly

[Laurent]

But I'd like to have the formula for 16-bit signed integers

I don't think that it would be hard to translate, but it requires some thinking.
But is there any good reason you don't want the float version at all? You'll notice the performance difference only if you apply this algorithm in real time. It's just a few CPU cycles more than the optimized version; for offline processing it's more than enough.

Also I have another quick question (maybe this one is stupid...): How do i allocate the array for the FinalSamples correctly?

This is very basic (yet very important) C++ stuff, so maybe you should read a book or a good tutorial about dynamic allocation.
Hint: std::vector.

[Foaly]

Yeah sorry never mind my question... It was too simply late. Of course I have to use a vector...
But that leads me to another problem. How do I load a soundbuffer from a vector?

It's not that I don't want to use the version that I have right now, it's just that I thought that while I'm still at it I might as well figure out the most efficient way to do this, because combining two soundbuffers is not that unusual of a task. I hoped to end up with some reusable code that other people might want to use too. But I couldn't figure it out by myself, because I am a little inexperienced with sounds and their mathematics, so I asked if somebody could help me

[Laurent]

But that leads me to another problem. How do I load a soundbuffer from a vector?

Code: [Select]

buffer.LoadFromSamples(&vector[0], vector.size());

I couldn't figure it out by myself, because I am a little inexperienced with sounds and their mathematics

This task has nothing to do with audio-related maths. It's just moving a formula from one range to another

[Foaly]

Alright here is a working example:

Code: [Select]

#include <SFML/System.hpp>
#include <SFML/Audio.hpp>
#include <iostream>
#include <vector>

int main()
{
    sf::SoundBuffer BufferOne;
    sf::SoundBuffer BufferTwo;

    // load the files
    if (!BufferOne.LoadFromFile("Sounds/Silence.wav"))
    {
        std::cout << "Error Loading First Sound File." << std::endl;
        return 1;
    }

    if (!BufferTwo.LoadFromFile("Sounds/02.wav"))
    {
        std::cout << "Error Loading Second Sound File." << std::endl;
        return 1;
    }

    int iLength;
    int iShortLength;
    const sf::Int16* LongerSamples;
    const sf::Int16* ShorterSamples;

    if(BufferOne.GetSamplesCount() > BufferTwo.GetSamplesCount())
    {
        LongerSamples = BufferOne.GetSamples();
        ShorterSamples = BufferTwo.GetSamples();
        iLength = BufferOne.GetSamplesCount();
        iShortLength = BufferTwo.GetSamplesCount();
    }
    else
    {
        LongerSamples = BufferTwo.GetSamples();
        ShorterSamples = BufferOne.GetSamples();
        iLength = BufferTwo.GetSamplesCount();
        iShortLength = BufferOne.GetSamplesCount();
    }

    std::vector<sf::Int16> FinalSamplesVector;
    FinalSamplesVector.reserve(iLength);

    for(int i = 0; i < iLength; i++)
    {
        if(i < iShortLength)
        {
            double dSampleOne = (LongerSamples[i] + 32768.) / 65535.;
            double dSampleTwo = (ShorterSamples[i] + 32768.) / 65535.;
            double dResult = 0;

            if(dSampleOne < 0.5 && dSampleTwo < 0.5)
            {
                dResult = 2 * dSampleOne * dSampleTwo;
            }
            else
            {
                dResult = 2 * (dSampleOne + dSampleTwo) - 2 * dSampleOne * dSampleTwo - 1;
            }

            FinalSamplesVector.push_back(static_cast<sf::Int16>(dResult * 65535. - 32768.));
        }
        else
        {
            FinalSamplesVector.push_back(LongerSamples[i]);
        }
    }

    sf::SoundBuffer FinalBuffer;
    FinalBuffer.LoadFromSamples(&FinalSamplesVector[0], FinalSamplesVector.size(), 2, 44100);
    FinalBuffer.SaveToFile("output.wav");

    sf::Sound FinalSound;
    FinalSound.SetBuffer(FinalBuffer);
    FinalSound.Play();

    // this is dirty!!!
    while(1)
    {
        if(FinalSound.GetStatus() == sf::Sound::Stopped)
            return 0;
    }
}
Again thank you so much for all the help! I hope somebody else might be able to use the code too

Well to be honest I am not that big of a math expert either... I've already tried to convert the formula, but it didn't work at all. So could someone please transform the formula for me or show me how it's done? I can't find anything that seems to help me. Thanks in advance!

Also another question how exactly are the samples stored in the array? I noticed that a sound with the length of 0.5 seconds and the sample rate of 44100 has exactly 44100 samples. I assume this is because it's stereo. So my question is how are the samples arranged in memory? Left channel first and then the right channel or the other way around? Or are they mixed?

Thanks, Foaly

[Laurent]

Samples are interleaved (mixed). So you have [s1-left, s1-right, s2-left, s2-right, ...].