Errors in Audio Spatialization code/documentation.

[nitram_cero]

1. The "Listener::SetTarget()" position (Listener's orietation) is RELATIVE.

This is not well documented, and in the tutorial is wrong:

http://www.sfml-dev.org/tutorials/1.4/audio-spatialization.php

sf::Listener::SetPosition(10.f, 0.f, 5.f);
sf::Listener::SetTarget(15.f, 0.f, 5.f);
Here, our listener is looking along the +X axis. This means that, for example, a sound coming from (15, 0, 3) will be heard from the right speaker.
By default, the listener is looking along the -Z axis (vector (0, 0, -1)).

The listener is actually looking to a right bottom (more right).

Quoting an example from the spec:

Code: [Select]


// place listener at camera
alListener3f(AL_POSITION, listenerPos[0], listenerPos[1], listenerPos[2]);

float directionvect[6];
directionvect[0] = (float) sin(listenerAngle);
directionvect[1] = 0;
directionvect[2] = (float) cos(listenerAngle);
directionvect[3] = 0;
directionvect[4] = 1;
directionvect[5] = 0;
alListenerfv(AL_ORIENTATION, directionvect);
}

(It's not necesary to pass it normalized, thou)


2. The coordinate system in the tutorial is misleading.

 First, you can set the listener's position in the scene :

sf::Listener::SetPosition(10.f, 0.f, 5.f);

If you have a 2D world you can just use the same Y value everywhere, usually 0.

It's insinuated to be X - horizontal, Y - depth, Z - vertical, while in the spec it states:

The OpenAL listener object and source objects have attributes to describe their position,
velocity and orientation in three dimensional space. OpenAL -- like OpenGL -- uses a
right-handed Cartesian coordinate system (RHS), where in a frontal default view X
(thumb) points right, Y (index finger) points up, and Z (middle finger) points towards the
viewer/camera.

3. Attenuation is too damn high.
I mean, even in the tutorial example you can't listen to anything.

   Factor = MinDistance / (MinDistance + Attenuation * (max(Distance, MinDistance) - MinDistance))

With defaults: 1 / D
If you try to use it out of the box, as a lot of people that seeks this libraries does, it will give you a headache.

I mean, if you use pixels as units... then when a sound is 20 pixels away you'll have an attenuation factor of 1/20!! 1/20 of the volume and it's just right there!!!

And on the next part of the tutorial it says to use:

Code: [Select]

3DSound.SetMinDistance(5.f);
3DSound.SetAttenuation(10.f);

Thats 5/(5+10*(D-5). If you're on x=20, then the gain goes to 3%
just 20 pixels away!!!!!

On the other hand, using a value like 100 will highly attenuate the sound, ie it wil be heard only if very close to the listener.
The default value for attenuation is 1.

(Maybe the creator of the tutorial meant 1.0 to highly attenuate and 0.01 to be the default?)

The tutorial should have a line to make a clear that metrics are NOT in pixels, and should be corrected so it at least can reproduce empirically the intended teoric behaviour.

A Sound::SetDefaultAttenuation(float) would be nice to centralize this behaviour instead of adding a "sound->SetAttenuation(n)" for every damn sound you create, as the metrics you choose for your game won't generally vary. And setting an attenuation or fixing the position for each call to SetPosition is counter-intuitive and counter-productive (error prone).

The possibility to choose the distance model would also be greatly appreciated.
If you just want some default padding, then a linear model would be much better than the inverse distance clamped model (OpenAL's default)




Thanks for your time
-Martín

[nitram_cero]

4. Listener::SetTarget is useless.

AL_ORIENTATION is a pair of 3-tuples consisting of an 'at' vector and an 'up' vector,
where the 'at' vector represents the 'forward' direction of the listener and the orthogonal
projection of the 'up' vector into the subspace perpendicular to the 'at' vector represents
the 'up' direction for the listener. OpenAL expects two vectors that are linearly
independent. These vectors are not expected to be normalized. If the two vectors are
linearly dependent, behavior is undefined.

void Listener::SetTarget(float X, float Y, float Z)
{
    float Orientation[] = {X, Y, Z, 0.f, 1.f, 0.f};
    ALCheck(alListenerfv(AL_ORIENTATION, Orientation));
}

This is useless. If you can't define what's your "up"... so "up" is always (0, 1, 0).

If you're in 3d and you want to look up, then your "up" would be linearly dependent with the target "at", messing everything.

"at" sets the direction you're facing, but without "up", you can't tell what's right from left.












That's why it only works for X and Z coords. The Y coord in set target gets eaten.


In this next graphical example is a more important issue for 2D games.









*Errata: When it passes thru L the LEFT speaker will be muted






Thats why the "up" vector shouldn't be hardcoded to (0, 1, 0)

-> Patch I'm using: http://nitram.cero.googlepages.com/patch_sfml_nitram_2009_05_04.patch
It just changes SetTarget(Target) for SetTarget(Target, Up=Vector3f(0,1,0))
(and it's 6-float version)

[nitram_cero]

Testing I found out that you can still get to use SFML as it is if you:
1. Only use X and Y from SetTarget as the real X and Y coordinates.
2. Never try to use it in 3D.
3. Put the listener position in the Z coordinate (positive)
4. Set the target in the Z coordinate as negative.

This way "Up" is the real up, using the Y screen-coordinate as up (sign doesn't matter).

Code: [Select]


float myMinDist=300.f;   //ye olde' MinDistance

#define SQRT_2                1.4142f
float listenerDist=SQRT_2 * 300.f;   //the one for the listener pos: sqrt(300² + 300²)

//listener away from screen
sf::Listener::SetPosition(centerx, centery, listenerDist);
//target pointing to the screen
sf::Listener::SetTarget(0, 0, -1.f);

//for each sound
sound.SetMinDistance(realMinDist);


This way, when the sound X/Y are on the min distance range of the listener's ... the sound is hearable at it's fullest... and the panning isn't only left or only right speaker.


I hope you hear me on this one, Mr. Laurent.

[Laurent]

I hope you hear me on this one, Mr. Laurent.

I hear you loud and clear dude
I'm glad to see someone going in depth into the sound module, and especially the spacialization features. I usually have only very little feedback about this stuff.

1. The "Listener::SetTarget()" position (Listener's orietation) is RELATIVE.

Damn, you're right. It's a shame I missed this point. Well, I guess the OpenAL documentation is badly written if you have to look at the specification to use OpenAL, instead of relying on the API documentation.
One point for you.

2. The coordinate system in the tutorial is misleading.
[...]
It's insinuated to be X - horizontal, Y - depth, Z - vertical

Yep. In the tutorial I'm talking about a 2D world viewed from top, like a RTS or RPG game. This is the default setup in SFML, if you want a side view and use (X, Y) then you have to call SetTarget(1, 0, 0).
I guess I should be more explicit in the tutorial.

3. Attenuation is too damn high
[...]
I mean, if you use pixels as units...

Nobody told you to do so. The audio module has nothing to do with pixels, it defines a 3D world and it's up to you to make it match your 2D scene. How could I assume any reasonable default value for that? 20 pixels may be a lot in someone else's game. It's just up to the user to setup its listener and sounds properly.

The possibility to choose the distance model would also be greatly appreciated.
If you just want some default padding, then a linear model would be much better than the inverse distance clamped model (OpenAL's default)

Probably. But before adding it I want to make sure it is really necessary. In what situation would you need to add padding? Who will really need to change the distance model?

4. Listener::SetTarget is useless.
[...]
If you're in 3d and you want to look up, then your "up" would be linearly dependent with the target "at", messing everything.

Absolutely. But do you mean I should change the global world up vector just because you're looking perfectly up? Even in real life, you can't look up 100%. And 99.9% will just work fine.
To me it's ok to have the up vector hard-coded, this is like a physical constant (just like gravity for example).
Besides, I don't understand your second graphical example (but this is probably just because I just woke up ).

I understand your point of view, but you have to understand that SFML is a user-friendly wrapper around OpenAL. What you're trying to do is forwarding the OpenAL programming interface to SFML, which doesn't make sense to me. If you really need every single feature of OpenAL then just use it directly
SFML's audio module doesn't claim to be a fully-featured professionnal sound API (FmodEx is good for that), it's goal is just to be simple and fast.

I really appreciate your feedback, don't hesitate to defend your point of view. This is what makes SFML become better

[nitram_cero]

Yey!  :lol:

I guess the OpenAL documentation is badly written

Yes, I found out testing, not by the docs... if you see above I quoted a code example because no text from the theory was specific enough.

In the tutorial I'm talking about a 2D world viewed from top, like a RTS or RPG game

I thought of that after my rant  :oops:
I was thinking a more super mario bros like.

(X, Y) then you have to call SetTarget(1, 0, 0).

Nope, this would get me +Z as right and -Z as left.

As Y is always facing up (+Y is OpenAL's "up"), and we are facing -X (OpenAL's "at")... then the plane that it's defined is XY... so right or left would be defined by the Z value.
(check yourself with the Right Hand Rule)

Nobody told you to do so

I know, but maybe a more pixel-friendly value would be best for new users of the library (as you said, nobody really goes in depth with this... I assume because of it's difficulty to get it working).

Nevertheless consider adding a SetDefaultAttenuation/MinDistance as a sf::Sound static function. It could be convenient to centralize initialization of this parameters in a couple of lines, instead of having to add a wrapper to load sounds.

Code: [Select]

Who will really need to change the distance model?
I don't have  a real reason, no.
Maybe if you're developing not a game but an application (Simple Fast Media Library), like a music player or whatever, it would work as an audio balance.

To me it's ok to have the up vector hard-coded, this is like a physical constant (just like gravity for example).

In 3D, gives you the hability to roll your head (e.g. Peeking from a corner of a wall).

In 2D gives you freedom to define which sound-coordinate axis is associated with a screen-coordinate counterpart. I like using x for x and y for y, it's more confortable to work this way.

Also to let you "reverse" the speakers (Up = -Up), for configuration and also could be used for a weird powerup effect.

What you're trying to do is forwarding the OpenAL programming interface to SFML... it's goal is just to be simple and fast.

Yeah, you're right. I also understand your point of view. Keeping it minimalistic is really a cool feature.

I really appreciate your feedback, don't hesitate to defend your point of view

Thanks, I can be a real pain in the ass  :lol:  

I came to this conclusion:
If you have a up vector +Y, and at(target) vector -Z... it leaves +X=right and -X=left.
The target vector is actually "depthness", which is played (or should be) as "further inside the monitor" (which we'll probably never use)

Y is Up, so if you wan't something making a falling sound, this is your axis.
And that's the same as a plataformer! You really want things falling (Y screen axis) to be represented like the're coming from "up" (Y sound axis). Actually it's opposite sign.
How it's now it's actually correct!
(But we can't test this without a quadraphonic system.)

So with all this whining I've been about, I wasn't even correct.

 :oops:

I don't understand your second graphical example

The only point left is the "sound going thru the listener's head" that mutes one speaker or the other.

It happens if a sound position relays in the axis orthogonal to target and up. It's not heard from both speakers, not even near the listener's position...

That's what the second graphic is about. But actually it makes no sense, don't try to figure it out.

I don't know if it's the way OpenAL handles distances for left and right speakers or what it is.

I think it has to do with the fact that there is no real life situation in which a sound passes thru your skull.

So my solution was to move away the listener from the "monitor".



Ok... I'm really over the edge... I'll go to sleep now.  

Good luck
-Martín

[Laurent]

I know, but maybe a more pixel-friendly value would be best for new users of the library

How am I supposed to find this value? One pixel may map to 1m, 1 cm or 1km. It's really up to every application.

Nevertheless consider adding a SetDefaultAttenuation/MinDistance as a sf::Sound static function. It could be convenient to centralize initialization of this parameters in a couple of lines, instead of having to add a wrapper to load sounds

Then I should consider writing a static function for every property of every class
Anyway, any serious application should have a sound manager, then this kind of stuff can be gathered in it.

I don't have a real reason, no.
Maybe if you're developing not a game but an application (Simple Fast Media Library), like a music player or whatever, it would work as an audio balance.

Yeah, maybe. But I'll wait until someone actually needs it, I don't like adding features that may not be useful.

I came to this conclusion:
If you have a up vector +Y, and at(target) vector -Z... it leaves +X=right and -X=left.
The target vector is actually "depthness", which is played (or should be) as "further inside the monitor" (which we'll probably never use)

Y is Up, so if you wan't something making a falling sound, this is your axis.
And that's the same as a plataformer! You really want things falling (Y screen axis) to be represented like the're coming from "up" (Y sound axis). Actually it's opposite sign.
How it's now it's actually correct!

I also realized that while reading your post :lol:

The only point left is the "sound going thru the listener's head" that mutes one speaker or the other

That's how spacialization work, I guess I can't help much on my side. But I don't think it's really an issue.

[nitram_cero]

How am I supposed to find this value? One pixel may map to 1m, 1 cm or 1km. It's really up to every application.

Yes, you're right I'm not discussing that.

For a lot of 2D games, metrics is not really important, so pixels are oftenly used (Specially by beginner game programmers).

I was just saying this to set friendlier default values so that it's easier to get it working without even caring for more than Listener::SetPosition and Sound::SetPosition.

The default values suck anyway... except if you're using 1 audio-unit per 30 "real life" meters.
This is to the application to define, I understand... but I think it would be just friendlier the pixel way.

Then I should consider writing a static function for every property of every class
Anyway, any serious application should have a sound manager, then this kind of stuff can be gathered in it.

Yeah, maybe. But I'll wait until someone actually needs it, I don't like adding features that may not be useful.

Ok, ok. Keeping it minimal, I get it.

The only point left is the "sound going thru the listener's head" that mutes one speaker or the other

That's how spacialization work, I guess I can't help much on my side. But I don't think it's really an issue.

Yes, I came to notice that. It's what's expected.

But oftenly 2D games sound search for a more confortable than physically realistic audio.

I found a "solution" and I'm writing a tutorial on that.
The solution is really practical (2D only): Putting the listener away from the plane where the sound sources are.
The expected behavior (talking about how the game feels, not aritmetic and logic) would be to have the listener where "your head" is, and the sounds passing in the "monitor's plane".


If you put the listener back in Z, and target towards Z... then you hear from both speakers.
And it works like a charm. It feels correct

I've made a testing application, there you can see this yourself. I can upload the code if you want.

OH! Before I forget, sombething really really really important about OpenAL...

Stereo data is expressed in an interleaved format, left channel sample followed by the
right channel sample.
Buffers containing audio data with more than one channel will be played without 3D
spatialization features – these formats are normally used for background music.

I tested it and, as expected, stereo sounds are played at full gain always.
There should be a really big emphasys on this on the documentation/tutorials.

Well, that's all.
Regards
-Martín

[Laurent]

I tested it and, as expected, stereo sounds are played at full gain always.
There should be a really big emphasys on this on the documentation/tutorials.

I think there is, isn't it?

[nitram_cero]

http://www.sfml-dev.org/tutorials/1.4/audio-spatialization.php

Nope, there is an good explanation about what mono and stereo is, but not an explanation on it's behaviour with spatialization.

When using spatialization, stereo sources sound with full gain, indiferent of the distance to the listener.


EDIT: http://blog.tbam.com.ar/2009/05/sound-spatiaiization-for-2d-games-in.html

link to the guide

It's kind of long, but if you can read it and give some feedback it would be great.

[Laurent]

I thought I mentioned it in the tutorial, but yeah there's nothing about it actually. I'll add it to the tutorial and to the documentation. Thanks for your feedback, again

[nitram_cero]

No problem, thank you for the patience

[WitchD0ctor]

Well thank you, I was wondering why the Listener was doing Nothing.

your incite nitram_cero, on the code and using MONO sound helped immensely, thank you!