SFML community forums

Help => Audio => Topic started by: therocode on March 03, 2014, 08:10:57 am

Title: Bug in the Audio Module with listeners?
Post by: therocode on March 03, 2014, 08:10:57 am
I hope this hasn't been discussed before, I couldn't search because I get an error every time I try (DATABASE ERROR Please try again. If you come back to this error screen, report the error to an administrator.).

Anyway, I am curious on this thing which might be a flaw in SFML.

It is regarding the function Listener::setDirection which is implemented like this:
void Listener::setDirection(float x, float y, float z)
{
    priv::ensureALInit();

    float orientation[] = {x, y, z, 0.f, 1.f, 0.f};
    alCheck(alListenerfv(AL_ORIENTATION, orientation));
}
 

Now, what it does is defining two vectors that are passed to OpenAL. The first vector (which is the one given by the user) is the "at" vector which is the direction the listener is facing in the scene. The second vector is the "up" vector which points to the upside relative to the listener. This can be thought of as having the "at" vector pointing out from your nose, and the "up" vector pointing up the top of your head.

Anyway, the function _always_ uses the (0, 1, 0) vector for up which to me looks like it poses a problem. Because the OpenAL specification states this about setting the orientation:
. OpenAL expects two vectors that are linearly
independent. These vectors are not expected to be normalized. If the two vectors are
linearly dependent, behavior is undefined.

The important bit being that they have to be linearly independent. However, with the implementation in SFML, if you would call setDirection(0.0f, 1.0f, 0.0f) or even setDirection(0.0f, 0.5f, 0.0f) etc, they would be linearly dependent and would according to the OpenAL specification cause undefined behaviour.

Is there something I am missing or is this just overlooked by SFML?
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 03, 2014, 08:39:59 am
Yes, there's a limitation. And what? What is your feature request?
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 03, 2014, 09:17:38 am
Ah, dang I must have clicked the wrong subforum! Meant to put it in "general discussion". :D Sorry about that.

Feel free to move it if you please.

But then is this limitation intended? And why so?
And shouldn't it at least be mentioned in the documentation?
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 03, 2014, 10:33:37 am
Quote
Ah, dang I must have clicked the wrong subforum! Meant to put it in "general discussion".
"Audio" would be even better :P

Quote
But then is this limitation intended? And why so?
There must be an "up" vector, so there must be a direction which is forbidden for the other axis. That's how it works. If your question is "why is the up vector hard-coded?", then it's to make the API simpler, as in most cases it is +Y.

Quote
And shouldn't it at least be mentioned in the documentation?
Yep.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 03, 2014, 10:57:16 am
There must be an "up" vector, so there must be a direction which is forbidden for the other axis. That's how it works. If your question is "why is the up vector hard-coded?", then it's to make the API simpler, as in most cases it is +Y.

Yep, I know they have to be different.

I suspected that the reason was to make a simpler API. Personally I disagree with that solution since I find it unacceptable to just accept that one arbitrarily chosen specific number is forbidden and just doesn't work. As you said the axes having to be different is "how it works" so why not let the API reflect this instead of letting the users being in a problematic "ignorance is bliss" situation? It is not much more compex anyway.

But yeah, that is just my opinion and how I would go about things. Fair enough if you pick the other approach. I was mostly just curious anyway. :)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 03, 2014, 12:09:17 pm
It's much simpler (to code and understand) to just have to choose a direction, rather than a pair of vectors that have to be linearly independent. Explaining what an "up" vector is, and what "linearly independent" involves, can be complicated for beginners. Explaining what the listener direction is, is really straight-forward to understand for everybody.

Of course I'll change my mind if one day I get complains about particular use cases that don't work because they need a listener direction on Y axis. But so far it has never been an issue.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 03, 2014, 12:26:28 pm
Yep, I know what you are saying, but that is where we disagree simply put.

I don't think that newbie-friendliness is an excuse in this case. It is not much to understand really, and going through that learning of how orientations work seems imo more worth than hiding it for the sake of newbies.

IMO programming is a lot about learning and if you abandon a library because it forces you to learn things, then the newbie is at fault, not the library API.

Also this doesn't mean that I don't see the value of simple APIs. I am very much for simple APIs, but not purely for the sake of newbies.

But yeah, as I said, I was mostly curious and I am not trying to argue to make you change your mind. I would have done it differently sure, but this is your library, not mine. :)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 03, 2014, 02:21:39 pm
I have to agree to therocode. Allowing undefined behavior without even documenting it should be avoided at all costs. Not only it's possible to have bugs, it might also lead to beginners not understanding why the hell their stuff is not working, even if they use SFML correctly.

Something as simple as "If you use (0, +x, 0) as direction it won't work" would at least be a workaround for the real problem of not explaining the user why there is the need for forward and up vectors.

Also I really wonder if SFML's only target audience is beginners? Because beginners won't be beginners forever, and not having the chance to do things right will probably make SFML being replaced by another library that's "better controllable".

Regarding this specific problem: What about providing default arguments/an overload to allow specifying an up vector?

Besides from that, therocode, I think this "request" will be one of those "won't fix"->"fixed" things, which is as follows:

SCNR :D
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 03, 2014, 03:06:30 pm
As I said, since so far (almost) nobody complained about the lack of control over the up vector, I don't see any reason to add it.

Of course the documentation will be improved ;)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: eXpl0it3r on March 03, 2014, 03:32:07 pm
As I said, since so far (almost) nobody complained about the lack of control over the up vector, I don't see any reason to add it.
Ever thought about that maybe nobody has been making something more advanced with SFML until now? Thus the fixed default was enough for all the simple games.
So is this really your way of arguing? Seeing such "logic" makes me rather sad to be honest. :(
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 03, 2014, 03:39:37 pm
Quote from: Laurent
As I said, since so far (almost) nobody complained about the lack of control over the up vector
therocode just did and gave a good reason.

Quote from: Laurent
I don't see any reason to add it.
"Undefined behavior" is a good reason in my opinion. What's the reason to ignore this? Please no "too complicated", there can be at least an overloaded function that allows eliminating the bug (that could be easily avoided for everybody). Or an assert. Or an exception. Anything, but please do not remain silent in case of a KNOWN error, which, in my eyes, feels negligent in terms of code safety.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 03, 2014, 03:55:08 pm
Adding a settable up vector in the function with 0,1,0 as default value would probably be good enough. However _just_ mentioning that 0,1,0 as a value for the setDirection function is undefined behaviour is imo more confusion inducing than just explaining what the point of the up-vector is and then not hide it.

I can't even think how one would formulate it to make sense. The best I can think of would be "setDirection sets the direction of the listener blah blah. Due to internal details, the value 0,1,0 is undefined behaviour". That would make me think "uuhh why? and can i reliably use this function to orient my player?".

And the reason that it should not be fixed just because no one had complained about it seems like a weird approach to make a stable bug free library.

Furthermore, I really don't think that explaining the up-vector is complicated. I mean, most users who are newbies would not even change the listener orientation. They would make simple 2D games with the default listener. Someone who changes the listener orientation is someone making something more complicated, like a 3D camera or something, and if you do that and cannot understand an up-vector, you're going to have many more difficult issues to address. :D

I can't imagine why a newbie would want to change the orientation at all, honestly so i don't think the "too complicated for the newbies"-argument even applies for this part of the API.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 03, 2014, 04:06:34 pm
So is this really your way of arguing? Seeing such "logic" makes me rather sad to be honest. :(
From an design standpoint, it makes much more sense to start with a minimal API and extend it when the need comes, instead of providing a whole lot of things that might be useful some time in the future, but contribute to an confusing and cluttered API. This is not about this specific feature, rather in general.

Quote from: Tank
"Undefined behavior" is a good reason in my opinion.
I agree that it's a good reason to add at least an error check (assert or alike) and a better documentation, and I'm sure Laurent is not against that. To implement a new feature I would say there need to be concrete use cases. I'm not saying there are none, but there have still not been any arguments in that direction.

So is anybody of you actually requring that feature, or is the main point of this discussion to criticize the way how feature requests are handled in SFML? I mean that's also a valid discussion, but then keep it separate from the audio up vector.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: eXpl0it3r on March 03, 2014, 04:13:34 pm
From an design standpoint, it makes much more sense to start with a minimal API and extend it when the need comes, instead of providing a whole lot of things that might be useful some time in the future, but contribute to an confusing and cluttered API. This is not about this specific feature, rather in general.
I can fully agree with that, but that's not what's happening here. Some are making a push for changing something, thus making the API not really more complex, but instead of discussing the request at hand, the only counter argument is "nobody wanted it in the past". Personally I wouldn't even call that an argument - the past is the past, we're in the present now, so let's evaluate now and here. If that's the level of arguing we're going for here, then sorry that can only make me sad.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 03, 2014, 04:27:19 pm
Quote
Some are making a push for changing something
I don't see anybody here who needs this feature. Even the OP said it was just for curiosity.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 03, 2014, 04:30:57 pm
To implement a new feature I would say there need to be concrete use cases. I'm not saying there are none, but there have still not been any arguments in that direction.

You are talking as if it is a feature request as in extending what is possible with SFML and even if that is technically right (since making it possible to change the up-vector is a new feature) that is not the main point. It is not a feature request, it is addressing an existing problem with the library which is fact. I don't think there should be a need for "specific use cases" to fix a problem that an arbitrary magic number in a function breaks the application without good reason.

It is like saying "oh the add(a, b) function doesn't work with the numbers add(3453, 2344)? well i haven't seen a use case for those particular numbers so I won't fix it." imo.

Quote
Some are making a push for changing something
I don't see anybody here who needs this feature. Even the OP said it was just for curiosity.

Again, it is not a feature request, it is a problem that is fact. And yes I asked out of curiousity, not because I need it because I am not making a 3D application in SFML. But know that if I would make a 3D application I would be wary about using SFML since with this issue, I could get random crashes if I turn the camera in a funny angle. ;)

But even if I ask out of curiousity and that it is up to you of course, I am still very much for the opinion that to go about it properly is to fix it.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 03, 2014, 05:00:27 pm
I think this quote summarizes the problem well enough:

Quote from: therocode
It is like saying "oh the add(a, b) function doesn't work with the numbers add(3453, 2344)? well i haven't seen a use case for those particular numbers so I won't fix it."

But I will still add some more words. ;-)

Quote from: Nexus
From an design standpoint, it makes much more sense to start with a minimal API
I fully agree. But this time it's a broken API. This is not a feature request.

Quote from: Nexus
but contribute to an confusing and cluttered API.
A "clean" and "not cluttered" API which is broken is not useful however. And it's not like adding "upVector" to the list of arguments with a description similar to "Vector pointing upwards from the listener" is rocket science. Seriously, if someone does not understand but needs that (-> 3D), then he will probably simply go and learn it -- if not, then I'm not sure if "saving" the people from learning the stuff, which is *required* to make things work, is the way to go.

I sometimes have the impression that the "S" in "SFML" does not mean "Simple" but "In no way try to confuSe the people who are programming for their firSt time in life and are not willing to read about Stuff that they haven't diScovered before and that'S not explained in the tutorialS" (I've capitalized all S's because I wasn't sure which one shall represent the SFML's "S").

Quote from: Nexus
I agree that it's a good reason to add at least an error check
An error check is needed, yes, but it seems weird to not have a chance to avoid the error case. SFML forces me to risk undefined behavior, and that's known. I call that a no-go.

Quote from: Nexus
So is anybody of you actually requring that feature, or is the main point of this discussion to criticize the way how feature requests are handled in SFML? I mean that's also a valid discussion, but then keep it separate from the audio up vector.
Being a member of this community for a couple of years makes me a bit biased of course, because the numbered list I posted in a previous post is how I see it. But this is really about the audio up vector issue. Experience showed that with enough pressure things can change. ;)


Title: Re: Possible bug in the Audio Module with listeners?
Post by: select_this on March 03, 2014, 05:06:55 pm
Extrapolating from the discussion above, a possible use case would be using the audio listener in a 3D application where the listener's attached to something freely rotatable on the three axes. I could see that happening in first person games quite easily, e.g. someone moving the camera to look along the Y axis. You might even want to roll the 3D listener on all three axes for something like a space combat simulator.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 03, 2014, 05:17:11 pm
Extrapolating from the discussion above, a possible use case would be using the audio listener in a 3D application where the listener's attached to something freely rotatable on the three axes. I could see that happening in first person games quite easily, e.g. someone moving the camera to look along the Y axis. You might even want to roll the 3D listener on all three axes for something like a space combat simulator.

Yes that is a perfectly valid use case. Also observe that in FPS games, the camera is often capped so that you can't look more upwards than perfectly straight up and no further down than straight down. Both of those capped orientations would cause undefined behaviour if the listener is set to follow the orientation of the player.

Also, this person making this 3D game would not think an extra up-vector argument is hard to understand. :)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: select_this on March 03, 2014, 05:26:48 pm
Also observe that in FPS games, the camera is often capped so that you can't look more upwards than perfectly straight up and no further down than straight down. Both of those capped orientations would cause undefined behaviour if the listener is set to follow the orientation of the player.

To quote Laurent from a previous discussion on this topic:

Quote
Absolutely. But do you mean I should change the global world up vector just because you're looking perfectly up? Even in real life, you can't look up 100%. And 99.9% will just work fine.
To me it's ok to have the up vector hard-coded, this is like a physical constant (just like gravity for example).
Besides, I don't understand your second graphical example (but this is probably just because I just woke up ;)).

Here's the topic in question:

http://en.sfml-dev.org/forums/index.php?topic=1225.0 (http://en.sfml-dev.org/forums/index.php?topic=1225.0)

I would argue that the fully rotatable space combat simulator example still stands, though.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 03, 2014, 05:45:31 pm
It is not a feature request, it is addressing an existing problem with the library which is fact. I don't think there should be a need for "specific use cases" to fix a problem that an arbitrary magic number in a function breaks the application without good reason.
The "fix" would be an error detection and better documentation. In fact, letting the user set the up vector doesn't fully solve the issue. There is still an error when up and direction are linearly dependent, there would still be a need to both check for it and mention it in the documentation. What's different is that the user can set the up vector to avoid this problem. But, the up vector is not necessary to avoid this problem, that's why adding it because of the problem doesn't make sense.

The main reason to set the up vector is not to avoid this specific problem, but rather to implement a specific coordinate system for the listener. You're argumenting from a workaround perspective, and API design should not be driven by workarounds. ghostmemory's argument is better in this respect: he shows a scenario where the up vector is useful from a functionality standpoint.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: select_this on March 03, 2014, 05:47:46 pm
Out of interest, and apropos of this topic (thought slightly off-topic, admittedly), is there a standard solution for when vectors reach a point of linear dependency? I would've thought the simplest solution would be to just adjust the vectors slightly but I imagine in some situations that could produce noticeable artefacts. I must admit I've never given it thought before.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: binary1248 on March 03, 2014, 05:56:49 pm
Out of interest, and apropos of this topic (thought slightly off-topic, admittedly), is there a standard solution for when vectors reach a point of linear dependency? I would've thought the simplest solution would be to just adjust the vectors slightly but I imagine in some situations that could produce noticeable artefacts. I must admit I've never given it thought before.
http://en.wikipedia.org/wiki/Gimbal_Lock
http://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation

Welcome to the wonderful world of Computer Graphics.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: select_this on March 03, 2014, 06:02:08 pm
http://en.wikipedia.org/wiki/Gimbal_Lock
http://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation

Welcome to the wonderful world of Computer Graphics.

Thank you, these are really informative (and reaffirms my belief that I'm better off leaving these sorts of things for those mad folk who love writing abstractions!)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 04, 2014, 06:24:59 am
Alright, I have done some research and as I suspected, this whole issue with a hard coded up-vector is far more problematic than I first thought; it is not just a matter of code safety, but the whole feature of orienting the listener is broken. Let me explain.

What made me thinking is that why is the up-vector there in OpenAL in the first place? Maybe it is actually used for something? And if you think about it, of course it is. A sole vector pointing where you look is not enough to describe orientation. Consider if you are looking forward and you have one blue fire to the front-left and one orange fire to the front-right like so (pardon my bad drawing skills :D):

(http://i.imgur.com/tdyWwcF.png)

The red arrow is the at-vector, the green is the up-vector.

Frame1: this is a first person view, the red at-vector is going straight forward and the up-vector is going straight up
Frame2: this is a top down view of the same scene. The player is looking straight at the fires
Frame3a: this is a side view of the same scene

In all those frames, the sound of the blue fire is heard through the left speaker and the yellow fire is heard through the right. This is all good. However, now consider that the player turns around with his back towards the fires and bends his neck backwards like crazy so that he looks at the fires straight, but is upside down. This would be what the Frame3b picture shows. The at-vector is still at the fires but the up-vector is now pointing downwards. And if you imagine doing this, the blue fire should now be heard through the right speaker and the yellow should be heard through the left. This is not possible in SFML since the up-vector is hard coded.

This might seem like a longshot situation but if you mean to develop applications with free moving 3D cameras, then it isn't really. Even the space simulator example from before would be enough. If you go forward, and then make a half-circle loop upwards to go the other way upside down, all of a sudden all sounds would be inverted because the up-vector is pointing in the wrong direction! Not good.

Furthermore, it is even more problematic, because as the OpenAL specification says, it expects the vectors to be orthogonal. i.e. having only right angles. Not having orthogonal vectors is not undefined behaviour however, but it creates unintended behaviour: A skewed space for the sound calculations. I verified this by asking in the #OpenAL channel:
http://thunked.org/p/view/pub/1xjz28abo

And I looked in the OpenAL source code to verify this:
http://thunked.org/p/view/pub/mjljisdt9

This is verifies what the person in the channel said: The up-vector and at-vector are used to calculate an orthogonal right-vector and these three vectors are then normalised and used to construct the listener-space coordinate system which is used for calculations with the sources sounds.

In other words if the vectors are not orthogonal, the calculations will be off due to the skewed space. In practice this means that the SFML 3D audio spatialisation only works as intended when the listener look-at is perfectly straight ahead. If the player looks up or down in an FPS game for instance, the audio calculations are skewed accordingly.

This imo is a major flaw of the 3D spatialisation. If you would use SFML for anything but the most basic spatialisation tasks, the result will be off. What is even worse is that it is not even mentioned in the documentation which could cause a lot of headaches for people trying to use it and wondering why sounds are not behaving correctly.

All of these issues are in the library apparently because it is too difficult for newbie programmers to understand that proper 3D orientation requires at least 2 orthogonal vectors. This is to me absurd, and the argument is voidified by the fact that newbies would not even make 3D applications in the first place. And starting out with a small API to avoid clutter and add features later etc is surely fine and all, but this feature doesn't even work properly without two vectors which is why that argument doesn't even apply here either.

If you still after this think that this issue is not worth fixing (it can even be done without breaking the API by adding a default up-vector argument like tank said), then at the very least explain in the documentation that most values you give to the setDirection() function will produce skewed and flawed output and that (0, 1, 0) will totally break it. That way people who want to have proper audio spatialisation can look for another library to use. ;)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 04, 2014, 07:57:01 am
Thanks for the detailed research ;)

Don't get me wrong, I know exactly what the up vector is for, and what limitations the current API has. My main argument is not that it is unfriendly for beginners, it's just that in practice nobody ever complained about this so that goes directly to the bottom of my todo-list. But if you implement it with a nice API, and submit a pull request, there's a chance that it doesn't get forgotten ;)

(the extra argument with default value is not a solution, because with need the corresponding getter anyway)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 04, 2014, 08:23:24 am
Yeah, well I could implement it for you but the problem is that we surely have different definitions of nice API. I would just go with setOrientation(atVector, upVector) (or even until 3.x to not break the api, i would do setDirection(atVector, upVector = (0,1,0)) and add a corresponding getUpVector() getter.

Your priorities are surely your business and I won't try to meddle with them, but I must say that I am surprised that fixing it or at least adding the info to the documentation is not a high prio since it doesn't involve much coding/writing and as long as it is not done, leaves the 3D spatialisation feature pretty broken imo. But you're the boss.

edit:
Btw, curious question: What was the idea with the setDirection() function in the first place if you know all the limitations with it? What is a proper use case that would not suffer from the skewed space problem?
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 04, 2014, 08:46:46 am
Quote from: Laurent
My main argument is not that it is unfriendly for beginners, it's just that in practice nobody ever complained about this
This sounds like a bug is "okay" as long as nobody seriously trapped into it. Personally I would try to fix it ASAP as long as it hasn't caused damage to something. And I think purely wrapping the forward and up vectors in the call is absolutely fine (unless you want to go to a even higher "newbie-friendly" level and add a camera to SFML or something to abstract the "complicated" vector algebra away).
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 04, 2014, 09:07:10 am
Quote
Your priorities are surely your business and I won't try to meddle with them, but I must say that I am surprised that fixing it or at least adding the info to the documentation is not a high prio since it doesn't involve much coding/writing and as long as it is not done, leaves the 3D spatialisation feature pretty broken imo. But you're the boss.
You know, when you don't even have the time to write a single line of code, you start to adopt a very pragmatic point of view. Sure it is broken and in a perfect world it would be changed, but since nobody is actually waiting for it, I prefer not to waste too much time on it and focus on things that have been pending for a long time and that are awaited by many users. You are focused in this particular issue right now, and might think that it very important, but don't forget that I have hundreds to handle, and this one is a drop in the ocean of tasks.

Quote
Btw, curious question: What was the idea with the setDirection() function in the first place if you know all the limitations with it? What is a proper use case that would not suffer from the skewed space problem?
Again, pragmatism. And you may disagree, but I think this was the right choice since so far everybody can use the audio API happily. Sure in the future this may not be true anymore, but future is the future, there are already too many things to handle in the present.

Quote
This sounds like a bug is "okay" as long as nobody seriously trapped into it.
I would just replace "okay" with "low priority", and yes, as long as there are other more important tasks to work on, this is true.

Quote
Personally I would try to fix it ASAP as long as it hasn't caused damage to something.
Personally I'd love to be able to do anything on SFML again ;D

Quote
And I think purely wrapping the forward and up vectors in the call is absolutely fine (unless you want to go to a even higher "newbie-friendly" level and add a camera to SFML or something to abstract the "complicated" vector algebra away).
To be consistent with the other "properties" of the Listener class, it would have to be implemented with its own pair of getter/setter. And maybe we can check if there would be a more friendly name than "up vector", which is a purely mathematical description of it and not connected to audio/listener semantic field.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: eXpl0it3r on March 04, 2014, 09:11:03 am
So I guess it has come to this:

(http://i.imgur.com/CnJR67g.jpg)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 04, 2014, 09:12:25 am
LOL
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 04, 2014, 09:32:36 am
To push this discussion here's an idea (I haven't thought about naming much, that's more up to the people who know how "newbie-friendly naming" works -- for me "forward and up vectors" make perfect sense and are easily understandable -- at last chance give the people an illustration and they'll get it immediately, I'm sure):

class Listener {
        public:
                struct Direction {
                        Direction( sf::Vector3f forward, sf::Vector3f up );

                        sf::Vector3f mForward;
                        sf::Vector3f mUp;
                };

                void setDirection( Direction );
                Direction getDirection() const;

        private:
                Direction mDirection;
};
 

That way you can't change any of both values independently and you are very near the current API (those functions could be even provided as overloads for adding it in 2.2).

Another option would be to rename "setDirection" to "pointAt" or "lookAt" or "listenAt" or even "listenTo" (native English speakers please). Same structure applies.

Edit: therocode suggested to use "orientation" instead of "direction". I agree to that.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 04, 2014, 09:36:41 am
Yes, I too would agree that Tank's API proposal is nice. I also agree on the fact that "up" is as semantically descriptive as "at". "at" is where the listener looks at, and "up" is what is up to the listener. Can't get more simple than that I think.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 04, 2014, 09:39:00 am
Direction won't be changed. Up is a separate attribute (and making a structure would break existing code anyway).

The API would rather look as follows:
class Listener
{
public:
     static void setDirection(const Vector3f& direction);
     static void setUp(const Vector3f& up);

     static Vector3f getDirection();
     static Vector3f getUp();
};

And yes, there might be a more expressive name than "up" (or "upVector"). But it seems like "up" is commonly used.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 04, 2014, 09:51:41 am
I'm not a native english speaker, but with this API I immediately thought about this :P
sf::sleep(sf::hours(7));
sf::Listener::getUp();
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 04, 2014, 09:56:48 am
Indeed ;) also "setUp" could be confused with "setup".

Maybe "set/getUpVector" or "set/getUpward" or similar. But also something different (e.g. referring to the listener's head, such as overhead, if it just didn't have this ambiguous meaning) would be possible.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Lo-X on March 04, 2014, 10:15:16 am
Quote
Get up, stand up
Stand up for your rights

Isn't it possible to have a overload or a new setDirection like

void sf::Listener::setDirection(const Vector3f& direction, const Vector3f& up = sf::Vector3f(0, 1, 0))

?
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 04, 2014, 10:24:52 am
Of course it would be possible, but it's a very unintuitive API because:
I don't understand why everybody wants to mix two attributes.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Lo-X on March 04, 2014, 10:57:05 am
Hum, you're right :) it doesn't make sense
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 04, 2014, 11:31:00 am
Quote from: Nexus
Direction won't be changed. Up is a separate attribute
Quote from: Nexus
I don't understand why everybody wants to mix two attributes.
In this case "up" shouldn't be seen as a separate attribute, because the forward and up vectors together make the orientation, and they are highly dependent on each other. For properly setting an orientation you have to provide both -- they both form the data type. (Compare it to sf::Color for example: r, g, b and a all form the compound "Color"; having sf::Sprite::setR(), sf::Sprite::setG() etc. would not make sense)

Quote from: Nexus
(and making a structure would break existing code anyway).
Could we please put that argument aside? I think you should also consider a solution that breaks the current (and broken, by the way) API. We don't want another WinAPI, do we?
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 04, 2014, 11:49:30 am
In this case "up" shouldn't be seen as a separate attribute, because the forward and up vectors together make the orientation, and they are highly dependent on each other.
The same applies for texture and texture rect, view center and view size, and so on. There are many examples where you have to call multiple methods to set the correct state.

Could we please put that argument aside? I think you should also consider a solution that breaks the current (and broken, by the way) API.
No. It can be discussed for SFML 3, but until then there won't be breaking changes. I don't know why this has to be stated again and again...

I listed three points why the proposed API is unintuitive (to which you didn't respond). On the other side, there is a very clean solution to add an up vector to the existing API.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 04, 2014, 12:22:43 pm
Quote from: Nexus
The same applies for texture and texture rect, view center and view size, and so on. There are tons of examples where you have to call multiple methods to set the correct state, and I see no reason to deviate from that scheme.
Then those should change as well, thanks for pointing them out. :) It's like your often mentioned RAII: Why allow dangerous situations if they can be avoided by design?

The problem I see with such things is that it's easy to use them wrong. And in case of SFML especially, it even remains silent if you do so (no assert, no exception, no message) -- or even does not allow to use it right, like this particular audio bug.

Quote from: Nexus
No. It can be discussed for SFML 3, but until then there won't be breaking changes. I don't know why this has to be stated again and again...
I don't care for what version, I try to find a good solution. This shouldn't have to do something with any version number at all: There's a bug, there should be a clean fix.

2.x can live with a workaround, 3 should have a clean solution. Personally I would go with a clean solution immediately, because I don't find backwards-incompatible API changes problematic (if they happen in 2.2 or 3.0 does not really matter in my opinion -- sooner or later it will happen, so why postponing it for an unnecessary long time?).

I've personally never heard something like "Oh no, those API changes are so massive, I will stop using SFML because of it!". (Except the naming scheme change, which surely pissed off some people. ;)) It's not like that it takes ages to adjust code to work with newer versions. And you have always the option to stay with an older version if it works for you.

Quote from: Nexus
I listed three points why the proposed API is unintuitive (to which you didn't respond). On the other side, there is a very clean solution to add an up vector to the existing API.
I did not respond because I did not propose it, you replied to Lo-X.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 04, 2014, 12:39:52 pm
I don't consider separate methods for up vectors a workaround, it's very probable that it would have been done so in the first place. It's a common way of providing access to attributes, but of course there may be different opinions.

The problem with your API (which is almost the same as Lo-X', as there would also be a setDirection overload with 2 parameters) is that it's not very intuitive. You have to know all the involved types exactly, while the API with separate methods allows people knowing the OpenAL concept, but not SFML, to work directly with SFML's listeners.

Compare the following codes:
sf::Listener::setOrientation(sf::Listener::Orientation(a, b));
// vs.
sf::Listener::setDirection(a);
sf::Listener::setUpVector(b);
Which one is more expressive and easier to read?

Even if you provided a direct overload to allow the call
sf::Listener::setOrientation(a, b);
it is not clear what a and b mean. It's not simply done by naming the variables differently, because often you work with temporary expressions.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 04, 2014, 12:52:01 pm
Quote
Then those should change as well, thanks for pointing them out.
No they shouldn't. I can move my view without changing its size, or I can zoom it without changing its position. I can change a texture while keeping the same rect, and I can change the rect within the same texture. Same for the listener orientation: in most cases (all 2D and most 3D) the up vector will be fixed to +Y and never change after that. So let's use a new property, which works well, is consistent with the current API, and has no constraint. A good documentation will then do the rest.

Quote
I don't care for what version, I try to find a good solution.
It's important to have in mind if a modification has to be scheduled for SFML 3, or can/must be applied immediately. The design considerations are not the same.

Quote
Personally I would go with a clean solution immediately, because I don't find backwards-incompatible API changes problematic (if they happen in 2.2 or 3.0 does not really matter in my opinion -- sooner or later it will happen, so why postponing it for an unnecessary long time?).
I really don't want to start discussing the benefits of versioning rules, I think you're smart and experienced enough to understand why they are needed, let's focus on the initial issue please.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 04, 2014, 12:55:50 pm
For me setOrientation() is easier to read, because it's what's more natural. I want to change the orientation, so I directly set it, not its component. I also rather do object.setPosition() instead of object.setX(), object.setY() etc. Or object.setColor() instead of object.setRedColor(), object.setGreenColor() etc. Orientation is a compound of forward and up vectors, that's how it works. I think you can't abstract it furthermore.

Also the concept of having a forward and up vector is nothing OpenAL invented or which is typical for it, it's basically mathematics, a way to express an orientation.

The problem with your separated functions is that the user has to know that he has to call two functions in order to correctly set the orientation. Auto completion heroes might simply stop by setDirection() and call it. In 2D environments in might even work if the up vector is by default set to {0, 1, 0}. But they completely miss the concept of how positioning the listener really works.

So, even if the compound is not intuitive (which it is in my eyes though), it at least makes sure that the user has to face the "forward/up vector" thing. Without knowing it I'd bet that the most will try to find out what it's all about. Result is: Lesson learned, no bug, everybody's happy.

The argument that it's not clear what specific arguments mean is not valid in my opinion. At first you say that you don't know what "a" and "b" means. Yes, that's right, if you name the parameters like that, then you should consider choosing a more meaningful name. And second, the same could apply to setDirection(): Is the parameter an angle? A relative vector? Absolute vector? Polar coordinate?

In the end the user has to learn the concept behind the code. Then using the code is as easy as a) using auto completion or b) using Doxygen.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 04, 2014, 01:03:23 pm
Quote from: Laurent
No they shouldn't.
Then Nexus' argument was wrong, not mine. He said that it's the same with other properties, not me.

Quote from: Laurent
I can move my view without changing its size, or I can zoom it without changing its position.
Absolutely, in all cases, because there are no constraints, restrictions, pitfalls etc. But:

Quote from: Laurent
Same for the listener orientation: in most cases (all 2D and most 3D)
In most cases? Personally I don't like programming with a time bomb that works "in most cases". Seriously, why can't all cases be simply fixed? I don't get it.

Quote from: Laurent
It's important to have in mind if a modification has to be scheduled for SFML 3, or can/must be applied immediately. The design considerations are not the same.
No, it's not important. Again: I'm looking for the best solution. That's my intention. It's then up to you and your version policies to decide how, when and if it all the changes are being implemented.

Quote from: Laurent
I really don't want to start discussing the benefits of versioning rules, I think you're smart and experienced enough to understand why they are needed, let's focus on the initial issue please.
I am indeed experienced and smart enough (thanks) to know that, that's why I am not talking about whether stuff should be implemented in version X or Y. I'm adjusting my arguments to your versioning scheme and rules, that's why I'm splitting the solution up into a workaround for 2.x and the "normal clean good solution" for 3. Like said, personally I would not care of any version at all, I would simply release the fix to a bug. I don't get why version numbers are so important for some people.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 04, 2014, 01:23:22 pm
Same for the listener orientation: in most cases (all 2D and most 3D) the up vector will be fixed to +Y and never change after that.

Sorry but this is just plainly wrong.

If you have a static +Y up vector, you are only allowed to set the at-vector to any values where Y is 0. In other words, this is DOOM movement where you can't tilt the camera up or down. In any basic 3D navigation application when you do any form of 3D calculations, you have to keep your at-vector and up-vector orthogonal. There is no way to hack around this and there is no way to hide this from the "noobs" who would use this feature. You have to when you navigate the camera set both the at-vector and the up-vector to keep the orthogonality, or the audio space will be skewed.

Hence it doesn't make sense to set them separately. I would find that way more awkward to have to call two functions whenever I set the orientation. It would also be as tank said a security flaw when someone doesn't understand that they have to set both orthogonally and they would only use one of the setters and ignore the other and things would not work properly.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 04, 2014, 01:24:47 pm
Quote
Then Nexus' argument was wrong, not mine. He said that it's the same with other properties, not me.
I know. I was quoting you but actually answering to both ;)

Quote
In most cases? Personally I don't like programming with a time bomb that works "in most cases". Seriously, why can't all cases be simply fixed? I don't get it.
All cases are fixed, since we give access to the up vector. That was the main point I think.
The design discussion that we have now will only change how easy/intuitive the new API will be. And all I was saying is that there are many cases where we don't need to deal with the up vector; we're not talking about "broken" cases.

Quote
If you have a static +Y up vector, you are only allowed to set the at-vector to any values where Y is 0.
This is called 2D, and is the main target of SFML ;)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 04, 2014, 01:32:43 pm
Quote
If you have a static +Y up vector, you are only allowed to set the at-vector to any values where Y is 0.
This is called 2D, and is the main target of SFML ;)

Okay so you're saying that the 3D audio spatialisation API of SFML is designed in a particular way because 2D is the main target of SFML? But you even said  "in most cases (all 2D and most 3D)" which is still plainly wrong because "most 3D cases" are not DOOM controls. :D

Also I must say that I am personally a bit disappointed since I didn't know that SFML focussed on 2D even in the 3D parts of the API and I would have assumed that it would be fine to use SFML for 3D audio since that is what is says in the documentation/tutorials etc, but I guess if I want to do 3D audio I am better off using something else indeed. Too bad it isn't that apparent though, since people might actually happen to use it for 3D before realising.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Laurent on March 04, 2014, 01:49:58 pm
No... I'm just saying that 2D represents most of the use cases of the audio module (this is easy to understand... most people use the audio API that goes with the graphics API that they chose, and SFML is often chosen for 2D because of its graphics module -- have a look at the Projects forum and tell me how many 3D projects you find), so this consideration must be taken in account when designing the modification. In other words, if I can design something simpler for 2D then most of SFML users will benefit from it and be happy. Whereas 3D users will still be able to achieve what they want. What's wrong with this reasoning?
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 04, 2014, 02:35:43 pm
No... I'm just saying that 2D represents most of the use cases of the audio module (this is easy to understand... most people use the audio API that goes with the graphics API that they chose, and SFML is often chosen for 2D because of its graphics module -- have a look at the Projects forum and tell me how many 3D projects you find), so this consideration must be taken in account when designing the modification. In other words, if I can design something simpler for 2D then most of SFML users will benefit from it and be happy. Whereas 3D users will still be able to achieve what they want. What's wrong with this reasoning?

The thing that is wrong with the reasoning imo is that the sound stuff claims to be a full 3D sound API with 3D sound spatialisations. It never says anything about being 2D specialised, which made at least me think that it was meant to work properly with 3D, but i guess I was wrong.

True you don't see many 3D projects being talked about on the forum. Maybe because newbies cannot do 3D and that more advanced users have realised the various limitations that SFML poses for more advanced use (context management, and until recently the GLEW issue, and similar topics) and moved on to libraries which does things in a way a decent programmer would expect.

I am just speculating here of course,  but more to the point I definitely think that a sound module which is advertised as capable of doing 3D sound spatialisation should do 3D sound spatialisation properly, regardless of if the library aims for 2D or not.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 04, 2014, 02:39:37 pm
therocode, what exactly is wrong? With the up vector, 3D spatialization will be possible. That was your main point of criticism, wasn't it? Or are you just denying that because you don't like the API design?

Maybe this whole thing is a misunderstanding :)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 04, 2014, 02:44:19 pm
therocode, what exactly is wrong? With the up vector, 3D spatialization will be possible. That was your main point of criticism, wasn't it? Or are you just denying that because you don't like the API design?

Maybe this whole thing is a misunderstanding :)

No I am not talking about you guys' recent talk about the API, I just replied to Laurent's posts where he said that most 3D cases use a static Y+ up vector and that the 3D sound API is affected by the 2D focus of SFML which are two things I simply disagree with. :)

As for the API I have stated my opinions already.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 04, 2014, 03:04:08 pm
Laurent's point was that the API with separate methods is simpler for 2D users because they don't need the up vector.

And 3D users can still achieve the whole functionality in a straightforward way. It's not as if the API with a compound would clearly be the better approach even for a 3D sound spatialization library, or that the API with separate methods limits 3D users in any way.

Anyway, I think we can summarize this discussion as follows:
The only question left is the name of the attribute. "UpVector" would be a possibility, but also "Upward(s)" or something completely different like "Overhead" (but a better word for it). Suggestions?
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 04, 2014, 03:15:23 pm
Laurent's point was that the API with separate methods is simpler for 2D users because they don't need the up vector.

Ah right, well that is a more fair point, even though I personally would not take that approach.

Quote
And 3D users can still achieve the whole functionality in a straightforward way. It's not as if the API with a compound would clearly be the better approach even for a 3D sound spatialization library, or that the SFML API limits 3D users in any way.

I would argue that it is for the same reasons as tank mentioned, you wouldn't have a separation of RGB. Object oriented ways teach to make structures of things that are conceptually tied together, and the at-vector and up-vector surely is; the term is "orientation" and that is why you set it as a single property in openAL. There is only SetSourcefv(sourceId, AL_ORIENTATION, eightFloats) and not AL_ATVECTOR and AL_UPVECTOR. That would have cluttered the API.

But true it doesn't limit the 3D users and if SFML would take this approach it would still be usable indeed.

Quote

Anyway, I think we can summarize this discussion as follows:
  • There is a problem whenever up and direction are linearly dependent, in both past and future. It was already possible to avoid the error in the past.

Correction: It is not only a problem when they are linearly dependent, but also when they are not orthogonal with each other, which is in all other calls to setDirection which does not look like this: setDirection(X, 0, Y).

Quote
The only question left is the name of the attribute. "UpVector" would be a possibility, but also "Upward(s)" or something completely different like "Overhead" (but a better word for it). Suggestions?

I still say go with upVector. Everyone knows what the direction of "up" is, you don't need to be into maths to get it. If you can understand what "a vector pointing forwards" is, then you can understand "a vector pointing upwards".
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Foaly on March 04, 2014, 03:31:21 pm
I'm just saying that 2D represents most of the use cases of the audio module

I now this might be a little off topic, but I have to to say that I don't agree with that. I have used the audio module multiple times, without using any other of SFML's modules. It is quiet hard to find a good open source (c++) library for sound. For me SFML provides the cleanest API (compared to SDL, allegro or pure OpenAL) while still being simple (compared to PortAudio) while still having a permiting license (compared to irrKlang). So for me SFML's audio module is pretty much the top of the line (unless I want to do some really freaky stuff, then I use PortAudio).
That is why I think it should be viewed independently of the other SFML parts. Quiet a few SFML users will use the audioo module to simply playback (2D) sound, but there is probably quiet a lot of people like myself, that use the audio module as a standalone "library". After all SFML is a multimedia library.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 04, 2014, 03:31:43 pm
Object oriented ways teach to make structures of things that are conceptually tied together, and the at-vector and up-vector surely is; the term is "orientation" and that is why you set it as a single property in openAL.
Yes, and that's a valid point. What I meant with "not clearly" is that both approaches have their advantages and disadvantages, it's not as if the compound blatantly outperformed separate methods. There are multiple design considerations which can have different weight depending on the library or context where they are used.

It is not only a problem when they are linearly dependent, but also when they are not orthogonal with each other, which is in all other calls to setDirection which does not look like this: setDirection(X, 0, Y).
You mentioned the skewed coordinate system. Out of curiosity, do you know what practical implications this has on sound? Will sounds simply appear in the "wrong" place?

The analogy to the human head sounds straightforward at first, because the front and top of our head point to different directions separated by 90°. But when you only consider the eyes that look at different heights without moving the head, a non-orthogonal system would make sense. I think the eyes are quite a good example: you cannot look straight up or down, so you'll never encounter the linear dependent vectors. Sorry for offtopic ;)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 04, 2014, 03:57:58 pm
Yes, and that's a valid point. What I meant with "not clearly" is that both approaches have their advantages and disadvantages, it's not as if the compound blatantly outperformed separate methods. There are multiple design considerations which can have different weight depending on the library or context where they are used.

Yep this is true which is why it isn't the crucial part of the discussion for me. I could live with such an API even if i myself would have done it differently. The issue at hand with broken spatialisation is more important and as long as that is fixed, it would be enough for me. :)

Quote
You mentioned the skewed coordinate system. Out of curiosity, do you know what practical implications this has on sound? Will sounds simply appear in the "wrong" place?

The analogy to the human head sounds straightforward at first, because the front and top of our head point to different directions separated by 90°. But when you only consider the eyes that look at different heights without moving the head, a non-orthogonal system would make sense. I think the eyes are quite a good example: you cannot look straight up or down, so you'll never encounter the linear dependent vectors. Sorry for offtopic ;)

Yeah, I am not sure how it works on the most detailed level, but as far as I've understood, the "listener" is mererely a coordinate system defined by these three vectors (at, up, right) where "right" is calculated to be orthogonal to the other two using a simple cross product. These vectors are analogous to how a normal coordinate system is defined by the vectors: (1,0,0) (0,1,0) (0,0,1). If you have done 3D rendering, these concepts will be familiar to you, it is how cameras work pretty much. Anyway, sound sources are calculated along this listener-space, which will make them be "oriented" after it. So if the space is skewed, the sounds' positions would end up skewed as well.

I am not sure if I managed to explain it well, but depending on how much the skew is (the skew is greater, the closer in angle these two vectors get) the sounds would be positioned a bit wrong. It would probably not be noticed in most cases since sound locations are hard to verify with 2 speakers, but they would be off none-theless. The equivalent in 3D graphics would be if you tried to render a straight square on a skewed 2D space, it would look something like this on the screen: (http://www.mathatube.com/sitebuilder/images/area_of_a_RHOMBUS-55-418x295.jpg)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 04, 2014, 04:12:53 pm
So what do you think, aren't there cases where such a skewed coordinate system is meaningful (see the eyes example in my last post)?

Even if an orthogonal system is mostly the default and will be encouraged in the SFML documentation, I don't think we should take any measures to prevent or detect non-orthogonal systems, as its utility depends on the application.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 04, 2014, 04:18:04 pm
Well the eye analogy doesn't really work since when you as a human "skew the coordinate system" by looking up with your eyes but not with your head, you don't hear sounds any different than if you would just look straight ahead do you? But in this case if you only tilt the at-vector upwards and keep the up-vector straight up, the sounds will change.

This is an undesired thing in most cases since you want the sound positioning to make sense, but it is true that there should be no measures to prevent it. It is fully legal as far as OpenAL Soft is concerned. The OpenAL specification however says that it "expects the vectors to be orthogonal" whatever that means.

But just to clarify (although you might already have gotten it) the issue right now makes it impossible to not have skewed audio positioning unless you look completely straight ahead like in Doom and such games.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 05, 2014, 08:21:21 am
Quote from: Laurent
This is called 2D, and is the main target of SFML ;)
What about this then?

void setOrientation( sf::Vector2f position );

It's for 2D only (so SFML's scope) and does simply not allow to specify wrong values. If you want to allow 3D spartialisation, then I'd still suggest to force setting the compound of forward and up vectors, like:

void setOrientation( sf::Vector3f forwardVector, sf::Vector3f upVector );

Edit: This is an overload of course!

And, of course, the getters:

sf::Vector3f getForwardVector() const;
sf::Vector3f getUpVector() const;

Personally I would still use a compound data type, but I guess that's out of question already, because you've decided not to use such.

I'm also aware that you will for sure disagree on the fact that there are different setters and getters, i.e. setOrientation() vs. getForwardVector()/getUpVector(). That's why setOrientation() might be renamed to something like orientTo(), making it clear that this is an operation and not setting a property.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 05, 2014, 08:48:38 am
I still think that having an Orientation property should be considered. Even the OpenAL API treats it as a single property, which makes sense since that is how it was meant to be used. So if SFLM decides to _not_ treat it like a property there should be a good reason for separating it imo.

I also had a new idea on how the API could be which is quite different to what has been proposed. My idea is based on the fact that SFML seems to want to have a simple/consistent 2D API but also have 3D spatialisation capability, so why try to come with a weird compromise API instead of actually treating them differently? I got the idea from tank's void setOrientation( sf::Vector2f position ); overload.

What is actually needed for the listener on a 2D scene? Not much. Only positioning (i.e. walking around with the player) and rotating (i.e. the player flips upside down with the whole view). Positioning can be set with a vec2 overloaded setPosition and rotation only sets the angle of the upvector. the at-vector is always looking straight at the scene. This causes proper audio positioning in _all_ cases and there are no cases with undefined behaviour. The functions could be:

void setPosition(sf::Vector2f position);
void set2DRotation(float angle); //this internally sets the at-vector to 0,0,-1 and angles the upvector accordingly

Then for 3D users, there is simply another overload of the setPosition function with a 3D vector, and then a setOrientation(at, up) to set a standard 3D orientation and this one can have an assert for linear dependency. Then for the 3D case, the users get the expected functionality and have full control over the vectors. This group of users will also not be confused by 2 vectors since they are making a 3D application. The orientation function of course mentions which at/up values are forbidden. In other words:

void setPosition(sf::Vector3f position);
void set3DOrientation(sf::Vector3f at, sf::Vetor3f up);

Now there are no weird half-way inbetween functions and the 2D user can use the simple functions and be happy with a safe easy API whilst the 3D users get the expected functionality.

What about backwards compatibility?
Well, that is possible if SFML accepts deprecation. If it does, then the current Listener:: functions can all become deprecated and the new ones can be under Listener2D:: and Listener3D:: which even further emphasises the idea that there are two different ways to go about audio in SFML: 2D and 3D. Which would ease confusion even more, and this would mean that the fix would be fully backwards compatible and SFML could get this issue fixed pretty soon.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 05, 2014, 09:17:24 am
therocode, what about the getters?
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 05, 2014, 09:23:02 am
Ah right. Not sure.  Possibly:

Vector2f getPosition() //in Listener2D::
float getRotation() //in Listener2D::
Vector3f getPosition() //in Listener3D::
Vector3f getAtVector() //in Listener3D::
Vector3f getUpVector() //in Listener3D::

And if that is too inconsistent, having 2 getters for one setter, I would again propose having a struct Orientation for the 3D case, leaving the 2D case as proposed.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 05, 2014, 11:37:07 am
Using a listener for 2D space is not as simple as setting it to a position in the z=0 plane. You need 3D spatialization even for 2D games. See this article (http://blog.tbam.com.ar/2009/05/sound-spatiaiization-for-2d-games-in.html) for more information.

I don't like having two classes if one does its job perfectly without a bloated interface or too many responsibilities, it only complicates the API. Furthermore, it introduces issues because there is only a single global listener.

SFML should provide a simple API that provides enough functionality for complex scenarios, and this works well with the new API. Users that need specific features (such as a simplified 2D sound spatializer front-end) can write their own abstractions on top of SFML. It might even be a candidate for a future version of Thor.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: therocode on March 05, 2014, 12:27:42 pm
Using a listener for 2D space is not as simple as setting it to a position in the z=0 plane. You need 3D spatialization even for 2D games. See this article (http://blog.tbam.com.ar/2009/05/sound-spatiaiization-for-2d-games-in.html) for more information.

The article only mentions that an additional parameter depth is needed. He always orients the listener to look right at the scene with the at vector being (0,0,1) so real orientation is not needed. However it would indeed be good to in that case add a depth parameter which imo would still make sense for the API.

Quote
I don't like having two classes if one does its job perfectly without a bloated interface or too many responsibilities, it only complicates the API. Furthermore, it introduces issues because there is only a single global listener.
Sure if it does its job perfectly without a bloated interface but it doesn't right now. The listener is broken due to the issue this thread is about, and IMO your suggested API which tries to mix the 3D and 2D case by using 3D input but still focussing on 2D-friendliness is not the way to go to have a clean design.

True with issues with a global listener but that is another issue in itself. I would personally scrap the global listener (and a lot of other global things in SFML like the problematic global rendering context) and make a listener class that can be set to be active, but this would need a lot of rework in many parts of the API/style of SFML. This won't of course happen.

This is also why I find it hard to partake in a discussion about an API with you guys, because many parts of SFML are designed with a "user friendly and simple" approach instead of approaches that I would take, so there are fundamental disagreements.

Quote
SFML should provide a simple API that provides enough functionality for complex scenarios, and this works well with the new API.
This is what imo my API proposal does as well.

Quote
Users that need specific features (such as a simplified 2D sound spatializer front-end) can write their own abstractions on top of SFML. It might even be a candidate for a future version of Thor.

So you are saying that it should _not_ have simplified 2D features now? I thought the whole deal with the standard 3D based API proposed before was that SFML focusses mainly on 2D and should hence aim to make it simple for 2D users?

But anyway, I am getting tired of discussing this and I don't think I can contribute to the discussion anyway due to aforementioned disagreements in how to design an API. Best of luck in fixing!
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 05, 2014, 12:28:03 pm
Quote from: Nexus
Using a listener for 2D space is not as simple as setting it to a position in the z=0 plane. You need 3D spatialization even for 2D games. See this article for more information.
Just like it's not as simple as hardcoding the up vector to {0, 1, 0}. Are you seriously linking to an article that states the following as a requirement in order to understand what's being taught?

Quote
Have basic algebra notions (3D, what a vector is, right hand rule...).

And that also explains the forward and up vectors? May I remind you that it's what you described as being "too complicated for newbies" in the first place?

Quote from: Nexus
I don't like having two classes if one does its job perfectly without a bloated interface or too many responsibilities, it only complicates the API. Furthermore, it introduces issues because there is only a single global listener.
I kinda agree that the API would be bloated in that case, because one listener can easily handle both the "easy" and "complex" cases. The single global listener (just like the global OGL context), while that's another topic, is an issue in itself however. Maybe I'll open a thread for discussing that at some time, not sure though as I don't expect it to turn out in any productive way. ;)

Quote from: Nexus
SFML should provide a simple API that provides enough functionality for complex scenarios, and this works well with the new API.
It still allows undefined behavior and doesn't make clear that you have to call two functions in order to make 3D spatialization work properly (documenting it is all good, but in this case we indeed have the option to solve this issue in code).

Quote from: Nexus
Users that need specific features (such as a simplified 2D sound spatializer front-end) can write their own abstractions on top of SFML. It might even be a candidate for a future version of Thor.
I'm really confused... Aren't those "specific 2D features" exactly what SFML has been doing until, well, now? You even argued that you don't want an OpenAL wrapper, but something that fits into SFML's 2D scope. At least that's what Laurent said. And now you argue the exact other way around, i.e. that all complex scenarios are possible, and that 2D stuff can be added on top..?

Spatialisation that's limited to 2 axes is probably what "most SFML games" need. They don't have depth -- hence 2D.. I'm just saying this because it's what has driven me to provide a Vector2f overload.

Could you please exactly state what you want to achieve now? Your opinions have changed during this thread massively, and I'm unsure what you are after right now.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 05, 2014, 01:11:51 pm
Quote
So you are saying that it should _not_ have simplified 2D features now? I thought the whole deal with the standard 3D based API proposed before was that SFML focusses mainly on 2D and should hence aim to make it simple for 2D users?
It should not have an full-featured 2D listener as described in the article I linked, with all the distance and depth stuff in the background. It should only provide a minimal API that allows to achieve sound spatialization -- 2D and 3D.

Are you seriously linking to an article that states the following as a requirement in order to understand what's being taught?
Quote
Have basic algebra notions (3D, what a vector is, right hand rule...).
And that also explains the forward and up vectors? May I remind you that it's what you described as being "too complicated for newbies" in the first place?
The article explains why a z=0 listener is a bad idea and why a Listener2D won't work as easily as one might imagine. It was an answer to therocode's suggestion and has nothing to do with "too complicated", because it's not what SFML will provide in any way. Please don't take sentences out of context; furthermore I wasn't the one who wrote it would be too complicated for beginners.

I kinda agree that the API would be bloated in that case, because one listener can easily handle both the "easy" and "complex" cases.
Just like when the up vector is implemented -- you can achieve everything, both easy and complex cases.

And it would be nice if we could avoid that aggressive tone. I always get the feeling that the motif is not progress in SFML, but showing as much things as possible that have been done wrong -- especially since nobody was directly concerned by the original problem, but suddenly everyone used it as an opportunity to criticize SFML in general. I really like discussing about API design, but please keep it objective and focused (flaws in other parts of SFML don't belong into this thread, nor do they make arguments any better). Thank you :)
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Tank on March 05, 2014, 02:52:57 pm
Quote from: Nexus
It should not have an full-featured 2D listener as described in the article I linked, with all the distance and depth stuff in the background. It should only provide a minimal API that allows to achieve sound spatialization -- 2D and 3D.
Good, then we have the same goal in mind. The only difference then is your 2 functions compared to one function.

Quote from: Nexus
furthermore I wasn't the one who wrote it would be too complicated for beginners.
True.

Quote from: Nexus
Just like when the up vector is implemented -- you can achieve everything, both easy and complex cases.
That's what I suggested in my first post(!) in this thread. My only additional request is a function that makes it clear that the orientation is built from two vectors, not one, and that error checking is done. (I proposed several approaches: two arguments, a compound, default argument.)

Quote from: Nexus
And it would be nice if we could avoid that aggressive tone. I always get the feeling that the motif is not progress in SFML, but showing as much things as possible that have been done wrong -- especially since nobody was directly concerned by the original problem, but suddenly everyone used it as an opportunity to criticize SFML in general. I really like discussing about API design, but please keep it objective and focused (flaws in other parts of SFML don't belong into this thread, nor do they make arguments any better). Thank you :)
Sorry if I sound aggressive -- I can assure you I am not, I'm sitting in a nice chair, in front of a screen while the sun is touching my face -- I'm relaxed. But I don't see why I should step away from my opinion or principles when I want to bring them into the solution? You do that as well, and even some sentences of you sound sarcastic as well, but that's alright.

The motif for me has always been to get this bug out of the library and replace it with the best solution. When I recognize the well-known SFML-ish reaction to this, then yes, I will criticize exactly that. There have been similar discussions in the past years. Discussions about requests that have been rejected in the first place but implemented in one of the later versions. So I would find it nice if, before rejecting things, either by saying "too complicated", "out of scope" or "nobody complained so far", you could consider a change.

Especially in requests as this one, where a bug has been found. I really can't and will never accept the fact that a bug is kept just because nobody has complained so far, and that a bug itself is not a valid reason to change an API. I'm so much against that, does that explain my attitude? I'm too pragmatic to ignore that.

Feel free to ignore my posts however, if you think they don't help with the discussion or are not productive at all. It's okay. But I won't stop to use "uncomfortable" arguments if I think they fit. None of them are meant to be personal.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: Nexus on March 05, 2014, 03:29:20 pm
Okay, then it's fine :)

I'm not uncomfortable with the arguments, as stated earlier I do see the advantage of a compound (namely that it forces the user to think about up vector as well when setting the direction), but there are also other considerations (expressivity of code for example). I'm sure that if we implemented a different API there would be another user complaining ;)
It's always a trade-off between different designs.

Concerning the "out of scope" or "too complicated" reaction: I can't say too much about this since I haven't been in charge of SFML for a long time, but there are indeed many features that are completely off (the popular ones being a 3D engine, GUI library and filesystem). On the other hand, there are also a lot of feature requests that actually improve the library, and if you look at GitHub, you'll see that I'm currently negotiating several pull requests. The "too complicated" argument is not always a definitive and absolute statement; as you noticed, some features were still implemented later. I think this has to do with two points: ensuring that there is an actual need and convincing arguments before simply accepting a feature, in order to ensure a high-quality API; as well as setting its priority with respect to other open issues -- there are tons of bug reports and feature requests pending, but often people who request a new feature only have their specific use case in mind. However I agree that especially for bugs we can't just say "it's out of scope" and not consider a fix at all, but here there will be a change. Apart from that, in my opinion, the listener up vector rather a feature than a bugfix. The problem of linearly dependent vectors exists in the past and future, and it can be avoided in both. The difference is that in the future, the user has more freedom to set a coordinate system.

...this is at least my interpretation of the situation.
Title: Re: Possible bug in the Audio Module with listeners?
Post by: wintertime on March 05, 2014, 09:55:21 pm
The article explains why a z=0 listener is a bad idea and why a Listener2D won't work as easily as one might imagine. It was an answer to therocode's suggestion and has nothing to do with "too complicated", because it's not what SFML will provide in any way. Please don't take sentences out of context; furthermore I wasn't the one who wrote it would be too complicated for beginners.
I would think when implementing a 2D listener one would automatically map the 2D vector to x and z, if the up vector is still hardcoded to (x,y,z)=(0,1,0), to make it possible to position it anywhere on the y=0 plane.


Btw., if you dont want to change the interface atm, I think fixing up weird inputs in a way similar to this might be a good idea:
void Listener::setDirection(float x, float y, float z)
{
    priv::ensureALInit();

    y=0.0f;
    if(x==0.0f && z==0.0f)
        z=-1.0f;
    float orientation[] = {x, y, z, 0.f, 1.f, 0.f};
    alCheck(alListenerfv(AL_ORIENTATION, orientation));
}
 
Title: Re: Bug in the Audio Module with listeners?
Post by: eXpl0it3r on March 08, 2014, 10:23:15 pm
So for all the people saying "nobody complained" and "the main purpose is 2D" here you go:

Quote from: Twitter - kooschi (https://twitter.com/koonschi/status/442326267728519169)
I can't be the only one being annoyed, am I? I'm talking about the Listener up vector in #sfml always being (0, 1, 0). #programming #gamedev

Quote from: Twitter - kooschi (https://twitter.com/koonschi)
For the next version I'll be dropping SFML audio, and writing my own sound lib again. No real 3D sound is simply a no-go.
...
On the other hand, it offers so much utility. Argh I don't know what to do.
...
I think I'll just make a patch and hope he'll accept it. That's much easier than writing my own sound stuff.

And I can kind of agree with this:
Quote from: Twitter - kooschi (https://twitter.com/koonschi/status/442327541622513664)
I really have to say that I don't like the maintainer's attitude about this. It was requested multiple times but he refused to add it.

Let's see what happens, because I doubt he will have the possibility to just wait for SFML 3 to come around... in 2 years or so...
Title: Re: Bug in the Audio Module with listeners?
Post by: Tank on March 08, 2014, 10:46:43 pm
Well there's still the chance of pulling out a minor version. Git branching is really easy and useful for exactly such things.
Title: Re: Bug in the Audio Module with listeners?
Post by: therocode on March 09, 2014, 04:08:48 am
That tweet.

Given the issue in the audio module I am not surprised at all. And I also agree that this shouldn't have been an issue in the first place. But oh well.
Title: Re: Bug in the Audio Module with listeners?
Post by: Laurent on March 09, 2014, 09:41:04 am
I think you can stop bashing SFML and myself. A new feature (https://github.com/SFML/SFML/issues/545) has been scheduled.
Title: Re: Bug in the Audio Module with listeners?
Post by: Tank on March 10, 2014, 09:10:29 am
Thank you!