Bug in the Audio Module with listeners?

[therocode]

To implement a new feature I would say there need to be concrete use cases. I'm not saying there are none, but there have still not been any arguments in that direction.

You are talking as if it is a feature request as in extending what is possible with SFML and even if that is technically right (since making it possible to change the up-vector is a new feature) that is not the main point. It is not a feature request, it is addressing an existing problem with the library which is fact. I don't think there should be a need for "specific use cases" to fix a problem that an arbitrary magic number in a function breaks the application without good reason.

It is like saying "oh the add(a, b) function doesn't work with the numbers add(3453, 2344)? well i haven't seen a use case for those particular numbers so I won't fix it." imo.

Some are making a push for changing something

I don't see anybody here who needs this feature. Even the OP said it was just for curiosity.

Again, it is not a feature request, it is a problem that is fact. And yes I asked out of curiousity, not because I need it because I am not making a 3D application in SFML. But know that if I would make a 3D application I would be wary about using SFML since with this issue, I could get random crashes if I turn the camera in a funny angle.

But even if I ask out of curiousity and that it is up to you of course, I am still very much for the opinion that to go about it properly is to fix it.

[Tank]

I think this quote summarizes the problem well enough:

It is like saying "oh the add(a, b) function doesn't work with the numbers add(3453, 2344)? well i haven't seen a use case for those particular numbers so I won't fix it."

But I will still add some more words. ;-)

From an design standpoint, it makes much more sense to start with a minimal API

I fully agree. But this time it's a broken API. This is not a feature request.

but contribute to an confusing and cluttered API.

A "clean" and "not cluttered" API which is broken is not useful however. And it's not like adding "upVector" to the list of arguments with a description similar to "Vector pointing upwards from the listener" is rocket science. Seriously, if someone does not understand but needs that (-> 3D), then he will probably simply go and learn it -- if not, then I'm not sure if "saving" the people from learning the stuff, which is *required* to make things work, is the way to go.

I sometimes have the impression that the "S" in "SFML" does not mean "Simple" but "In no way try to confuSe the people who are programming for their firSt time in life and are not willing to read about Stuff that they haven't diScovered before and that'S not explained in the tutorialS" (I've capitalized all S's because I wasn't sure which one shall represent the SFML's "S").

I agree that it's a good reason to add at least an error check

An error check is needed, yes, but it seems weird to not have a chance to avoid the error case. SFML forces me to risk undefined behavior, and that's known. I call that a no-go.

So is anybody of you actually requring that feature, or is the main point of this discussion to criticize the way how feature requests are handled in SFML? I mean that's also a valid discussion, but then keep it separate from the audio up vector.

Being a member of this community for a couple of years makes me a bit biased of course, because the numbered list I posted in a previous post is how I see it. But this is really about the audio up vector issue. Experience showed that with enough pressure things can change.

[select_this]

Extrapolating from the discussion above, a possible use case would be using the audio listener in a 3D application where the listener's attached to something freely rotatable on the three axes. I could see that happening in first person games quite easily, e.g. someone moving the camera to look along the Y axis. You might even want to roll the 3D listener on all three axes for something like a space combat simulator.

[therocode]

Extrapolating from the discussion above, a possible use case would be using the audio listener in a 3D application where the listener's attached to something freely rotatable on the three axes. I could see that happening in first person games quite easily, e.g. someone moving the camera to look along the Y axis. You might even want to roll the 3D listener on all three axes for something like a space combat simulator.

Yes that is a perfectly valid use case. Also observe that in FPS games, the camera is often capped so that you can't look more upwards than perfectly straight up and no further down than straight down. Both of those capped orientations would cause undefined behaviour if the listener is set to follow the orientation of the player.

Also, this person making this 3D game would not think an extra up-vector argument is hard to understand.

[select_this]

Also observe that in FPS games, the camera is often capped so that you can't look more upwards than perfectly straight up and no further down than straight down. Both of those capped orientations would cause undefined behaviour if the listener is set to follow the orientation of the player.

To quote Laurent from a previous discussion on this topic:

Absolutely. But do you mean I should change the global world up vector just because you're looking perfectly up? Even in real life, you can't look up 100%. And 99.9% will just work fine.
To me it's ok to have the up vector hard-coded, this is like a physical constant (just like gravity for example).
Besides, I don't understand your second graphical example (but this is probably just because I just woke up ).

Here's the topic in question:

http://en.sfml-dev.org/forums/index.php?topic=1225.0

I would argue that the fully rotatable space combat simulator example still stands, though.

[Nexus]

It is not a feature request, it is addressing an existing problem with the library which is fact. I don't think there should be a need for "specific use cases" to fix a problem that an arbitrary magic number in a function breaks the application without good reason.

The "fix" would be an error detection and better documentation. In fact, letting the user set the up vector doesn't fully solve the issue. There is still an error when up and direction are linearly dependent, there would still be a need to both check for it and mention it in the documentation. What's different is that the user can set the up vector to avoid this problem. But, the up vector is not necessary to avoid this problem, that's why adding it because of the problem doesn't make sense.

The main reason to set the up vector is not to avoid this specific problem, but rather to implement a specific coordinate system for the listener. You're argumenting from a workaround perspective, and API design should not be driven by workarounds. ghostmemory's argument is better in this respect: he shows a scenario where the up vector is useful from a functionality standpoint.

[select_this]

Out of interest, and apropos of this topic (thought slightly off-topic, admittedly), is there a standard solution for when vectors reach a point of linear dependency? I would've thought the simplest solution would be to just adjust the vectors slightly but I imagine in some situations that could produce noticeable artefacts. I must admit I've never given it thought before.

[binary1248]

Out of interest, and apropos of this topic (thought slightly off-topic, admittedly), is there a standard solution for when vectors reach a point of linear dependency? I would've thought the simplest solution would be to just adjust the vectors slightly but I imagine in some situations that could produce noticeable artefacts. I must admit I've never given it thought before.

http://en.wikipedia.org/wiki/Gimbal_Lock
http://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation

Welcome to the wonderful world of Computer Graphics.

[select_this]

http://en.wikipedia.org/wiki/Gimbal_Lock
http://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation

Welcome to the wonderful world of Computer Graphics.

Thank you, these are really informative (and reaffirms my belief that I'm better off leaving these sorts of things for those mad folk who love writing abstractions!)

[therocode]

Alright, I have done some research and as I suspected, this whole issue with a hard coded up-vector is far more problematic than I first thought; it is not just a matter of code safety, but the whole feature of orienting the listener is broken. Let me explain.

What made me thinking is that why is the up-vector there in OpenAL in the first place? Maybe it is actually used for something? And if you think about it, of course it is. A sole vector pointing where you look is not enough to describe orientation. Consider if you are looking forward and you have one blue fire to the front-left and one orange fire to the front-right like so (pardon my bad drawing skills ):



The red arrow is the at-vector, the green is the up-vector.

Frame1: this is a first person view, the red at-vector is going straight forward and the up-vector is going straight up
Frame2: this is a top down view of the same scene. The player is looking straight at the fires
Frame3a: this is a side view of the same scene

In all those frames, the sound of the blue fire is heard through the left speaker and the yellow fire is heard through the right. This is all good. However, now consider that the player turns around with his back towards the fires and bends his neck backwards like crazy so that he looks at the fires straight, but is upside down. This would be what the Frame3b picture shows. The at-vector is still at the fires but the up-vector is now pointing downwards. And if you imagine doing this, the blue fire should now be heard through the right speaker and the yellow should be heard through the left. This is not possible in SFML since the up-vector is hard coded.

This might seem like a longshot situation but if you mean to develop applications with free moving 3D cameras, then it isn't really. Even the space simulator example from before would be enough. If you go forward, and then make a half-circle loop upwards to go the other way upside down, all of a sudden all sounds would be inverted because the up-vector is pointing in the wrong direction! Not good.

Furthermore, it is even more problematic, because as the OpenAL specification says, it expects the vectors to be orthogonal. i.e. having only right angles. Not having orthogonal vectors is not undefined behaviour however, but it creates unintended behaviour: A skewed space for the sound calculations. I verified this by asking in the #OpenAL channel:
http://thunked.org/p/view/pub/1xjz28abo

And I looked in the OpenAL source code to verify this:
http://thunked.org/p/view/pub/mjljisdt9

This is verifies what the person in the channel said: The up-vector and at-vector are used to calculate an orthogonal right-vector and these three vectors are then normalised and used to construct the listener-space coordinate system which is used for calculations with the sources sounds.

In other words if the vectors are not orthogonal, the calculations will be off due to the skewed space. In practice this means that the SFML 3D audio spatialisation only works as intended when the listener look-at is perfectly straight ahead. If the player looks up or down in an FPS game for instance, the audio calculations are skewed accordingly.

This imo is a major flaw of the 3D spatialisation. If you would use SFML for anything but the most basic spatialisation tasks, the result will be off. What is even worse is that it is not even mentioned in the documentation which could cause a lot of headaches for people trying to use it and wondering why sounds are not behaving correctly.

All of these issues are in the library apparently because it is too difficult for newbie programmers to understand that proper 3D orientation requires at least 2 orthogonal vectors. This is to me absurd, and the argument is voidified by the fact that newbies would not even make 3D applications in the first place. And starting out with a small API to avoid clutter and add features later etc is surely fine and all, but this feature doesn't even work properly without two vectors which is why that argument doesn't even apply here either.

If you still after this think that this issue is not worth fixing (it can even be done without breaking the API by adding a default up-vector argument like tank said), then at the very least explain in the documentation that most values you give to the setDirection() function will produce skewed and flawed output and that (0, 1, 0) will totally break it. That way people who want to have proper audio spatialisation can look for another library to use.

[Laurent]

Thanks for the detailed research

Don't get me wrong, I know exactly what the up vector is for, and what limitations the current API has. My main argument is not that it is unfriendly for beginners, it's just that in practice nobody ever complained about this so that goes directly to the bottom of my todo-list. But if you implement it with a nice API, and submit a pull request, there's a chance that it doesn't get forgotten

(the extra argument with default value is not a solution, because with need the corresponding getter anyway)

[therocode]

Yeah, well I could implement it for you but the problem is that we surely have different definitions of nice API. I would just go with setOrientation(atVector, upVector) (or even until 3.x to not break the api, i would do setDirection(atVector, upVector = (0,1,0)) and add a corresponding getUpVector() getter.

Your priorities are surely your business and I won't try to meddle with them, but I must say that I am surprised that fixing it or at least adding the info to the documentation is not a high prio since it doesn't involve much coding/writing and as long as it is not done, leaves the 3D spatialisation feature pretty broken imo. But you're the boss.

edit:
Btw, curious question: What was the idea with the setDirection() function in the first place if you know all the limitations with it? What is a proper use case that would not suffer from the skewed space problem?

[Tank]

My main argument is not that it is unfriendly for beginners, it's just that in practice nobody ever complained about this

This sounds like a bug is "okay" as long as nobody seriously trapped into it. Personally I would try to fix it ASAP as long as it hasn't caused damage to something. And I think purely wrapping the forward and up vectors in the call is absolutely fine (unless you want to go to a even higher "newbie-friendly" level and add a camera to SFML or something to abstract the "complicated" vector algebra away).

[Laurent]

Your priorities are surely your business and I won't try to meddle with them, but I must say that I am surprised that fixing it or at least adding the info to the documentation is not a high prio since it doesn't involve much coding/writing and as long as it is not done, leaves the 3D spatialisation feature pretty broken imo. But you're the boss.

You know, when you don't even have the time to write a single line of code, you start to adopt a very pragmatic point of view. Sure it is broken and in a perfect world it would be changed, but since nobody is actually waiting for it, I prefer not to waste too much time on it and focus on things that have been pending for a long time and that are awaited by many users. You are focused in this particular issue right now, and might think that it very important, but don't forget that I have hundreds to handle, and this one is a drop in the ocean of tasks.

Btw, curious question: What was the idea with the setDirection() function in the first place if you know all the limitations with it? What is a proper use case that would not suffer from the skewed space problem?

Again, pragmatism. And you may disagree, but I think this was the right choice since so far everybody can use the audio API happily. Sure in the future this may not be true anymore, but future is the future, there are already too many things to handle in the present.

This sounds like a bug is "okay" as long as nobody seriously trapped into it.

I would just replace "okay" with "low priority", and yes, as long as there are other more important tasks to work on, this is true.

Personally I would try to fix it ASAP as long as it hasn't caused damage to something.

Personally I'd love to be able to do anything on SFML again

And I think purely wrapping the forward and up vectors in the call is absolutely fine (unless you want to go to a even higher "newbie-friendly" level and add a camera to SFML or something to abstract the "complicated" vector algebra away).

To be consistent with the other "properties" of the Listener class, it would have to be implemented with its own pair of getter/setter. And maybe we can check if there would be a more friendly name than "up vector", which is a purely mathematical description of it and not connected to audio/listener semantic field.

[eXpl0it3r]

So I guess it has come to this: