Z ordering

[Tank]

Hello guys,

I recently faced the lack of a z ordering mechanism in SFML and want to start a little discussion about

• the need of such a feature and
• if it would be still in SFML's scope.

A brief overview of the problem

SFML's graphics module is written for 2D graphics. Although using an OpenGL backend, which is per se a 3D graphics API, it uses 2D vectors for all its high-level features.

One difference between 3D and 2D is that using the third dimension (depth), objects can be in front or behind other objects. That effect is also widely used in 2D applications/games to suggerate depth (e.g. the player in front of a fancy mountain parallax background).

To make objects appear in front of others, SFML makes use of overdrawing: Whatever is rendered last will overdraw what has been rendered before. That, in turn, means that you have to do the depth ordering (or z ordering) yourself, by taking care of the order you render your drawable things.

OpenGL (and other APIs) however offers a so called depth buffer, which will give exactly that task to the graphics processing unit. There are some special cases where you still have to take care of what you send first, but generally you don't have to pay that much attention anymore. The whole process gets easier and also (theoretically) faster.

My question is: Are there users (besides Nexus ) who already missed that feature badly? Laurent, would you consider z ordering to be in SFML's scope at all? If not, what can we do to externally extend SFML in a way to support that? (e.g. in extension libraries like Thor, which already has plenty of neat additional SFML candy)

[ZackTheHuman]

I would find such a feature extremely valuable for the kind of games I'm working on. Right now I am doing manual "z-ordering" but it would be nice if the GPU could handle it for me.

[Cornstalks]

I requested a depth axis once, only to realize depth testing is completely useless once you start using textures that have transparency, which make up the majority of my sprites (and many other peoples').

I think too many people use alpha blending too often to make this feature very useful. I imagine it would require a substantial change for little net gain.

[Tank]

Well, translucent stuff just needs to be rendered front-to-back, that's it. That's what I meant with "special cases". But those could be of course considered, too.

[binary1248]

I can't believe you forgot SFGUI, back then when we still used SFML for rendering . It was so cool having to sort widgets back to front before flushing the RenderQueues every frame. I even wrote a special bubblesort implementation for that. If SFML had real depth support it would make life easier for me when I write the SFML VA renderer... some time in the future .

The whole process gets easier and also (theoretically) faster.

Depends on what you consider "easier". Sure for people who understand how the depth buffer works, they just specify an extra coordinate and thats it. But for beginners in graphics programming, who Laurent holds so dear, the "painter's algorithm" way of doing things is more intuitive and hence easier to learn. If this becomes a feature then at least make it optional and off by default, so that the learning curve doesn't become too horrible.

(theoretically) faster: in theory it only makes things faster if you already had a high degree of overdraw before turning depth testing on. Because the render pipeline can discard fragments earlier if it notices that geometry exists with a higher depth value than the fragment it currently wants to render, it just saves the fragment shader and buffer op steps of the pipeline. This is so unpredictable because early depth test implementation differ so much, you really can't tell if depth testing will even make things faster in all cases. In some it might even slow things down because e.g. you have an extra buffer to clear every frame etc. You need to conduct application specific testing to determine whether depth testing is worth it on a case by case basis. Another reason to make this optional.

http://www.opengl.org/wiki/Early_Depth_Test

[Laurent]

I'm not against this feature. And it can definitely not be implemented externally. Using a sorted list of drawables could work, but it's not the best solution. And it doesn't solve your initial problem: having different depth values within the same entity (the particle system's vertex array).

But there are negative points (most important last):

1. Depth buffer may not always be available: today's GPUs all have it, but I don't know, maybe a crappy driver or weird configuration can lead to a lack of depth buffer

2. What if user disables it in its sf::ContextSettings? SFML would not be able to work correctly. And if it changes it to 8 or 24 bits per pixel instead of 32? The valid range for Z values is changed, and with 8 bits is becomes really low (only 256 Z values available).

3. While it's ok for high-level entities to have a 'z-index' property that may or may not be used, sf::Vertex would have a sf::Vector3 for its position, even if you don't use depth. It's not a big deal but it's less convenient when your app has sf::Vector2 everywhere else.

4. The default behaviour (i.e. not using depth) would result in all vertices having the same Z component. Which leads to the well known problem of Z-fighting. The only solution would be to disable depth testing/writing for entities that don't use depth, but it makes the API even more complicated since you have two properties: z-enabled and z-index. And to be applicable at low-level (with vertex arrays), I'd have to add it to the render states as well.

5. Transparent entities. This is a major problem, since most sprites will have transparency, as already mentioned by Cornstalks. I wouldn't call this a "special case", but rather the common case. I expect many messages on the forum about this, if we do it

If you have an answer to all these problems, I'm ready to work on it

[binary1248]

1. Provide something like sf::isDepthBufferAvailable() or something similar to how you check for shader support.

2. If the depth buffer is disabled in the sf::ContextSettings, just don't enable depth testing. Nothing will happen with the third coordinate then, if you are using an orthographic projection. Your conception of the depth buffer precision is also not correct. You assume there is a direct uniform mapping from our values to the "true" internal values that the GPU uses during rendering. If you read this, they explain that it is in fact a non-uniform mapping from your values to whatever values are available from the range that the size of the depth buffer provides. At smaller buffer sizes, the chances of Z-fighting is higher as a consequence.

3. Provide a "copy constructor" that initializes an sf::Vector3 from an sf::Vector2 and sets the z coordinate to 0. For the performance freaks like me, this can be even done using a move when applicable.

4. Like I said in my previous post, make it optional so that the people who have a clue what they are doing can solve this problem on their own. If you want to use the depth buffer, you will have to make semantic changes to your application to deal with this problem and it isn't the responsibility of SFML to make sure the values that are specified are sane. Garbage In Garbage Out.

5. Same argument as 4. People have to give it a bit more thought instead of just doing sf::enableDepth( true ) and expecting the application to behave like what they had in mind instantly from the beginning. SFML is a mid-level API and shouldn't try to do too much thinking for the user. They should also have a bit of code to solve their specific problems.

[Laurent]

1. Provide something like sf::isDepthBufferAvailable() or something similar to how you check for shader support.

It will be like sf::RenderTexture: the (crappy Intel) driver says it can handle it, but you just see garbage on your screen
But ok, let's say that this point won't cause any problem.

2. If the depth buffer is disabled in the sf::ContextSettings, just don't enable depth testing. Nothing will happen with the third coordinate then, if you are using an orthographic projection.

That's what I'd like to avoid. By changing an indirect property, which is normally only for your own OpenGL rendering, you disable a major feature of SFML.
But, again, let's say it's ok...

Your conception of the depth buffer precision is also not correct.

I know Z values are not mapped uniformly, but with 8 bits the number of available Z values will be ridiculously low anyway, regardless how they are mapped to those 8 bits (ok, I shouldn't have said "256").

3. Provide a "copy constructor" that initializes an sf::Vector3 from an sf::Vector2 and sets the z coordinate to 0.

I don't like this kind of implicit conversions. And making it explicit won't be less verbose than the regular constructor.

4. Like I said in my previous post, make it optional so that the people who have a clue what they are doing can solve this problem on their own. If you want to use the depth buffer, you will have to make semantic changes to your application to deal with this problem and it isn't the responsibility of SFML to make sure the values that are specified are sane. Garbage In Garbage Out.

I'd like to make depth testing optional, but implicitly: assign a depth value or leave the default one if you don't care. Any solution that makes activation of depth testing explicit is already too bloated.
At this point I'd like to see proposals for a clean and simple API. Otherwise I'll hardly be convinced

5. Same argument as 4. People have to give it a bit more thought instead of just doing sf::enableDepth( true ) and expecting the application to behave like what they had in mind instantly from the beginning.

This is not my philosophy. I'd like things to work out of the box, in all cases, and as expected as soon as you activate them. If users need to figure out a specific context with certain conditions to have it working as expected, it's already too complicated. Especially for this feature, which is really basic and simple (as opposed to shaders, for example).
SFML would just be FML if I just implemented a thin wrapper on top of OpenGL without dealing with the annoying rendering issues.

[binary1248]

This is not my philosophy. I'd like things to work out of the box, in all cases, and as expected as soon as you activate them. If users need to figure out a specific context with certain conditions to have it working as expected, it's already too complicated. Especially for this feature, which is really basic and simple (as opposed to shaders, for example).
SFML would just be FML if I just implemented a thin wrapper on top of OpenGL without dealing with the annoying rendering issues.

Well, good luck then. Depth testing wasn't around from the start of computer graphics and can be considered an "advanced" technique. That being said, I don't see any way to provide transparent support for depth testing without jumping through loops.

As an example, initially, I wanted to optimize the new SFGUI renderer to make use of depth testing because of the high levels of overdraw that was occurring. This meant that I had to create a workaround for transparent GUI themes because the blending got messed up. So I detected whether any blending had to be performed (by explicitly checking for an alpha value < 255 or > 0) and disabled depth testing for the whole process if even one primitive had to be rendered with blending. The fun part really started there... I noticed somehow the fonts started looking weird. After browsing through SFML code I noticed that the font textures needed to be blended into the background because that's the way you implemented it with FreeType smoothing activated as opposed to simple masking. After tens of hours of trying to work around that I just gave up and disabled depth testing because it broke too much stuff. There was simply no way to do it transparently because of the nature of the concept. The code is still in the renderer but has been deactivated for a very long time .

I think that will be the case with SFML as well. If the user does not consciously make use of depth testing there will always be cases where something will get broken. There are probably ways to work around this, but I'm sure you are less interested in spending a significant amount of time on something that could be considered a "little optimisation".

[Laurent]

Your experience confirms that enabling depth testing in SFML is probably not a good idea. It feels like it will cause more problems than it solves.

[Tank]

Your experience confirms that enabling depth testing in SFML is probably not a good idea. It feels like it will cause more problems than it solves.

I really don't agree with that. I'm doing a really simple game and already have to think about using a custom OpenGL renderer instead of SFML's graphics module because there's no easy way to do my z ordering. As long as I'm in control of everything I can implement my own, but when using external libraries (like Thor's particle system), I'm at a dead end.

The depth buffer is definitely not something I'd recommend for a beginner, that's why it should be disabled by default. But not having the option to enable it makes me feel quite uncomfortable, because it's such an elemental feature of rendering in general.

What about moving the z component to a new render state?

window.draw( stuff, sf::Depth( 1234.0f ) );

SFML could enable depth testing when it detects such a state automagically, thus removing the explicit part of it.

Regarding the "What if the user chooses 8 bpps?": Your own fault. It's like being surprised that colors look wonky when setting a low color depth.

[foobarbaz]

Probably a bit late to the party, but I'd definitely be interested in this feature. In my game engine, I handle it with this code:

void Scene::render(sf::RenderTarget *target, sf::RenderStates states)
{
    int topLayer = 0;
    for (unsigned int o = 0; o < mGameObjects.size(); o++)
        topLayer = std::max(topLayer, mGameObjects[o]->getRenderLayer());


    for (int l = 0; l <= topLayer; l++)
    {
        for (unsigned int o = 0; o < mGameObjects.size(); o++)
        {
            if (mGameObjects[o]->getRenderLayer() == l)
                mGameObjects[o]->onRender(target, states);
        }
    }
}
 

But if there's some legendary, mega-fast OpenGL depth buffer then that sounds awesome!

EDIT: I agree with Tank, the particle systems example hits home for me. I haven't even started thinking about particles yet, but I'll have to do my own custom particle engine to make the depth work with my game engine's render layers.

Double edit: Although, I suppose I could just call the render function for whatever external library wherever I want, so maybe it isn't such an issue.

[Tank]

Although, I suppose I could just call the render function for whatever external library wherever I want, so maybe it isn't such an issue.

If particles stay in one specific layer, then it's not a problem. But just imagine smoke particles that go up in a top-down game. They might start below other objects, but after some time occlude them due to moving up.

Or the problem I currently face: You look along the z axis and simulate sprite depth by scaling them. Sprites need to be ordered so they occlude what's behind them, and the same applies to particles: If an object explodes at z=10, the explosion effect particles have to occlude stuff with z<10 and be occluded by objects with z>10.

It's impossible to do except you implement your own particle system that does z ordering. Considering that Thor already provides a nice particle system, I find it an unacceptable solution (rather a workaround for something that should probably be supported out of the box, because the underlying graphics backend supports it).

[foobarbaz]

Although, I suppose I could just call the render function for whatever external library wherever I want, so maybe it isn't such an issue.

If particles stay in one specific layer, then it's not a problem. But just imagine smoke particles that go up in a top-down game. They might start below other objects, but after some time occlude them due to moving up.

Or the problem I currently face: You look along the z axis and simulate sprite depth by scaling them. Sprites need to be ordered so they occlude what's behind them, and the same applies to particles: If an object explodes at z=10, the explosion effect particles have to occlude stuff with z<10 and be occluded by objects with z>10.

It's impossible to do except you implement your own particle system that does z ordering. Considering that Thor already provides a nice particle system, I find it an unacceptable solution (rather a workaround for something that should probably be supported out of the box, because the underlying graphics backend supports it).

Touché.

[MorleyDev]

Possibly one of my only real design criticism of SFML: It doesn't it lend itself well to replacing internal implementations without...well, rewriting the internal implementations more or less completely.

If it was possible for the underlying code to swap out the rendering system of SFML on the fly you could just replace the "unordered OpenGL Renderer" with a "Depth-Buffered OpenGL renderer" or "Sorting OpenGL renderer" as needs be. Having to have a bunch of if statements related to this kind of thing is a bit of a code smell. It's also why a DirectX port of SFML would be incredibly difficult.

But maybe this is just the different way I code. I also regard functions larger than 5 lines as "too big" and classes that need a bunch of private functions to do it's job as "having too many responsibilities".