Shader Architecture for 3D in SFML

[Groogy]

Yo! This might more be of a brainstorming/discussion than an actual feature request.

As some might know there is an upcoming huge change to the Graphics module where it will include Vertex Shader capability to SFML.

If using a Vertex Shader it would for us be possible to create 3D graphics in SFML without actually concerning ourselves with any OpenGL stuff at all more than the language GLSL. the main requirements would be a vertex shader that looks something like this I guess:

Code: [Select]


uniform mat4 toWorldSpace;
uniform mat4 toViewSpace;
uniform mat4 projection;
 
// I know OpenGL has ModelView matrix instead of separating
// it into two seperate but I'm writing with what I am comfortable with.

void main() {
    vec4 vertex = gl_Vertex
    vertex *= toWorldSpace;
    vertex *= toViewspace;
    vertex *= projection
    gl_Position = vertex;
}

Have never used GLSL so somethings might be wrong but the math should be the same as my software renderer and as in HLSL.

Though this requires us to be able to pass 3D vertexes to OpenGL to use it which SFML at the moment don't support and I don't expect it to either since it's a 2D library.

Also from what I know using a Shader makes most of the states in OpenGL unused unless you use it in your own Shader. So from my point of view and the knowledge I got of OpenGL is that it would make it possible for us to generate 3D graphics with SFML, no custom OpenGL calls and no need of maintaining the state machine right?

My feature request is that somehow be able to pass on 3D vertexes to OpenGL trough SFML. Maybe by defining a custom drawable and have the sf::Renderer extended to take in 3D coordinates instead of only 2D for placing a vertex? This request is not related with: http://www.sfml-dev.org/forum/viewtopic.php?t=5729 What I want is the possibility to create 3D graphics using only SFML, no custom OpenGL and no change in how the drawing interface works now or will work after the change.

Note even if Laurent decides no he don't want this, I still won't give up I'll try and hack something together as an extension that will first be available for rbSFML and then C++ :twisted: One of the reasons I want this is because I have trouble getting ruby-opengl installed on my Windows machine and this would make things SO much simpler.

Anyone else that might have more experience with 3D and OpenGL/Shaders than me that might be able to give more detailed information?

[Laurent]

Basically, this is not a bad idea. Indeed, when you write a shader-only rendering architecture, you can get rid of most of the fixed pipeline states -- but not all of them. So the first task would be to list them, and see if we could really get rid of OpenGL calls, or if there's too much stuff to wrap in addition to what SFML provides.

The new graphics API will not allow 3D of course, but it should be very easy to support it: transform the vertices' position from Vector2f to Vector3f, and allocate/configure a depth-buffer when you create the render-target. This won't be possible from the public API, but hacking it should be really easy

I should be able to tell you more about the new graphics API in a few days (hopefully), which in my opinion is mandatory before continuing this discussion (from an implementation point of view).

[Groogy]

This won't be possible from the public API, but hacking it should be really easy

Wondering what you classify as public API. Is sf::Drawable part of the public API? In that case with hacking it, does that mean going into your source and actually change physically your code? Or would it be available trough classes not part of the "public API" like the Matrix class and sf::Drawable?

I should be able to tell you more about the new graphics API in a few days (hopefully), which in my opinion is mandatory before continuing this discussion (from an implementation point of view).

I totally agree with you and I'm super excited

Though this thread is still open for discussion for others who are interested and that might know the exact effect this would have and what states are concerned and so on?

Also the main pro for me on this is that it would kind of give us a perfectly nice wrap on OpenGL on this and also give a perfect Ruby interface to it(The OpenGL bindings is not that ruby-like/object oriented). Yay

[Laurent]

Wondering what you classify as public API. Is sf::Drawable part of the public API? In that case with hacking it, does that mean going into your source and actually change physically your code? Or would it be available trough classes not part of the "public API" like the Matrix class and sf::Drawable?

All classes that you can access in namespace sf:: are part of the public API. What I mean is that you will have to change/add some functions, and of course implementation code.
By the way, sf::Drawable will probably disappear and sf::Matrix will be replaced by sf::Transform and be totally public.

Though this thread is still open for discussion for others who are interested and that might know the exact effect this would have and what states are concerned and so on?

You can have a look at Qt3D (or whatever it is called now), it's a Qt module that is in development and which is very similar to what you want -- a thin portable/object-oriented wrapper on top of OpenGL.

[Groogy]

All classes that you can access in namespace sf:: are part of the public API. What I mean is that you will have to change/add some functions, and of course implementation code.
By the way, sf::Drawable will probably disappear and sf::Matrix will be replaced by sf::Transform and be totally public.

Hmm :?
I guess it's a step in the direction I want ^^
We'll see how it becomes when we get to that point. Also it will be perfect for what I am doing to prove Ruby's worth

Just took a quick glance at Qt3D, not so familiar with Qt but looking from the examples/tutorials... No I do not like :roll:
Though there is of course lessons to take from it on what to do.

All of this will probably make me rewrite the entire engine, make it into an extension to a modified SFML(as you said) and write it entirely for Ruby  :twisted:

[Groogy]

Just want clarification on one thing...

allocate/configure a depth-buffer when you create the render-target

Can't you already do that by passing a context settings when creating the render window for instance? You can specify the depth bits there. Or you maybe meant that I will have to manually enable it trough glEnable before drawing?

Actually after you have set your graphics design in stone, will it be possible to have a dialogue of how SFML works with the OpenGL state machine or are you too busy so I'll have to figure it out for myself while hacking around in your code?

[Laurent]

Can't you already do that by passing a context settings when creating the render window for instance? You can specify the depth bits there. Or you maybe meant that I will have to manually enable it trough glEnable before drawing?

You need both:
- force using a depth-buffer in ContextSettings (user must not be able to disable it!)
- enable depth-testing with glEnable, set depth range in projection matrix, add depth-buffer bit in glClear

Actually after you have set your graphics design in stone, will it be possible to have a dialogue of how SFML works with the OpenGL state machine or are you too busy so I'll have to figure it out for myself while hacking around in your code?

I can already tell you what the lowest-level will be:
- there will be a thin layer on top of OpenGL vertex arrays, so basically it wil be a dynamic array of vertices (position, color, texcoords) that can be rendered
- RenderTarget will have functions to set global states: SetBlendMode, SetView, SetTransform, SetTexture, SetShader
- RenderTarget will have a single drawing function: Draw(VertexArray)

The drawing of higher-level objects should look like this:

Code: [Select]

void MyObject::Draw(sf::RenderTarget& target)
{
    target.PushStates();
    target.SetTexture(myTexture);
    target.SetTransform(myTransform);
    target.Draw(myVertexArray);
    target.PopStates();
}

[Groogy]

You need both:
- force using a depth-buffer in ContextSettings (user must not be able to disable it!)
- enable depth-testing with glEnable, set depth range in projection matrix, add depth-buffer bit in glClear

Check, will it be possible to have depth-testing enabled while normal SFML 2D graphics is rendered without it behaving weirdly? I.E so there is no need for "prepare 2D/3D" functions.

I can already tell you what the lowest-level will be:
- there will be a thin layer on top of OpenGL vertex arrays, so basically it wil be a dynamic array of vertices (position, color, texcoords) that can be rendered
- RenderTarget will have functions to set global states: SetBlendMode, SetView, SetTransform, SetTexture, SetShader
- RenderTarget will have a single drawing function: Draw(VertexArray)

The drawing of higher-level objects should look like this:

Code: [Select]

void MyObject::Draw(sf::RenderTarget& target)
{
    target.PushStates();
    target.SetTexture(myTexture);
    target.SetTransform(myTransform);
    target.Draw(myVertexArray);
    target.PopStates();
}

Alright so from what I can see what I have to do is more or less change position to take a sf::Vector3 instead, have it be so any original SFML 2D graphics are defaulting to some z-value, I guess 1 or something? Or maybe it's smartest if I allow the user to change this outside the vertex shader.

Anyway those are the changes I can see that I will have to do. Modify so a 3D vector is used instead and make sure that the depth testing is always enabled.