How should SFML handle non-legacy shaders?

[binary1248]

One of the roadblocks on the way to providing newer, more "modern" and less "legacy" rendering implementations is the way shaders would be used.

The problem is that Khronos decided to make a clean cut when jumping from the legacy to the modern pipeline in regards to shader usage. There is no backwards compatible way to cover both implementations.

For those who don't understand what I mean, let me demonstrate.

This is what legacy GLSL (the kind SFML currently uses) would look like:

uniform float wave_phase;
uniform vec2 wave_amplitude;

void main()
{
    vec4 vertex = gl_Vertex;
    vertex.x += cos(gl_Vertex.y * 0.02 + wave_phase * 3.8) * wave_amplitude.x
              + sin(gl_Vertex.y * 0.02 + wave_phase * 6.3) * wave_amplitude.x * 0.3;
    vertex.y += sin(gl_Vertex.x * 0.02 + wave_phase * 2.4) * wave_amplitude.y
              + cos(gl_Vertex.x * 0.02 + wave_phase * 5.2) * wave_amplitude.y * 0.3;

    gl_Position = gl_ModelViewProjectionMatrix * vertex;
    gl_TexCoord[0] = gl_TextureMatrix[0] * gl_MultiTexCoord0;
    gl_FrontColor = gl_Color;
}

This is what the non-legacy variant would look like:

#version 130

uniform mat4 sf_ModelViewMatrix;
uniform mat4 sf_ProjectionMatrix;
uniform mat4 sf_TextureMatrix;

in vec4 sf_Vertex;
in vec4 sf_Color;
in vec4 sf_MultiTexCoord;

out vec4 sf_FrontColor;
out vec2 sf_TexCoord;

uniform float wave_phase;
uniform vec2 wave_amplitude;

void main()
{
    vec4 vertex = sf_Vertex;
    vertex.x += cos(sf_Vertex.y * 0.02 + wave_phase * 3.8) * wave_amplitude.x
              + sin(sf_Vertex.y * 0.02 + wave_phase * 6.3) * wave_amplitude.x * 0.3;
    vertex.y += sin(sf_Vertex.x * 0.02 + wave_phase * 2.4) * wave_amplitude.y
              + cos(sf_Vertex.x * 0.02 + wave_phase * 5.2) * wave_amplitude.y * 0.3;

    gl_Position = sf_ProjectionMatrix * sf_ModelViewMatrix * vertex;
    sf_TexCoord = sf_TextureMatrix * sf_MultiTexCoord;
    sf_FrontColor = sf_Color;
}

Almost all the built-in variables have been removed. The only one remaining which is relevant to us is gl_Position.

Because all the built-ins were removed, this means that SFML would have to explicitly transfer the uniform data as well as specify the attribute bindings using the in and out attribute names as can be seen in the second snippet. For demonstration purposes, I just replaced the gl_ prefix with sf_ to make the changes clearer and obvious that they are no longer GL built-ins.

The question now is:

Should SFML just provide a fixed table of legacy replacements so that the user can write GLSL like they used to even using the non-legacy pipeline? This would force the user to follow the given convention and if they don't, errors will occur and drawing will simply not produce the expected output.

Or should SFML allow the user to specify what they are going to name all of the required uniforms and attributes in their GLSL using some global/static function once at the start of their application? If the user doesn't follow what they specify, drawing will also not produce the expected output.

Maybe a mixed solution containing both of the options is the best. SFML would use the pre-specified table of replacements as default values and allow the user to override entries as they wish.

The next question would be: How does SFML guarantee that the GLSL the user provides will work no matter what the target platform supports? The problem is that the user might not know whether the target platform can support non-legacy shaders beforehand. In order to handle both scenarios, the user would have to specify 2 sets of shader source: 1 for the legacy shader implementation and 1 for the non-legacy shader implementation. They could always opt to reject using the non-legacy implementation and stick to legacy for maximum compatibility and not having to write GLSL twice, however this prevents them from using the non-legacy pipeline which might be the better choice if available. On the mobile platforms, this isn't an issue since legacy shaders don't exist.

Once these questions are answered, SFML might be able to move on and provide a few newer rendering backends and finally offer support for shaders on mobile platforms.

So, what would you, the community, like to see/use?

[Gambit]

Personally I would be an advocate for moving to the modern pipeline however I would put it at the same priority as moving sfml to modern c++. Both changes will make sfml non backwards compatible as soon as they are implemented.

If I'm not mistaken, glsl offers the use of layout specifiers for variables (in, out and uniform)  which would mean that internally sfml shaders can be named whatever you want to name them and when users want to alter uniforms and in variables they can use the layout position specified in the shader as opposed to doing a name lookup.

That aside, I would consider myself an expert on the topic by any means, however I'm curious as to how a user would go about using their own shaders in conjunction with sfml ones. For example, say I want to draw a sprite but maybe I want some wavy effect going on. Sfml provides the absolute minimal shaders required to render the sprite to the screen so I don't want to write my own, instead I want to use the sfml one us my own.

I believe shader programs accept multiple shaders of the same type which could come in handy but would that mean that I would have to create a shader program every time I wanted to use my own shaders?

[victorlevasseur]

I definitely think SFML should switch to OpenGL 3 at least in the next major version.

[binary1248]

Personally I would be an advocate for moving to the modern pipeline however I would put it at the same priority as moving sfml to modern c++. Both changes will make sfml non backwards compatible as soon as they are implemented.

Supporting non-legacy GLSL doesn't have to break API or ABI compatibility if it doesn't need to, whereas supporting C++11 will by definition break it. For this reason, support for non-legacy GLSL can be added separate from (and possibly earlier than) C++11 support.

If I'm not mistaken, glsl offers the use of layout specifiers for variables (in, out and uniform)  which would mean that internally sfml shaders can be named whatever you want to name them and when users want to alter uniforms and in variables they can use the layout position specified in the shader as opposed to doing a name lookup.

This doesn't change the fact that the binding still has to be made between attribute location or uniform binding point and application data input. Instead of binding by string, the user would then bind by number instead. This doesn't really address much of the original problem.

That aside, I would consider myself an expert on the topic by any means, however I'm curious as to how a user would go about using their own shaders in conjunction with sfml ones. For example, say I want to draw a sprite but maybe I want some wavy effect going on. Sfml provides the absolute minimal shaders required to render the sprite to the screen so I don't want to write my own, instead I want to use the sfml one us my own.

SFML will have to provide a default "legacy emulation" shader that does the same as the legacy pipeline used to do in the cases where the user doesn't care about shaders or effects. If however, they want to implement some effect that is only possible with a shader, they will have to completely override the default one. Reusing/Modifying it doesn't make sense, it's all or nothing.

I believe shader programs accept multiple shaders of the same type which could come in handy but would that mean that I would have to create a shader program every time I wanted to use my own shaders?

Each shader program consists of up to a single shader instance for each programmable stage of the pipeline. For stages that are optional (such as the geometry shader or even the fragment shader in special cases), OpenGL lets the user omit them. Per pipeline, there can only be a single program active at any given time, and per program, there can be only a single shader active per stage. As such, yes, you would have to create a shader program every time you want to use your own shaders. This is no different in the legacy world.

I definitely think SFML should switch to OpenGL 3 at least in the next major version.

This is not a question of if, but how.

[Jabberwocky]

Maybe a mixed solution containing both of the options is the best. SFML would use the pre-specified table of replacements as default values and allow the user to override entries as they wish.

This is exactly what I was thinking while reading up to this line.  Seems like a perfect solution.  Full flexibility either way, but still works in a "simple" (SFML) manner by providing reasonable default values.

The next question would be: How does SFML guarantee that the GLSL the user provides will work no matter what the target platform supports?

Again, the solution you've proposed seems completely logical. 

Yes, if the programmer wants to support both legacy and non-legacy shaders, he/she will have to write both.  No way around that, of course.

Yes, if the programmer wants to be a bit more lazy and only write legacy shaders, that's fine.  This decision of simplicity (1 set of legacy shaders) vs. optimization (2 sets of shaders) is completely up to them.
... which, incidentally, seems to mean you can continue using SFML exactly as you do today, and ignore this new functionality.  Or do some more work and take advantage of it.  That's perfect - I see no downside here at all.  Give us (SFML users) the power to do whichever, then it's our problem (or "non-problem" if I choose to ignore the new functionality). 

I assume SFML will be able to tell us whether non-legacy shaders are supported, right?  Is it as simple as a GL_VERSION check? 

If so, I could imagine things working two ways.

1.  SFML tells you whether non-legacy shaders are supported.  Then, in (non-SFML) application code, it is up to the programmer to load and use the appropriate glsl shader. 

2.  SFML actually has some kind of "unified" shader API which allows 2 implementations:  legacy and non-legacy.  The programmer provides sources of one or both, and then SFML takes care of everything from there by using the appropriate shader source depending on the opengl implementation on that machine.  Nowhere in application code do we have to think about legacy vs. non-legacy, except for providing the source of one or both.

Off the top of my head - I think I'd lean more towards solution 1:

• It keeps the SFML API simpler.
• It's not that difficult to deal with in application code, if you need it.  (Or you can just provide legacy shaders if you don't care).
• Maybe the application code needs to know whether it's using legacy vs. non-legacy shaders anyway.  e.g. because those shaders are using different shader params which need to be set.  So trying to get SFML to do this shader-switching work for us under the hood doesn't actually accomplish it's goal of allowing the application code to be agnostic towards legacy vs. non-legacy shaders, anyway.

[Mario]

Should SFML just provide a fixed table of legacy replacements so that the user can write GLSL like they used to even using the non-legacy pipeline? This would force the user to follow the given convention and if they don't, errors will occur and drawing will simply not produce the expected output.

Or should SFML allow the user to specify what they are going to name all of the required uniforms and attributes in their GLSL using some global/static function once at the start of their application? If the user doesn't follow what they specify, drawing will also not produce the expected output.

Maybe a mixed solution containing both of the options is the best. SFML would use the pre-specified table of replacements as default values and allow the user to override entries as they wish.

I'd consider something like:

sf::Shader::bindLegacyInputs()
sf::Shader::bindLegacyOutputs()
sf::Shader::unbindLegacyInputs()
sf::Shader::unbindLegacyOutputs()

sf::Shader::bindInput(sf::Shader::Binding, sf::String)
sf::Shader::unbindInput(sf::Shader::Binding)

Where sf::Shader::Binding is an enum holding the classic parameters. The legacy options enable/disable all the classic bindings, while the more specific (un)bindInput() options allow you to specifically opt-in/out to only use whatever you need.

This also has the neat side-effect that it's a lot easier to write a program that's compatible with the names used by some other program (e.g. ShaderToy and similar tools coming to my mind).

The next question would be: How does SFML guarantee that the GLSL the user provides will work no matter what the target platform supports?

That's no new problem, since you can already feed a shader that's too complex for the actual hardware. Maybe some sf::Shader::getVersion() thing?

[binary1248]

Setting the bindings via the Shader class is all nice and good, however, the actual binding takes place in the RenderTarget. Therefore, either the RenderTarget has to check if custom binding is required by inspecting the Shader objects every draw, or the bindings become part of the RenderTarget instead. The latter variant would make the inner workings of the RenderTarget transparent to the users who don't care one bit about shaders in their application. They would automatically benefit from non-legacy if it is supported.

That's no new problem, since you can already feed a shader that's too complex for the actual hardware. Maybe some sf::Shader::getVersion() thing?

In "proper" implementations, the supported GLSL version is always coupled to the supported OpenGL version (3.30 = 3.3, 4.50 = 4.5). As such, the user is already now capable of determining the GLSL version that is supported by the created context. The question is: How does the user determine/control which rendering backend SFML chooses to use? If multiple are available, it would be nice if the newest were to be chosen, but there might be times when the user wants to suppress selection of newer backends for whatever reason.

[Jabberwocky]

Setting the bindings via the Shader class is all nice and good, however, the actual binding takes place in the RenderTarget. Therefore, either the RenderTarget has to check if custom binding is required by inspecting the Shader objects every draw, or the bindings become part of the RenderTarget instead. The latter variant would make the inner workings of the RenderTarget transparent to the users who don't care one bit about shaders in their application.

To clarify, is the concern here having a public function in the RenderTarget class for setting these shader param bindings?

If so, one way around that is to make those functions private (so not exposed in the public API), have the RenderTarget friend the Shader class, make a public API in the Shader class to set the bindings, which then calls the private function on the RenderTarget.
... A little convoluted, but it at least keeps the public API for the RenderTarget clean of any shader-related stuff.

(edit) For this to work, the binding vars would need to be static, and shared across all RenderTargets, but that's likely what you'd want anyway.

How does the user determine/control which rendering backend SFML chooses to use? If multiple are available, it would be nice if the newest were to be chosen, but there might be times when the user wants to suppress selection of newer backends for whatever reason.

I would think an sf::ContextSettings variable would do the trick.
Perhaps something like

// in sf::ContextSettings:
unsigned int maxMajorVersion
unsigned int maxMinorVersion

or alternatively

bool useHighestAvailableGLVersion

[Mario]

Setting the bindings via the Shader class is all nice and good, however, the actual binding takes place in the RenderTarget. Therefore, either the RenderTarget has to check if custom binding is required by inspecting the Shader objects every draw, or the bindings become part of the RenderTarget instead.

My bad then, so imagine those being part of sf::RenderTarget

[binary1248]

To clarify, is the concern here having a public function in the RenderTarget class for setting these shader param bindings?

Neither nor really.

Conceptually, yes... Shader "stuff" should go into the Shader class. However, this is less about setting attributes of the shaders as much as it is about simply defining a convention that is to be used between SFML and the developer. I also wouldn't go as far as to store the attribute names in non-static storage. As I said, it's about defining a convention, and typically, there should really only be a single convention in use at a time. This speaks strongly in favour of having static functions/variables to manage the bindings.

As for the location of these static methods/variables, I am really indifferent (this is where the community comes in ). Either way, the RenderTarget will have to access this data and bind the attributes accordingly. From a performance stand point there is absolutely no difference between having them in RenderTarget or Shader. Semantically, and from an OpenGL perspective, the binding (as its name implies) is the interface between the data source (in our case the RenderTarget) and the shader program (which needs to know which attributes to assign the input data streams to). As such, both sides would be equally valid locations for binding data. One thing we must not forget is that VAOs (which are required in modern OpenGL) will need to be part of the RenderTargets since they are not shareable. Having the static binding data in the RenderTarget would ease maintenance of the VAOs without having to introduce coupling between RenderTarget and Shader classes.

I would think an sf::ContextSettings variable would do the trick.
Perhaps something like

// in sf::ContextSettings:
unsigned int maxMajorVersion
unsigned int maxMinorVersion

or alternatively

bool useHighestAvailableGLVersion

The OpenGL specification allows implementations to give us any version it wants that satisfies the requirements imposed by the version we request. SFML can request version 2.1 and will get a 4.5 compatibility context. On the other hand, if SFML requests 2.1 and the implementation doesn't support compatibility contexts (such as OS X and Mesa3D), it will give us a 3.0 context since 3.1+ would be core only.

The version numbers the user specifies in ContextSettings are merely a minimum required version (which is why the check that is built-in only warns when the version returned might be incompatible i.e. lower or core when compatibility is requested). There is no notion of "maximum version".

What I was referring to however, was not the OpenGL context version. SFML will always request the highest context version that it can if the user does not care (the implementation already does this anyway as I mentioned above). Based on the version of the created context, SFML will compile a list of which rendering implementations are supported on the user's system. From the set of supported implementations, SFML will have to decide which one to use. It is this decision that the user should be able to influence somehow, since the decision will have implications on what kind of GLSL is expected/supported. If the user doesn't influence the decision in any way, SFML would just go for the newest/optimal implementation that can deliver the best performance on the given hardware.

[eXpl0it3r]

This topic has been lingering in the depth of the forum unresolved with little attention.
Yet it's essential in pushing SFML forward towards modern OpenGL.

I've moved it into the SFML Development forum and thus hope to get more feedback on the topic, so we can make decisions on this.

The previous posts are short and informative enough, so I don't feel there's a need to summarize anything at this point.

[eXpl0it3r]

People seem to be afraid to post here, so I'll just quote some comments.

On #raylib I created an abstraction layer to support multiple GL versions: https://github.com/raysan5/raylib/blob/develop/src/rlgl.c it includes one default internal shaders but also exposes some shader control functions for users that create their own shaders. Maybe you can get some ideas...

I am not sure maintaining 2 api's is a good idea
My personal opinion on it is to leave it the way it is and switch entirely to modern shaders for SFML 3

SFML can provide default inputs and uniforms
it's up to the user to use them
e.g. setting the colour of a shape requires the shader read the sfml_Color uniform or something like that
or "color" becomes entirely a shader property
But that requires more design around the API, what are the valid material properties? Are they arbitrary strings?
Because you of course want to keep the boilerplate low, there needs to be an easy API for trivial usage

[MorleyDev]

I think that Shaders are an advanced feature by default. If you're just starting out, one of the nice things SFML has going for it over diving into full OpenGL is that it abstracts those away until you are ready to deal with them. I'd go far as to say the argument could be made that Shaders are the most advanced feature covered by the SFML API.

Being advanced features I also think that this means there's more leeway in how much elbow grease is required. Given that, I would suggest that by default SFML 3 should opt for the modern non-legacy shader model only, and the API should be built with that in mind.

Legacy Shader support can then be provided by a 3rd party community-driven library. So long as there are sufficient hooks for SFML, which since you can get at the underlying OpenGL context there should be, these kinds of legacy features can be supported without needing to 'dirty' the main branch with them.

That way, if you know what you're doing well enough to use Shaders and to know you need the legacy shaders in particular, then SFML provides the tools you need to get at that, but without impacting the cleanliness of the main SFML APIs.

Yes, requiring an extra library does complicate the user experience some-what (though SDL gets away with it...like, a lot), but for an advanced feature like shaders I'm not sure that matters as much.

[Chaia*]

My personal opinion is: skip the legacy stuff. To be honest, whose computer does not support OGL 3(.3) in 2018? The first graphics cards supporting OGL 3.3 were the HD2000 and GTX8000 series (as well as iGPUs from Ivy Bridge on). According to the steam hardware survey, 98,11% (not current values, I looked them up like half a year ago) use a graphics card capable of DX10 and thus also modern OGL. Why not making a survey here in the forums to see if there is really still a need for the legacy part?

I'm using SFML since many years and I actually love its simplicity, especially for beginners. But over the years, I noticed many limitations of the graphics module. For some of those I found posts in the feature requests forum but most of them were rejected because "too modern" or "would not be available to all users because it is not possible in legacy". I understand that you do not want to exclude users without modern graphics card/iGPU but as explained above - are there still such users today? With skipping the legacy part, the API would not get more difficult - simiplicity is one of the main advantages of SFML - but in my opinion would make many things faster, easier and more flexible for advanced users.

I think the current SFML is great for quick prototyping or very small games but as soon as you want to do something more advanced (e.g. (2D) game with more advanced graphics or a (2D) game where generally rendering performance matters) you basically have to recode half of the graphics module just because of the legacy focus. Furthermore you have to be careful when combining SFML's legacy rendering with more modern features (e.g. vertex buffer objects combined with vertex array objects).

Don't get me wrong - I don't want to be SFML a game engine or graphics engine but just suggesting to make the SFML framework more "modern OGL friendly". And the easiest way to do this without crippling the API is to get rid of all the legacy stuff.

If you only want to do legacy stuff, then the current SFML is perfectly fine, there is not much need for improvement, you can do everything you could think of for a small game or a quick prototype. But if you want to do more modern stuff it gets nasty. You have to ask yourself: Assuming SFML 3 would still support all the legacy stuff, would a user of SFML 3 who targets legacy platforms profit from all the planned new features and the greater flexibility regarding modern OGL? Probably not. Would a user of SFML3 who uses modern OGL be somehow limited or confused or have more work to do just because there is this big rat tail of legacy stuff which he does not need? Probably yes. So why not keeping the current SFML for legacy users and for SFML3 focusing on modern OGL only?

[binary1248]

Nobody said a modern sfml-graphics implementation couldn't co-exist with the current legacy implementation. That is the aim of this discussion. Once we figure out how to support modern shaders through the API, anybody is free to start work on a modern sfml-graphics backend.

The whole idea of sfml-graphics is to provide a simple abstraction of 2D graphics concepts so that beginners can jump in and get stuff done without all the boilerplate associated with handling the lower level stuff themselves. People who use sfml-graphics should not have to know how it is implemented. Whether it runs using OpenGL 1.2, OpenGL 4.5, Direct3D or Vulkan should just be an implementation detail. There are certain places where the implementation seeps through the abstraction API, such as with shaders. Short of SFML inventing its own shader language, which some engines have actually done, there is no way to abstract away GLSL, HLSL and SPIR-V.

The fact that many parts of sfml-graphics can be used as wrappers around OpenGL objects does not mean that was its primary purpose. It is merely a side-effect of the API design. While you can mix OpenGL and sfml-graphics code, it can get really messy, and I would recommend either going full OpenGL or full sfml-graphics. It will stay this way in the foreseeable future and even into SFML 3. There are enough OpenGL wrapper libraries out there, SFML doesn't have to focus on reinventing that wheel.

Because SFML 3 is realistically still quite a bit away for a myriad of reasons, I wanted to focus this discussion on implementing a modern shader API side-by-side with the current API. The fact is that it can already be implemented in SFML 2, but that also means that the legacy sfml-graphics implementation is here to stay for a bit.

P.S. The Steam hardware survey is far from what one can consider representative. Firstly, it is voluntary, and seems to pick users out at random. Secondly, and more importantly, Steam is considered a desktop application targeted at PC gamers who like to buy/play games over that platform. If you aren't a gamer, or don't have a reason to game over Steam, chances are you are not going to install it. Being a gamer who installs Steam, you are disproportionately more likely to invest a bit of money in better hardware. Steam will likely not be installed on any embedded platforms, resource constrained platforms or systems that are dedicated to a single purpose (kiosk applications, slot machines, POS systems, etc.). You'd be surprised what kind of hardware is still being sold in "new" systems in industrial applications to this day. Certifying a new platform is so cumbersome that OEMs just settle with 15 year old hardware that is proven to work. In the industrial sector, it's the hardware people that tell the software people what they can and can't do, which is the complete opposite of a consumer system that would end up running Steam.