What's the future of SFML's OpenGL-based renderer?

[Ceylo]

Hello,

Context

As you may know, Apple is pushing its own GPU technology, Metal, which was made available to iOS in 2014 and to macOS in 2015. In 2017 Metal 2 was released both to macOS and iOS.

On the other hand, macOS supports up to OpenGL 4.1 (spec released in 2010) and iOS supports up to OpenGL ES 3.0 (spec released in 2012).

On macOS, although OpenGL 4.2 (2011), OpenGL 4.3 (2012), OpenGL 4.4 (2013), OpenGL 4.5 (2014), OpenGL 4.6 (2017), Vulkan 1.0 (2016) specs have been published, they've been ignored by Apple.
On iOS, OpenGL ES 3.1 (2014) and OpenGL ES 3.2 (2015) are also not supported.

Issue

You can more or less see that since Metal was released in 2014, no progress has been made on OpenGL (ES) support. This is not an issue for short-term, except if you're targetting better performance. But the more SFML waits, and the more it'll sink in legacy (at least in mid-term regarding Apple platforms, and in long-term on other platforms too).

Potential solutions

Here are a few suggestions about what looks possible:
1. Have a platform-specific renderer : I don't know the amount of tweaks and the burden of current support for OpenGL (ES) in SFML on all current platforms, but having a renderer per platform is very likely to be an awful huge pile of work, both for development and maintenance.

2. Switch to OpenGL ES 2 on macOS through MoltenGL : this would potentially improve performance on macOS, and ease code sharing between macOS and iOS implementation, but on the other hand, OpenGL ES 2 is old (2007) and the fact that MoltenGL doesn't support more recent versions makes me think it will also die in a few years. Additionally it's not free (150$/user …) so… most likely not worth it.

3. Switch to Vulkan for all platforms : this is maybe a bit early but looks to be the most promising solution. According to https://en.wikipedia.org/wiki/Vulkan_(API)#Compatibility and a few other sources here is what's supported:

Platform	Intel	AMD	NVIDIA	Qualcomm	ARM
Linux	2012+	2014+	2012+	N/A	N/A
Windows	2015+	2012+	2012+	N/A	N/A
macOS	macOS 10.11+*	macOS 10.11+*	N/A	N/A	N/A
iOS	N/A	N/A	N/A	N/A	iOS 8*
Android	N/A	N/A	N/A	2013+	2013+

*through MoltenVK.

The worst case is Intel integrated GPU which requires at least a Skylake (6th gen) CPU.
On macOS and iOS, MoltenVK (which provides an implementation of Vulkan that is based on Metal) is open source and under the Apache 2 license.
Note that Metal support is related to the supported macOS/iOS version, not to a hardware, which is why I didn't put a year in macOS/iOS table cells.

Additionally LunarG provides a SDK for using Vulkan with validation layers, for all the platforms that SFML is using except Android which looks to have its own official Vulkan SDK.

Opinion

Considering the points given above, to me the best solution looks to start relying on Vulkan, while still keeping OpenGL (ES) for now. That means supporting 2 (and a half if you count OpenGL ES) renderers for a transition period and giving up on OpenGL in ~2020, so that SFML can remain compatible with hardware from less than 5 years (the main issue being Intel GPUs). Considering that OpenGL is not immediately an issue the beginning of this transition can most likely wait one more year. Of course it depends on how long the SFML community is willing to maintain 2 renderers.

[binary1248]

The question is not if... but how and when.

I would have already started with this years ago if it were not for this unresolved discussion.

[Laurent]

And what about OpenGL stuff that leaks through the public API? Things like sf::Texture, sf::Shader, which can give their native GL handle, be activated in the current context, etc. And what about GlContext anyway? If we start supporting multiple rendering back-ends, won't we have to hide/abstract a lot of currently exposed stuff?

And what about using SFML as an OpenGL window API?

[binary1248]

Well... nothing in getNativeHandle suggests that it has to be an OpenGL object ID.

If you look at the Pong example, we can already write applications without having to care about how it is actually rendered underneath. In typical applications you would also not have to touch sf::Context in any way either.

The only hurdle left to deal with is specification of the shader source. In OpenGL GLSL is used, in Direct3D HLSL is used and in Vulkan... SPIR-V is "used" in the sense that it is the stuff you feed into the API. All the examples you see in the wild will have generated the SPIR-V from GLSL using a compiler (in fact Vulkan has gotten to the point where it can accept GLSL directly now). It has become so easy nowadays to translate from one shader language to another shader language that this isn't even a problem that is impossible to solve any more.

The fact that you can use SFML as an OpenGL capable window isn't going to disappear. If you wanted to you could even attach an OpenGL context, a Direct3D device and a Vulkan device to the same window simultaneously if you really wanted to. One doesn't exclude the other.

Like I said in my previous post, the first step in this direction is to solve the problem of non-legacy shaders. Once that is done we can deal with the other smaller problems on the way.

[Laurent]

Well... nothing in getNativeHandle suggests that it has to be an OpenGL object ID.

But if it is something not well-defined (ie. something that depends on the chosen back-end) then it is useless.

If you look at the Pong example, we can already write applications without having to care about how it is actually rendered underneath. In typical applications you would also not have to touch sf::Context in any way either.

Yes... I was not talking about the cases where this is not an issue; I was trying to raise a concern about cases where this will be an issue -- ie. cases where sf::Context & friends are involved.

The fact that you can use SFML as an OpenGL capable window isn't going to disappear. If you wanted to you could even attach an OpenGL context, a Direct3D device and a Vulkan device to the same window simultaneously if you really wanted to. One doesn't exclude the other.

But the API will have to change anyway. This will require at least to explicitly choose which back-end you want.

[Ceylo]

The question is not if... but how and when.

I would have already started with this years ago if it were not for this unresolved discussion.

I understand this is part of the problem, but only a (big?) part. Anyway will check this topic

And what about using SFML as an OpenGL window API?

Unless I miss something there is no question on this matter: if the user chooses the OpenGL backend he can use OpenGL, otherwise he can't. If/when OpenGL support is dropped, then SFML can't be used as an OpenGL window API anymore, but at that time OpenGL will most likely be obsolete.

The fact that you can use SFML as an OpenGL capable window isn't going to disappear.

It should! At least once Vulkan is widely enough supported. You don't want to maintain 2 backends forever.