SFML community forums

General => SFML development => Topic started by: binary1248 on October 02, 2014, 04:06:50 am

Title: SFML Context Management
Post by: binary1248 on October 02, 2014, 04:06:50 am
Hello awesome SFML forum! :D

Today, I (We? Can't speak for the other team members..) approach all of you to ask for your opinion on the matter of:

SFML Context Management

This has been an issue that I have personally been involved in for a long time, and I can imagine many others, including Laurent himself of course, have expressed their concern in regards to this matter time and again.

With the SFML 3 vision thread (http://en.sfml-dev.org/forums/index.php?topic=15027.0) up in full swing for just over 5 months, there is no doubt that we are going to start work on SFML 3 sooner or later.

As you all know one of the biggest uses of SFML, is in the area of 2D graphics. To realize this, SFML builds on top of the cross-platform graphics API OpenGL. As with any hardware accelerated API, there needs to be some form handle to identify the accelerator device in question, whether it is really the physical device or a virtual device that is emulated by the driver. DirectX names their handles, devices, fittingly, and OpenGL decided to name their handles contexts. OpenGL has had a long history, much longer than DirectX, and needless to say, the concept of contexts as a container for GL state has existed since the beginnings of OpenGL.

For those familiar with OpenGL, you know that calls to the OpenGL API will only have an effect if a context is currently active, to keep it simple for now. Any state modifying calls will modify the state of the currently active context and no other. Since OpenGL used to do most of its work via bindings (i.e. state variables), understanding how contexts work is essential if you plan on using multiple of them, and that is where the difficulties arise.

There are a certain set of constraints that have to be met when dealing with contexts. Many of them are probably the result of operating system constraints from the time when the first specifications were written, but they haven't changed much since then. I will try to list most of the important ones here so that you can understand what I mention later better.

SFML makes extensive use of OpenGL for drawing throughout its graphics module, and marginal use of OpenGL in its window module in order to provide context support for developers who aim to use OpenGL directly.

The technical details come now.

Due to historical reasons, Laurent chose to have a single context that serves no other purpose than to be shared with any context that is created after it. In SFML code, this context is referred to as the "shared context". The shared context is the first context to be created in an SFML application, due to the fact that any future contexts have to be shared with it. It is created as part of the global GL context initialization which happens as soon as the first sf::GlResource is created. It relies on reference counting in order to track how many objects there are that still rely on having an active context. Once the last sf::GlResource is destroyed, the global GL context cleanup is performed, which gets rid of the shared context as well. In addition to that, the cleanup function also gets rid of all "internal contexts".

SFML sets up a sort of "invariant" that states that as soon as a sf::GlResource has to perform any operation within any thread that thread will have to constantly have a context active at all times, even when not required. Maybe this was done for performance reasons? I do not know (only Laurent knows :P), and the difference was barely measurable in my tests. As opposed to "standard practice" of deactivating contexts when required, SFML "deactivates" contexts on the current thread by activating a so called "internal context", a thread local context that makes sure that the invariant of having a context active at all times is upheld. These internal contexts are created the first time they are required (i.e. a context was activated and requested to deactivate) and due to the way the context management works, they cannot be destroyed until global cleanup, which typically happens when the application closes. This results in what is now known as the "context leak" that has been discussed on the forum and tracker often enough. Applications spawning temporary threads to perform sfml-graphics operations will notice a constant rise in memory usage due to this, and at some point, the driver will also give up in trying to manage the excessive number of contexts.

Are these internal contexts ever used when (and if) they are active? Not very often in typical code. If an application doesn't make use of sfml-graphics in secondary threads without an explicit context object and never explicitly calls .setActive(false) from within the main thread (the one that deals with the main window) or creates sfml-graphics objects before the main window is created, it will likely never be used. Nonetheless, it is created, all the time. If you are proficient in OpenGL debugging tools, you will notice that even the simplest SFML application, such as those found in the beginner tutorials will end up creating 3 OpenGL contexts, although the shared context is unnecessary (according to my tests) and the internal context is rarely ever used.

Additionally, because framebuffer objects cannot be shared between contexts as stated above, in order for SFML to guarantee that a sf::RenderTexture can be used in any thread after it is created, it has to contain its own context that is activated solely when something is to be done on the sf::RenderTexture and deactivated thereafter. This means that in a single-threaded application with a "large" number of sf::RenderTextures, the same number of additional contexts will have to be created, one for each sf::RenderTexture. The alternative implementation of sf::RenderTexture using a secondary context's default framebuffer on systems that don't support framebuffer objects is more justified, but the systems that cause this to be the case are getting less and less common.

The reasoning behind the current design stems from the fact that SFML tries to hide context management from its users. The assumption is that context management shouldn't be something that beginners have to struggle with when they already have so much to learn. This is true, and maybe it should be something that is kept for understanding at a later time, but I personally find that learning that a context has to be active in order to play with sfml-graphics is something that shouldn't be too hard to grasp. The problem is that I, Laurent and many others already have a lot of experience with graphics programming, so what might seem easy for us might seem like a nightmare for beginners. Conversely, what might seem like a boon to beginners is a nightmare to advanced users (e.g. threaded sfml-graphics).

SFML 3 will probably bring with it a lot of changes, and Laurent has already proposed his ideas on separating the sf::RenderTarget concept from the window and offscreen rendering surface objects (currently sf::RenderTexture). Conceptually, a sf::RenderTarget is nothing other than a context that is used to target a surface to render to, so in that respect a change in regards to context management wouldn't be as drastic as some might expect.

I would have provided mockup examples of what explicit context management might look like, but I decided against it since it is only one of many possible ways of exposing it through the API and I don't want to influence the outcome of the poll because of that.

We really need community feedback on this matter. Should SFML make context management more explicit? Or do you think that it should stay implicit since this is what makes SFML different from (and better than?) all other multimedia libraries?

Please try to be honest and give the poll a bit of thought before voting. In the case you retrospectively change your mind, please try to factor in the gut feeling you had when you voted previously. We want the API to feel good and easy to use, yet powerful at the same time, and there is no better way to judge the "feel" of the API than relying on your gut feeling. Also, since I hope to get a representative opinion on this matter, it is important to motivate as many of SFML's users to vote as possible, especially those that don't visit the forum very often. SFML will always try to be beginner friendly and thus their opinions are worth just as much (if not more) than those of us who have been using SFML for years, and yet they are probably the ones who are the hardest to get a hold of for these kinds of polls.

For those who are afraid/shy of exposing yourself, keep in mind that voting is anonymous (posting is not mandatory), and only those with database access (i.e. those who really don't care ;D) can check what you voted for, if they even invest the amount of effort required to do that.

I would be happy for your feedback on this matter :).
Title: Re: SFML Context Management
Post by: Tex Killer on October 02, 2014, 05:18:58 am
From what I could understand from your post, apart from that "context leak" issue, I see no reason at all to even consider those contexts. Maybe SFML's internal code could be improved to remove those useless contexts as you've said, but why would I want to manually use them? What could I do with direct access to them that I can't do now?

Aside from that, what about DirectX rendering support? If it is implemented (and I hope it gets into SFML3), won't contexts only exist natively on OpenGL supported devices? In my oppinion, DirectX support is much more appealing to me than those contexts (I might change my mind depending on your answers to my previous paragraph, but I find it unlikely).
Title: Re: SFML Context Management
Post by: Ruckamongus on October 02, 2014, 05:55:29 am
I have to agree with Tex Killer; I was attracted to SFML because of its hardware accelerated graphics and abstraction over the nasty OpenGL stuff. I feel that SFML should continue to manage contexts internally, but if you're considering an internal rewrite of the context management perhaps it's time to think about rendering backend abstraction as well. It would be a massive amount of work I'm sure, but being able to choose a renderer (OpenGL, DirectX, Apple's Metal?, etc) could attract a wider audience. Also this may allow development on different platforms to be easier; Android with ES, Apple with Metal (have they concluded if they're dropping OpenGL support fully in the future?), Windows with DirectX, OpenGL with everything else.

If you guys aren't going to consider going down that road my vote is to let SFML keep handling the context(s). Nothing stops people from using OpenGL directly in SFML, and those that don't want to don't have to.
Title: Re: SFML Context Management
Post by: eXpl0it3r on October 02, 2014, 08:35:59 am
I think it's important to state, that this thread shouldn't turn out into yet another "rendering backend" discussions, we already had that on the SFML 3 thread, so please don't go into any further - mentioning it as argument for context management is a valid point, but please no "I want feature X more than Y".

If you guys aren't going to consider going down that road my vote is to let SFML keep handling the context(s). Nothing stops people from using OpenGL directly in SFML, and those that don't want to don't have to.
That's one of the main problem with contexts managed by SFML. If you want to use OpenGL directly, you're still bound to how SFML handles the contexts internally. You'll have to hack around whatever SFML does with the contexts and you can't handle things the way you want.

I don't have experience with OpenGL, but talking to different people and understanding as many of the pros and cons, it seems that the current system is not optimal. Until now it more or less successfully hide the context systems from the casual user (I'm sure many didn't even know that they existed). But at the same time it makes the life of the advanced users a nightmare and I've seen posts here and there which were surprised about SFML's strangely high number of contexts and some even switched to SDL since they couldn't handle contexts explicitly.

Thus while SFML's audience consists currently mostly of new beginners, I'd really like to see SFML being used in a lot of advanced code bases as well. Context management should in my opinion be explicit, however if somehow possible it would be nice, if when operating within one thread on one window (most basic use cases of SFML), one would have to deal with contexts.
Title: Re: SFML Context Management
Post by: select_this on October 02, 2014, 09:58:33 am
Personally I'm in favour of explicit context management being at least possible, if not the default. Perhaps there could be some mechanism to decouple SFML's concepts from the contexts.

I'd be wary about ditching SFML's context management entirely; the forum is already filled with tons of posts about textures not being drawn correctly due to scoping issues, as an example, so I can imagine the flood of angst that will be received should SFML stop behaving in a way that people have come to expect ;)
Title: Re: SFML Context Management
Post by: Laurent on October 02, 2014, 10:19:58 am
Before thinking about what we would like to do, we should focus on what we can do. binary1248, your tests showed that the limitations the current code works around no longer apply. But this is not enough to draw conclusions, I think we should write and publish an application that runs the same tests, and have as many users as possible to test it, to validate these assumptions on a wide range of OSes, GPUs and drivers.

I also remember that you already came up with significant improvements to the internal code (based on the above assumptions), so with these improvements, would we still need explicit context management?
Title: Re: SFML Context Management
Post by: binary1248 on October 02, 2014, 10:43:30 am
But this is not enough to draw conclusions, I think we should write and publish an application that runs the same tests, and have as many users as possible to test it, to validate these assumptions on a wide range of OSes, GPUs and drivers.
I could do that... But, you have to remember, the way SFML handles context management is so unique, that when we say we consider changing it to look different, it might in fact end up looking like what other libraries have already done for a very long time. If some of the assumptions are not true, those libraries would have already ceased to function properly long ago.

I guess writing simple test cases isn't too hard, but considering that SFML normally takes care of the cross-platform part, in this scenario we would have to write different code to test the same feature for different platforms, and that can become... quite tedious :P.

I also remember that you already came up with significant improvements to the internal code (based on the above assumptions), so with these improvements, would we still need explicit context management?
If you are referring to gl_dev, then no, that branch doesn't touch context management in any way. It just focuses on replacing GLEW without changing any functionality, and after that, adding support for things people have wanted for a long time. The context management still looks the same as it did without the changes.

I've also experimented in the past with cutting down the number of contexts SFML internally uses and I would have had a working implementation if I gave it a bit more effort, but that branch also assumed that many of the things you also mentioned back then that had to be tested were simply given. I recently browsed over the changes again before starting this thread, because, contrary to what people might think of me, while I would prefer having explicit control of contexts, I also have a somewhat working solution if it is to stay implicit as well. I guess I can finish up on it when I next find time and provide it to SFML users for testing. This would also alleviate the need to write individual tests as described above.

Of course, this should not prevent people from taking part in this poll :P.
Title: Re: SFML Context Management
Post by: Nexus on October 02, 2014, 10:49:10 am
binary1248, thanks a lot for the detailed and insightful explanation, it really helped me understand some important details.

I have a question similar to Laurent's: you present implicit and explicit context management as alternatives. What limitations would implicit management imply, the same as currently? Or, is implicit management even a true option when we want to defeat the context leak and reduce the number of unused contexts? I'm not sure whether such designs are still being investigated, or whether there is a fundamental constraint preventing them.
Edit: Ah, I saw you referred to that point in your last reply.

And as I've understood the issue, it will be very difficult to provide an API that handles things implicitly for most users, but allows more control for users with specific requirements, right?
Title: Re: SFML Context Management
Post by: binary1248 on October 02, 2014, 11:25:00 am
Currently, this only really applies to users of sfml-graphics. OpenGL programmers who only use sfml-window (like me) are unaffected by context management, because well... context management is only done when sfml-graphics resources request that contexts are active to perform any OpenGL operation.

The reason why it might also affect people like me, is because, simply put: I am lazy :P. I use sf::Texture a lot instead of manually managing OpenGL texture objects myself, and the same with sf::Shader. They fit my needs perfectly most of the time, but they also bring with them the whole context management issue. The "issue" with the shared context also affects people who don't use sfml-graphics in any way as well, since sf::Context is also a sf::GlResource, and will cause the whole global initialization/cleanup like any other sf::GlResource.

Take this really simple example:
#include <SFML/Window.hpp>
#include <SFML/OpenGL.hpp>

int main()
{
    sf::Window window(sf::VideoMode(200, 200), "Test");

    while (window.isOpen())
    {
        sf::Event event;

        while (window.pollEvent(event))
        {
            if (event.type == sf::Event::Closed)
                window.close();
        }

        glClear(GL_COLOR_BUFFER_BIT);
        window.display();
    }
}
This is a barebones OpenGL skeleton that every OpenGL application will start out with. How many contexts do you think are created just for these 21 lines of code, not even using sfml-graphics?

1? Nope...
2? Wrong again...
3...

When the sf::Window is created, it causes global context initialization since it is a sf::GlResource. This in turn creates the shared context, and in the current implementation, the shared context is immediately deactivated after it is created, which, as I described in the original post, is done by activating the internal context (which is created the first time it is required). After that is done, the actual context that is used for the rest of the application is created by the window.

This means that no matter how you try to do things, there is no way to use SFML for OpenGL without having to create at least 3 contexts, and I can understand why some more experienced OpenGL coders might already seek greener pastures when they discover this.

It is probably easier to solve this issue first before tackling the whole sfml-graphics context management, since we can assume that in this use case the user really only wants a single context. But in the end, I think any fundamental changes we make to either this use case or the sfml-graphics use case will affect the other anyway. In that sense, I don't think it makes sense to provide 2 different paths (one simple and one advanced) for users to choose from. The way I see it, sfml-graphics has to be rewritten to build on top of sfml-window more than it does now. There is still too much coupling between the modules for my taste. Hopefully with the module restructuring in SFML 3, we can try to solve this issue as well.
Title: Re: SFML Context Management
Post by: Ruckamongus on October 02, 2014, 12:42:25 pm
binary1248, I'm still not fully grasping the contexts fully. Are you saying that every sf::GlResource will not be freed until the termination of the program? I get what you're saying about having the internal context and looking at ways to prevent these unnecessary globals, but does the context sharing create serious memory leaks?

The way I see it, sfml-graphics has to be rewritten to build on top of sfml-window more than it does now. There is still too much coupling between the modules for my taste. Hopefully with the module restructuring in SFML 3, we can try to solve this issue as well.

Wouldn't that mean sfml-graphics would depend on sfml-window? That seems counterintuitive to the goal of reducing coupling?
Title: Re: SFML Context Management
Post by: binary1248 on October 02, 2014, 01:20:43 pm
Are you saying that every sf::GlResource will not be freed until the termination of the program?
No... You control the lifetime of your sf::GlResources, and they will be freed when you choose to do so. It is the internal contexts that will not be freed until the termination of the program (or at least the destruction of every sf::GlResource, which ends up being the same in typical cases), due to the threading constraints I mentioned. The problem is that the internal contexts are managed in part using thread-local storage, and at least on Windows, the assumption is that the code which spawns the thread should also be responsible for allocating and deallocating objects stored in TLS. This is broken in SFML, since it uses TLS "behind the scenes" and lets the user create threads at will. Windows doesn't provide a simple way for SFML to detect when a thread is going to terminate from within the terminating thread. This last part is important, because it is that thread that has to deactivate the context so it can be freed. Strictly speaking, the way SFML destroys its internal contexts during global cleanup in the main thread isn't necessarily correct, since they weren't deactivated in the threads that had already terminated a long time ago. Why doesn't this cause an error to be returned by the operating system? Maybe they decided to be a bit lax regarding this matter, who knows...

I get what you're saying about having the internal context and looking at ways to prevent these unnecessary globals, but does the context sharing create serious memory leaks?
The context sharing does not create leaks itself, it is the fact that the way SFML asserts that there always has to be a context active on every thread that makes it impossible to free the internal contexts until SFML is sure that no more sf::GlResources depend on contexts any longer (which can only happen in the scenario I just mentioned). Context sharing is only insofar an "issue", due to the fact that SFML currently uses a dedicated context to share with the others, although this dedicated context doesn't do anything useful otherwise.

Quote
Wouldn't that mean sfml-graphics would depend on sfml-window? That seems counterintuitive to the goal of reducing coupling?
sfml-graphics already depends on sfml-window. But as I see it now, certain things, like extension loading and raw OpenGL object management should be moved out of sfml-graphics and perhaps into a dedicated opengl module. That in turn would be what people that want simplified access to OpenGL objects in a RAII manner would be using, and that would also alleviate them of having to rely on SFML context management when they choose to do it themselves.

Ideally, sfml-graphics shouldn't have to be worried about the underlying graphics library or objects in use. It would rely on a module that serves no other purpose than to generalize and simplify library specifics so that people who use that library (and not sfml-graphics) would have the same access to those objects that sfml-graphics provides now, with as much flexibility as e.g. raw OpenGL code would provide them as well.

When I mentioned coupling, I meant bidirectional coupling. Certain concepts, such as sf::GlResource are used almost exclusively by sfml-graphics, and yet are part of sfml-window, probably because sf::Window and sf::Context were designed to be sf::GlResources themselves, although I personally don't consider them to fit into that idea as well as the other objects do. Extension loading code should technically also be part of the sfml-window module, since that module occasionally has to load extensions itself as well. I actually ended up doing this in gl_dev, since it made sense to me. The glLoadGen implementations that are still part of sfml-graphics no longer contain operating system specific code. That was moved into sfml-window so it could be reused by modules that are to be built on top of it.
Title: Re: SFML Context Management
Post by: wintertime on October 02, 2014, 03:09:45 pm
I already said somewhere that I'm in favor of ripping out all those hidden contexts.

For a typical single threaded SFML program that beginners and intermediates produce it would be just:
- create (and maybe activate) the window before using any other object from the graphics part
- keep the window object alive (and activated) until you stop using any objects from the graphics part
(- it may or may not be needed for the user to create a context object after that and also keep it alive, depending on if windows continue to contain one, which only depends on how SFML implementers want it done)

For an advanced multithreaded program it would just require additionally to create a context object on each thread that needs to access graphics and keep that alive and active there until the thread is stopped or does not need to access graphics anymore.
Everything else could still be handled internally by SFML although it does not create any hidden contexts anymore.

Therefore, people would not need to be afraid of explicit context management, because it comes down to very simple requirements on the user, which all people are already following with Textures that need to stay alive while they are used for a Sprite and fonts that need to stay alive while used for text. ;)
Title: Re: SFML Context Management
Post by: Ixrec on October 02, 2014, 11:52:17 pm
Well, I've never written a line of OpenGL code, and despite all my lurking on the forum reading threads like this I still don't really understand this context stuff, so hopefully I count as a "beginner".

Personally I don't think I can separate the question "Should SFML context management be more explicit?" from the question "What would the concrete API changes be?"  If it was impossible to expose manual context management without making the API as off-putting as raw OpenGL, then I'd be firmly against it.  But if it was possible to expose explicit context management with a minor change such as requiring an extra argument to all your RenderTargets (which shouldn't be all that numerous), then I'd be all for it.  That said, I'm under the (possibly naive) impression that the latter is far more likely.

As a semi-competent noob, the thing I really like about SFML is not that it's simple: it's that it lets you choose exactly how simple or complicated each part of your app should be.  In C++, you only pay (in performance) for what you use.  In SFML, things are only complicated if you want to do something complicated with them.  As a simple example, if you want to do 3D graphics, SFML lets you go "down" to raw OpenGL for that without complicating the window, audio, and networking APIs in the slightest.  Context management is one of the few things that does not currently work this way, and even if I never need it, that should change somehow.

But even after reading all of the above discussion (and many past threads about this), I still have very little confidence that I understand this context stuff.  So, here are a bunch of high-level claims I think are true about SFML and context managment, but probably aren't.  How close am I?
Quote
  • If all your rendering is done in a single thread, then in principle you can create a single context at the beginning of the thread, destroy it when the thread dies (or just use RAII), and never think about contexts once in the rest of your code.
  • For the vast majority of simple apps, and probably a lot of complex ones, rendering is best done in a single thread anyway,
  • The people who are interested in using raw OpenGL or multiple threads or multiple OpenGL contexts are sophisticated users that can and want to manage such things themselves.
  • If the shared context and internal contexts and RenderTexture contexts were simply erased from SFML, it would have no real effect on a program with only one thread.
  • Even if all implicit context management were erased from SFML, the burden to the beginner would probably consist of nothing more than writing "sf::Context context;" at the start of his main() function.
  • Even for advanced users, there aren't very many things an explicit context management API would have to include.  All I can think of is: create and destroy a context, share contexts with each other, activate and deactivate contexts for the current thread, pass contexts to another thread, and construct framebuffers/vertex array objects bound to a given context.  And sf::Context already has some of that.
  • It's possible to have implicit and explicit context management coexist in a relatively simple and intuitive way, such as having sf::GlResources secretly construct one (and only one!) context if they find that the user never created one themselves in that thread.
  • Certain methods like the static sf::Shader::isAvailable() that (somewhat unintuitively) require an OpenGL context might be better off as non-static methods like sf::Context::areShadersAvailable().
  • The only "benefit" of the way SFML context management currently works is that if you're passing drawable objects between threads, a draw() command will never fail because you made a contex-related error.  But beginners probably shouldn't do multithreading in the first place, and it does nothing to stop the "white square problem", so it's a pretty weak benefit.
Title: Re: SFML Context Management
Post by: binary1248 on October 03, 2014, 01:28:30 am
If all your rendering is done in a single thread, then in principle you can create a single context at the beginning of the thread, destroy it when the thread dies (or just use RAII), and never think about contexts once in the rest of your code.
This is true.
For the vast majority of simple apps, and probably a lot of complex ones, rendering is best done in a single thread anyway
For almost all apps, and especially ones that exploit the latest and greatest OpenGL. Even guys from Nvidia and AMD say that internally, the driver has to synchronize anyway and you would not gain much if anything at all from putting much into a secondary thread. People who still believe stuff like this are simply misguided and too stubborn to realize things have changed over the years.
The people who are interested in using raw OpenGL or multiple threads or multiple OpenGL contexts are sophisticated users that can and want to manage such things themselves.
You only really need multiple OpenGL contexts if you multi-thread your OpenGL which doesn't offer that many benefits as I already mentioned. Often people are too lazy to separate out the non-OpenGL stuff from the OpenGL stuff because "putting OpenGL stuff into a separate thread works anyway right?".
If the shared context and internal contexts and RenderTexture contexts were simply erased from SFML, it would have no real effect on a program with only one thread.
This is true. In fact, getting rid of the shared context is fairly trivial. One could just use a guarded std::set containing all currently existing contexts to share with future ones. Laurent chose to have a separate shared context, because according to him, sharing with contexts that were active didn't work for him back then...
Even if all implicit context management were erased from SFML, the burden to the beginner would probably consist of nothing more than writing "sf::Context context;" at the start of his main() function.
The context is used to choose a target for draw commands, thus it would have to be bound to the window that is created before it. Currently every context that isn't bound to a visible window has an "in-memory window". This is also the reason why creating/destroying FBO sf::RenderTextures in fullscreen causes flicker, windows are created and destroyed in the background.
Even for advanced users, there aren't very many things an explicit context management API would have to include.  All I can think of is: create and destroy a context, share contexts with each other, activate and deactivate contexts for the current thread, pass contexts to another thread, and construct framebuffers/vertex array objects bound to a given context.  And sf::Context already has some of that.
This is true. In my modern GL test sandbox I also use for testing gl_dev, I run all graphics in a single thread, and I also resort to semi-manual context management by instantiating my own sf::Contexts from time to time.
It's possible to have implicit and explicit context management coexist in a relatively simple and intuitive way, such as having sf::GlResources secretly construct one (and only one!) context if they find that the user never created one themselves in that thread.
This is not that simple. The whole idea of this discussion is to get rid of all the stuff that happens secretly, especially because they are the things causing problems. SFML wouldn't be allowed to assume the user would have did it anyway, because getting rid of contexts that were unintentionally created is a very difficult task. I still agree though, that if a user decides that they really want to go down the path of doing sfml-graphics in a separate thread, they should be required to understand that they need to manage the secondary context on their own.
Certain methods like the static sf::Shader::isAvailable() that (somewhat unintuitively) require an OpenGL context might be better off as non-static methods like sf::Context::areShadersAvailable().
This is also something I considered a long time ago. Instead of having each class probe for specific capabilities on their own, there should be a centralized way of doing so. Many other libraries already allow the user to use a single point to probe any/all capabilities the card supports. As an example of stuff an advanced SFML user (who doesn't use any raw OpenGL whatsoever) would want to probe: How many uniforms does the current hardware support? If they did not know this, some shaders would simply not work on specific hardware.
The only "benefit" of the way SFML context management currently works is that if you're passing drawable objects between threads, a draw() command will never fail because you made a contex-related error.  But beginners probably shouldn't do multithreading in the first place, and it does nothing to stop the "white square problem", so it's a pretty weak benefit.
This is true, and probably the most important point to understand. SFML currently makes a lot of sacrifices in order to hide the true complexity of multi-threading graphics related development. It isn't a good idea to start with, and if you ask me, SFML shouldn't hide its true nature from the beginner only so that they can discover its problems further down the road.
Title: Re: SFML Context Management
Post by: Tex Killer on October 03, 2014, 02:29:01 am
Ok, since you seem to know contexts in detail, can you confirm those points for me as well?

• In order to have a render target (Window or RenderTexture, for example) you must have a context bound to that target.
• Every context is bound to a render target, even if to a "virtual" and useless one.
• Each context can only be used by one thread at a time, so you can't use multiple threads to draw to the same render target.

If those points are all true and if I understood the "context leaking" problem right, then the simplest and cleanest way to solve those problems would be to have an internal SFML graphical thread (like the internal audio thread) for each render target, and all drawing operations on said render targets would be performed by that thread. You can have a poll of drawing operations for the render target and have that thread go over it whenever a new operation pops up.

You said it yourself that multithreading is useless to gain performance on the context operations, so this would keep contexts hidden and only create contexts when neccessary. That is, assuming I understood these contexts right. If the drawing operations have useful returning values, it is always possible to block the calling thread untill the drawing thread finishes the task, and get the result back afterwards.
Title: Re: SFML Context Management
Post by: Hapax on October 03, 2014, 03:06:40 am
The Q&A with Ixrec and Binary was massively informative. The whole thread is, really. I can't say that I understand it all though.
As Ixrec was saying, if it's more complicated that just adding an extra line or parameter to a constructor, it'll take newbies (like me) a while to grasp and the newer ones might not stay if it's too hard to get something going immediately.
Personally, I'd like to learn everything about it but I just don't think I'm that good and I know I, for one, would struggle if the interface got closer to raw OpenGL. That said, if we're hand-held through the process and the beginner tutorials are as good as they are now, anything's learnable.

Binary, I think you should transfer your brain to the internet. Come on; share it. Don't be selfish!
(you could clear out some space to learn new stuff...)
That said, I think you're on your way to doing that via this forum  ;D
Title: Re: SFML Context Management
Post by: binary1248 on October 03, 2014, 03:17:27 am
In order to have a render target (Window or RenderTexture, for example) you must have a context bound to that target.
This is only partially true.

For GPUs that don't support framebuffer objects, SFML has to resort to emulation of offscreen framebuffers through a separate context. In that case, you would select which to render to via activation of the respective contexts.

For GPUs that do support framebuffer objects, as long as they are only used in the same thread as the window whose context was already created, they do not require their own context. Selection of where to draw to is done via the OpenGL function glBindFramebuffer (https://www.opengl.org/wiki/GLAPI/glBindFramebuffer). Since framebuffer objects cannot be shared between contexts, SFML would have to somehow see to it that the window's context is activated in the secondary thread in order to use the framebuffer object there. This would mean deactivating it in the main thread, and the whole issue becomes very very complex.

If the user could promise that they would not transfer sf::RenderTextures between threads, then the whole situation would become much simpler to implement, but SFML does not force this restriction on them, which is why we have the current implementation.

Every context is bound to a render target, even if to a "virtual" and useless one
If by that you mean "has a framebuffer", then yes... This is one of the things I mentioned in my original post.

Each context can only be used by one thread at a time, so you can't use multiple threads to draw to the same render target
You can use multiple threads to draw to the same render target. What people don't fully understand is that you have to manually call .setActive(false) in the previous thread before drawing from a secondary thread. There is currently no thread-to-thread communication to "inform the previous thread to relinquish its active context" automatically. This has to be done by the user. If this is not done, .setActive(true) (called internally by sfml-graphics as well) in the secondary thread would return false to notify of failure, and nothing will be drawn.

If those points are all true and if I understood the "context leaking" problem right, then the simplest and cleanest way to solve those problems would be to have an internal SFML graphical thread (like the internal audio thread) for each render target, and all drawing operations on said render targets would be performed by that thread. You can have a poll of drawing operations for the render target and have that thread go over it whenever a new operation pops up.
It's not as simple as you think, not to mention the overhead would be astronomical and simply kill performance.

Audio processing is very very different from graphics processing. What is a sample in audio processing can probably be compared to a frame in graphics processing. Since audio plays at a fixed sample rate, and you often have the data beforehand, it is common to pass a whole chunk of samples to the sound card to be played over a period of time. Since the idea behind graphics rendering is that the frame is constructed from the data you pass to the GPU, frame data isn't readily available in chunks beforehand, and even if it was, it would lag behind the true state of the application so bad that it can't even be considered real-time any longer. This is the case when you play movies, the frame data is already available beforehand, but it also isn't considered real-time rendering.

What you term as the "graphical thread" is in reality the driver. It runs in its own thread and queues operations from multiple contexts into a single command stream that is fed to the GPU. There is no reason to introduce another synchronization point that wouldn't even benefit that many people in the end. All people have to be aware of is that it really only makes sense doing graphics related stuff from a single thread. It isn't impossible to do, I do it all the time. It just takes a bit of getting used to and also requires that they acknowledge the fact that they indeed do not gain from multi-threading in this sense.

You said it yourself that multithreading is useless to gain performance on the context operations, so this would keep contexts hidden and only create contexts when neccessary. That is, assuming I understood these contexts right. If the drawing operations have useful returning values, it is always possible to block the calling thread untill the drawing thread finishes the task, and get the result back afterwards.
I don't understand much of what you said there. What I have said time and again is that multi-threading doesn't provide the benefits many people think they are getting when offloading graphics operations to separate threads. This has nothing to do with "context operations" whatever you meant by that. As soon as SFML has to go to OpenGL to get stuff done, if that is happening in a thread other than the main one, you are doing something wrong.

And if you think drawing "takes time", that is another misconception that beginners seem to have. When you tell OpenGL to "draw" stuff, what you are doing is queuing a command for it to render that stuff at a later point in time. It doesn't happen immediately. Obviously if you have multiple threads all issuing draw commands, whoever issues first will get their command executed first by the GPU, followed by the second command that got issued, all according to an absolute time scale. This is why multi-threaded OpenGL makes almost no sense in almost all cases and only serves to confuse beginners even more than necessary. Also, drawing doesn't "produce results" in terms of return values, if that's how you understand it. The only results that it produces are what you end up seeing on your monitor. OpenGL doesn't care whether it is what you expected or not. It just does what you tell it to do, and if that was successful it will not return any error which implies success. Garbage In Garbage Out.

Also, trying to reduce the number of times contexts have to be created and destroyed is beneficial for performance. Creating contexts on my machine takes something on the order of milliseconds, and if you imagine doing that 10s of times per frame, it will surely kill performance. The key is not to make sure a context exists by creating and destroying them just for single use, it is about reusing contexts that were already created prior in an intelligent manner.
Title: Re: SFML Context Management
Post by: Tex Killer on October 03, 2014, 03:45:49 am
Ok, but everything you said just confirmed that my suggestion would be great. Since graphical operations are "returnless", you can just put them on the poll, wake the drawing thread up if it is sleeping and return. It wouldn't have any delay whatsoever for the user's code, and since the operations are going to be done sequentialy anyway, you won't lose any performance (by having only one thread doing all the drawing sequentialy for each render target). You've said that it would be so slow it would cease to be realtime, but why?
Title: Re: SFML Context Management
Post by: zsbzsb on October 03, 2014, 04:04:22 am
@Tex Killer: just stop painting yourself into another hole (like on the OpenAL thread). If you don't believe us - then fine, but if you want to do stuff so wrong you can, we won't stop you. Having internal threads won't help performance and it also has absolutely nothing to do with this thread.
Title: Re: SFML Context Management
Post by: binary1248 on October 03, 2014, 04:21:51 am
Ok, but everything you said just confirmed that my suggestion would be great.
I would like to see a statement by statement reasoning for why you think this. It seems you didn't fully understand what I said.

Since graphical operations are "returnless", you can just put them on the poll, wake the drawing thread up if it is sleeping and return. It wouldn't have any delay whatsoever for the user's code, and since the operations are going to be done sequentialy anyway, you won't lose any performance (by having only one thread doing all the drawing sequentialy for each render target).
It is ludicrous to think that dedicating another thread just to synchronize access to the driver would help solve this problem. I don't know how much multi-threading you have done, but you do understand that you would have to take care of data synchronization yourself in this case right? Not to mention that there are also functions that don't draw stuff but query the driver for data as well. What would you do in that case? Copy the OpenGL function signatures so you can relay these data structures back and forth as well? What about selecting the right context to draw to in multi-window applications? Would you really have to pass the context to activate along with each function that you call? What benefit does that provide over merely activating the contexts in the main thread and doing all the work there? And the part that I really don't understand is: How would this solve the problem that we are already currently having?

You've said that it would be so slow it would cease to be realtime, but why?
You've taken this statement out of context. Please reread that post again if you don't understand.

SFML, and other middleware has done this a lot. They are so desperate to appeal to developers with bad practices or little understanding of the subject matter, that they end up having to hide the real problems from them. Instead of saying "don't worry, you can do what you have always been doing although it is a bad idea" why not say "you know... that is not such a good idea, why not try it like this instead"? Sure some will always complain and threaten to go to another library that keeps pampering them. But trying to keep a hold of those with objectively very very bad practices to the detriment of others willing to improve is not something I feel good about either.

I'm not saying that we need to change the way SFML does context management overnight. I'm pretty sure it is possible to keep it the way it is while dealing with some, but not all of the problems that arise. Going out of our way by introducing an additional thread just for rendering, is unheard of, and I've seen a lot over the last years.

Also, what Ixrec said should be universally true. The amount of effort required to get something done should always be proportional to the complexity of what you want done. Reducing the effort by a flat amount regardless of the complexity leads to situations like the one we have now. If the user wants to multi-thread sfml-graphics, I will tell them that I do not recommend doing it, but it is up to them what their code looks like, and if they are that desperate to do it, they can read a bit more about multi-threaded context management. If you ask me, the effort required to multi-thread properly is much much larger than the effort required to understand how contexts are used in different threads. Most of the time, unfortunately, people fail to do the former already, and we are trying to help hide the latter from them.
Title: Re: SFML Context Management
Post by: Tex Killer on October 03, 2014, 07:14:03 am
Ok, but everything you said just confirmed that my suggestion would be great.
I would like to see a statement by statement reasoning for why you think this. It seems you didn't fully understand what I said.
Ok, I'll quote your post and answer it part by part. I didn't do that before because the topic had too much text for that, and because I wasn't sure how contexts work, even after reading everything. I don't remember ever hearing anything about OpenGL contexts before reading this topic, so I am taking all of my information from here.

Since graphical operations are "returnless", you can just put them on the poll, wake the drawing thread up if it is sleeping and return. It wouldn't have any delay whatsoever for the user's code, and since the operations are going to be done sequentialy anyway, you won't lose any performance (by having only one thread doing all the drawing sequentialy for each render target).
It is ludicrous to think that dedicating another thread just to synchronize access to the driver would help solve this problem. I don't know how much multi-threading you have done, but you do understand that you would have to take care of data synchronization yourself in this case right?
What I understood from previous posts is this: OpenGL uses contexts to manage render targets, and those contexts are not "safe" for multithreading, so they have restricted each context to a single active thread. You can deactivate one thread and activate another, but you can't have two active threads on the same context, so in other words it just forces the user to do the synchronization. My suggestion is that the synchronization is done hidden inside SFML, so that the API can stay the same. The back-end particularities shouldn't have effect on SFML's public API, in my opinion (specially when we consider possible future DirectX support, for example, but lets not diverge from the topic at hand).

Not to mention that there are also functions that don't draw stuff but query the driver for data as well. What would you do in that case? Copy the OpenGL function signatures so you can relay these data structures back and forth as well?
I didn't know we had to synchronize data retrival functions as well. As I've said before, I only know this issue from what I've read here, so if I don't know all of the problems my suggestions will be flawed from the start. How does data retrieval work on OpenGL? Is it thread-safe? Does it support parallel data retrival on multiple threads? Does it use the contexts for that? In what way? I am realy clueless to this point.

What about selecting the right context to draw to in multi-window applications? Would you really have to pass the context to activate along with each function that you call? What benefit does that provide over merely activating the contexts in the main thread and doing all the work there? And the part that I really don't understand is: How would this solve the problem that we are already currently having?
You wouldn't have to pass the context, since the Window object would have its context internaly. The benefit of my suggestion would be that you wouldn't have to worry about the synchronization at all when drawing stuff, the Window's internal thread will do the "talking" with OpenGL and everything will just work. Also, we wouldn't need more contexts than the number of render targets, if my understanding of contexts is correct, but I might be wrong.

You've said that it would be so slow it would cease to be realtime, but why?
You've taken this statement out of context. Please reread that post again if you don't understand.

SFML, and other middleware has done this a lot. They are so desperate to appeal to developers with bad practices or little understanding of the subject matter, that they end up having to hide the real problems from them. Instead of saying "don't worry, you can do what you have always been doing although it is a bad idea" why not say "you know... that is not such a good idea, why not try it like this instead"? Sure some will always complain and threaten to go to another library that keeps pampering them. But trying to keep a hold of those with objectively very very bad practices to the detriment of others willing to improve is not something I feel good about either.
You say that drawing stuff on multiple threads is bad practice, but it is only bad practice if it creates unsolveable issues or if it affects resource usage significantly. Sure, you shouldn't draw from multiple threads on the same context in parallel in OpenGL, but we are talking about SFML, and as long as we can solve OpenGL's issues ourselves without affecting the performance significantly, we can choose our own way of presenting features to the user. In my opinion, the interface should only show an issue to the user if it can't be solved internaly.

I'm not saying that we need to change the way SFML does context management overnight. I'm pretty sure it is possible to keep it the way it is while dealing with some, but not all of the problems that arise. Going out of our way by introducing an additional thread just for rendering, is unheard of, and I've seen a lot over the last years.
Ok, it is unheard of, but would it work? What are the possible drawbacks? Is the number of threads limited to less than the number of possible contexts on an application? Would multiple sleeping threads (since the threads will be mostly sleeping, except when waken to draw stuff and sleep again) affect the performance of the user's thread? Would those sleeping threads compromise memory in any significant way? You have also mentioned data retrieval, and I would like to know how that is done on OpenGL (to try and adapt the "internal thread solution", if it passes the previous questions).

Also, what Ixrec said should be universally true. The amount of effort required to get something done should always be proportional to the complexity of what you want done. Reducing the effort by a flat amount regardless of the complexity leads to situations like the one we have now. If the user wants to multi-thread sfml-graphics, I will tell them that I do not recommend doing it, but it is up to them what their code looks like, and if they are that desperate to do it, they can read a bit more about multi-threaded context management. If you ask me, the effort required to multi-thread properly is much much larger than the effort required to understand how contexts are used in different threads. Most of the time, unfortunately, people fail to do the former already, and we are trying to help hide the latter from them.
So, you are suggesting that since multithreading OpenGL contexts efficiently is hard, you should just force SFML users to do it themselves and give the "OpenGL context" problem to them? I might well be underestimating the problems here (and I probably am), but until I understand the problem I won't assume it is unsolveable, so I am sorry if I am talking nonsense.
Title: Re: SFML Context Management
Post by: binary1248 on October 03, 2014, 11:40:00 am
What I understood from previous posts is this: OpenGL uses contexts to manage render targets, and those contexts are not "safe" for multithreading, so they have restricted each context to a single active thread. You can deactivate one thread and activate another, but you can't have two active threads on the same context, so in other words it just forces the user to do the synchronization.
All that is important to know, I already mentioned in the list in the original post. You can multi-thread with contexts safely, you just have to keep those constraints in mind. Instead of activating and deactivating threads, which I never talked anywhere about, you activate and deactivate contexts. Even if a thread is "inactive" whatever that means, if a context is still active on that thread, it won't be able to be activated anywhere else. You have to consider things from the context's point of view and not the thread's point of view.

And when you say that "it forces the user to do the synchronization", I have to add: It only forces them to do the synchronization if they already made the bad decision of trying to multi-thread OpenGL in the first place. Maybe this is the key. SFML users don't know how ugly the internals are that they take for granted and everything works. You don't find many OpenGL applications, either written using platform code or other libraries that end up multi-threading. The reason for this is simple: It is very very ugly. If it was hard for SFML users to do it from the beginning, I am sure they would consider better alternatives sooner in their development cycle.

My suggestion is that the synchronization is done hidden inside SFML, so that the API can stay the same.
This is the way it currently is, and if you've read what has already been posted in this thread again, you will see that this leads to problems. And before you say anything else, no, your solution will not solve any of these problems. It will merely create more of them.

The back-end particularities shouldn't have effect on SFML's public API, in my opinion (specially when we consider possible future DirectX support, for example, but lets not diverge from the topic at hand).
The aim of this discussion is to try and find a solution for the current situation. Simply saying "hide all the ugly stuff for me so that alternate renderers will look just as simple" still doesn't change the fact that that "stuff" that is hidden is problematic. And, in case people didn't know yet, SFML was designed with OpenGL in mind. I have not made many thoughts on how it would have to change to support multiple renderers, but I am pretty sure that you won't be able to recognize similarities between the new and old implementation when it gets done one day.

Please, for future posters, do not mention DirectX any more. It shouldn't even be a factor in this discussion. We are trying to fix a system that was not designed for it, and all things DirectX related can have its place in a separate thread.

I didn't know we had to synchronize data retrival functions as well. As I've said before, I only know this issue from what I've read here, so if I don't know all of the problems my suggestions will be flawed from the start. How does data retrieval work on OpenGL? Is it thread-safe? Does it support parallel data retrival on multiple threads? Does it use the contexts for that? In what way? I am realy clueless to this point.
Data retrieval isn't as complicated as you think, but that still doesn't mean you don't have to give it a bit of thought. It can have many manifestations. The most common and simple one is getting a single state value from the driver. This can be different depending on which context is active when you call the respective function. For things like taking screenshots, it is a bit more complicated. You will have to pass in a correctly sized array so that the GL can write the data you request from it into the array. This is also context dependent, and is a rather slow operation since we are reading data directly from the GPU.

If you heed what I said in the original post in regards to contexts and threads, then yes, it is thread-safe. Yes it supports parallel retrieval on multiple threads, also constrained by the same requirements. Yes it uses contexts, everything in OpenGL uses contexts, didn't you read the original post? Really... just read the original post, and especially the list, over and over again until you understand it. From what you have said so far, I highly doubt you understood even a bit of it, and it is necessary for continuing this discussion in a meaningful way.

You wouldn't have to pass the context, since the Window object would have its context internaly. The benefit of my suggestion would be that you wouldn't have to worry about the synchronization at all when drawing stuff, the Window's internal thread will do the "talking" with OpenGL and everything will just work. Also, we wouldn't need more contexts than the number of render targets, if my understanding of contexts is correct, but I might be wrong.
This is worse than I expected... Now I understand that you were aiming to have a separate "rendering thread" for each context that can be used in some way... just so you can omit having to constantly select which one to activate? There are beginners on this forum... that just start out with game development, and after not too long, even they understand that having more threads than necessary is not a good thing. I don't know where you are coming from, but if you think that resorting to creating an excessive number of threads to get anything done is a good idea, I am sorry to have to tell you that you are mistaken. The same is said to beginners as well as advanced developers: Avoid threads while you can. Unless you know exactly why they provide a benefit, you won't gain much from them.

You say that drawing stuff on multiple threads is bad practice, but it is only bad practice if it creates unsolveable issues or if it affects resource usage significantly.
If that is the case, then you must also oppose usage of smart pointers over raw pointers too right? Raw pointers don't create unsolvable issues and have better resource usage than smart pointers, and yet the universal opinion from any established C++ programmer is that you should rarely ever use raw pointers (and certainly not owning ones) in modern C++ code.

Bad practice doesn't always have to result in catastrophic results. This is the reason why it is bad practice (http://en.wiktionary.org/wiki/practice).

A customary action, habit, or behavior; a manner or routine.

While it is legitimate, it is something the general majority recommend against, because there are known issues tied to it. In the case of context management, these issues might not be of interest to you or anyone else, but they are still there. Letting people know that there are difficulties with the way they planned to do things (even if not within their code) is the first step in helping them consider an alternative, and often superior, solution. If this has to be done through the API, then so be it. The last thing I want to hear is people blaming the API for their misinformed decisions, and "had they known earlier..." etc.

Sure, you shouldn't draw from multiple threads on the same context in parallel in OpenGL, but we are talking about SFML, and as long as we can solve OpenGL's issues ourselves without affecting the performance significantly, we can choose our own way of presenting features to the user. In my opinion, the interface should only show an issue to the user if it can't be solved internaly.
Sure... here at SFML, we like to create our own concepts that users have to embrace to use SFML efficiently. But that still doesn't change the fact that those concepts have to map to "real-world" implementations. There are still connections between the two. And as uncomfortable as it is, it is not possible to hide every single implementation detail from seeping through the API to the user. Maybe SFML users are merely using SFML as a stepping stone on their way towards more advanced graphics programming? You never know. But twisting reality just because we can isn't something we do. We simplify it, where we can. Where we can't, we have to pass it on to the user. And the whole point of this discussion is that initially, SFML thought that it could hide the complexities of multi-threaded OpenGL without causing too many problems, but over time that assumption has lost more and more weight, which is why we are in our current situation.

Ok, it is unheard of, but would it work?
You tell me... I am not going to go ahead and spend 10s maybe even 100s of hours implementing something I find problematic from the start. From reading what you have proposed, I would say, yeah, it can work, but have fun with your 2 FPS applications. Ultimately, most ideas in software development, no matter how crazy they are, can be implemented. You just have to spend a varying amount of time dealing with the effects you did not anticipate during brainstorming.

What are the possible drawbacks? Is the number of threads limited to less than the number of possible contexts on an application? Would multiple sleeping threads (since the threads will be mostly sleeping, except when waken to draw stuff and sleep again) affect the performance of the user's thread? Would those sleeping threads compromise memory in any significant way?
I've already reiterated over the drawbacks enough times in the last posts and this one. If you still don't understand them, please read them again. I assume that you are somewhat familiar with threading (and more importantly efficient multi-threading), since you often propose "solutions" to problems that involve them in some way. So can't you just draw your own conclusions? If this is not the case, i.e. you are not experienced in threading, then I highly recommend you refrain from using them in any future solutions you propose anywhere. I don't mean to be offensive, but these solutions don't necessarily make you look good...

You have also mentioned data retrieval, and I would like to know how that is done on OpenGL (to try and adapt the "internal thread solution", if it passes the previous questions).
I already mentioned this above. And instead of relying on my spending well over an hour typing a reply for you, why don't you do a bit of research on your own? ;) OpenGL isn't that hard to understand, especially if you concentrate on specific parts of the API.

So, you are suggesting that since multithreading OpenGL contexts efficiently is hard, you should just force SFML users to do it themselves and give the "OpenGL context" problem to them? I might well be underestimating the problems here (and I probably am), but until I understand the problem I won't assume it is unsolveable, so I am sorry if I am talking nonsense.
What you misunderstand, is that the majority of users probably won't be affected code-wise by any changes. They already assume that performing all OpenGL related stuff within the main thread is the way to go, and like I said, there is no reason to annoy these users. It is the ones who already resort to bad practice that will have to potentially make small adjustments to their code. Maybe this is a signal to them that what they are doing is not a good idea. If this motivates them to change the way they think of using sfml-graphics to single threaded, then I am nothing but happy.

Merely dismissing this discussion as "passing the OpenGL problem on to the user" is not correct in any way. We strive to keep it as simple to write code as it currently is, but if we notice that some parts of the API allow or even motivate people to engage in bad practices, then that is something we should consider changing for the better.
Title: Re: SFML Context Management
Post by: Tex Killer on October 03, 2014, 09:08:49 pm
What I understood from previous posts is this: OpenGL uses contexts to manage render targets, and those contexts are not "safe" for multithreading, so they have restricted each context to a single active thread. You can deactivate one thread and activate another, but you can't have two active threads on the same context, so in other words it just forces the user to do the synchronization.
All that is important to know, I already mentioned in the list in the original post. You can multi-thread with contexts safely, you just have to keep those constraints in mind. Instead of activating and deactivating threads, which I never talked anywhere about, you activate and deactivate contexts. Even if a thread is "inactive" whatever that means, if a context is still active on that thread, it won't be able to be activated anywhere else. You have to consider things from the context's point of view and not the thread's point of view.
I meant to say that you can deactivate a context on a thread and activate it on another thread, but you can't activate it on multiple threads at the same time. I though it was obvious, but sorry for the bad wording.

My suggestion is that the synchronization is done hidden inside SFML, so that the API can stay the same.
This is the way it currently is, and if you've read what has already been posted in this thread again, you will see that this leads to problems. And before you say anything else, no, your solution will not solve any of these problems. It will merely create more of them.
I know it is already like this, that is why I've said that the API would stay the same. What I am discussing here is possible implementation changes that could solve the said problems without affecting the API.

I didn't know we had to synchronize data retrival functions as well. As I've said before, I only know this issue from what I've read here, so if I don't know all of the problems my suggestions will be flawed from the start. How does data retrieval work on OpenGL? Is it thread-safe? Does it support parallel data retrival on multiple threads? Does it use the contexts for that? In what way? I am realy clueless to this point.
Data retrieval isn't as complicated as you think, but that still doesn't mean you don't have to give it a bit of thought. It can have many manifestations. The most common and simple one is getting a single state value from the driver. This can be different depending on which context is active when you call the respective function. For things like taking screenshots, it is a bit more complicated. You will have to pass in a correctly sized array so that the GL can write the data you request from it into the array. This is also context dependent, and is a rather slow operation since we are reading data directly from the GPU.
Ok, the data depends on the active context, so we have to synchronize data retrieval as well. That could be done by blocking the calling thread until the "drawing thread" retrieves the data, and sharing the return value between threads, but I agree that it is not as clean as it could be.

If you heed what I said in the original post in regards to contexts and threads, then yes, it is thread-safe. Yes it supports parallel retrieval on multiple threads, also constrained by the same requirements. Yes it uses contexts, everything in OpenGL uses contexts, didn't you read the original post?
If it uses the active context, it can't be used in parallel. Or am I missunderstanding something? If you can't have two different threads retrieving data from the same context at the same time, I wouldn't consider it thread-safe, but that might be bad wording, so I am sorry for the missunderstanding. In the end, we have to synchronize drawing and data retrieval functions to the same context ourselves.

This is worse than I expected... Now I understand that you were aiming to have a separate "rendering thread" for each context that can be used in some way... just so you can omit having to constantly select which one to activate? There are beginners on this forum... that just start out with game development, and after not too long, even they understand that having more threads than necessary is not a good thing. I don't know where you are coming from, but if you think that resorting to creating an excessive number of threads to get anything done is a good idea, I am sorry to have to tell you that you are mistaken. The same is said to beginners as well as advanced developers: Avoid threads while you can. Unless you know exactly why they provide a benefit, you won't gain much from them.
Ok then, lets avoid multiple meaningless threads. I'll suggest another solution at the end of the post that doesn't create any aditional thread.

If that is the case, then you must also oppose usage of smart pointers over raw pointers too right? Raw pointers don't create unsolvable issues and have better resource usage than smart pointers, and yet the universal opinion from any established C++ programmer is that you should rarely ever use raw pointers (and certainly not owning ones) in modern C++ code.
Using raw pointers directly is much harder than using smart pointer. It is bad practice because unless you pay attention to all of the allocation and freeing of memory, you will create serious problems. Sure, if you don't pay attention to the synchronization of OpenGL operations it will create serious problems, but as long as we manage to take care of those problems internally, it will cease to be bad practice to the final user. If SFML used raw pointers on some internal sections of code, for example, but used them correctly, the user code wouldn't be affected by it in the least. The user wouldn't even see the pointers.

Ok, it is unheard of, but would it work?
You tell me... I am not going to go ahead and spend 10s maybe even 100s of hours implementing something I find problematic from the start. From reading what you have proposed, I would say, yeah, it can work, but have fun with your 2 FPS applications. Ultimately, most ideas in software development, no matter how crazy they are, can be implemented. You just have to spend a varying amount of time dealing with the effects you did not anticipate during brainstorming.
I still fail to see the reason for the drop in performance. Sleeping threads shouldn't affect the performance of the active threads. Anyway, I'll suggest another solution at the end of this post, please take a look.

What are the possible drawbacks? Is the number of threads limited to less than the number of possible contexts on an application? Would multiple sleeping threads (since the threads will be mostly sleeping, except when waken to draw stuff and sleep again) affect the performance of the user's thread? Would those sleeping threads compromise memory in any significant way?
I've already reiterated over the drawbacks enough times in the last posts and this one. If you still don't understand them, please read them again. I assume that you are somewhat familiar with threading (and more importantly efficient multi-threading), since you often propose "solutions" to problems that involve them in some way. So can't you just draw your own conclusions? If this is not the case, i.e. you are not experienced in threading, then I highly recommend you refrain from using them in any future solutions you propose anywhere. I don't mean to be offensive, but these solutions don't necessarily make you look good...
From what I did understand from your previous posts, the main drawback was that SFML created a lot of useless contexts so that every operation performs on a different context and we don't have to synchronize anything. I might have missunderstood the issues, but my suggestion was made considering this scenario. Please elucidate me if I am mistaken, but after reading the first post of the thread again this is still what I understood.



Ok, so I agree that creating a dedicated thread for each render target and having the context active on that thread might not be a very clean solution. An alternative solution would be to always deactivate contexts after each OpenGL operation, and block threads that want to do OpenGL operations until the previous one stops.
I am not very familiar to C++ mutexes and synchronization, but on Java the code would be something like this for every OpenGL operation:

synchronize (context) {
    // activate context
    // perform OpenGL operation
    // deactivate context
}
 

I assume it would be similar on C++. That way the context will only be active during the operation, and multiple threads trying to use the same context in parallel would be executed one at a time. What I don't know is: does activation and deactivation of contexts take much time, or is it instantaneous? This solution would be bad if activating and deactivating contexts multiple times would lead to performance issues, but if it is as trivial as changing one state, it won't matter much.



Another possible variant of the dedicated internal threads suggestion would be to have only one internal thread, and have it doing the OpenGL operations for all of the contexts. If I understood what you've said on the first post, we might gain some performance if driver operations are issued in parallel to different contexts, so this suggestion would remove this possible performance gain, but you make it sound like driver operations are so fast that the gain is trivial. If that is the case, then this solution will be viable.
Title: Re: SFML Context Management
Post by: binary1248 on October 04, 2014, 02:18:54 am
That way the context will only be active during the operation, and multiple threads trying to use the same context in parallel would be executed one at a time. What I don't know is: does activation and deactivation of contexts take much time, or is it instantaneous? This solution would be bad if activating and deactivating contexts multiple times would lead to performance issues, but if it is as trivial as changing one state, it won't matter much.
This was also a solution I presented to the other team members a while ago. At first glance, it seems like something that might actually work and isn't too complex to implement, but the devil is in the details. I was already working on a prototype of such an implementation, but switched to solving other issues a while ago before it was finished. I guess I could finish it off now and see what it performs like. The performance of these things, and if the drivers play along, is always hard to estimate solely based on the initial design. Often, reality isn't as clean as what the API documentation has you believe. Even when it is finished and seems to work, extensive tests have to be conducted on multiple systems before even considering it for inclusion into SFML.

Another possible variant of the dedicated internal threads suggestion would be to have only one internal thread, and have it doing the OpenGL operations for all of the contexts. If I understood what you've said on the first post, we might gain some performance if driver operations are issued in parallel to different contexts, so this suggestion would remove this possible performance gain, but you make it sound like driver operations are so fast that the gain is trivial. If that is the case, then this solution will be viable.
Any time a worker thread does nothing but execute stuff that could have been done in the invoker thread while blocking the invoker thread, you end up with serial execution. This defeats the whole purpose of multi-threading. The fact that contexts can't arbitrarily be passed between threads without some non-trivial synchronization routines does not mean that we keep it in a worker thread just so we don't have to perform those routines.
Title: Re: SFML Context Management
Post by: Lolilolight on October 04, 2014, 06:24:11 pm
Personnaly I've choosen SFML to go faster, if I had to manage those contexts explicitly, it would take a lot of time.

And this is the real strength of SFML.

Moreover, the context creation is different depending on the plateform and the number of plateforms are dangerously increasing, the drivers too and now we even have opengl and gles which use differents functions.

I really don't want to have to deal with that.

I know I'm probably not the better one to discuss about this, I've never used context explicitly so..., but, What'll happen for beginners like me ?

Even for a rendertexture I was not able to deal with multiple contexts with the others multi media librairies, this is why I've choosen SFML for my projects, and it's also because SFML is written in c++ and not in C.  (It's easier for enum when handling events with the automatic typedef)

You should use RAII, you have a bunch of memory leaks, if you don't want to use c++14 you can at least make your own classes for resource management like I does : http://lolilolightdevblog.wordpress.com/le-module-core-partie-3/ (http://lolilolightdevblog.wordpress.com/le-module-core-partie-3/)
The article is in French.

PS : One day I've also tried to display something on an embedded system by creating render context explicitly with EGL and GLES, it was a nightmare. :X

PS 2 : Thread management is not

Title: Re: SFML Context Management
Post by: zsbzsb on October 04, 2014, 07:45:28 pm
Lolilolight, if you don't read the thread then don't bother posting. This has nothing to do with requiring users to write platform specific code or do anything complex. On top of that, why are you mentioning RAII and memory leaks? Once again it has nothing to do with the thread. If you have nothing constructive to say it is best to say nothing at all.
Title: Re: SFML Context Management
Post by: Juhani on October 07, 2014, 01:48:24 pm
Personally I don't care how chaotic context management gets with threads, for multithreading isn't for beginners anyway. IMHO context management should be simplest for a single thread.

With these opinions, which option should I vote?
Title: Re: SFML Context Management
Post by: binary1248 on October 07, 2014, 01:54:52 pm
Personally I don't care how chaotic context management gets with threads, for multithreading isn't for beginners anyway. IMHO context management should be simplest for a single thread.

With these opinions, which option should I vote?
Since you seem to value the simplicity of single threaded context management more, implicit context management.
Title: Re: SFML Context Management
Post by: Juhani on October 07, 2014, 02:52:58 pm
Ok so I voted it, but for a clarification, it doesn't mean that I would like to keep the current system where SFML creates extraneous contexts even for a simple single-threaded app, just to pamper multithreaders, wherefore I also said that I don't care how painful context management becomes to them. SFML shouldn't create more contexts than what is needed in a single-threaded app, and multithreaders can handle contexts manually.