[Win32] Multithreaded rendering + multiple windows + recording software = fail

[Suslik]

Ok, here's the minimal code:

#include <SFML/Graphics.hpp>

sf::RenderWindow *window1, *window2;
bool isRunning = 1;

void ThreadFunc1()
{
  while(isRunning)
  {
    window1->clear();
    window1->display();
  }
}

void ThreadFunc2()
{
  while(isRunning)
  {
    window2->clear();
    window2->display();
  }
}

int main()
{
  window1 = new sf::RenderWindow(sf::VideoMode(800, 600), "Awesome window");
  window1->setActive(false);
  sf::Thread thread1(ThreadFunc1);
  thread1.launch();

  window2 = new sf::RenderWindow(sf::VideoMode(800, 600), "Not as awesome window as the first one but still great");
  window2->setActive(false);
  sf::Thread thread2(ThreadFunc2);
  thread2.launch();

  while(isRunning)
  {
    sf::Event event;

    while(window1->pollEvent(event))
    {
      if(event.type == sf::Event::Closed)
      {
        isRunning = 0;
      }
    }
    while(window2->pollEvent(event))
    {
      if(event.type == sf::Event::Closed)
      {
        isRunning = 0;
      }
    }

    printf("we're alive\n");
  }
}
 

It creates 2 windows and a separate rendering thread for each of them while processing window events in the main thread. It works just fine if you compile and run it. However if you have some kind of recording software that captures OpenGL or D3D context running in your system, after running this test program for a few secods you get this:

or this:

but most often this:

When the crash happens, stack pointer always points either to window1->display() or window2->display().

The crash happens with different kinds of recording software like Bandicam, Action! and a few others when they're in OpenGL/D3D recording mode. It happens when these programs are just running and not even recording anything. The test program may run for a few seconds, sometimes it may semi-freeze for a second or two and reliably crashes after 3-10 seconds of running.

Any ideas how to work this around or reason why this happens?

[eXpl0it3r]

I'm not sure if you know how capture software work, but what they usually do is injecting themselves into your application and grabbing back buffer directly from your application.
Now if you use multiple threads and multiple windows chances are pretty high that their injection code isn't made for this kind of scenario, I mean which game does use multiple windows to rendering from different threads?
And don't forget that you're using global RenderWindows which itself can in certain cases already create issues. Plus you're not deleting the windows at the end.

Anyways I tried it with Open Broadcaster Software and yes, as soon as I start the preview the application will crash. Since I ran it in debug mode, I got some glCheck error:

An internal OpenGL call failed in Texture.cpp (526) : GL_INVALID_FRAMEBUFFER_OPERATION, the object bound to FRAMEBUFFER_BINDING is not "framebuffer complete"

No idea if that is of any help.

I went ahead and made everything single threaded. This at least didn't lead to any crashes, but OBS still couldn't get an image from it and the capture instead stayed black.

#include <SFML/Graphics.hpp>


bool isRunning = 1;


int main()
{
  sf::RenderWindow *window1, *window2;
  window1 = new sf::RenderWindow(sf::VideoMode(800, 600), "Awesome window");
  window2 = new sf::RenderWindow(sf::VideoMode(800, 600), "Not as awesome window as the first one but still great");

  while(isRunning)
  {
    sf::Event event;

    while(window1->pollEvent(event))
    {
      if(event.type == sf::Event::Closed)
      {
        isRunning = 0;
      }
    }

        window1->clear(sf::Color::Red);
        window1->display();


    while(window2->pollEvent(event))
    {
      if(event.type == sf::Event::Closed)
      {
        isRunning = 0;
      }
    }

    window2->clear(sf::Color::Blue);
    window2->display();

    printf("we're alive\n");
  }

  delete window1;
  delete window2;
}
 

And if I render just one window, either with the multi-threaded or the single threaded code, i.e. just calling clear() and display() on one window, the OBS can capture the rendering and nothing crashes.

My conclusion: Capturing software is not made for this kind of scenario, i.e. multiple window with multiple rendering "loops".
If you really need to windows, you might want to think about splitting up the windows in two applications, maybe then the capture software will again be able to differentiate the rendering.

[binary1248]

An internal OpenGL call failed in Texture.cpp (526) : GL_INVALID_FRAMEBUFFER_OPERATION, the object bound to FRAMEBUFFER_BINDING is not "framebuffer complete"

That error is probably generated inside the .dll that OBS injects at this line. OBS hooks certain functions in order to have a reference point to know when to copy over framebuffer data into its memory space. Their code also only really functions for applications where there is only a single framebuffer to copy from. As soon as you have multiple framebuffers (because of multiple windows) and even worse, they run unsynchronized on their own threads, the OBS code will not perform the correct operations in the correct contexts at the correct time, and all of that has to happen in order for capturing to work.

Multi-threaded OpenGL is also kind of like playing with fire, avoid it if you can, it will save you a lot of headaches.

Like eXpl0it3r said, OBS and in general recording software was not made for this kind of use case. This isn't even specific to SFML, you can try to find other multi-window software that uses independent loops and you will encounter the same problem. Thus this isn't per se a "bug" but rather a limitation of the way recording software works. There is no uniform and standardized way to do such things (it's even worse with OpenGL than with Direct3D), hence every implementation is more or less a hack that "might work" most of the time and for typical applications, but is not guaranteed to work in every scenario.

[Suslik]

This isn't even specific to SFML, you can try to find other multi-window software that uses independent loops and you will encounter the same problem.

A proof please? If it is indeed so I suppose the behaviour is acceptable. Advice like "avoid using multithreaded rendering" is not, because it's supposed to work and I have my reasons for using it.

[binary1248]

A proof please? If it is indeed so I suppose the behaviour is acceptable.

I already linked you to the OBS source code. You can see how they do stuff. If that doesn't convince you that this can't work then you can look for such applications yourself. Like I said, the reason why recording software choose not to explicitly support this is because it is are rare occurrence. Maybe there is software that does support it? Perhaps closed source? Maybe they even explicitly mention that they support such applications... I don't have the time to look for them for you unfortunately.

As I already said in my previous post, recording software never guarantee that they will work with all applications. If it does, great. If it doesn't, you can approach them and ask them if they are willing to rectify the problem. It is the responsibility of the maintainers of the recording software to make sure their software works with applications and not the other way around.

If an application developer wanted their users to be able to record at all costs, they would be able to build such a feature directly into the application which would alleviate the need for external software. I wouldn't say that most applications actively try to prevent external software from recording, but they aren't going out of their way to make it as simple as possible either.

Advice like "avoid using multithreaded rendering" is not, because it's supposed to work and I have my reasons for using it.

The fact that the low level API allows for something doesn't mean that it is well supported. The exact same API came out more than 15 years ago and hasn't changed one bit, whereas multi-core CPUs have become commonplace now. Sure, you hear about drivers supporting "multi-threaded OpenGL" etc. but to what extent do they really "support" it? We've had many reports of multi-threaded drivers breaking even the most simple applications, and that isn't specific to SFML either. I've seen posts on other forums where people have had issues with multi-threaded driver implementations when playing AAA OpenGL games.

This might not be what you want to hear, but it is common knowledge by now that the OpenGL API simply does not support efficient multi-threaded use. Of course there are going to be edge cases where one does see a minor boost in performance, but to exploit these scenarios, one has to have in depth knowledge about how the driver works and shape the workload accordingly.

I'm interested, what are your reasons for using it? Performance? Code simplicity? Maybe both? Because I can tell you that you aren't going to get both. It's always going to be a trade-off between simplicity and performance, as is typical with low level APIs.

[Suslik]

I don't have the time to look for them for you unfortunately.

I thought it's just a common sense to prove your point if you claim a very debatable statement to be a fact. I have never encountered any problems with other programs crashing while recording so apparently I'm curious that you did.

I'm interested, what are your reasons for using it? Performance? Code simplicity? Maybe both? Because I can tell you that you aren't going to get both. It's always going to be a trade-off between simplicity and performance, as is typical with low level APIs.

I'm writing a physics simulation plugin for some program. The plugin is single-threaded CPU-bounded due to heavy physical calculations. Debug renderer is GPU-bounded but takes considerable CPU time as well because it processes a lot of info to visualize it. If I run them both in one thread their computation time adds and debug renderer has a significant impact on plugin performance overall. If I run them in separate threads, debug renderer does its thing independently and since it's read-only for physical data I barely need any syncronizations -> sync overhead is negligible.

[binary1248]

The plugin is single-threaded CPU-bounded due to heavy physical calculations. Debug renderer is GPU-bounded but takes considerable CPU time as well because it processes a lot of info to visualize it. If I run them both in one thread their computation time adds and debug renderer has a significant impact on plugin performance overall. If I run them in separate threads, debug renderer does its thing independently and since it's read-only for physical data I barely need any syncronizations -> sync overhead is negligible.

But... this isn't multi-threaded rendering... unless I am understanding you wrong. If there is only a single instance of your renderer, then running it in parallel to your physics simulation doesn't mean you are rendering from multiple threads simultaneously. I assumed from the thread title and your initial example that you had multiple truly independent loops rendering simultaneously, which doesn't provide a performance benefit over simply rendering both windows from the same thread.

[Suslik]

The plugin is single-threaded CPU-bounded due to heavy physical calculations. Debug renderer is GPU-bounded but takes considerable CPU time as well because it processes a lot of info to visualize it. If I run them both in one thread their computation time adds and debug renderer has a significant impact on plugin performance overall. If I run them in separate threads, debug renderer does its thing independently and since it's read-only for physical data I barely need any syncronizations -> sync overhead is negligible.

But... this isn't multi-threaded rendering... unless I am understanding you wrong. If there is only a single instance of your renderer, then running it in parallel to your physics simulation doesn't mean you are rendering from multiple threads simultaneously. I assumed from the thread title and your initial example that you had multiple truly independent loops rendering simultaneously, which doesn't provide a performance benefit over simply rendering both windows from the same thread.

The debug renderer consists of multiple windows and every window is rendered from a separate thread.