[SOLVED] gameSWF started corrupting the rendering since SFML 2.5.0

[oomek]

I know this might be a painful question since there are countless topics regarding mixing SFML and OpenGL that suggests wrapping SFML draw calls in pushGLStates() popGLStates(), but this is a bit more complicated matter.

The project Attract Mode I'm contributing to uses SFML framework, but also gameSWF that draws swf files on rendertextures using plain OpenGL. The SFML and gameSWF were playing nicely until SFML 2.5.0, so I'm wondering what has changed in the SFML that made the Attract Mode corrupting the rendering ( and even fonts in MSI Afterburner's OSD and crashing the VMWare sessions ) whenever the swf file is loaded.

The advices regarding usage of push/popGLStates() were based on a situation when you draw SFML stuff surrounded by OpenGL, but what if it's other way round? AM uses mainly SFML for rendering, but the Image class sometimes loads swf instead of an image, and the order of draws is based on the dynamic script written in Squirrel language. In this case OpenGL is surrounded by SFML code. Using push/popGLStates() might be a bit problematic since it would require wrapping every draw call in this and it's quite expensive as the documentation explains.

I know, no one has time for digging through some random repo, but maybe some kind soul would find some spare time to drop some ideas, or give some directions to what we should check to make AM back on it's track.

https://github.com/mickelson/attract

the files of interest are swf.cpp and fe_image.cpp

[eXpl0it3r]

There was a bigger refactoring of the RT to bring FBO support and thus major performance improvement. Naturally it brought quite a few issues with, which is largely why we released 2.5.1.

Can you give SFML 2.5.1 a try?

[oomek]

Yes, I've tried both 2.5.0 and 2.5.1

[oomek]

I was suspecting this might have something to do with the FBOs. Is there any way to force the old RenderTexture implementation from within the api?

[binary1248]

The big difference between 2.4.x and 2.5.x is that OpenGL contexts are reused where it makes sense. Performance-wise, this saves a lot of time because context switching is an expensive operation. The problem that you have now is that you are no longer working in a sandboxed environment like you were before.

This is a common "problem" I have seen when different software libraries are combined that all make use of OpenGL. Because OpenGL relies on shared state, they all end up stepping on each other's toes. To make sure that they themselves keep working in such an environment, some libraries go through the extra effort to reset all state to whatever it is that they expect it to be before they start to do their stuff. This has the side-effect that you get abysmal performance because of the redundant state changes that are introduced because of this paranoia (and probably the cause of the "OpenGL is slower that D3D" myths in circulation).

In the case of SFML and gameswf, neither bothers to clean up after the other (in SFML's case for performance reasons) so they just end up clobbering the global OpenGL state. As I said, this was not a problem before because SFML isolated each RenderTexture into a sandbox. It was an implementation detail and never something people should rely on. pushGLStates() and popGLStates() only takes care of the OpenGL state that SFML cares about. If gameswf touches anything else it will also have to be reverted to whatever SFML is expecting it to be before continuing to render.

If this is too much to ask, you could try emulating the old sandbox behaviour by creating an explicit sf::Context and activating it before activating the sf::RenderTexture. You will suffer from performance degradation relative to the current implementation, but this should not be any slower than what it was like in 2.4.x.

[oomek]

Thank you for your input. I've tried your suggestion and created a new context in the FeSwf class and indeed it fixed the issue. I've also tried forcing the RenderTextureImplFBO::isAvailable() function to always return false and it's also good. I'm yet to measure what is faster as I'm worried that it mignt have a negative impact when I create a new context for every FeSwf instance.

[oomek]

I've done some benchmarks and the context switching seems to be a little faster than the old RenderTexture implementation, we are talking about 1-2% less on the GPU with around 10% utilisation.

Thank you one more time for solving our problem. I think this tip should end up somewhere on the wiki page.

[oomek]

I've had to unmark this topic as Solved unfortunately.

Attract Mode creates SWF instances at run-time and for each instance there is a private member sf::Context. It creates huge lag in the constructor, over 1 second for 5 swf instances created. I've tried making one global context for all SWF instances and pass a reference to the constructor, but then the lag is even worse and it moves to one of the internal gameSWF functions, when it starts queueing OpenGL. It only happens during initialization, context switching during frame rendering is working fast.

Do you think if there is another way around it, or we would have to revert back to SFML 2.4.2 to get rid of those vram corruptions?

[binary1248]

Since your code should take care of cleaning its own state up itself (if it doesn't... well...) having a single context for all instances should be adequate. Debug/Trace/Profile your code to see where all the time is being spent and if it is being spent in your own code you need to come up with a way to optimize it.

[oomek]

I've moved shared context for gameSWF from the stack to the heap and the loading of 20 SWFs went down from 4000ms to 150ms, so what I believe was happening is that the context was being created before main window's context. Unfortunately I traded it for a vram leak.

[oomek]

I've check the output of the OpenGL queue with glItercept and it seems that this context switching prevents calling glDeleteFramebuffersEXT on each created RenderTexture. There is a flood of glDeleteFramebuffersEXT on closing the app.

[oomek]

I've written a small example that is showing the vram leak if I use the binary1248's suggestion. Even without drawing with OpenGL the leak is present and it's caused just by activating the context.

Please take a look, maybe I'm still doing something wrong.

#include <SFML/Graphics.hpp>

void main()
{
        // true - vram leak
        // false - ram leak, fps degradation over time, delay on exit ( comment setFrameLimit() )
        bool leak = true;

        sf::RenderWindow window(sf::VideoMode(800, 600), "");
        window.setFramerateLimit(30);
        sf::Context context;
        sf::RenderTexture *rtexture;

        if (!leak) rtexture = new sf::RenderTexture();

        sf::Event event;
        while (window.isOpen())
        {
                if (leak) rtexture = new sf::RenderTexture();

                rtexture->create(rand() % 100 + 400, rand() % 100 + 400);
                rtexture->clear(sf::Color::Cyan);

                context.setActive(true);
                rtexture->setActive(true);
                //drawing OpenGL stuff here
                rtexture->setActive(false);
                context.setActive(false);

                rtexture->display();
                window.clear();
                sf::Sprite sprite(rtexture->getTexture());
                window.draw(sprite);
                window.display();

                if (leak)
                {
                        delete rtexture;
                        rtexture = NULL;
                }

                window.pollEvent(event);
                if (event.type == sf::Event::KeyPressed && event.key.code == sf::Keyboard::Escape)
                {
                        if (!leak)
                        {
                                delete rtexture;
                                rtexture = NULL;
                        }
                        window.close();
                }
        }
}
 

In case you are wondering why I call new before create(), rtexture is a member of a class that gets destroyed every time AttractMode reloads SWF. And that's because the artwork type can be one the following ( an image, a video, or swf )

[eXpl0it3r]

Afaik this is expected behavior. Due to some reason (binary1248 can provide that) the RT contexts can only be cleaned at the end of application. It's not "really" leaking, it's just not cleaned during runtime.

It's generally not advised to re-create a lot of render textures. Instead, it's better to just reuse existing ones.

[oomek]

I could define the RenderTexture as a static class member, but unfortunately I have no control over how many instances of FeSWF the script generates.

[binary1248]

It is important that you "jail" the RenderTexture to that one context that you made for it whenever you do anything with the RenderTexture, this includes destroying it. We can't release the corresponding FBO while you are still working in a context other than the one you created it in, which is the case in your example. This leads to an accumulation of FBOs over time that can never get released because you are explicitly blocking us from doing so.

The corrected code would look like this:

#include <SFML/Graphics.hpp>

void main()
{
    // true - vram leak
    // false - ram leak, fps degradation over time, delay on exit ( comment setFrameLimit() )
    bool leak = true;

    sf::RenderWindow window(sf::VideoMode(800, 600), "");
    window.setFramerateLimit(30);
    sf::Context context;
    sf::RenderTexture *rtexture;

    if (!leak) rtexture = new sf::RenderTexture();

    sf::Event event;
    while (window.isOpen())
    {
        // jail begin
        context.setActive(true);

        if (leak) rtexture = new sf::RenderTexture();

        rtexture->create(rand() % 100 + 400, rand() % 100 + 400);
        rtexture->clear(sf::Color::Cyan);

        rtexture->setActive(true);
        //drawing OpenGL stuff here
        rtexture->setActive(false);

        rtexture->display();

        // jail end since window.clear() will activate another context
        window.clear();
        sf::Sprite sprite(rtexture->getTexture());
        window.draw(sprite);
        window.display();

        if (leak)
        {
            // jail begin since window's context is active
            context.setActive(true);
            delete rtexture;
            rtexture = NULL;
        }

        window.pollEvent(event);
        if (event.type == sf::Event::KeyPressed && event.key.code == sf::Keyboard::Escape)
        {
            if (!leak)
            {
                delete rtexture;
                rtexture = NULL;
            }
            window.close();
        }
    }
}