Blur shader confusion (Update: full source provided)

[SteelGiant]

I'm trying to repeatedly blur an image to get it to dissolve. I'm doing this by drawing into one sf::RenderTexture and then drawing from that into another sf::RenderTexture with a blur fragment shader, then drawing that image back to the first RenderTexture (with no shader) ready to draw again.

//drawRenderTexture starts off clear at the start of the program

drawRenderTexture.draw(drawVertices, nVertices, sf::PrimitiveType::Points);
drawRenderTexture.display();

blurRenderTexture.clear(sf::Color::Transparent);//Don't need to clear, as when using sf::BlendNone the original is entirely overdrawn
blurRenderTexture.draw(drawSprite, sf::RenderStates(sf::BlendNone, sf::Transform(), NULL, &SFMLBlur));
blurRenderTexture.display();

//Cycle texture back to image:
drawRenderTexture.clear(sf::Color::Transparent);//Don't need to clear, as when using sf::BlendNone the original is entirely overdrawn
drawRenderTexture.draw(blurSprite, sf::RenderStates(sf::BlendNone));
//TODO: Why doesn't it dissipate? Seems to get to a certain distance and minimal density, then neither get more spread out, nor get less dense...??

//Rendering:
window.clear();

window.draw(blurSprite);

window.display();

 

The blur fragment shader here is the one from the SFML shader examples https://github.com/SFML/SFML/blob/master/examples/shader/resources/blur.frag

I have also tried this with a shader I wrote myself, which also behaves the same way.

What I had expected would be that if I drew a white square and then repeatedly applied a blur shader every frame, then the square would spread out and become more transparent until it disappeared.

What I'm actually seeing is that the shape spreads out a bit and then stops spreading out, which really confuses me. Similarly if I draw something every frame moving around the screen, the trail the object leaves only spreads out a finite amount, then stops spreading. Even quite high alpha bits of the trail don't seem to be getting blurred.

This seems very strange, as I would either expect the blur to not work at all, and for no trail to be created, or for it to dissipate the image successfully. This strange inbetween where it manages to blur but only for a while seems insane behaviour.

If in the shader I deliberately set alpha values that are less than about 0.02 to zero then the trail disappears, although it doesn't look great as it doesn't fade out at the edges, but choosing a lower threshold doesn't seem to eliminate any pixels at all (if there was a threshold I would expect it to be at about 1.0/256.0 or something).

It feels like I have forgotten to clear some buffer somewhere, yet as far as I can see I am clearing them all, and if that was the case then chopping low alpha values in the shader would not eliminate things...

Attached png shows two successive screenshots several hundred frames apart. Note how the trail is somehow the same size and intact hundreds of frames later.

Anyone have any ideas?

[eXpl0it3r]

Just blurring something doesn't mean it must become more transparent or spread out infinitely. It really depends on the shader used to do this and I'm not sure what the linked shader does exactly (don't know GLSL very well).

If something doesn't work the way you assumed, then more often than not, the assumption was likely wrong. For a images to dissipate over time, you'll have to reduce the alpha value over time. And for something to spread out, you'd have to move pixels out further and further.

Many blur filters will simply change the pixels around a certain pixel to a mixed value, but that way it won't really spread out and the center will never really vanish as pixels aren't weakened enough.

[SteelGiant]

Just blurring something doesn't mean it must become more transparent or spread out infinitely. It really depends on the shader used to do this and I'm not sure what the linked shader does exactly (don't know GLSL very well).

If something doesn't work the way you assumed, then more often than not, the assumption was likely wrong. For a images to dissipate over time, you'll have to reduce the alpha value over time. And for something to spread out, you'd have to move pixels out further and further.

Many blur filters will simply change the pixels around a certain pixel to a mixed value, but that way it won't really spread out and the center will never really vanish as pixels aren't weakened enough.

I totally agree with you, somehow, somewhere one of my assumptions is wrong.

I tried something else just now: I completely reimplemented the effect I'm trying to do without using a shader and doing all the work manually on the CPU drawing between sf::Images.

As you can see in the attached image, the CPU/Image version does exactly what I would expect, while the shader/RenderTextures is not. Mathematically and logically everything works as it should. There is some problem coming in somewhere in my implementation going via RenderTextures and shaders.

So somehow there is something that is managing to store a ghost image somewhere.

I have a window and two RenderTextures: these are all cleared every frame. Then I have two sprites (one for each RenderTexture) and a shader. As far as I know there is no information stored in the shader or sprited directly, so nothing to clear here. What could possibly be causing this?

[Chaia*]

Is this really what you expect on the CPU?
Maybe I understood you wrong but when you just draw a square or any other shape and then apply the blur shader, the original shape (with maximum "intensity") should always be in the center and fading out around. To me it looks like your CPU version has the same problem as the GPU version but less strongly, maybe because it runs much slower.

Looking at the image you definitly forgot to clear something somewhere. Can you provide a complete minimal example? Probably you will find the error yourself when preparing this minimal example. Is the tail fading out after some time or does it stay forever?

Alternatively start from the SFML example again. Reading your initial post I think you just simply want to apply the blur shader multiple times to get it to fade out more than with only one pass?

[SteelGiant]

Is this really what you expect on the CPU?
Maybe I understood you wrong but when you just draw a square or any other shape and then apply the blur shader, the original shape (with maximum "intensity") should always be in the center and fading out around. To me it looks like your CPU version has the same problem as the GPU version but less strongly, maybe because it runs much slower.

Looking at the image you definitly forgot to clear something somewhere. Can you provide a complete minimal example? Probably you will find the error yourself when preparing this minimal example. Is the tail fading out after some time or does it stay forever?

Alternatively start from the SFML example again. Reading your initial post I think you just simply want to apply the blur shader multiple times to get it to fade out more than with only one pass?

I'm actually rendering the CPU and GPU versions in the same program, doing all the calculations in lockstep and rendering to two separate windows. That screenshot is how the windows look after about 100 frames.

The slug trail in the GPU version essentially never goes away.

I'm not just rendering a blurred image once, I am rendering an image with a blur, then rendering that resulting blurred image with a blur and so on. If you just had an initial image then it would dissolve into nothingness (or uniform intensity depending on if your boundary is lossy or not). If you draw a small shape moving then it should leave a diffuse trail that dissipates over time. I have implemented this in another language in the past.

I'll post a minimal example tomorrow. Unfortunately my test code is already quite minimal, i only have a 40 line shader and about 100 lines of main loop.

[SteelGiant]

Ok, here is the full source for a minimal example that shows both (broken) GPU and (working) CPU versions side by side. The full cpp code is only about 100 lines for each implementation, and I have annotated which blocks of code are for CPU and GPU. There is also a flag, cpu_enable, at the top that controls if the CPU version is done - I have made no effort to make the CPU implementation efficient, so it is incredibly slow.

The cpp code:

#include "stdafx.h"
#include <SFML/Graphics.hpp>

int main(int argc, char *argv[])
{

        unsigned int defaultSize = 160;

        bool cpu_enable = true;

        //Window
        //GPU:
        sf::RenderWindow window(sf::VideoMode(defaultSize, defaultSize), "SFML test");

        //CPU:
        sf::RenderWindow cpu_Window(sf::VideoMode(defaultSize, defaultSize), "CPU");

        window.setFramerateLimit(60);
       
        //Draw layer
        //GPU
        sf::RenderTexture drawRenderTexture;
        drawRenderTexture.create(window.getSize().x, window.getSize().y);

        drawRenderTexture.clear(sf::Color::Transparent);

        sf::Sprite drawSprite;
        drawSprite.setTexture(drawRenderTexture.getTexture());

        sf::RenderTexture blurRenderTexture;
        blurRenderTexture.create(drawRenderTexture.getSize().x, drawRenderTexture.getSize().y);

        blurRenderTexture.clear(sf::Color::Transparent);

        sf::Sprite blurSprite;
        blurSprite.setTexture(blurRenderTexture.getTexture());

        sf::Shader drawBlur;
        if (!drawBlur.loadFromFile("drawBlur.frag", sf::Shader::Fragment)) {
                return EXIT_FAILURE;
        }
        if (!drawBlur.isAvailable()) {
                return EXIT_FAILURE;
        }
        drawBlur.setUniform("texture", sf::Shader::CurrentTexture);
        drawBlur.setUniform("xResolution", float(1.0f / float(drawRenderTexture.getSize().x)));
        drawBlur.setUniform("yResolution", float(1.0f / float(drawRenderTexture.getSize().y)));

       
        //CPU
        sf::Image cpu_drawImage;
        cpu_drawImage.create(cpu_Window.getSize().x, cpu_Window.getSize().y, sf::Color::Transparent);

        sf::Image cpu_blurImage;
        cpu_blurImage.create(cpu_drawImage.getSize().x, cpu_drawImage.getSize().y, sf::Color::Transparent);

        sf::Texture cpu_texture;
        cpu_texture.create(cpu_blurImage.getSize().x, cpu_blurImage.getSize().y);

        sf::Sprite cpu_drawSprite;
        cpu_drawSprite.setTexture(cpu_texture);


        //Main loop:
        int iFrame = 0;

        while (window.isOpen())
        {


                //Input:
                sf::Event event;
                while (window.pollEvent(event))
                {
                        if (event.type == sf::Event::Closed) {
                                window.close();
                        }
                        if (event.type == sf::Event::Resized) {
                                window.setView(sf::View(sf::FloatRect(0.f, 0.f, window.getSize().x, window.getSize().y)));
                        }
                }


                //Logic:
                //Draw circling trail
                double radius = 0.7 * double(window.getSize().x) / 2.0;
                double prec = 1.0;
                double angle = double(iFrame % int(360 * prec)) / prec;

                double degreesToRadians = std::acos(-1) / 180.0;


                sf::Vertex drawVertices[1000];
                int nVertices = 0;
                for (int iX = 0; iX < 10; ++iX) {
                        for (int iY = 0; iY < 10; ++iY) {
                                sf::Color drawColour = sf::Color::White;

                                float xLoc = window.getSize().x / 2 + radius * cos(angle * degreesToRadians) + iX;
                                float yLoc = window.getSize().y / 2 + radius * sin(angle * degreesToRadians) + iY;

                                //GPU:
                                drawVertices[nVertices++] = sf::Vertex(sf::Vector2f(xLoc, yLoc), drawColour);

                                //CPU:
                                cpu_drawImage.setPixel(unsigned int(xLoc), unsigned int(yLoc), drawColour);
                        }
                }


                //GPU:
                drawRenderTexture.draw(drawVertices, nVertices, sf::PrimitiveType::Points);
                drawRenderTexture.display();

                blurRenderTexture.clear(sf::Color::Transparent);//Don't need to clear, as when using sf::BlendNone the original is entirely overdrawn
                blurRenderTexture.draw(drawSprite, sf::RenderStates(sf::BlendNone, sf::Transform(), NULL, &drawBlur));
                blurRenderTexture.display();

                //Cycle texture back to image:
                drawRenderTexture.clear(sf::Color::Transparent);//Don't need to clear, as when using sf::BlendNone the original is entirely overdrawn
                drawRenderTexture.draw(blurSprite, sf::RenderStates(sf::BlendNone));
                //TODO: Why doesn't it dissipate? Seems to get to a certain distance and minimal density, then neither get more spread out, nor get less dense...??

                //CPU:
                //Draw blurred version of the image to the blur texture:
                if (cpu_enable) {
                        for (unsigned int x = 0; x < cpu_drawImage.getSize().x; ++x) {
                                for (unsigned int y = 0; y < cpu_drawImage.getSize().y; ++y) {

                                        float weightCentre = 1.0;//TODO: Could it be an issue of float precision when numbers are summed together and then divided at the end?
                                        float weightAdj = 0.5;
                                        float weightDiag = 0.25;

                                        float weights[3] = {weightCentre, weightAdj, weightDiag};

                                        float alphaWeight = 0.0;
                                        float colourWeight = 0.0;


                                        float colour[4] = {0.0, 0.0, 0.0, 0.0};

                                        for (int dx = -1; dx <= 1; ++dx) {
                                                for (int dy = -1; dy <= 1; ++dy) {

                                                        unsigned int tX = x + dx;
                                                        unsigned int tY = y + dy;

                                                        sf::Color inColour(0, 0, 0, 0);
                                                        if (tX < 0 || tX >= cpu_drawImage.getSize().x || tY < 0 || tY >= cpu_drawImage.getSize().y) {
                                                                //Out of bounds defaults to transparent black
                                                        } else {
                                                                inColour = cpu_drawImage.getPixel(tX, tY);
                                                        }

                                                        float texColour[4] = {float(inColour.r) / float(255.0), float(inColour.g) / float(255.0), float(inColour.b) / float(255.0), float(inColour.a) / float(255.0)};

                                                        int ds = dx*dx + dy*dy;//This works nicely because numbers have magnitude 0 or 1, but won't stretch further
                                                        float weight = weights[ds];

                                                        float effectiveWeight = texColour[3] * weight;

                                                        colour[3] += effectiveWeight;
                                                        alphaWeight += weight;

                                                        colour[0] += texColour[0] * effectiveWeight;
                                                        colour[1] += texColour[1] * effectiveWeight;
                                                        colour[2] += texColour[2] * effectiveWeight;
                                                        colourWeight += effectiveWeight;

                                                }
                                        }

                                        //Make certain we never divide by zero
                                        colour[3] = alphaWeight > 0.0 ? colour[3] / alphaWeight : 0.0;
                                        colour[0] = colourWeight > 0.0 ? colour[0] / colourWeight : colour[0];
                                        colour[1] = colourWeight > 0.0 ? colour[1] / colourWeight : colour[1];
                                        colour[2] = colourWeight > 0.0 ? colour[2] / colourWeight : colour[2];

                                        sf::Color outColour(sf::Uint8(colour[0] * 255), sf::Uint8(colour[1] * 255), sf::Uint8(colour[2] * 255), sf::Uint8(colour[3] * 255));

                                        //Apply to blur image:
                                        cpu_blurImage.setPixel(x, y, outColour);
                                }
                        }

                        //copy the blurred version back to the original image:
                        //(could probably get double the performance if we alternated which image we drew to/from rather than doing a copy, but performance doesn't matter here: only correctness)
                        for (unsigned int x = 0; x < cpu_drawImage.getSize().x; ++x) {
                                for (unsigned int y = 0; y < cpu_drawImage.getSize().y; ++y) {
                                        cpu_drawImage.setPixel(x, y, cpu_blurImage.getPixel(x, y));
                                }
                        }

                        //Push blurred image to renderable texture:
                        cpu_texture.update(cpu_blurImage);

                        //Render the cpu blurred image:
                        cpu_Window.clear();
                        cpu_Window.draw(cpu_drawSprite);
                        cpu_Window.display();
                } else {
                        //Dont render Render the cpu blurred image:
                        cpu_Window.clear(sf::Color::Red);
                        cpu_Window.display();
                }


                //Rendering:
                window.clear();
                window.draw(blurSprite);
                window.display();

                if (iFrame % 60 == 0) {
                        printf("%d\n", iFrame);
                }

                ++iFrame;
        }

        return EXIT_SUCCESS;
}

 

and the shader:

uniform sampler2D texture;
uniform float xResolution;
uniform float yResolution;

void main()
{
    vec2 offx = vec2(xResolution, 0.0);
    vec2 offy = vec2(0.0, yResolution);

        float weightCentre = 1.0;//TODO: Could it be an issue of float precision when numbers are summed together and then divided at the end?
        float weightAdj    = 0.5;
        float weightDiag   = 0.25;

        float weights[3] = {weightCentre, weightAdj, weightDiag};

        //Alpha weighted colour blending: (maybe there is some built in way to achieve what I'm doing here trivially)
        float alphaWeight = 0.0;
        float colourWeight = 0.0;

        vec4 colour = vec4(0.0, 0.0, 0.0, 0.0);

        for(int dx = -1; dx <= 1; ++dx) {
                for(int dy = -1; dy <= 1; ++dy) {
                       
                        vec4 texColour = texture2D(texture, gl_TexCoord[0].xy + float(dx) * offx + float(dy) * offy);

                        int ds = dx*dx + dy*dy;//This works nicely because numbers have magnitude 0 or 1, but won't stretch further
                        float weight = weights[ds];

                        float effectiveWeight = texColour.a * weight;

                        colour.a += effectiveWeight;
                        alphaWeight += weight;

                        colour.rgb += texColour.rgb * effectiveWeight;
                        colourWeight += effectiveWeight;

                }
        }
       
        //Make certain we never divide by zero
        colour.a = alphaWeight > 0.0 ? colour.a / alphaWeight : 0.0;
        colour.rgb = colourWeight > 0.0 ? colour.rgb / colourWeight : colour.rgb;

    gl_FragColor = colour;
}

 

[SteelGiant]

I have found the discrepancy between the CPU and GPU versions. It seems that if in the GPU version I convert everything to int and back as is done in the CPU version, then they match up exactly, so this is "solved".

colour.a = float(int(colour.a * 255)) / 255.0;
colour.r = float(int(colour.r * 255)) / 255.0;
colour.g = float(int(colour.g * 255)) / 255.0;
colour.b = float(int(colour.b * 255)) / 255.0;
 

I'm still very confused how without the rounding the trail was maintaining alpha values of over 0.5, when they should have been decaying exponentially.

Along the way in my tests I wrote a shader that just translated the image by a couple of pixels, and that worked perfectly, showing that the RenderTextures weren't doing anything strange, so the problem had to be in the shader. I then wrote another shader that coloured the image differently depending purely on alpha values, so I could verify that regions were somehow maintaining high alpha values without decaying.

If anyone can figure out how the rounding fixes things then I would be glad to hear it. Having done the mathematical calculations by hand I can't see a way for it to work though...

[FRex]

I'm interested and was already trying to figure it out myself but didn't have time to yet. Maybe it's something with the fact that float has much more precision than just 256 values that a byte has.

[FRex]

After looking into it for a while I'm not sure. But I found that any number can be put in that's 255 place and it'll affect the length of the tail and that calculation needs to be done only on the a. I didn't look into the math that would cause this any further though and I don't think I will because it's so much work for so little gain.

[SteelGiant]

After looking into it for a while I'm not sure. But I found that any number can be put in that's 255 place and it'll affect the length of the tail and that calculation needs to be done only on the a. I didn't look into the math that would cause this any further though and I don't think I will because it's so much work for so little gain.

Interesting, thanks for taking another look at it. Good to confirm that it was only alpha that needed to be transformed.

It is especially strange that the problems are solved by adding any amount of rounding. I had tried rounding down low alpha values, and scaling all alpha values down by a factor more aggressively that than this rounding would do and that didn't fix it.

Hopefully I'll have a chance to look at it more in the near future.