CPU/Pixel based "Post processing"

[AncientGrief]

I wonder if there's a more performant way of doing this:

        //Draw everything
        renderTex.clear(sf::Color(200,200,200));
        renderTex.draw(shape);
        renderTex.display();
       
        //Do "Post-Processing"
        window.clear();
       
        sf::Image img = renderTex.getTexture().copyToImage();
       
        //Manipulate the pixels in any way, this is just an example
        for(unsigned int i=0; i < img.getSize().x; i+=8)
        {
            for(unsigned int j=0; j < img.getSize().y; j++)
            {
                img.setPixel(i,j,sf::Color::White);
            }
        }
       
        //Draw result
        postTexture.loadFromImage(img);
        sf::Sprite sprite(postTexture);
        window.draw(sprite);
        window.display();
 

Full code:

(click to show/hide)

#include <SFML/Graphics.hpp>

int main()
{
    sf::RenderWindow window(sf::VideoMode(800, 600), "SFML works!");
    sf::RenderTexture renderTex;
    sf::Texture postTexture;
    if (!renderTex.create(800, 600))
        return -1;
       
    sf::CircleShape shape(50.f);
    shape.setFillColor(sf::Color::Black);
    shape.setOrigin(shape.getRadius(), shape.getRadius());
   
    while(window.isOpen()) {
        sf::Event event;
        while(window.pollEvent(event)) {
            if(event.type == sf::Event::Closed)
                window.close();
        }
       
        sf::Vector2i mousePos =  sf::Mouse::getPosition(window);
        shape.setPosition(sf::Vector2f(static_cast<float>(mousePos.x), static_cast<float>(mousePos.y)));
       
       
        //Draw everything
        renderTex.clear(sf::Color(200,200,200));
        renderTex.draw(shape);
        renderTex.display();
       
        //Do "Post-Processing"
        window.clear();
       
        sf::Image img = renderTex.getTexture().copyToImage();
       
        //Manipulate the pixels in any way, this is just an example
        for(unsigned int i=0; i < img.getSize().x; i+=8)
        {
            for(unsigned int j=0; j < img.getSize().y; j++)
            {
                img.setPixel(i,j,sf::Color::White);
            }
        }
       
        //Draw result
        postTexture.loadFromImage(img);
        sf::Sprite sprite(postTexture);
        window.draw(sprite);
        window.display();
    }

    return 0;
}
 

I am drawing everything to a RenderTexture, then I retrieve a copy of the underlying texture as an Image.
Then I manipulate the image (e.g. do xBRZ scaling) and render the result to the screen.

Rendering resolution will be around 660x360px.

The pixel manipulation will be done using the Uint8 array, not with setPixel.

And yes, I am not able to create the GLSL shader to do the resizing ... lack of knowledge atm 

[eXpl0it3r]

You could also work with a pixel vector and call update() on the texture.

[AncientGrief]

Mhh I think this could work.

- I take the texture from my RenderWindow
- Pass it to my xBRZ scaling function
- And pass the new Pixels to the texture.

I guess the update() method will not resize the texture so I would have to create a bigger image (i.e. 1920x1080) and pass only 660x330 to my scaler (where the actual game graphics are placed). This could work and should be faster.

Thanks eXpl0it3r, I will try it

Edit: Meh doesn't help, since I still have to call copyToImage() to get access to the RenderWindow texture pixels.

Oh an this does not work, since getTexture is const:

        sf::Uint8 bla[10];
        renderTex.getTexture().update(bla);
 

Too bad m_Texture is private and not protected in RenderTexture x_X

[Hapax]

Texture update() should be a little better than texture loadFromImage() (update gets called from loadFromImage anyway).
I'm referring to the use on postTexture, by the way.

If you're processing the pixels in the kind of loops that you use in the example, the shader should be fairly straight-forward, unless you're relying on other, previously processed pixels.

[artelius]

Hi! The GLSL may be easier than you think.

I created a little proof-of-concept for 4x upscaling. So a 4x4 block of screen pixels corresponds to 1 texture pixel. My code just does some smoothing in the y direction but it should be enough to give you an idea of how to do more sophisticated upscaling.

To get your head around it, you just need to understand that the main() function is called once for each pixel on screen (as if in a pair of for loops; although in reality they run in parallel), and the colour of that pixel is determined by what ends up in gl_FragColor.

Any questions, just ask.

Here is the shader code:

#version 130

uniform sampler2D texture;

void main () {

    //get window co-ordinates
    int window_x = int(gl_FragCoord.x); //varies from 0 to 799 (0,0 is bottom-left)
    int window_y = int(gl_FragCoord.y); //varies from 0 to 599

    //get the index into the texture corresponding to this pixel
    int tex_x = window_x / 4;
    int tex_y = window_y / 4;

    //figure out which "subpixel" this one is
    int xoff = window_x % 4;
    int yoff = window_y % 4;

    //get colour of corresponding texture pixel
    vec4 here = texelFetch(texture, ivec2(tex_x,tex_y), 0);
    //get colour of texture 1 pixel up
    vec4 up1 = texelFetch(texture, ivec2(tex_x,tex_y+1), 0);
    //get colour of texture 1 pixel down
    vec4 down1 = texelFetch(texture, ivec2(tex_x,tex_y-1), 0);
   
    if (yoff == 0) //mix with below colour
        gl_FragColor = (here + down1) / 2.0;
    else if (yoff == 3) //mix with above colour
        gl_FragColor = (here + up1) / 2.0;
    else  //mix them all!
        gl_FragColor = (2.0 * here + up1 + down1) / 4.0;

    //How the heck do you debug shaders??
    //You can't print stuff but you CAN colour pixels instead
    //try uncommenting the following to get an idea:
    //gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0); //will make all pixels red
    //if (window_x >= 400 && window_x < 404)
    //    gl_FragColor = vec4(0.0, float(window_y)/600.0, 0.0, 1.0); //a bar ranging from black to green
}
 

Here a full example:

(click to show/hide)

#include <SFML/Graphics.hpp>
#include <SFML/Graphics/Image.hpp>



const std::string FRAG_SHADER = "#version 130\n\
\
   uniform sampler2D texture; \
\
   void main () { \
       int window_x = int(gl_FragCoord.x); \
       int window_y = int(gl_FragCoord.y); \
       int tex_x = window_x / 4; \
       int tex_y = window_y / 4; \
       int xoff = window_x % 4; \
       int yoff = window_y % 4; \
       vec4 here = texelFetch(texture, ivec2(tex_x,tex_y), 0); \
       vec4 up1 = texelFetch(texture, ivec2(tex_x,tex_y+1), 0); \
       vec4 down1 = texelFetch(texture, ivec2(tex_x,tex_y-1), 0); \
       if (yoff == 0) \
           gl_FragColor = (here + down1) / 2.0; \
       else if (yoff == 3) \
           gl_FragColor = (here + up1) / 2.0; \
       else \
           gl_FragColor = (2.0 * here + up1 + down1) / 4.0; \
   }\
";


int main()
{
    sf::RenderWindow window(sf::VideoMode(800, 600), "SFML works!");
    sf::RenderTexture renderTex;
    if (!renderTex.create(200, 150))
        return -1;
       
    sf::CircleShape shape(10.f);
    shape.setFillColor(sf::Color::Black);
    shape.setOrigin(shape.getRadius(), shape.getRadius());

    sf::Shader upscaleShader;
    upscaleShader.loadFromMemory(FRAG_SHADER, sf::Shader::Fragment);
    //set up the 'texture' variable in the shader
    upscaleShader.setParameter("texture", renderTex.getTexture());
    sf::RenderStates upscaleRenderStates(&upscaleShader);

    //FIXME: hardcoding
    sf::Vertex winRect[4] = {
        sf::Vector2f(0,0),
        sf::Vector2f(800,0),
        sf::Vector2f(0,600),
        sf::Vector2f(800,600)};

   
    while(window.isOpen()) {
        sf::Event event;
        while(window.pollEvent(event)) {
            if(event.type == sf::Event::Closed)
                window.close();
        }
       
        sf::Vector2i mousePos =  sf::Mouse::getPosition(window);
        //shape.setPosition(sf::Vector2f(static_cast<float>(mousePos.x)/4.f, static_cast<float>(mousePos.y)/4.f));
        shape.setPosition(sf::Vector2f(static_cast<float>(mousePos.x/4), static_cast<float>(mousePos.y/4)));
       
       
        //Draw everything
        renderTex.clear(sf::Color(200,200,200));
        renderTex.draw(shape);
        renderTex.display();
       
        window.clear();

        //this is where the upscaling happens
        window.draw(winRect, 4, sf::TrianglesStrip, upscaleRenderStates);
        window.display();
    }

    return 0;
}