Drawing to a RenderTexture with sRGB conversion enabled results in banded colors

[Fewes]

Hi there! I was very excited to see that support for native sRGB handling was added to the repo a few weeks back. After doing some testing however I noticed that if you use sf::RenderTexture to draw anything anywhere between sampling the texture and drawing to the window, you get terrible color banding.

Here are three images rendered with different setups to illustrate the problem. The texture drawn is a simple 8 bits/channel gradient.

1. Texture is drawn to the RenderTexture via a Sprite, then the RenderTexture is drawn to the window via a Sprite.
ContextSettings.sRgbCapable is set to false and Texture.setSrgb is set to false.



2. Texture is drawn directly to the window via a Sprite.
ContextSettings.sRgbCapable is set to true and Texture.setSrgb is set to true.



3. Texture is drawn to the RenderTexture via a Sprite, then the RenderTexture is drawn to the window via a Sprite.
ContextSettings.sRgbCapable is set to true and Texture.setSrgb is set to true.



As you can see the result suffers from banded colors, as expected of when storing gamma decoded values improperly.

I'm not that high on color space but it seems there is something missing where the RenderTexture should convert the input and output when rendering, just like the window does. I'm not sure if it can be remedied somehow, but I thought I should bring it to light.

Here's the code for the program shown above. Change the boolean at the top to toggle between using sRGB conversion or not.

#include <SFML/Graphics.hpp>

int main()
{
        bool srgb = true;

        sf::RenderWindow window;
        sf::VideoMode videoMode;
        videoMode.width = 720;
        videoMode.height = 405;
        sf::ContextSettings contextSettings;
        contextSettings.sRgbCapable = srgb;

        window.create(videoMode, "Linear Color Space Testing", sf::Style::Default, contextSettings);

        sf::Texture texture;
        texture.setSmooth(true);
        texture.setSrgb(srgb);
        texture.loadFromFile("Gradient.png");

        sf::Sprite sprite;
        sprite.setTexture(texture);

        // Make sure the sprite fills the screen
        float scale_x = (float)videoMode.width / (float)texture.getSize().x;
        float scale_y = (float)videoMode.width / (float)texture.getSize().y;
        sprite.setScale(scale_x, scale_y);

        sf::RenderTexture renderTexture;
        renderTexture.setSmooth(true);
        renderTexture.create(videoMode.width, videoMode.height);

        sf::Sprite renderTextureDrawable;
        renderTextureDrawable.setTexture(renderTexture.getTexture());

        while (window.isOpen())
        {
                // Poll events
        sf::Event event;
        while (window.pollEvent(event))
                {
            // Window closed
            if (event.type == sf::Event::Closed || (event.type == sf::Event::KeyPressed && event.key.code == sf::Keyboard::Key::Escape))
                        {
                window.close();
                        }
        }

                // Clear render texture
                renderTexture.clear(sf::Color(0, 0, 0, 255));
                // Draw gradient sprite
                renderTexture.draw(sprite);
                // Finished drawing to render texture
                renderTexture.display();

                // Clear window
                window.clear(sf::Color(0, 0, 0, 255));
                // Draw render texture drawable
                window.draw(renderTextureDrawable);
                // Finished drawing to the window
                window.display();

        }

    return 0;
}

And here's the gradient texture:

[Fewes]

Showing the results of #1093

[eXpl0it3r]

Are you rendering the same thing to the render texture or why is the black part in the last image larger then on the other two?

[Fewes]

I am drawing the same thing, the black part is bigger because the sRGB conversion seemingly happens twice. Once when the texture is loaded and once when drawing to the RenderTexture. Disabling sRGB conversion on the texture fixes this, but I'm not sure if this is intended behaviour or not.

[binary1248]

Maybe I might have been a bit too hasty.

The fix I pushed addresses the issue that developers cannot enable the sRGB conversion to happen within the RenderTexture itself.

According to the OpenGL extension specification, image data is meant to be converted from its assumed sRGB source format into the linear colour space when it is consumed by the application. Within the application itself, the data should be kept in linear space. Once the image data is to be output to the screen, it has to be converted back to sRGB again.

Conversion from sRGB to linear really shouldn't take place twice as that can lead artefacts as can be seen in the last image. The conversion from and to sRGB in your original example is correct, and my fix doesn't address the colour banding issue, merely the lack of sRGB support in RenderTextures.

I'll have to investigate a bit further.

[Sakarah]

I am necro-bumping this thread because the issue is still not solved and I have recently investigated it further since it prevents me from using sRGB in my project.

The problem is that once converted from sRGB space, colors take more space than before to be kept accurately.
Actually to keep a smooth gradient in linear space that can be converted back to the full sRGB scale, one would need to use more bits per pixel (as a simple approximation we can think of sRGB encoded data as pixel values between 0 and 1 raised to the power 2.2, we can then convince ourselves that if we do not increase the number of decimals stored, the conversion function will not be injective).

Currently, sf::Texture's are either represented as 8 bit linear RGBA (when isSrgb is false, always the case for a rendered Texture) or as 8 bit sRGB + 8 bit alpha (when isSrgb is true).
This means that after decoding a 8 bit sRGB color we want to store it in a linear 8 bit space, and it obviously fails.

It seems to me that all solutions for using sRGB decoding along RenderTexture rely on letting advanced user choose internal formats for the sf::Texture of a sf::RenderTexture. The amount of control to actually give, and the preservation of the Texture interface with weirder formats is the real question.

A texture format choice (like current call to setSrgb) need to be done before loading pixel data or rendering.
When using multisampling, the color renderbuffer of the RenderTextureImpl must be tweaked as well, and some formats might not be available for this purpose.
The number of internal texture formats that OpenGL offers is huge, some seem very old or exotic (like the tiny GL_R3_G3_B2), and platform support for these vary a lot. For our problem, most relevant ones are:

• GL_RGBA8: This is the current SFML default on most modern computers, it has data that is nicely represented by sf::Image. It is treated by OpenGL as linear RGB when performing operations.
• GL_SRGB8_ALPHA8: This is the same as the previous one but before performing an operation, OpenGL decode the color from sRGB to linear space. This is currently used by SFML when isSrgb is true.
• GL_RGBA16: This format use 16 bits for each color value. It seems that sRGB decoded values almost fit there.
• GL_RGBA16F / GL_RGBA32F: Stores the value as (half-)floats, should also map sRGB more accurately, supported only with the GL_ARB_texture_float extension (core since OpenGL 3.0).
  
• GL_RED/GL_RG/GL_RGB: Variants that store less color channels, might feel even weirder for sf::Image.

On desktop OpenGL, we can feed our internal Image pixels to any of the formats, and channels will be extended/discarded/recreated as needed. However, this means that we cannot expect copyToImage and loadFromImage to be lossless operations anymore. Moreover, it seems that OpenGL ES, does not support pixel data input in a different format than the internal representation. This is somehow not very satisfying but stems from not also supporting multiple formats in Image.

If we want to deal with the banding problem of this thread, I see four main options to be considered:

• Option full sRGB: We consider that encoding intermediate RenderTexture in sRGB is not that bad since it only prevents textures to slowly accumulate data, as it never exceeds 8 bits per channel. Support just need to be added in RenderTexture, to set the sRGB mode to the underlying texture when requested (maybe through ContextSettings). I think I can very quickly send a PR for this option.
• Option RenderTexture format support: We could allow RenderTexture to select the format of its underlying texture. This might be RGBA16, RGBA16F or RGBA32F. This is a more powerful option. Though it does create an inconsistency if the user calls copyToImage because the returned image will be in linear space (while loaded image supposedly were in sRGB), and might be troublesome with OpenGL ES or with old implementations that do not support float formats. The actual choice of supported format must also be decided
• Option Texture format support: We could let the user choose the format of any Texture. Though Image stays identical. This is basically the same as the previous option but we also allow users to also change the format of loaded textures. This might feel weird since all the pixel transfer operations will still be in 8bit RGBA format. This will definitely cause trouble with OpenGL ES.
• Option Image format support: We could go even further and allow sf::Image to have different formats, that can be given to Textures. If this is surely the most powerful option, it is also the most complex and pervasive. I that if such a change is desired it should be postponed until SFML 3 to be able to fully rework the Image API.

These options are not mutually incompatible and in my quick and dirty test it seems that all four of them would actually solve the banding problem.
My current favorite option is full sRGB for its simplicity and the internal coherency it can create. It means that all Image's will be in sRGB color space, and maybe enabling us to have a simple compile time option to make sRGB the default for a given project.