Convert from one pixel format to another

[natinusala]

Hello,

I've been annoying you with this issue on the IRC Chat, I'll continue here instead, I find it more practical for me and more people might help.

So here is the deal : I am trying to create a tiny libretro frontend using SFML, which point is to eventually run on a Raspberry Pi. I have an emulation core which gives me every frame the image that I'm supposed to display. The thing is that the pixel format is not the same as the one SFML's using.

libretro can support several emulation consoles, which can use three different pixel formats. For now I'll just focus on one : RGB565.

The core provides me, every frame, an uint16_t * containing the pixels this way : { rrrrrggggggbbbbb, rrrrrggggggbbbbb, rrrrrggggggbbbbb, etc...}.

On the other hand, SFML requires an RGBA8888 format, plus a weird uint8_t * array disposition : {rrrrrrrr, gggggggg, bbbbbbbb, aaaaaaaa, rrrrrrrr, gggggggg, bbbbbbbb, aaaaaaaa, rrrrrrrr, gggggggg, bbbbbbbb, aaaaaaaa, etc...}.

On the IRC we eventually came to a working method to convert each pixel to three uint8_t containing the red, green and blue channels (well, I think it works).

Here is the snippet I have - pixelsBuffer is the final array containing the pixels to be displayed, and videoRefreshCallback is the function called every frame to update pixelsBuffer :

//The final pixels buffer (the size is fixed for now), which should be RGBA8888
uint8_t pixelsBuffer[229376];

void videoRefreshCallback(const void *data, unsigned width, unsigned height, size_t pitch)
{
    uint16_t * pixels = (uint16_t *) data; //the uint16_t array of RGB565 pixels

    for (int y = 0; y < height; y++)
    {
        for (int x = 0; x < width; x++)
        {
            uint16_t actualPixel = pixels[(pitch*y)/2 + x];

            uint8_t r = (actualPixel >> 8) & 0xF8;
            uint8_t g = (actualPixel >> 3) & 0xFC;
            uint8_t b = (actualPixel) << 3;

            pixelsBuffer[?] = r;
            pixelsBuffer[? * 4 + 1] = g;
            pixelsBuffer[? * 4 + 2] = b;
            pixelsBuffer[? * 4 + 3] = 0xFF;
        }
    }

    gameTexture.update(pixelsBuffer, width, height, 0, 0); //we update the texture drawn in the main loop
}
 

Okay, so the pixel conversion works here (I guess), but I don't really know how to put them in pixelsBuffer (where the ? are).

Could you help me ? I also have few other questions :

• isn't there a more optimized solution to do this ? i think the double-loop solution is heavy, it's supposed to run on a Raspberry and I might need to implement some more high-qualities consoles (with more pixels to process each frame)
• wouldn't be more easy to extend SFML to support more pixels formats ? I looked at the Texture::update() code, and I think we can change GL_ARGB to pretty much every format supported by GL (RGB565 included), but this doesn't solve the array format problem, switching from one array of pixels (one entry per pixel) to one array of pixel colors (3 or 4 entry per pixel, depending on the alpha channel)

Thanks a lot !

[eXpl0it3r]

isn't there a more optimized solution to do this ? i think the double-loop solution is heavy, it's supposed to run on a Raspberry and I might need to implement some more high-qualities consoles (with more pixels to process each frame)

You convert n pixels as such you need to go over n pixels, thus you end up with O(n). There's nothing you can do about it.

wouldn't be more easy to extend SFML to support more pixels formats ? I looked at the Texture::update() code, and I think we can change GL_ARGB to pretty much every format supported by GL (RGB565 included), but this doesn't solve the array format problem, switching from one array of pixels (one entry per pixel) to one array of pixel colors (3 or 4 entry per pixel, depending on the alpha channel)

You can try modifying things, but there's no real gain for SFML itself. I mean you can ask yourself why does SFML have to adapt the RGB565 format instead of liretro providing an RGBA output?

isn't there a more optimized solution to do this ? i think the double-loop solution is heavy

Don't think about performance. Profile!
You won't gain anything in trying to optimize parts that aren't really the bottleneck.

[natinusala]

You can try modifying things, but there's no real gain for SFML itself. I mean you can ask yourself why does SFML have to adapt the RGB565 format instead of liretro providing an RGBA output?

The thing is that libretro is quite a huge project, and there is more that 70 different consoles cores that can use 3 different pixel formats, RGB565 is more accurate and faster for most of the old consoles (like the SNES), but for the PSX for example it's XRGB8888, so... there isn't much I can do about it, apart from taking each one of them and creating a conversion method for SFML's format =/

[FRex]

You can bind the sf::Texture yourself, and then transfer data to it in your format:

sf::Texture::bind(&tex);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, width, height, GL_RGB, GL_UNSIGNED_SHORT_5_6_5, myarrayofunisgnedshorts);

I have no idea if this will work on raspberry.

[natinusala]

You can bind the sf::Texture yourself, and then transfer data to it in your format:

sf::Texture::bind(&tex);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, width, height, GL_RGB, GL_UNSIGNED_SHORT_5_6_5, myarrayofunisgnedshorts);

I have no idea if this will work on raspberry.

That looks super interesting, is the array format the same as the one I'm using (arrays of pixels, one entry per pixel) or is it the one currently used by SFML (array of pixel channels, several entry per pixel) ?

[FRex]

GL_UNSIGNED_SHORT_5_6_5 is obviously "your" format, there are plenty of color and data formats but I have no idea which will and which won't work on raspberry and I have no idea how fast this is.
You can read more yourself about glTexImage2D and glTexSubImage2D.
Also, I don't know what pitch in your code is, and if the way your pixels are packed will work with these functions.

[natinusala]

I've tried this :

void videoRefreshCallback(const void *data, unsigned width, unsigned height, size_t pitch)
{
    Texture::bind(&gameTexture);
    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, width, height, GL_RGB, GL_UNSIGNED_SHORT_5_6_5, data);
}
 

It doens't seem to work, the pixels are showing on the screen but with a messed up aspect (close to the result I have by using gameTexture.update(data); without converting anything).

The pitch is the size of one line of pixels, in bytes

[FRex]

If you do a rectangle then pixels must be tightly packed, one row after another, but if you want, you can try send them in 1 pixel high lines, each pitch/2 pixels wide, like SFML loadFromImage does.
Also, try to make a screenshot, because "messed up aspect" is very cryptic.

[natinusala]

Here is what the game looks like (the window is too big that's normal) :

[FRex]

Did you try to transfer only single lines? And did you get the expected result ever? And which of these is which (or are you programming a DS, but then when does this happen, with update or with GL?).
Also, your pictures are now gone and there is a French error message instead.

[natinusala]

It works, here is the code :

    const uint16_t* pixels = (uint16_t*) data;

    Texture::bind(&gameTexture);
    for (int i = 0; i < height; i++)
    {
        glTexSubImage2D(GL_TEXTURE_2D, 0, 0, i, width, 1, GL_RGB, GL_UNSIGNED_SHORT_5_6_5, pixels);
        pixels += pitch/2;
    }
 

I love you <3