Per-pixel Transparent window with SFML content (win32 API) - The Final Solution

[UltraNyan]

This thread... again...

I have done many searches and read different forums, this is how I intend to do it on Windows and with SFML.

1. Render my SFML content to a transparent render texture and convert it to image.
2. Use UpdateLayeredWindows to create the transparent window based on that transparent render texture.

My main concern is that the program has to process the whole texture each frame.

I had some great satisfactory performance using TGA files with transparency and overwriting the SFML window:

(click to show/hide)

Taken from: (http://www.dhpoware.com/demos/layeredWindows.html)

#include <string.h>
#include <string>
#include <cstring>
#include <cstdlib>
#include <ctime>
#include <vector>
#include <sstream>
#include <SFML/Graphics.hpp>
#include <SFML/Audio.hpp>
#define _WIN32_WINNT 0x0501
#include <windows.h>


/* TGA file header structure. This *must* be byte aligned. */
#pragma pack(push, 1)
typedef struct
{
    BYTE idLength;
    BYTE colormapType;
    BYTE imageType;
    USHORT firstEntryIndex;
    USHORT colormapLength;
    BYTE colormapEntrySize;
    USHORT xOrigin;
    USHORT yOrigin;
    USHORT width;
    USHORT height;
    BYTE pixelDepth;
    BYTE imageDescriptor;
} TgaHeader;
#pragma pack(pop)

/* Generic wrapper around a DIB with a 32-bit color depth. */
typedef struct
{
    int width;
    int height;
    int pitch;
    HDC hdc;
    HBITMAP hBitmap;
    BITMAPINFO info;
    BYTE *pPixels;
} Image;

Image g_image;

void ImageDestroy(Image *pImage)
{
    if (!pImage)
        return;

    pImage->width = 0;
    pImage->height = 0;
    pImage->pitch = 0;

    if (pImage->hBitmap)
    {
        DeleteObject(pImage->hBitmap);
        pImage->hBitmap = NULL;
    }

    if (pImage->hdc)
    {
        DeleteDC(pImage->hdc);
        pImage->hdc = NULL;
    }

    memset(&pImage->info, 0, sizeof(pImage->info));
    pImage->pPixels = NULL;
}

BOOL ImageCreate(Image *pImage, int width, int height)
{
    /* All Windows DIBs are aligned to 4-byte (DWORD) memory boundaries. This
     * means that each scan line is padded with extra bytes to ensure that the
     * next scan line starts on a 4-byte memory boundary. The 'pitch' member
     * of the Image structure contains width of each scan line (in bytes).
     */

    if (!pImage)
        return FALSE;

    pImage->width = width;
    pImage->height = height;
    pImage->pitch = ((width * 32 + 31) & ~31) >> 3;
    pImage->pPixels = NULL;
    pImage->hdc = CreateCompatibleDC(NULL);

    if (!pImage->hdc)
        return FALSE;

    memset(&pImage->info, 0, sizeof(pImage->info));

    pImage->info.bmiHeader.biSize = sizeof(pImage->info.bmiHeader);
    pImage->info.bmiHeader.biBitCount = 32;
    pImage->info.bmiHeader.biWidth = width;
    pImage->info.bmiHeader.biHeight = -height;
    pImage->info.bmiHeader.biCompression = BI_RGB;
    pImage->info.bmiHeader.biPlanes = 1;

    pImage->hBitmap = CreateDIBSection(pImage->hdc, &pImage->info,
                        DIB_RGB_COLORS, (void**)&pImage->pPixels, NULL, 0);

    if (!pImage->hBitmap)
    {
        ImageDestroy(pImage);
        return FALSE;
    }

    GdiFlush();
    return TRUE;
}

void ImagePreMultAlpha(Image *pImage)
{
    /* The per pixel alpha blending API for layered windows deals with
     * pre-multiplied alpha values in the RGB channels. For further details see
     * the MSDN documentation for the BLENDFUNCTION structure. It basically
     * means we have to multiply each red, green, and blue channel in our image
     * with the alpha value divided by 255.
     *
     * Notes:
     * 1. ImagePreMultAlpha() needs to be called before every call to
     *    UpdateLayeredWindow().
     *
     * 2. Must divide by 255.0 instead of 255 to prevent alpha values in range
     *    [1, 254] from causing the pixel to become black. This will cause a
     *    conversion from 'float' to 'BYTE' possible loss of data warning which
     *    can be safely ignored.
     */

    BYTE *pPixel = NULL;

    if (pImage->width * 4 == pImage->pitch)
    {
        /* This is a special case. When the image width is already a multiple
         * of 4 the image does not require any padding bytes at the end of each
         * scan line. Consequently we do not need to address each scan line
         * separately. This is much faster than the below case where the image
         * width is not a multiple of 4.
         */

        int i = 0;
        int totalBytes = pImage->width * pImage->height * 4;

        for (i = 0; i < totalBytes; i += 4)
        {
            pPixel = &pImage->pPixels[i];
            pPixel[0] *= (BYTE)((float)pPixel[3] / 255.0f);
            pPixel[1] *= (BYTE)((float)pPixel[3] / 255.0f);
            pPixel[2] *= (BYTE)((float)pPixel[3] / 255.0f);
        }
    }
    else
    {
        /* Width of the image is not a multiple of 4. So padding bytes have
         * been included in the DIB's pixel data. Need to address each scan
         * line separately. This is much slower than the above case where the
         * width of the image is already a multiple of 4.
         */

        int x = 0;
        int y = 0;

        for (y = 0; y < pImage->height; ++y)
        {
            for (x = 0; x < pImage->width; ++x)
            {
                pPixel = &pImage->pPixels[(y * pImage->pitch) + (x * 4)];
                pPixel[0] *= (BYTE)((float)pPixel[3] / 255.0f);
                pPixel[1] *= (BYTE)((float)pPixel[3] / 255.0f);
                pPixel[2] *= (BYTE)((float)pPixel[3] / 255.0f);
            }
        }
    }
}

BOOL ImageImportTga(LPCSTR lpszFilename, Image *pImage)
{
    /* Load the 32-bit TGA file into our Image structure. Only true color
     * images with a 32 bit color depth and an alpha channel are supported.
     * Loading all other formats will fail.
     */

    int i = 0;
    int scanlineBytes = 0;
    BYTE *pRow = NULL;
    FILE *pFile = NULL;
    TgaHeader header = {0};

    if (!pImage)
        return FALSE;

    if (!(pFile = fopen(lpszFilename, "rb")))
        return FALSE;

    fread(&header, sizeof(TgaHeader), 1, pFile);

    /* Skip over the TGA image ID field. */
    if (header.idLength > 0)
        fseek(pFile, header.idLength, SEEK_CUR);

    if (header.pixelDepth != 32)
    {
        fclose(pFile);
        return FALSE;
    }

    if (!ImageCreate(pImage, header.width, header.height))
    {
        fclose(pFile);
        return FALSE;
    }

    if (header.imageType != 0x02)
    {
        ImageDestroy(pImage);
        fclose(pFile);
        return FALSE;
    }

    scanlineBytes = pImage->width * 4;

    if ((header.imageDescriptor & 0x30) == 0x20)
    {
        /* TGA is stored top down in file. */
        for (i = 0; i < pImage->height; ++i)
        {
            pRow = &pImage->pPixels[i * pImage->pitch];
            fread(pRow, scanlineBytes, 1, pFile);
        }
    }
    else
    {
        /* TGA is stored bottom up in file. */
        for (i = 0; i < pImage->height; ++i)
        {
            pRow = &pImage->pPixels[(pImage->height - 1 - i) * pImage->pitch];
            fread(pRow, scanlineBytes, 1, pFile);
        }
    }

    fclose(pFile);
    ImagePreMultAlpha(pImage);

    return TRUE;
}

void InitLayeredWindow(HWND hWnd)
{
    /* The call to UpdateLayeredWindow() is what makes a non-rectangular
     * window possible. To enable per pixel alpha blending we pass in the
     * argument ULW_ALPHA, and provide a BLENDFUNCTION structure filled in
     * to do per pixel alpha blending.
     */

    HDC hdc = NULL;

    if (hdc = GetDC(hWnd))
    {
        HGDIOBJ hPrevObj = NULL;
        POINT ptDest = {0, 0};
        POINT ptSrc = {0, 0};
        SIZE client = {g_image.width, g_image.height};
        BLENDFUNCTION blendFunc = {AC_SRC_OVER, 0, 255, AC_SRC_ALPHA};

        hPrevObj = SelectObject(g_image.hdc, g_image.hBitmap);
        ClientToScreen(hWnd, &ptDest);


        UpdateLayeredWindow(hWnd, hdc, &ptDest, &client, g_image.hdc, &ptSrc, 0, &blendFunc, ULW_ALPHA);

        SelectObject(g_image.hdc, hPrevObj);
        ReleaseDC(hWnd, hdc);
    }
}




std::string toadArray[10] = {
"assets/rainbowtoadtga/0.tga",
"assets/rainbowtoadtga/1.tga",
"assets/rainbowtoadtga/2.tga",
"assets/rainbowtoadtga/3.tga",
"assets/rainbowtoadtga/4.tga",
"assets/rainbowtoadtga/5.tga",
"assets/rainbowtoadtga/6.tga",
"assets/rainbowtoadtga/7.tga",
"assets/rainbowtoadtga/8.tga",
"assets/rainbowtoadtga/9.tga"
};


int main()
{



    sf::RenderWindow window(sf::VideoMode(winWidth, winHeight), "Transparent Window", sf::Style::None);

    //Set window to WS EX LAYERED
    SetWindowLong(window.getSystemHandle(), GWL_EXSTYLE, GetWindowLong(window.getSystemHandle(), GWL_EXSTYLE) | WS_EX_LAYERED);


    int ayyy= 0;
    window.setFramerateLimit(24);


    while (window.isOpen())
    {
        sf::Event event;
        while (window.pollEvent(event))
        {
            // "close requested" event: we close the window
            if (event.type == sf::Event::Closed)
                window.close();
        }


    ayyy +=1;
    if (ayyy > 9)
       ayyy = 0;



    if (!ImageImportTga(toadArray[ayyy].c_str(), &g_image))
    {
        MessageBox(0, TEXT("Failed to load"),
            TEXT("Layered Window Demo"), MB_ICONSTOP);

        return 0;
    }



     InitLayeredWindow(window.getSystemHandle());



   window.display();



    }



    return 0;
}

 

The video actually uses per pixel transparency but the image I was using was just bad 

... any thoughts?

[Hapax]

Are you attempting to animate pixel transparency of a window?

[UltraNyan]

Are you attempting to animate pixel transparency of a window?

Yes.

And ideally I would like to split the updating the window into 2 parts. First part would update the window contents itself, regardless of the shape or transparency. The second part would update the transparency mask, that way you would be able to skip creating a new dataset for transparency and animate inside of the non transparent area.

Currently I have some difficulties understanding how exactly the UpdateLayerdWindows works because i cant avoid drawing the window over with the new picture used to generate the transparency.

[Hapax]

I'm not sure how well an operating system can actually animate a window's transparency.
If it's in real-time, you need to process every pixel manually and create regions (you mentioned this was for Windows only) etc. for every frame.
If it's not in real-time, though, you can store combined regions (effectively a frame's alpha map) and recall them as needed.

[UltraNyan]

I'm not sure how well an operating system can actually animate a window's transparency.
If it's in real-time, you need to process every pixel manually and create regions (you mentioned this was for Windows only) etc. for every frame.

Well the youtube video I included, it was running locked at 24 frames per second on my laptop(i7) on one core at about 40%. It was much more effective than setting the window shape that used about 100% of cpu and ran at about 15 fps. In both cases, it was loading the image file each frame (very ineffective) to simulate off screen rendering.

If it's not in real-time, though, you can store combined regions (effectively a frame's alpha map) and recall them as needed.

If I used a single sprite for the whole window, I could have some of the data for textures split in two: one for image and other for transparency, would be able to speed stuff up a bit by avoiding to process the whole image file each time i use it.

PS.
Im kinda busy atm with some other stuff, if I figure out how to properly use UpdateLayeredWindow then I will try to implement it with the render texture and post the results here.