Issue with RenderTexture to Image copy

[gex]

Hey there

First of all I try to explain what I am doing before going into the specific issue I encountered and where I will appreciate any help and experience.

SFML version 2.5.1 on windows, dynamic linked in c++ 11

I am using SFML Graphics for creating a rasterized output containing text and a QR code, which can be added to print output (yeah, not best for print, but reality for a lot of legacy applications, but we use high resolutions to get best quality). When everything is drawn, it is exported/saved as PNG/JPEG or whatsoever. Initially I created one "large" RenderTexture and used is one canvas where I drew all the stuff. when finished I copied the texture to an Image and saved it to a file. This worked fine on Desktop machines.

So now for the fun part:
Then we made tests on target machines and some of them were Windows Server (2012 / 2016) machines, where no RenderTexture larger than 1000x1000 or maybe 1024/1024 could have been created. So the obvious thing to do was to split it up to multiple RenderTextures and then combined it into the final Image. Now it still worked on my Machine and it did on the target Windows Server machines. However it did only work properly on the windows server machine if our executable was executed directly (both traditional win32 exe and console application). As soon as the same binaries with same commands and data were executed as a subprocess of another programm, parts of the output image went black (/or gray when saved as png). This messed up part did only occur in the largest RenderTexture, so I like to illustrate this with two pictures

layout_lines.png is created using "debug layout" option, which draws green lines around all layout areas, which currently match a RenderTexture each.

https://en.sfml-dev.org/forums/index.php?action=dlattach;topic=25561.0;attach=4391;image


and as you can see in the following example.jpg, in the largest layout area, the end / bottom is messed up.
https://en.sfml-dev.org/forums/index.php?action=dlattach;topic=25561.0;attach=4389;image



From a code perspective the following pattern is followed:

sf::RenderTexture rt;
if (!rt.create(width, height)) {
            throw ...;
}
rt.clear(sf::Color::White);

// multiple draw calls
rt.draw(...);

rt.display();
const sf::Texture &texture = rt.getTexture();
img.copy(texture.copyToImage(), an_x_pos, a_y_pos);
 

I am passing the RenderTexture as a reference to other functions in order to draw to it.

At the moment I am unable to reproduce the issue on any of my machines at hand - I ran the examples on azure virtual machines (server) and was able to reproduce the RenderTexture size limit but not black/gray-area issue. What am I missing, does any one have an idea what goes wrong there?

[binary1248]

This seems a bit over-engineered to me.

If all you want to do is combine text, rectilinear lines and a generated QR code bitmap into a single bitmap, you don't even need SFML to do this. Just use FreeType directly to rasterize the text for you and you will have all the bitmap data you need to combine into the final image.

SFML is a library aimed at real-time GPU accelerated rendering. If you know that your target system will typically not have a (usable) GPU then SFML probably isn't the right library for your work. In your case you don't even want to render/draw more than a single frame. The time it takes to set up the graphics pipeline far outweighs the time it would take it to render that singular frame let alone transfer the data back to main memory to save as an image. Performance-wise it is a nobrainer that this is better done purely on the CPU.

If FreeType seems daunting to use at first, that might be so. Just have a look at the tutorials and familiarize yourself with the API. Rasterizing non-rotated LTR western script text to a bitmap can be done in 100-200 lines of code. From there you can combine the bitmap data with the rest of your elements into the final image.

[gex]

First of all, thank you binary1248 for spending your time to answer me, I really appreciate it.

I expected this kind of reply - I am aware what SFML is intended for and that it is too much for my use case. We were looking for a library that abstracts some details away and it worked well - until this issue.

Still, I would be very interested in any hints that can point to the actual issue I am running into so that I do not need to rewrite the code at this moment. At the same time I will consider rewriting it...

[binary1248]

Since you are targeting systems without real GPUs, we can assume that SFML is choosing the FBO emulation path via pbuffers or hidden windows. In this case it is up to the system how large the pbuffers are allowed to be and if it really had to fall back to creating hidden windows there might be some strange interactions with the fact that it is a sub-process of another process. pbuffers were never really a clean thing either.

As I already said, it might be comfortable for you as the application developer, but from an implementation point of view, so many things are being used in ways they were never meant to be used that nobody is going to guarantee any kind of behaviour in this scenario. You are basically on your own at this point.

Realistically, the time you would invest in figuring this problem out (if you can actually resolve it) would be better spent in coming up with a clean, supported solution from the get go. The solution I detailed above is purely CPU based, requiring nothing more than CPU and memory. It would even run on headless systems without a desktop environment, which makes it more amenable to being run in the cloud. Considering your target platforms, it is the only path I can recommend.

[gex]

At first we only expected the code to run on desktop systems, where we did not expected those issues.
But from a retrospective point of view it would indeed have been less effort to go the way you outlined. I will do some PoC coding and see how things go there. I already have stbi included for saving data as PNG/JPG included.
I'm just wondering if there is a thin abstraction library over FreeType or not. Furthermore I will need to include a small scissor image which needs to be scaled.

Anyway I appreciate your explanation on what could have gone wrong.

[binary1248]

I'm just wondering if there is a thin abstraction library over FreeType or not.

When I first had to do TrueType things myself I was also looking for such a library for quite a while.

Over the years I have come to the conclusion that there is a reason why wrapper/abstraction libraries don't exist for certain C libraries. It is probably because their API is so "simple" to use that a wrapper or additional abstraction wouldn't add much value that it offsets any potential disadvantages (like performance overhead, additional dependency, etc.) that might be incurred. FreeType is such a library. It is used everywhere, by many people and there is still no widely used wrapper of it.

Furthermore I will need to include a small scissor image which needs to be scaled.

If the target resolution stays the same throughout the life cycle of your application, just pre-scale the image yourself with a high quality filter and bake in the final bitmap data. Any algorithm that is optimized for real-time scaling, up/down sampling such as those found on GPUs will never achieve the quality of a good offline algorithm.

[gex]

Just made first steps with FreeType. I have to confess this does feel too low level to me, however I guess I will follow this path and complete my PoC. By now I was able to compile a simple exe which renders a simple text string and outputs it into a BMP file using stb_image_write.h

[gex]

Although this is out of SFML scope and thus of this forum you might still have a hint for me. The QR code, which I need to include into the image, varies in terms of encoded data length and thus in the number of modules. In the final image the QR code need to have always the same size (+/- some pixels is okay). Therefore I assume I well get into situations where I need to scale the qr code at runtime before integrating it into my output bitmap. What algorithm or even small lib is best suited for such a job? I guess if I get this done, then the rest is just coding effort.

[binary1248]

https://en.wikipedia.org/wiki/QR_code

The size and thus maximum payload length depends among other things on the "version" of QR code that is used. I don't know how you are generating these, but if you make sure the version stays the same all the time then the size of the output image will not change. You just have to pick a version that is big enough to fit all your data.

Although good code readers have to put up with a lot of broken stuff, I wouldn't recommend trying to perform non-integer scaling on the final output image. This will lead to some squares being bigger/smaller than others and might throw off the reader in certain situations where it can't synchronize with the timing pattern.

If you really need to do it anyway and are confident that readers will be able to read a scaled image, then since you are already using STB you can use stb_image_resize.h for your purposes...

[gex]

I use qrencode to generate the QR Code. I am aware of the QR code, version, module error correction level et al, so there I won't have issue.

If I could choose, I would only do integer scaling, however I'm not free to. However quality will be okay as I'm going to generate "high" resolution images, I just need to run some examples for different QR versions with different scaling factors in order to check if there are particular bad combinations.

Oh my, "stb_image_resize.h" could have seen this myself.