sRGB/RGB conversions

[kfj]

Dear all!

I have just discovered that SFML can handle textures in linear RGB (I'll use 'LRGB' in this post, in contrast to SRGB). You may say this is old hat, but the capability is not glaringly obvious from the documentation. I'll reiterate what I have understood so far:

There are two components involved, the texture and the window. The texture may contain LRGB or SRGB data, and to tell the system which type of data the texture holds, you call sf::Texture::setSrgb() and you can inquire with sf::Texture::isSrgb() what the current state is. The second component is the window, and there it depends on the ContextSettings passed to the window's c'tor. The ContextSettings have a boolen field 'sRgbCapable'. You can pass true here if you want an SRGB-capable frame buffer, and false if you don't. AFAICT passing false will certainly succeed, but passing true may not succeed - you have to inquire about the state after creating the window to figure out what type of frame buffer you have.

Now let's assume you're working in LRGB. To offload LRGB to SRGB conversion to the GPU, you'd call setSrgb() with a false argument to tag the texture as holding LRGB data and work with an SRGB-capable frame buffer. If the latter can't be had (or the window was deliberately created without SRGB capability), you have to 'manually' convert your data to SRGB on the CPU before storing it to the texture.

I assume that passing in LRGB data should be preferable: first, the time to convert the texture data to LRGB (which is used by the GPU) is saved, and second, since the texture data is quantized to 8 bits already, doing the conversion from SRGB to LRGB will produce loss of image quality. Is this assumption correct?

How about transparency? Is the alpha value processed as-is, regardless of the texture's property?

I'm processing photographic images, SRGB/LRGB is an issue for me - in fact, my data are initially in single precision float LRGB, and I'd be happy if I could offload the float-to-uchar conversion to the GPU as well. Is there a way to do that simply with SFML? And if not, maybe this would make a nice new feature?

[eXpl0it3r]

I'm out of my depth here, so all I can say is, why don't you give it a try and see if you end up with the result that you expect?
Maybe then you can point of some behavior that doesn't work the way you expect and I can with someone more knowledgeable in the topic if this is expected or should be adjusted.

[kfj]

So far sending linear RGB to the openGL code works just fine, and for me it creates a significant performance improvement.[lux][https://bitbucket.org/kfj/pv/] now does this by default (needs to be built from master; the precompiled binaries still pass SRGB). How about my proposal to allow passing in single precision float data, rather than 8bit RGBA?

[eXpl0it3r]

There's mostly the question of API design and technical implementation in the way of adding single precision float data support to SFML. You can find among others, this discussion on the topic (it's only 10 years old ): https://github.com/SFML/SFML/issues/166

Btw. I gave lux a quick try and I can still not figure out what the selling point is over other image viewers.
Maybe there isn't a killer feature over other image viewers, or only if you're interested in some very specific rendering settings, but it wasn't clear for me from the readme.
Like I don't see why it would be important for an image viewer to support SIMD op codes etc.

I also got a "nice" crash, while viewing a panorama from my Galaxy S20.
Not sure if it's a feature or a bug

[kfj]

When I scanned the openGL documentation, I found that openGL supports float textures, so I thought that passing in float data and the appropriate tag might be easy, but I haven't investigated further. My rendering pipeline directly produces 8bit RGBA in a final stage, where it converts the float vectors to uchar and interleaves them. The alternative would be to directly interleave the float vectors to memory, but the memory footprint would be four times as large for a float texture, and the question is whether the added memory traffic outweighs the float-to-uchar conversion, which can be done quite efficiently with SIMD operations since the data is in the vector registers already. When handling large amounts of data the best approach is to get memory access down to a minimum, because that's usually the slowest bit. If I had a simple way of passing float textures to openGL, I'd give it a try and see if it's any faster than what I use now - hence my request.

Thanks for giving lux a try! What you see in your abortive attempt is not a crash but a message box stating that lux can't figure out the FOV. Oftentimes the metadata in panoramas generated with smartphones don't work with lux without passing additional information. Try and pass both projection and HFOV (in degrees) on the command line, like

lux --projection=spherical --hfov=200 pano.jpg

The projection and field of view are vital for a correct display, and if lux can't figure them out that's a show-stopper and lux terminates, emitting the message box as it's 'last gasp'. Sadly, panorama-specific metadata aren't standardized well. lux should be able to process GPano metadata and metadata in the UserComment tag, as written by hugin. It also supports it's own flavour, have a look at https://kfj.bitbucket.io/README.html#image-metadata.

lux can do much more beyond simply displaying panoramas, but most 'advanced' capabilities (like stitching or exposure fusions) require PTO input and the use of additional command line parameters. Being FOSS, lux does not have 'selling points', but I'd say that my b-spline-based reimplementation of the Burt&Adelson image splining algorithm makes it unique. I am a panorama photographer and hugin contributor, and lux is the result of many years of personal reasearch into b-splines and image processing - mainly 'scratching an itch', but, feeling generous, I publish my software for everyone who wants to give it a try. My approach uses SIMD to do efficient rendering on the CPU, hence the use of SIMD-specific code. If you're working on an intel machine, you can figure out what difference SIMD makes. Let's assume you have an AVX2-capable machine. lux will figure out that your processor has AVX2 units and use AVX2 ops, but you can tell it to switch back to lesser ISAs (try --isa=fallback which uses some 'lowest common denominator' SSE level). If you do a 1000-frame sweep over your image and look at the average frame calculation duration (echoed near the end of the console output) you'll see the difference. Try

lux -ps -h360 -z1000 -A.05 some_image.jpg

lux is a large program with tons of features, but most of it's capabilities aren't immediately obvious, and many require the use of command line parameters. Please give it a bit more time, and if you find something amiss, post an issue to it's issue tracker!

[kfj]

There is one issue with using an sRGB-capable frame buffer and SFML as it is: AFAICT I can only pass uchar data to create a texture. If I do so, the resolution is insufficient to display dark areas without banding. So this would be another reason to allow passing in other data types.
In the context of my image/panorama viewer, the gain in rendering speed when passing LRGB data is significant - somewhere in the OOM of 1 msec per frame on my machine, but the banding can be annoying for some content. So in lux, I use a non-sRGB-capable frame buffer per default and pass in sRGB data (this is also SFML's default), but I allow the user to override that via --gpu_for_srgb=yes (currently master branch only, the binaries are still 1.1.4)
Thanks @eXpl0it3r for the link to the discussion about textures with different data types - I wonder if this is still an issue? The discussion went on over a long time and never really seemed to get anywhere. How about this: rather than creating openGL textures with a different internal data type, one might overload the SFML texture's c'tor to accept several data types and subsequently convert the incoming data to a uchar sRGB texture. The conversion could be done on the GPU, and the resulting texture would be the same as the textures created by SFML's usual texture creation routines, so no issues would arise from a multitude of texture data types, and the remainder of the code wouldn't be affected. The overloads would not interfere with the status quo.