SFML community forums

Help => Graphics => Topic started by: zcream on October 08, 2021, 01:51:25 pm

Title: Use SFML in a video processing pipeline - use offscreen buffer
Post by: zcream on October 08, 2021, 01:51:25 pm

In my application, I originally obtain an image buffer on the CPU. This is then copied over to SFML - processed with overlays, and then copied back to CPU image buffer.
I looked at sf::Image and sf::RenderTexture. If I understand this correctly, I need to
a. Copy CPU Image Buffer to sf::Image
b. Copy sf::Image to sf::RenderTexture
c. Process overlays in sf::RenderTexture
d. Copy sf::RenderTexture to sf::Image
e. Copy sf::Image to CPU Image Buffer

Does this sound correct ? Is there a sample code for someone using SFML in an image processing pipeline ?

Title: Re: Use SFML in a video processing pipeline - use offscreen buffer
Post by: fallahn on October 09, 2021, 11:09:09 am

If you can, use the sf::Image as the buffer, it is essentally a wrapper for std::vector<uint8>, or update an sf::Texture directly from your existing buffer, to prevent making an unnecessary copy of it:

https://www.sfml-dev.org/documentation/2.5.1/classsf_1_1Texture.php#ae4eab5c6781316840b0c50ad08370963

Once your buffer is copied to the texture it can be rendered in the same way as any other sf::Texture/sf::Sprite combo, presumably including your video overlays.

You can return the render texture to an image with sf::Texture::copyToImage() although it's not particularly swift:

https://www.sfml-dev.org/documentation/2.5.1/classsf_1_1Texture.php#a77e18a70de2e525ac5e4a7cd95f614b9

At this point I'd be tempted to use some OpenGL directly to copy the output stright to your buffer - although it's worth trying first to see what kind of performance you get.

Title: Re: Use SFML in a video processing pipeline - use offscreen buffer
Post by: zcream on October 11, 2021, 04:04:32 pm

Thats an image buffer of

uint8_t * pixels

a. Copy image buffer to Texture
 

void sf::Texture::update        (       const Uint8 *   pixels  )

b. Render using SFML to texture

c. Copy sf::Texture to sf::Image

or
a. Use sf::Image with you buffer copied to sf::Image
b. Copy sf::Image to sf::Texture
c. Copy sf::Texture to sf::Image


Are these the two options you described ?

Title: Re: Use SFML in a video processing pipeline - use offscreen buffer
Post by: zcream on October 11, 2021, 04:32:50 pm

There is a comparison of direct OpenGL methods here - https://stackoverflow.com/questions/68791782/fastest-way-to-transfer-buffer-texture-data-from-gpu-to-cpu

1)Using glReadPixels - < 3.1ms

   glBindTexture(GL_TEXTURE_2D,depthTexture);
   glReadPixels(0, 0, width, height, GL_DEPTH_COMPONENT, GL_FLOAT,mat.data);
 

2)Using glGetTexImage - <2.9ms

glBindTexture(GL_TEXTURE_2D,depthTexture);
glGetTexImage(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, GL_FLOAT, mat.data);
 

3)Using PBO with glGetTexImage - <2.3ms

glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo);
glBindTexture(GL_TEXTURE_2D, depthTexture);
glGetTexImage(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, GL_FLOAT, 0);
mat.data = (uchar*)glMapBuffer(GL_PIXEL_PACK_BUFFER, GL_READ_ONLY);
 

Title: Re: Use SFML in a video processing pipeline - use offscreen buffer
Post by: fallahn on October 11, 2021, 11:02:05 pm

Quote from: zcream on October 11, 2021, 04:04:32 pm

Thats an image buffer of

uint8_t * pixels

a. Copy image buffer to Texture
 

void sf::Texture::update        (       const Uint8 *   pixels  )

b. Render using SFML to texture

c. Copy sf::Texture to sf::Image

or
a. Use sf::Image with you buffer copied to sf::Image
b. Copy sf::Image to sf::Texture
c. Copy sf::Texture to sf::Image


Are these the two options you described ?

That's correct. SFML uses glGetTexImage under the hood:
https://github.com/SFML/SFML/blob/master/src/SFML/Graphics/Texture.cpp#L369

(or glReadPixels if using GLES)

and glTexSubImage2D for uploading
https://github.com/SFML/SFML/blob/master/src/SFML/Graphics/Texture.cpp#L425

This might be fine for your needs (I've used it for rendering the output of emulators for example) but it will of course start to suffer with larger video resolutions. You can only really know for sure by testing it out  8)