Greater Pixel Access in sf::Image

[Clairvoire]

Hello!  Long time user, first time poster.  

I switched to SFML from SDL quite a while ago, and it was a great move up!  But there has always been something of SDL's that I really miss sorely, and that's the ability to perform pixel level modifications efficiently.

I'm aware of SetPixel(), but it has more overhead than I'm sure anyone would care for in a function whom will be called more than a million times a second.  (500 x 500 image, at 60 frames a second, is 15,000,000, so 'a million' is actually a modest accessment.  xD)

What I'm doing right now to circumvent much of that, is this:

Code: [Select]

sf::Uint8 * p = const_cast<sf::Uint8*> (image->GetPixelsPtr());

// ... mess with the pixels here ...

image->SetPixel(0, 0, image.GetPixel(0, 0));

A const cast, and a backwards function call at the end simply for the sake of setting Image::myTextureUpdate to true.  It's not very pretty, even if it works.


Maybe perhaps providing for, say... a public 'Finalize()' function in Image that would set myTextureUpdate to true (and anything else that'd have to be done, in the future), and perhaps another function that returns a non const pointer into the pixel array, that would require Finalize() be called for the image to be valid for rendering?  

It could be done with a boolean flag, that'd be set to false in the pointer retrieving function, set to true in Finalize(), and if it's not true, it can set off an Assert or something if someone attempts to render it.  I'm sure you'd think of a more elegant way, but that's probably how I'd implement it.  


This can't be the first time this has come up, but if it counts at the very least as a vote in the hat, then I'm happy~  Thanks for the awesome library!

[Laurent]

What you can do is using a temporary buffer instead of const_casting sf::Image's internal one.

Code: [Select]

std::size_t size = image->GetWidth() * image->GetHeight() * 4;
std::vector<sf::Uint8> buffer(size);
const sf::Uint8 * p = image->GetPixelsPtr();
buffer.assign(p, p + size);

// ... mess with buffer here ...

image->LoadFromPixels(image->GetWidth(), image->GetHeight(), &buffer[0]);
Image::LoadFromPixels is optimized for this kind of situations, and it will end up only updating the pixels in video memory.
You still pay for two extra copies of the whole array of pixels, but it may not be a problem depending on what you do.
You can avoid the first copy if your image doesn't change between two updates (in this case the temporary buffer is already up-to-date).

[Clairvoire]

Hm... Taking a closer look at LoadFromPixels(), it doesn't destroy and reassign a new texture or anything, like I thought it would.

This is great, since I can now keep my pixel modification code separated from the library too.  Haha, I can't believe I didn't think of that, I just assumed that Load function was an initialization function and nothing else.  xD

Thanks!

[Clairvoire]

I've replaced the code with the method you suggested, but noticed a considerable loss in performance.  Granted, the difference was from 'nonexistent' to 'trivial', but I'm pretty nit picky about making senseless optimizations.  xD

Most of the pixel data in the new array and old array are the same, with sparse modifications here and there.  Accessing the raw pixel array and making edits directly seems to cut a lot of the time LoadFromPixels() uses to reassign data that doesn't need to be updated, and to make assertions that don't need to be asserted.

For right now, I'm going to take the Strawman approach and add the two functions to sf::Image myself.  I realize this opens me up to being lambasted should SFML change how it internally represents pixels.  But then again, no one deviates from the 8bit RGBA quartet~

[Laurent]

Ok, fair enough

In the future I'll try to provide faster and more direct access to pixels. I think there is some room for improvement there.

[alienGhost]

Ok, fair enough

In the future I'll try to provide faster and more direct access to pixels. I think there is some room for improvement there.

Hi...I'm currently evaluating the potential of using either SDL or SMFL for a university project. I have been using java/Processing for my first year for image/audio convolution and general DSP techniques - we will now be moving on to C++ and we have been asked to research APIs to use for similar work in year 2, during our current holiday.

I need relatively painless/fast access to the pixel buffer. So far I like the look of SFML more, but because my project will involve direct access to pixels, SDL seems to have bonus points for this historically.

I can see SFML 1.6 has been released, has there been any improvements on direct pixel access in this new release?

thank you for any advice!

EDIT: I'm sorry for making this post long but additionally: does anyone have a good sense of which API (SDL or SFML) is better for fastest/simple access to raw data (audio/pixel) _overal_. in other words, which would be more suited to DSP techniques. It's very hard for a noivce like me to tell immediately, without wasting a lot of time testing different APIs...so if anyone has any hints on this.

Thank you!

[Laurent]

Nothing changed in SFML 1.6, but SFML 2.0 will probably soon have a much better API for dealing with pixels. It already implements a new Image::UpdatePixels function that avoids copying the pixels to the Image cache; instead they go straight to the video memory.

So, currently the best solution is to use SFML 2.0, an external raw array of pixels, and Image::UpdatePixels whenever you want to sync the image with it.

There's currently no similar option in the audio module, to avoid the internal copy of sound samples when updating a sf::SoundBuffer.

[alienGhost]

Nothing changed in SFML 1.6, but SFML 2.0 will probably soon have a much better API for dealing with pixels. It already implements a new Image::UpdatePixels function that avoids copying the pixels to the Image cache; instead they go straight to the video memory.

So, currently the best solution is to use SFML 2.0, an external raw array of pixels, and Image::UpdatePixels whenever you want to sync the image with it.

There's currently no similar option in the audio module, to avoid the internal copy of sound samples when updating a sf::SoundBuffer.

thank you very much Laurant! the new UpdatePixels in v2 sounds a lot more efficient...and updating each frame should be no problem for graphic works.

as for audio, this i think will be my problem in both SFML and SDL as i need access  realtime audio stream for dynamic sound manipulation.

i'm thinking i can hopefully use sfml in combination with something like RtAudio library http://www.music.mcgill.ca/~gary/rtaudio/.

bah, Processing and its 3rd party libs made this so much easier! I have a lot to take in learn.

thank you again!

[Laurent]

In SFML you can use sf::SoundStream for real-time audio manipulation. I don't know if it's a good candidate for what you need.

[Ashenwraith]

I used const_cast for my original pointer and and edit the pixel data with a variety of functions and and updatepixels in the end.

I wished there was no need for const_cast too, ie loading a pixel array and have the image point to that instead of the reverse.


BTW, after I finish writing my mutli image to pixel array loader I'm going to need to do the same thing with wavs, so hopefully there is a stream data or something I can drop it into.