How to play video from image-sequences (fast) onto the screen?

[tom64]

Hi everyone,

I have sequences of images (BMP or PNG) on my HDD and need to 'play' them as if they were a video.

Does anyone know the fastest way to:

1) Read file data from the HDD and store contents into a pixel array
2) Display that pixel array on the screen
... so that I can get up to 60 FPS with frames that are 2000x1000 resolution?

Any ideas or advice?

[Laurent]

The sfeMovie project does what you want with SFML. But it reads video files, not images. I doubt you'll ever succeed to read HD videos at 60 FPS from individual images... You should really make a proper video from them first.

[eXpl0it3r]

It is possible with SFML to update a texture fast enough to get a decent framerate, though the question is why you'd want a framerate above 30 for video playback.
The harder part is to stream the data from the disk. You might not be able to just call loadFromFile for every single frame, but you should be rather looking into making use of some streams. An even simpler way would be to use a well known format that ffmpeg can play and then use for instance sfeMovie.

Can you elaborate a bit more on your use case, maybe we can suggest another way to approach it.

The best thing you can do, is simply write code and test your idea. Unless we've written test on our own, we can only make educated guesses, which might well be wrong.

[tom64]

I'm not literally making a 'video player', rather, this program is needed for frame examination and manipulation purposes, and so I need to keep them as individual frames. But I may need to 'play' the frame sequence at real time (which may be 30-60 FPS depending on what I get)

What is the fastest way to store the pixel array from the image file and then display the array? This is important as I may just want to display a smaller portion of the frame on the screen, especially if the frame size is bigger than my monitor (and I can't have the frame scaled down as it must be 1:1 pixel/monitor ratio).

I doubt you'll ever succeed to read HD videos at 60 FPS from individual images...

Why? Videos are really just a container with all the individual frames in them and also get decoded , then displayed on the screen.

Isn't it the same thing? 

[Laurent]

What is the fastest way to store the pixel array from the image file and then display the array?

Load it to a texture then draw it with a sprite. But the loading time (with disk access) will kill your real-time performances. And you won't be able to pre-load all images in video RAM.

Why? Videos are really just a container with all the individual frames in them and also get decoded , then displayed on the screen.

Isn't it the same thing?

Not at all. Video formats are have much more complicated compression algorithms than individual images. For example, they can(and they probably do) only store the pixels that change between two adjacent frames. They also use jpeg-like algorithms rather than RLE (which is what BMP and PNG use), which provide a greater compression ratio. Compare the size of your set of images to the size of the corresponding video properly encoded, and you'll see that it's not the same

[tom64]

@Laurent: Sorry I meant compared to (equivalent) lossless videos.

This program will not be run on a standard PC anyway, as this is for work purposes, it will be high end (high enough that it could play lossless video) - but still needs to be as fast/efficient as possible.

Since I'm new to C++/SFML, could you help point me towards what the fastest method for file loading is? (assuming there is a 'faster' way to do it), and what kind of texture loading functions will I need to use? Do I need to set them to 'dynamic'? Would this be 'hardware accelerated'?

Sincerely,
Tom

[Laurent]

sf::Texture texture;
texture.loadFromFile("...");
...
window.draw(sf::Sprite(texture));

That's all you need to know

[tom64]

Thanks Laurent.

Could you recommend what method would be most efficient for displaying only a cropped portion of the image file?
e.g. If the image file is 3000x2000, only show a cropped portion from 0,0 to 1920,1080?

I could probably do this myself by only reading/storing the relevant pixel area from the bitmap (since bitmap data is stored predictably), but then once I have that pixel array in memory could I still use window.draw()? And if so, how?

PS: Is that script already dynamic and hardware accelerated? Looks almost too simple! (I was expecting a lot more coding)

Thanks,
Tom

[Laurent]

To load only a portion of an image, you can use the second argument of texture.loadFromFile, which is the rectangle to load.

To show only a portion of a loaded texture, you can call sprite.setTextureRect.

If I understand correctly, you need the first one.

And yes, SFML textures are hardware accelerated, everything is based on OpenGL. Why would it be more complicated?

[tom64]

Great, wow thanks. I'm glad I stumbled across SFML (I was planning on using DirectX before but it was really mind-numbing since I'm new).

I'm going to give your advice a shot and post here if I have any problems.

Thanks again

[Ceylo]

Hello,

The idea with movies is indeed to store only the changes from previous frames. This allows movie playing without reading too much data. That's not completely true because there are also key frames (usually every 10s) that contain a full image.

However considering the 3000x2000 JPEG compressed RGB images (even if you only use a part of it), it means around 650KB of data per image according to some pictures I have (note that it depends on which picture you choose and it may be heavier...), and almost 8MB for the 1920x1080 RGBA texture loaded in VRAM. If you want to have 60 images per second, that's around 40MB to read from disk each second, and ~500MB to upload in VRAM each second.

Depending on your hard disk read speed, and considering you also have to take in account image loading, uploading to VRAM, screen synchronization and other processes that want to use the hard disk, you may not reach the image rate you wish. That'd probably be much easier with a SSD. If your screen vertical synchronization rate is 60Hz and if you don't want to see half loaded images, you should understand that you have to perform the whole image process in less than 16ms. Parallelization can help at this but here come the headaches too.

As a comparison, a 24 FPS movie with a 1920x1080 resolution needs about 1.5MB of data per second. Even with 60 FPS and a higher resolution you can see it's really much lighter than with independent images.

[tom64]

It's kinda funny in a way, that even today computers (as strong as they are) are not very good at playing decent quality (raw) videos at decent frame rates.

Thanks for your input Ceylo.

Either way, I still have to make this program as it is part of my job. I'll just have to put up with less than real-time until computers get faster/cheaper.

If you guys think of any other ideas that might help please feel free to let me know.

Tom.

[Laurent]

It's not funny, it's just that you're trying to drive a screw with a hammer

[tom64]

Hammer: "Hurry up and get in there - I have another 59 frames!"
Screw: "Hang on hang on! I need to slowly screw myself"

Something like that.

Here is an alternative way to look at the problem.

[Laurent]