sfeMovie project [v1.0 RC1 available]

[Ceylo]

Time for some news!

What has been done since last time
- sfe::Movie::Stop() is working fine now. It goes back to the beginning and you can resume playing right after stopping the movie.
- I've been working a lot on performance optimization. Now if the video decoding is late, it is shortened as much as possible (the image is still decoded, but not resized, not converted, not loaded, and not synchronized ; in other words, it's skipped). Thus audio and video keep synchronized. See the second part of this post for more information about this point.
- I've added sfe::Movie::ResizeToFrame() to easily resize the movie. It's main interest is that you *can* preserve the original movie ratio. If you choose to do so, the movie will be centered according to the given frame. Thus you can really easily get the right position and size for your movie, so that it best fits your screen when you want to play a movie in full screen mode for example.
- I've also updated the Xcode (Mac OS X), and Code::Blocks (Windows) project files to match the file name changes.


Performances
I did a lot of testing to find which parts in the code were taking too much time. These tests have been done on my personal laptop :
Intel Core 2 Duo 2.4 GHz, Intel GMA X3100 graphic card with 144 MB, 2 GB of DDR2 RAM.

Here are the results :

On Mac OS X:
- 31.6% : sf::Sprite::Draw()
- 16% : video decoding (avcodec_decode_video2())
- 10.1% : synchronization between the main thread and the video decoding thread with mutexes (sf::Lock)
- 7.7% : image loading (sf::Image::LoadFromPixels())
- 4.4% : image conversion from YUV to RGBA format (sws_scale())

On Windows (same computer) :
- 38% : sf::Sprite::Draw()
- 18.3% : video decoding (avcodec_decode_video2())  
- 18.4% (!!!) : synchronization between the main thread and the video decoding thread with mutexes (sf::Lock)
- 6.1% : image loading (sf::Image::LoadFromPixels())
- 6.3% : image conversion from YUV to RGBA format (sws_scale()) (sws_scale())


As you can see, on Windows the synchronization is taking twice as much time as on Mac OS X. I've no explanation about this for now. But to sum it up : a 1280x544 definition movie plays fine on both Mac OS X and Windows, but as for the 2048x872 definition movie... it works fine on Mac OS X (no frame skipped) but not on Windows (frames are skipped every 1 or 2 seconds).

To my mind, this is still quite good, 1280x544 is still a good definition, and I don't have a very powerful laptop, this should be usable for higher definition movies with computers that are just a bit more powerful. I'll of course try to improve the performances for Windows if I can find a solution.

[Ceylo]

I've been doing a little bug fix : when pausing/resuming the movie playback several times, the audio track was getting late. This is due to the fact that sf::SoundStream often plays again the last samples that were being played right before performing the pause. Thus now audio and video should ALWAYS keep synchronized.

This was the only remaining bug I knew as far as Mac OS X and Windows are concerned. As for Linux, there's been a crash reported and I still need to fix it. After this is done, I think I'll post sfe::Movie on the wiki.


Now... I would like to get your feedback for a few questions :
- for now, Pause() pauses the movie playback if it's playing, and resumes it if it's paused. This is the behavior I find the most logical but I realised sf::Music was not working that way, and I would like to know which behavior you prefer.

- the second point is about the error/warning messages. With the last bug fix, there are even some more messages because the video track is catching up with the audio track. But... there messages are not really useful. They are actually mostly used for debug purposes. So, even if these messages are outputted to std::cerr only, I would like to know whether you think I should remove them. They could be useful when you're developping an application and want to check whether your movie definition is too high for a basic computer. What do you think of it ?

- and last but not least, I've start a blog in order to make some advertisement for me and SFML. Till now I wrote an article (in english) to sum up what is the sfe::Movie project. If you're interested, you can have a look at it at http://ceylow.wordpress.com/ . I'm probably going to reuse what I wrote in this article for the project's presentation on SFML's wiki. I also plan to write another article, more focused on how sfe::Movie works, rather than what is does. So... if you have any comment, don't hesitate, I'm listening to you! (I hope my English is not too bad :roll: )

[Ceylo]

Some news here .

1. As for Ubuntu, I wasn't able to make a bootable USB stick (it has several partitions and my laptop won't boot from it...). Thus I asked Ubuntu.org for a CD, and I'll get it.. when I get it.

2. I'm looking for two persons to help me to develop sfe::Movie. The first person must be a Windows developer and the second one a Linux developer. If one works regularly on both and can do both, it's even better.

The goal is to do testing to find out what's slowing down the video playback (this may depend on the OS and hardware). You'll also have to test sfe::Movie to the limit so that we can exactly know what's its behavior in these conditions and to find out possible bugs (one has been reported on Linux but I still don't know why it is happening). This testing will also help us to improve the overall performances with high resolution movies and/or with "slow" computers.
I'd also like to provide the simplest integration, thus the developer will choose the way the library should be distributed for the OS he's using. It may seem obvious to the one using the concerned OS, but to me it does not as for Windows and Linux.


Note that I *could* do most of that work, but it would take time.. that's I don't have. And as I want sfe::Movie to reach a mature state, I'll focus on Mac OS X and let the rest of it to one or two other developers.

To the one who may participate, note that's you'll have to use Git to post your work.


3. As for the improvements, here are my thoughts till now.

I'm thinking of decreasing the time during which the displaying and decoding threads are locked. I plan to lock the mutexes for the time of an image swap instead of an image loading. Thus there'll be a front and a back image directly handled by sfe::Movie. This should allow the class to better use the processing units and provide an even smoother video playback.

I'm also thinking of letting the possibility to define a target definition. For example if I have a movie with a 2048x872 definition and my screen's definition is only 1280x800, copying and transferring pictures of the original size is useless. This would also let the final user, if he's running a slow computer, and the developer does not want to provide video files for each definition, get better results. This is not as efficient as providing the video files for several definitions but it may still be a significant improvement with very few efforts.

[Ceylo]

News again .

1. The source files have been updated and can be used to compile sfe::Movie against SFML 2.0. I didn't make any change to the headers and libraries on the Git repository, thus you'll have to replace them with the SFML 2.0 version in order to compile sfe::Movie for this version.

2. I've also committed a very small change that greatly improves the fluidity* of the video playback. This change applies to both SFML 1 and SFML 2.

3. Danman is helping me on the Linux version of sfe::Movie (still crashes quite often for now).


Sooo.... still many things to do, but it's still getting better and better !


*Is it the right word to say the video playback is much more fluent ?

[Terrydil]

*Is it the right word to say the video playback is much more fluent ?

If you mean that the playback is smoother than before then "improves the fluidity" makes sense.  I wouldn't say playback is "fluent" though, just smooth, or maybe consistent.

[Ceylo]

Ok, so.. the fluidity of the video playback has been improved ^^.


(smoothness is fine too?)

[Ceylo]

Some news again

Right now I'm trying to provide binaries for each OS so that you do no more have to deal with compiling sfeMovie (I admit it's not so obvious). As for Mac OS X, the library is ready. As for Windows.. I'm still working on it and I'll try to get it ready next week-end.


Otherwise, I've noticed a few issues with audio/video sync. When video is late, it correctly catches up with the movie timeline, but not audio. I hadn't noticed this earlier because audio processing is much lighter and hardly ever blocked as samples are buffered and ready to use even if the program is freezed.

One way to "fix" this issue is pausing then playing. Because at playing time it'll synchronize according to the audio timeline. However I cannot always synchronize to the audio timeline because it's not precise enough (it's not updated often enough). Thus I would have to implement seeking.

I could also manually skip some samples, as I do for the video track. But I think I'll just implement seeking as it's also needed.

I guess that'll be the first task for v1.1.

[Haikarainen]

Where can i get my hands on this?

Im working on a project for my car, basically an embedded computer. Writing the shell in SFML, and i need video playback support. It will be running Linux Debian on ARM Architecture, is linux supported?

How much work would it be to be able to seek the videos? If its not too hardcore lowlevel coding i might be able to pull it off.

[Ceylo]

Hello Haikarainen,

Thanks for your interest .

You're quite true, I didn't even give a link to the Git repository...
Here it is : https://github.com/Yalir/sfeMovie

I didn't really work on the Linux port, but the developer who wanted to help told me it often crashed. Thus the short answer is no. However you can notice that this project relies on portable libraries only, thus you may be able to fix the bug. Note that I've never tested sfeMovie on an ARM architecture.

As for building sfeMovie, the shell script supposed to do so is not ready yet, so here is what you have to do :
1. Build FFmpeg static libraries
2. Put these libraries in sfeMovie/deps/ffmpeg-build (create the directory if needed)
3. Run CMake and the EDI/compiler you chose to use (probably make)

As for seeking, I don't know how much time you'll need. There is no hardcore coding, it's all about properly using FFmpeg and making the rights changes to sfeMovie.

[Ceylo]

As a side note, here is a wiki page I had started : https://github.com/SFML/SFML/wiki/ProjectsfeMovie

It's still incomplete but you may find some useful information.

[Haikarainen]

Hello Haikarainen,

Th.. blablab text.

Ok, shouldn't be too hard. If I ever get the code working perfectly on a linux platform I will notice you

[Ceylo]

Indeed it would be really nice of you!

I wish I had the time to get sfeMovie work perfectly everywhere and support every important features and get the world peaceful and.. but well no, for now I don't have enough time :oops: .

[Ceylo]

Hello everyone,

Information update from 2011-05-12
I'm writing here to give you the latest news about the sfeMovie project.

As you may know, sfeMovie is a small C++ library that lets you play movies in SFML based applications. It relies on SFML for the rendering process and FFmpeg for the decoding process. It supports both audio and video, and basic controls. This project has been written to work closely with SFML and tries to keep the same easy-to-use paradigm, while remaining coherent with SFML's conventions.


Information and download:
Most of the information is available through the wiki page, including the download links.

For now, sfeMovie is available to Windows and Mac OS X, and is in a beta state. Thanks to FFmpeg, sfeMovie can read many formats, including AAC, MP3, Vorbis, WMA, H.264, MPEG4, Theora, VP6 and WMV.
There is still a lot of work to do, but the current version already lets you use the main features.

I'm inviting you to test the library and write your feedback about bugs or any other issue you encountered. The bug tracker allows you to check the already-reported bugs or upcoming features and improvements.


As for the differences with the previous versions:
- all of the precompiled FFmpeg binaries have been dropped
- a script to compile everything for you has been added, it also allows you to exactly choose which decoders you want to enable
- FFmpeg is statically linked to sfeMovie, thus you don't have to care about FFmpeg dlls
- some minor bugs have been fixed


Ceylo

[slotdev]

I downloaded the binaries for Windows, but in the "lib" directory there is no libsfe-movie.lib file to link against. What am I missing? I really don't want to install GCC, MinGW etc to recompile this. I have no experience with them and it'll turn into a can of worms...

Thanks

[Ceylo]

For now the only supported compiler is GCC. Supporting the other compilers is among the many things to do. That's why there is no .lib. The provided package only contains the libraries for GCC.