SFML community forums

General => SFML projects => Topic started by: zsbzsb on July 20, 2013, 06:05:25 am

Title: VideoStreamer - .NET Video Player
Post by: zsbzsb on July 20, 2013, 06:05:25 am
What is it?

VideoStreamer is a simple video player I wrote in C++ with a C binding to .NET to provide a way for me to render video in C# with SFML 2.x. Internally it uses FFMPEG for reading/decoding the video clips.


Where is it used?

I use VideoStreamer in my own project that I hope to be releasing soontm. (I use videos as backgrounds)  :)


Special Thanks

I also want to give thanks for inspiration from Ceylo (http://en.sfml-dev.org/forums/index.php?action=profile;u=395)'s sfeMovie Project (http://en.sfml-dev.org/forums/index.php?topic=3463.15) that helped me write VideoStreamer. And of course big thanks to Laurent for SFML.


Licensing

VideoStreamer is licensed with the ZLIP/PNG, the license same as SFML. That way you can do whatever you want with it. Feel free to write patches and make pull requests on bitbucket. If you look at the code you will see that it is a bit ugly and that I am not a C++ developer. Personally I am happy with the performance, but if anyone wants to post improvements I am wide open to suggestions.


Features



Modules



Where do I get it?

Get if from here (https://bitbucket.org/zsbzsb/videostreamer/overview) and remember you will have to provide the paths to FFMPEG libs in order to compile it. There is also a sample project that shows how to use VideoStreamer and this following example is a snippet of the sample project.


How do I use VideoStreamer?

Here is a snippet from the sample program (see "Screenshots" below) on how to use VideoStreamer in your .NET application.


            RenderWindow window = new RenderWindow(new VideoMode(640, 480), "VideoStreamer Example");
            window.SetFramerateLimit(60);
            //Create our video stream
            stream = new VideoStreamerNET.VideoStream(@"VIDEO FILE PATH");
            //Create a clock to track elapsed time
            Clock frameclock = new Clock();
            //Create a specialized sprite renderer to draw our video frame
            VideoStreamerNET.VideoDrawer framesprite = new VideoStreamerNET.VideoDrawer(stream);
            //Start playing
            stream.Play();
            //Keep looping while our window is open
            while (window.IsOpen())
            {
                //Handle current events
                window.DispatchEvents();
                //Clear current window
                window.Clear();
                //Get our current frame time and restart the frame clock
                Time frametime = frameclock.Restart();
                //Update the current frame inside our video streamer
                stream.Update(frametime);
                //Draw the current video frame
                window.Draw(framesprite);
                //Now display everything on the screen
                window.Display();
            }


Screenshots

Here is the sample application playing back a test video.

(http://i40.tinypic.com/smf39x.png)

Also here is the breakdown of what is called when and where and as you can see SFML uses most of the processing time for drawing. Meaning vertical sync is working hard to slow down the application.  :)
Toshiba Satellite - Intel i7 2.40ghz 8gb Intel HD Graphics 4000

(http://i41.tinypic.com/24n442a.png)
Title: Re: VideoStreamer - .NET Video Player
Post by: Lo-X on July 20, 2013, 08:41:18 am
does this use SFML 1.6 ?  :-\
Title: Re: VideoStreamer - .NET Video Player
Post by: netrick on July 20, 2013, 11:05:30 am
Why not cross-platform P/Invoke but M$-only C++/CLI?  :-\

@up
I think it's 2.0, .NET just uses ugly UpperCamelCase convention.
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on July 20, 2013, 12:50:57 pm
Quote from: Lo-X on July 20, 2013, 08:41:18 am
does this use SFML 1.6 ?  :-\

No, it uses SFML 2.0.

Quote from: netrick on July 20, 2013, 11:05:30 am
Why not cross-platform P/Invoke but M$-only C++/CLI?  :-\

@up
I think it's 2.0, .NET just uses ugly UpperCamelCase convention.

Unfortunately I could not find any good solid .NET FFMPEG wrappers that are not abandoned or outdated so I started working on it originally in C# with P/Invoke. But I was having trouble with all of the structures FFMPEG was using so I went with the C++/CLI design. In the future when I add audio support I will probably have to rewrite some parts of it so I may change over to a P/Invoke design now that I have a working design.

And yes .NET uses the UpperCammelCase convention :D Personally I like it better than lowerCamelCase  8)
Title: VideoStreamer - Update Version 2.0
Post by: zsbzsb on July 21, 2013, 11:42:59 pm
Update - Version 2.0

I completely rewrote VideoStreamer without the C++/CLI interface. VideoStreamer now is cleanly split into 3 separate projects.


I am not on linux using mono so I can not provide cross platform checks for compatibility, but with this rewrite it should be cross platform once you set your compilers up with new projects.

If anyone has any questions/comments/suggestions please post them :D And I am planning on finishing audio support soon.
Title: VideoStreamer - Update Version 3.0
Post by: zsbzsb on July 24, 2013, 10:22:42 pm
Update - Version 3.0

This is a big update. I entirely redid the way I handle video packets/frames and changed it so I can get audio and video data in the same data frame. This way it keeps the C#/.NET side cleaner and easier to call into the C++ FFMPEG wrapper. As before full seeking support is included for both the video and audio.

Changes

Todo

Also I have updated and cleaned up the first/main post.
Title: VideoStreamer - Update Version 3.1
Post by: zsbzsb on August 01, 2013, 08:07:00 pm
Update - Version 3.1

This is a small update, the biggest part was I added CMakeLists.txt to the VideoStreamer and CVideoStreamer projects.

Changes


Todo

Title: Re: VideoStreamer - .NET Video Player
Post by: erthon on August 14, 2013, 07:15:27 pm
After searching for a SFML based ffmpeg wrapper I found your code and liked it despite it being C# and M$ dependend in the higher levels. What I'm going to do is to wrap it up in C++ and use it as a base for my own project which involves playing a number - up to 16 - video streams in parallel so performance will be an issue.

After browsing through the implementation of the VideoStream and DataFrame I found some places where there's some space for optimizations.

First I'd suggest not to do the copying "by hand" and byte by byte in the constructors of DataFrame. A simply memcpy does the job perfectly in less time. Initializing the array would be done using memset().

Second I am thinking about reusing the DataFrame in GetNextDataFrame() by passing it as a parameter - GetNextDataFrame(DataFrame*) - so there's no need to do the allocate - deallocate - 2step all the time. One could use some sort of a ring buffer and avoid trashing the heap to much over time. Maybe making the AVPacket used in GetNextDataFrame a class member and reusing it would go in the same direction.

Keep up the good work. I'm going to post my changes after testing and cleaning them up.
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on August 14, 2013, 08:19:52 pm
Quote from: erthon on August 14, 2013, 07:15:27 pm
After searching for a SFML based ffmpeg wrapper I found your code and liked it despite it being C# and M$ dependend in the higher levels.

I would love to know what is MS specific at the higher levels. The initial release contained MS specific code with C++/CLI code, but that has since been removed.

Quote
What I'm going to do is to wrap it up in C++ and use it as a base for my own project which involves playing a number - up to 16 - video streams in parallel so performance will be an issue.

Sounds very cool, I would love to see your project when it is done.

Quote
First I'd suggest not to do the copying "by hand" and byte by byte in the constructors of DataFrame. A simply memcpy does the job perfectly in less time. Initializing the array would be done using memset().

Yes this should be changed, if I ever get back around to working on this code I will do that.

Quote
Second I am thinking about reusing the DataFrame in GetNextDataFrame() by passing it as a parameter - GetNextDataFrame(DataFrame*) - so there's no need to do the allocate - deallocate - 2step all the time. One could use some sort of a ring buffer and avoid trashing the heap to much over time. Maybe making the AVPacket used in GetNextDataFrame a class member and reusing it would go in the same direction.

Well the reason a new DataFrame is created on every GetNextDataFrame is because in the C# code the frames are put in a queue and then read back when they are needed.

Also if you plan on just using this for C++ you should take a look at Ceylo's sfeMovie Project(linked in first post) as part of my video player is implemented just in C#.
Title: Re: VideoStreamer - .NET Video Player
Post by: erthon on August 14, 2013, 10:59:01 pm
Quote from: zsbzsb on August 14, 2013, 08:19:52 pm
I would love to know what is MS specific at the higher levels. The initial release contained MS specific code with C++/CLI code, but that has since been removed.

It's C#/.NET. They invented it. Maybe just prejudice. And there's MONO. Doesn't matter.

Quote
Yes this should be changed, if I ever get back around to working on this code I will do that.

Done.

Quote
Well the reason a new DataFrame is created on every GetNextDataFrame is because in the C# code the frames are put in a queue and then read back when they are needed.

What I'm planning to do is to have a pool queue of preallocated DataFrames which are then used to decode, taken from this queue and pushed at the end of the processing queue from where they are consumed and pushed back at the end of the pool queue. So it's going to be just moving some pointers around, not creating a new DataFrame every time GetNextDataFrame gets called. Shouldn't be to hard to implement in C#, either. And using tbb::concurrent_bounded_queue the threading overhead is marginal.

Quote
Also if you plan on just using this for C++ you should take a look at Ceylo's sfeMovie Project(linked in first post) as part of my video player is implemented just in C#.

Exactly this was my first start - but it simply does much more than I need - gimme a frame, next frame please etc. - so I stumbled upon your two classes which provide exactly what I need. So Thank you for their existence.
Title: Re: VideoStreamer - .NET Video Player
Post by: rcurtis on February 07, 2014, 05:15:28 pm
Is there any documentation for building this?  I have all the C++ code happily building DLLs but the test program blows up when it tries to load a video -
An unhandled exception of type 'System.DllNotFoundException' occurred in VideoStreamerNET.dll

Additional information: Unable to load DLL 'CVideoStreamer': The specified module could not be found. (Exception from HRESULT: 0x8007007E)

 I have included the CVideoStreamer.dll in the project and set its properties to "Copy if newer" and can see it sitting right next to the exe.  Not sure what do to.
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on February 07, 2014, 06:03:10 pm
There is no documentation/tutorials at the moment, but I hope soon to have my own site for this and my other projects up soon.  :)

Quote from: rcurtis on February 07, 2014, 05:15:28 pm
I have included the CVideoStreamer.dll in the project and set its properties to "Copy if newer" and can see it sitting right next to the exe.  Not sure what do to.

Make sure you are also copying the FFMPEG dlls.

Quote
avcodec-55.dll
avformat-55.dll
avutil-52.dll
swresample-0.dll
swscale-2.dll
Title: Re: VideoStreamer - .NET Video Player
Post by: rcurtis on February 07, 2014, 09:56:16 pm
Thanks for the quick reply, yes it was the ffmpeg dlls that were missing.  I used a dll dependency viewer to figure it out it was missing.  It seems to play videos much smoother than the one I was working with for Java.  I have not tested streaming with alpha channel yet, but the only other thing missing is documentation/tutorial type stuff.  Great project!
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on February 07, 2014, 10:03:06 pm
Glad it is working out well for you :D

And be sure you let us all see your finished project when it is done  ;)
Title: Re: VideoStreamer - .NET Video Player
Post by: rcurtis on February 11, 2014, 03:08:32 pm
zsbzsb,

  I have been playing with your video player for a while and have found it gets very unhappy with video's that do not have an audio track.  I am playing FLV files, the only ones that do not crash all contain an audio track.  Reading through the code, it looks like it should be ok with not having audio, but in practice it seems to insist.  I can supply you with test videos if your interested in having a look.

Thanks,
Robert
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on February 11, 2014, 04:32:22 pm
Robert,

Videos should play fine even if there is no audio track. If you could provide me with some test videos that do not work I would be glad to check them out. Also some example code and some more information about the crash would be helpful  ;)
Title: Re: VideoStreamer - .NET Video Player
Post by: rcurtis on February 11, 2014, 09:53:54 pm
zsbzsb,

  Thanks for the patience, I got the audio thing settled using the long winded constructor to turn it off when I need to.  On to the next problem!  It seems to 'flicker' for a moment before playing the loaded video.  It looks like it has not yet decoded a frame and fills the buffer with solid black.  If you are interested in seeing the project I am working with, it is located on github at https://github.com/rcurtis/SFMLVideoPlayer.  Thanks again for taking the time to answer my questions, I do my best to try and solve these on my own.

Thanks,
Robert
Title: Re: VideoStreamer - .NET Video Player
Post by: rcurtis on February 11, 2014, 10:17:09 pm
Update:  If I start play back and then give the thread a brief sleep it does not flash black.  These feels like a hack tho.  What are your thoughts?
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on February 11, 2014, 10:49:34 pm
First, let me explain the "flicker to black issue". In the constructor the texture is initially set to black. When you call Play() the decoding thread is started and begins decoding video/audio frames from the file. The texture will only be updated once enough time has passed and there is a frame ready. However directly after calling Play() there is no frames available (this is why I need to add the option to preload frames  :P).

On to the constructor, I am working on adding some more overloads and also adding an option to set the initial texture color (which would allow you to set an initial color of transparent so that you won't see the black flicker).  :D

As for the sleep hack, it isn't a good solution since you are pausing the entire clear/draw/display loop, it is possible to still see the black texture, and it is possible to actually miss parts of the video. Let me finish these few changes and you should have no more issues.


On a side note, you can edit posts  ;)


Update

Ok I made the changes, I updated the constructors with more overloads (no more long winded thing) and I also added the option to set the initial color of the buffer [requires recompile].  ;D
Title: Re: VideoStreamer - .NET Video Player
Post by: rcurtis on February 13, 2014, 01:08:41 am
Thanks!  I pulled the changes and they worked perfectly.  I used your new documentation to build it as well!

Edit: link to build docs: http://zbrown.net/projects/videostreamer/tutorials/build.php

Robert
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on February 13, 2014, 01:48:16 am
Quote from: rcurtis on February 13, 2014, 01:08:41 am
Thanks!  I pulled the changes and they worked perfectly.  I used your new documentation to build it as well!

Edit: link to build docs: http://zbrown.net/projects/videostreamer/tutorials/build.php

Robert

Glad it worked out well, and about the build documentation... I wasn't quite ready to publicly release it, but oh well   8)

Here it is peeps, the official site for NetEXT and VideoStreamer [WIP] (http://zbrown.net/projects/)  :D
Title: Re: VideoStreamer - .NET Video Player
Post by: rcurtis on February 15, 2014, 02:23:07 am
Ah, terribly sorry.  If I had known I wouldn't have posted   :-[
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on February 15, 2014, 04:26:57 am
Quote from: rcurtis on February 15, 2014, 02:23:07 am
Ah, terribly sorry.  If I had known I wouldn't have posted   :-[

No problem, all is cool  ;D
Title: Re: VideoStreamer - .NET Video Player
Post by: lilbuh on March 16, 2014, 11:46:52 am
Hey there,

i cant get videostreamer to compile properly even with the tutorial...
each time i correct an issue another one comes up:

videostreamer:

- inttypes.h missing ( found that the r29 of https://code.google.com/p/msinttypes/ fixes that issue )

Cvideostreamer:

- lots of error LNK2026: module unsafe for SAFESEH image. ( using libs from http://ffmpeg.zeranoe.com/builds/win32/dev/ ) had to disable SAFESEH option in project properties

and now:

Erreur   1   error LNK2038: discordance détectée pour '_ITERATOR_DEBUG_LEVEL' : la valeur '0' ne correspond pas à la valeur '2' in HDataFrame.obj   \VideoStreamer.lib(DataFrame.obj)   CVideoStreamer
Erreur   2   error LNK2038: discordance détectée pour 'RuntimeLibrary' : la valeur 'MD_DynamicRelease' ne correspond pas à la valeur 'MDd_DynamicDebug' in HDataFrame.obj VideoStreamer.lib(DataFrame.obj)   CVideoStreamer
Erreur   3   error LNK2038: discordance détectée pour '_ITERATOR_DEBUG_LEVEL' : la valeur '0' ne correspond pas à la valeur '2' in HDataFrame.obj   \VideoStreamer.lib(VideoStream.obj)   CVideoStreamer
Erreur   4   error LNK2038: discordance détectée pour 'RuntimeLibrary' : la valeur 'MD_DynamicRelease' ne correspond pas à la valeur 'MDd_DynamicDebug' in HDataFrame.obj    \VideoStreamer.lib(VideoStream.obj)   CVideoStreamer
Avertissement   5   warning LNK4098: conflit entre la bibliothèque par défaut 'MSVCRT' et les autres bibliothèques ; utilisez /NODEFAULTLIB:library    \LINK   CVideoStreamer
Erreur   6   error LNK1319: 4 discordances détectées   \Debug\CVideoStreamer.dll   1   1   CVideoStreamer

using visualstudio 2012 and windows 8 64

help!
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on March 16, 2014, 03:13:24 pm
Problem has been solved via IRC  :D
Title: Re: VideoStreamer - .NET Video Player
Post by: lilbuh on March 16, 2014, 06:49:58 pm
for the other errors i just had to recompile the 2 projects videostreamer and Cvideostreamer in release mode
( one was set to debug and the other one to release )

and my last error was: do not forget to copy the ffmpeg dlls in the program's folder



Title: Re: VideoStreamer - .NET Video Player
Post by: FallenWyvern on May 29, 2014, 09:24:51 pm
How does one get audio to playback in this? There's nothing in the tutorial/sample about that.
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on May 30, 2014, 02:39:05 am
Quote from: FallenWyvern on May 29, 2014, 09:24:51 pm
How does one get audio to playback in this? There's nothing in the tutorial/sample about that.

I'm not sure what you mean, audio is played whenever the file that is loaded has an audio stream. If you want to just play back audio without video you should probably just use SFML's audio classes directly then.
Title: Re: VideoStreamer - .NET Video Player
Post by: FallenWyvern on May 30, 2014, 04:21:09 am
Quote from: zsbzsb on May 30, 2014, 02:39:05 am
Quote from: FallenWyvern on May 29, 2014, 09:24:51 pm
How does one get audio to playback in this? There's nothing in the tutorial/sample about that.

I'm not sure what you mean, audio is played whenever the file that is loaded has an audio stream. If you want to just play back audio without video you should probably just use SFML's audio classes directly then.

That's weird. No audio is playing on my side. I'll play more with it tomorrow. I'm using the Pacific Rim Trailer (.mp4 format, h.264 encoding) as my test. It's a beautiful little library, with excellent performance. I tried using SDLdotNet to play video using raw FFMPEG output but SDLdotNet has a bad memory leak when applying frames to a surface so I'm very happy with this.
Title: Re: VideoStreamer - .NET Video Player
Post by: FallenWyvern on May 30, 2014, 03:42:09 pm
Quote from: FallenWyvern on May 30, 2014, 04:21:09 am
Quote from: zsbzsb on May 30, 2014, 02:39:05 am
Quote from: FallenWyvern on May 29, 2014, 09:24:51 pm
How does one get audio to playback in this? There's nothing in the tutorial/sample about that.

I'm not sure what you mean, audio is played whenever the file that is loaded has an audio stream. If you want to just play back audio without video you should probably just use SFML's audio classes directly then.

That's weird. No audio is playing on my side. I'll play more with it tomorrow. I'm using the Pacific Rim Trailer (.mp4 format, h.264 encoding) as my test. It's a beautiful little library, with excellent performance. I tried using SDLdotNet to play video using raw FFMPEG output but SDLdotNet has a bad memory leak when applying frames to a surface so I'm very happy with this.

So I used user rcurtis's project that he linked in an earlier post to get the compiled version of your dll's (I'm lazy and wanted to see how the project worked) and so I'm using his "Video Player Test" which has a png background and a flv with an alpha channel (it's a deer on a quad).

I've added to this four copies (to see performance) of the Pacific Rim MP4 trailer. Using a StreamVideoPlayer (the same sort as the deer), no sound happens. When using a videostreamernet.videostream there is also no audio.

I see classes in videostreamernet for audio... but if it is supposed to work automatically I presume there's another class I should be using...
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on May 30, 2014, 04:27:52 pm
No need to do full quotes  ;)


Quote
So I used user rcurtis's project that he linked in an earlier post to get the compiled version of your dll's

Those are really old binaries that contain several bugs that have been fixed since then. I really recommend that you compile the current master branch and see if that fixes your issue. If that doesn't fix the issue send me a download link to a video that I can test with.

Quote
I see classes in videostreamernet for audio... but if it is supposed to work automatically I presume there's another class I should be using...

The audio classes are for internal use only, the only classes that you should be dealing with are the ones that are shown in the example.
Title: Re: VideoStreamer - .NET Video Player
Post by: FallenWyvern on May 30, 2014, 04:35:47 pm
Figured it would be something like that. I'll keep you informed! Thanks for your time.
Title: Re: VideoStreamer - .NET Video Player
Post by: FallenWyvern on June 02, 2014, 04:19:35 pm
So I believe I'm in need of someone pointing out what I'm overlooking.

1. Downloaded the CMake, SFML.NET, NetEXT, FFMPEG - Dev and VideoStreamer Source files.
2. Following the tutorial on your site, using CMake (set to VS2012) I was able to generate VideoStreamer without errors, however it is failing to actually build. There is no release folder.
3. Build CVideoStreamer in the same way. Also no release folder.

I can open these up in VS, I can build VideoStreamer (although there are some deprecation warnings) but CVideoStreamer gives me an error "Cannot Open DataFrame.h" and "VideoStream.h"

So I rebuild CVideoStreamer with CMake, which I can now point to a release folder. However I get the "Cannot open the include file, no such file or directory" errors again.

So what am I overlooking? I keep staring at the tutorial, but I'm not seeing what I missed.

-----------

Got it built. I had to delete the failed cmake version and recompile again. Was getting some SAFESEH errors but they didn't stop me from building.

Now my problem is the same. No audio on the video. I've uploaded a rar of the test video as well as my binaries I built. https://drive.google.com/file/d/0BxanGGXXqJaSN1FmbXg4RnRKNm8/edit?usp=sharing
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on June 02, 2014, 09:05:23 pm
Quote from: FallenWyvern on June 02, 2014, 04:19:35 pm
Now my problem is the same. No audio on the video. I've uploaded a rar of the test video as well as my binaries I built. https://drive.google.com/file/d/0BxanGGXXqJaSN1FmbXg4RnRKNm8/edit?usp=sharing

I tried your test video and it all works fine on my end. Any chance of you being able to hop on IRC (http://en.sfml-dev.org/forums/index.php?topic=2997.0) as I feel that would be the best way to help sort it out on your end.
Title: Re: VideoStreamer - .NET Video Player
Post by: FallenWyvern on June 24, 2014, 03:05:56 pm
Did your new Motion library ever take care of that memory leak when dealing with many videos being loaded/unloaded?
Title: Re: VideoStreamer - .NET Video Player
Post by: zsbzsb on July 01, 2014, 03:03:35 pm
Some of you already know this, but I am doing a rewrite of this project which includes many improvements (including SFML and SFML.NET support). The new project is called Motion and I will be writing up a proper post for it as soon as I finish a few other things. For now you can check it out with the following links.

https://github.com/zsbzsb/Motion
https://bitbucket.org/zsbzsb/motion

Quote from: FallenWyvern on June 24, 2014, 03:05:56 pm
Did your new Motion library ever take care of that memory leak when dealing with many videos being loaded/unloaded?

Yes, the current project VideoStreamer has a small leak when you create and destroy many instances of the VideoStream class. It has been fixed in the new Motion project.