Welcome, Guest. Please login or register. Did you miss your activation email?

Author Topic: Video Integration using SFML 2.3.2 and FFMPEG 3.1.1  (Read 6016 times)

0 Members and 1 Guest are viewing this topic.

sglee

  • Newbie
  • *
  • Posts: 3
    • View Profile
Video Integration using SFML 2.3.2 and FFMPEG 3.1.1
« on: July 22, 2016, 12:02:43 pm »
Hi there,

I have managed to dig up an old article about video integration using SFML and FFMPEG.
Unfortunately it has been removed from the SFML wiki.
I have managed to update it to Latest SFML 2.3.2 and FFMPEG 3 code. Here are the source codes:

//----- Video.h File -----------------------------------------------------------------------------------------------------------
#ifndef VIDEO_H
#define VIDEO_H

#include <SFML/Graphics.hpp>
#include <iostream>
#include <string>

using namespace std;

typedef unsigned int nuint;

#define __STDC_CONSTANT_MACROS
extern "C"
{
    #include <libavformat/avformat.h>
    #include <libavcodec/avcodec.h>
    #include <libswscale/swscale.h>
}

class Video
{

private:
//Variables
    sf::Texture       m_Texture;
    bool            m_bVideoLoaded;
    bool            m_bImageNeedsUpdate;
    AVFormatContext *m_pFormatCtx;
    sf::Uint8       m_iVideoStream;
    sf::Uint32      m_iFrameSize;
    AVCodecContext  *m_pCodecCtx;
    AVFrame         *m_pFrame;
    AVFrame         *m_pFrameRGB;
    sf::Uint8       *m_pBuffer;
    AVPacket        m_Packet;
    AVCodec         *m_pCodec;
    SwsContext      *img_convert_ctx;
    string          m_sFilename;
    float           m_fSecondsPerFrame;
    float           m_fTimePassedSinceLastFrameUpdate;

//Functions
    void UpdateImage();
    void LoadNextFrame();
public:
    Video();
    explicit Video(const string& filename = "");
    bool LoadFromFile(const string& filename);
    void Update(float time);

    sf::Vector2i Size() const { return sf::Vector2i(GetWidth(), GetHeight()); }

    nuint GetWidth() const { return m_pCodecCtx->width; }
    nuint GetHeight() const { return m_pCodecCtx->height; }

    float GetFrameRate() const { return 1/m_fSecondsPerFrame; }

    operator const sf::Texture&() const {return m_Texture;}

    ~Video();

    sf::Color GetPixel(nuint x, nuint y) const;

    void CloseVideo();

private:
    Video(const Video& rhs);
    Video& operator=(const Video& rhs);
};

#endif // VIDEO_H

 

//----- END OF VIDEO.H FILE  ----------------------------------------------------------------------------------------------------

//----- START OF VIDEO.CPP FILE -----------------------------------------------------------------------------------------------
#include "video.h"

Video::Video() :    m_Texture(),
                    m_bVideoLoaded(false),
                    m_bImageNeedsUpdate(false),
                    m_pFormatCtx(NULL),
                    m_iVideoStream(0),
                    m_iFrameSize(0),
                    m_pCodecCtx(NULL),
                    m_pFrame(NULL),
                    m_pFrameRGB(NULL),
                    m_pBuffer(NULL),
                    m_Packet(),
                    m_pCodec(NULL),
                    img_convert_ctx(NULL),
                    m_sFilename(""),
                    m_fSecondsPerFrame(0),
                    m_fTimePassedSinceLastFrameUpdate(0)
{
    av_register_all();
}


Video::Video(const string& filename) :  m_Texture(),
                                        m_bVideoLoaded(false),
                                        m_bImageNeedsUpdate(false),
                                        m_pFormatCtx(NULL),
                                        m_iVideoStream(0),
                                        m_iFrameSize(0),
                                        m_pCodecCtx(NULL),
                                        m_pFrame(NULL),
                                        m_pFrameRGB(NULL),
                                        m_pBuffer(NULL),
                                        m_Packet(),
                                        m_pCodec(NULL),
                                        img_convert_ctx(NULL),
                                        m_sFilename(""),
                                        m_fSecondsPerFrame(0),
                                        m_fTimePassedSinceLastFrameUpdate(0)
{
    av_register_all();

    if (!filename.empty())
    {
        if (!LoadFromFile(filename))
        {
            cout << "Could not load video!" << endl; //Probably should throw exception.
        }
    }
}

bool Video::LoadFromFile(const string& filename)
{
    CloseVideo();
    m_sFilename = filename;
    const char * const file = m_sFilename.c_str();

    if(avformat_open_input(&m_pFormatCtx, file, NULL,NULL) != 0)
    {
        cout << "Unexisting file!" << endl;;
        return false;
    }

    if(avformat_find_stream_info(m_pFormatCtx,NULL) < 0)
    {
        cout << "Couldn't find stream information!" << endl;
        return false;
    }

    av_dump_format(m_pFormatCtx, 0, file, 0);

    m_iVideoStream = -1;
    for(nuint i = 0; i < m_pFormatCtx->nb_streams; i++)
    {
        if(m_pFormatCtx->streams[i]->codec->coder_type == AVMEDIA_TYPE_VIDEO)
        {
            m_iVideoStream = i;
            break;
        }
    }

    if(m_iVideoStream == -1)
        return false;

    m_pCodecCtx = m_pFormatCtx->streams[m_iVideoStream]->codec;

    m_pCodec = avcodec_find_decoder(m_pCodecCtx->codec_id);

    AVRational r = m_pFormatCtx->streams[m_iVideoStream]->avg_frame_rate;
    AVRational r2 = m_pFormatCtx->streams[m_iVideoStream]->r_frame_rate;
    if ((!r.num || !r.den) &&
        (!r2.num || !r2.den))
    {
        std::cerr << "Video_video::Initialize() - unable to get the video frame rate. Using 25 fps." << std::endl;
        m_fSecondsPerFrame = 1.f / 25;
    }
    else
    {
        if (r.num && r.den)
            m_fSecondsPerFrame = 1.f/((float)r.num / r.den);
        else
            m_fSecondsPerFrame = 1.f/((float)r2.num / r2.den);
    }

    if(m_pCodec == NULL)
    {
        cout << "Unsupported codec!" << endl;
        return false;
    }

    if(avcodec_open2(m_pCodecCtx, m_pCodec,NULL) < 0)
    {
        cout << "Could not open Codec Context" << endl;
        return false;
    }

    m_iFrameSize = m_pCodecCtx->width * m_pCodecCtx->height * 3;


    m_pFrame = av_frame_alloc();

    m_pFrameRGB = av_frame_alloc();

    if(m_pFrameRGB == NULL || m_pFrame == NULL)
    {
        cout << "Error allocating frame" << endl;
        return false;
    }

    int numBytes = avpicture_get_size(AV_PIX_FMT_RGBA, m_pCodecCtx->width, m_pCodecCtx->height);

    m_pBuffer = (sf::Uint8 *)av_malloc(numBytes * sizeof(sf::Uint8));

    avpicture_fill((AVPicture *)m_pFrameRGB, m_pBuffer, AV_PIX_FMT_RGBA,
                   m_pCodecCtx->width, m_pCodecCtx->height);

    img_convert_ctx = sws_getContext(m_pCodecCtx->width, m_pCodecCtx->height,
                                     m_pCodecCtx->pix_fmt,
                                     m_pCodecCtx->width, m_pCodecCtx->height,
                                     AV_PIX_FMT_RGBA, SWS_FAST_BILINEAR,
                                     NULL, NULL, NULL);
    m_bVideoLoaded = true;
    m_bImageNeedsUpdate = true;

    m_Texture.create(GetWidth(), GetHeight());

    Update(10000);

    return true;
} //Load From File

void Video::Update(float time)
{

    if (m_bVideoLoaded)
    {
        m_fTimePassedSinceLastFrameUpdate += time;
        UpdateImage();

        if (m_fTimePassedSinceLastFrameUpdate > m_fSecondsPerFrame)
        {
            m_fTimePassedSinceLastFrameUpdate = 0;
            LoadNextFrame();
        }
    }
}

void Video::LoadNextFrame()
{
    do
    {
        av_free_packet(&m_Packet);
        int result = av_read_frame(m_pFormatCtx, &m_Packet);
        if (result < 0)
        {
            CloseVideo();
            LoadFromFile(m_sFilename);
            m_fTimePassedSinceLastFrameUpdate = 0;
            return;
        }
    } while (m_Packet.stream_index != m_iVideoStream);

    int frameFinished = 0;
    avcodec_decode_video2(m_pCodecCtx, m_pFrame, &frameFinished, &m_Packet);
    if (frameFinished)
    {
        sws_scale(img_convert_ctx, m_pFrame->data, m_pFrame->linesize,
                  0, m_pCodecCtx->height, m_pFrameRGB->data, m_pFrameRGB->linesize);
        m_bImageNeedsUpdate = true;
    }

}

sf::Color Video::GetPixel(nuint x, nuint y) const
{
    nuint i = 3*(y*GetWidth()+x);
    sf::Uint8 red   = m_pFrameRGB->data[0][i];
    sf::Uint8 green = m_pFrameRGB->data[0][i+1];
    sf::Uint8 blue  = m_pFrameRGB->data[0][i+2];
    return sf::Color(red,green,blue,255);
}

void Video::UpdateImage()
{
    if (m_bImageNeedsUpdate)
    {
        m_Texture.update(m_pBuffer);
        m_bImageNeedsUpdate = false;
    }
}

void Video::CloseVideo()
{
    if (m_bVideoLoaded)
    {
        av_free_packet(&m_Packet);
        av_free(m_pBuffer);
        av_free(m_pFrameRGB);
        av_free(m_pFrame);
        avcodec_close(m_pCodecCtx);
        avformat_close_input(&m_pFormatCtx);
        sws_freeContext(img_convert_ctx);

        m_bVideoLoaded = false;
        m_bImageNeedsUpdate = false;
    }
}

Video::~Video()
{
    CloseVideo();
}

 

//----- END OF VIDEO.CPP FILE-------------------------------------------------------------------------------------------------

//----- MAIN.CPP FILE ------------------------------------------------------------------------------------------------------------
int main(int argc, char *argv[])
{
    Video viddy("C:/Qt/sample.avi");
    sf::Sprite sprite(viddy);

    int w = viddy.GetWidth();
    int h = viddy.GetHeight();

    sf::RenderWindow App(sf::VideoMode(w, h), "Video with SFML");

    sf::Clock clock;

    bool Running = true;
    while (Running)
    {
        float time = clock.getElapsedTime().asSeconds();
        clock.restart();

        viddy.Update(time);

        App.clear();

        App.draw(sprite);

        App.display();

        sf::Event Event;
        while(App.pollEvent(Event))
        {
            if (Event.type == sf::Event::Closed)
                Running = false;

            if ((Event.type == sf::Event::KeyPressed) && (Event.key.code == sf::Keyboard::Escape))
                Running = false;

        }

    }

    App.close();
}
 

//----- END OF MAIN.CPP FILE


I will continue to work on it maybe i can improve it better and make it faster. Check this thread for any update in the future.

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: Video Integration using SFML 2.3.2 and FFMPEG 3.1.1
« Reply #1 on: July 22, 2016, 12:08:28 pm »
Why don't you put it back on the wiki?
Laurent Gomila - SFML developer

eXpl0it3r

  • SFML Team
  • Hero Member
  • *****
  • Posts: 10801
    • View Profile
    • development blog
    • Email
AW: Video Integration using SFML 2.3.2 and FFMPEG 3.1.1
« Reply #2 on: July 22, 2016, 08:30:07 pm »
Also feel free to check out sfeMovie which is does something similar (I think): http://sfemovie.yalir.org/latest/
Official FAQ: https://www.sfml-dev.org/faq.php
Official Discord Server: https://discord.gg/nr4X7Fh
——————————————————————
Dev Blog: https://duerrenberger.dev/blog/

sglee

  • Newbie
  • *
  • Posts: 3
    • View Profile
Re: Video Integration using SFML 2.3.2 and FFMPEG 3.1.1
« Reply #3 on: July 23, 2016, 08:57:09 am »
Why don't you put it back on the wiki?

I will make sure i put it back to the wiki, i wanted to see if people are interested of it before putting it back there.

sglee

  • Newbie
  • *
  • Posts: 3
    • View Profile
Re: AW: Video Integration using SFML 2.3.2 and FFMPEG 3.1.1
« Reply #4 on: July 23, 2016, 08:58:38 am »
Also feel free to check out sfeMovie which is does something similar (I think): http://sfemovie.yalir.org/latest/

sfeMovie is great, but it has some restriction from playing MP3 files.
So i thought the barebone FFMPEG and SFML will be great to those who wish to explore the wild West! ;D

korczurekk

  • Full Member
  • ***
  • Posts: 150
    • View Profile
    • Email
Re: Video Integration using SFML 2.3.2 and FFMPEG 3.1.1
« Reply #5 on: July 25, 2016, 05:10:01 pm »
I can't check it right now, but getframerate method seems to always return 0, as 1 / x is an integer. Can anybody confirm that?

Hapax

  • Hero Member
  • *****
  • Posts: 3346
  • My number of posts is shown in hexadecimal.
    • View Profile
    • Links
Re: Video Integration using SFML 2.3.2 and FFMPEG 3.1.1
« Reply #6 on: July 25, 2016, 09:17:50 pm »
1 / x is an integer. Can anybody confirm that?
x is a float; as long as one of the operands is not an integer, it will not perform integer division.

The one thing I noticed about this code is that I think it's rude to use using namespace std in the header  :(
Selba Ward -SFML drawables
Cheese Map -Drawable Layered Tile Map
Kairos -Timing Library
Grambol
 *Hapaxia Links*

korczurekk

  • Full Member
  • ***
  • Posts: 150
    • View Profile
    • Email
Re: Video Integration using SFML 2.3.2 and FFMPEG 3.1.1
« Reply #7 on: July 27, 2016, 12:28:00 pm »
1 / x is an integer. Can anybody confirm that?
x is a float; as long as one of the operands is not an integer, it will not perform integer division.

The one thing I noticed about this code is that I think it's rude to use using namespace std in the header  :(
Yep, but after all it works and short script (or author?) can fix it.
I wasn't sure how c++ handles int/float division, so... thanks. :D