Welcome, Guest. Please login or register. Did you miss your activation email?

Author Topic: [SOLVED] Second OpenGL context for loading stuff in seperate thread  (Read 8449 times)

0 Members and 3 Guests are viewing this topic.

Osbios

  • Newbie
  • *
  • Posts: 23
    • View Profile
I'm using SFML 1.6 on XUbuntu LTS.

I like to get a second OpenGL context to run that shares resources with the render context, so I can use it to load stuff in the background.
My tests so far only got x11 to lockup and I hat to use magic keys to reboot. :{

My first try was to use sf::Context and create it after the real windows was created.
Then start a new thread and acticated the context there. As soon as the shader is marked as ready I try to use it in the main thread. Thats when I get the lockup.

I also tried this using the global shared context instead of my self created one.

How is this done right?

Some short non working example code:

Version where I use new created context
bool readyVariable = false;

sf::Context *context2;

void theThread(void* data)
{
    context2->SetActive(true);
    //... do some glsl compiling
    //compiling here works like a charm... no errors all ok
    readyVariable = true;
}

int main()
{
    App.Create( sf::VideoMode(640, 480, 32), "SFML OpenGL", sf::Style::Resize | sf::Style::Close, sf::WindowSettings(24, 8, 0));
    context2 = new sf::Context();
    sf::Thread thread(&theThread);
    thread.Launch();
    while(true)
    {
        while(App.GetEvent(e))
        {
            processEvent(e);
        }
        if(!App.IsOpened()) break;
        //Render insignificant stuff
        if (readyVariable)
        {
                //try to use the glsl shader that is now compiled
                //This will lockup x11
        }
        App.Display();
    }
}


Version where I use the global context
bool readyVariable = false;

void theThread(void* data)
{
    sf::Context::GetGlobal().SetActive(true);
    //... do some glsl compiling
    //compiling here works like a charm... no errors all ok
    readyVariable = true;
}

int main()
{
    App.Create( sf::VideoMode(640, 480, 32), "SFML OpenGL", sf::Style::Resize | sf::Style::Close, sf::WindowSettings(24, 8, 0));
    sf::Thread thread(&theThread);
    thread.Launch();
    while(true)
    {
        while(App.GetEvent(e))
        {
            processEvent(e);
        }
        if(!App.IsOpened()) break;
        //Render insignificant stuff
        if (readyVariable)
        {
                //try to use the glsl shader that is now compiled
                //This will lockup x11
        }
        App.Display();
    }
}
« Last Edit: April 20, 2013, 01:58:39 pm by Osbios »

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32498
    • View Profile
    • SFML's website
    • Email
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #1 on: April 17, 2013, 01:24:12 pm »
I don't remember how it works in SFML 1.6. You should really use SFML 2, the context handling has been rewritten and is much better, and some bugs were fixed too.

I can only give you a few tips:
- you must call XInitThreads() (include <Xlib.h>) to enable multithreading at the X level.
- you should explicitely deactivate your context after creating it, I think 1.6 contexts are enabled by default.
- you can create your sf::Context instance locally to the thread function; no need to declare it global and allocate it dynamically in the main thread
- even if you do everything right, there might be problems with the Linux implementation in SFML 1.6 ;D
Laurent Gomila - SFML developer

The Terminator

  • Full Member
  • ***
  • Posts: 224
  • Windows and Mac C++ Developer
    • View Profile
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #2 on: April 17, 2013, 01:26:33 pm »
Well for one, I see a lot of new and not any deletes ;). If you're going to use pointers, use smart pointers so that memory leaks don't happen. Also, you should transition yourself to SFML 2.0 because it fixes huge amounts of bugs, including a particulary nasty bug with ATI GPUs.
Current Projects:
Technoport

Osbios

  • Newbie
  • *
  • Posts: 23
    • View Profile
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #3 on: April 17, 2013, 05:45:12 pm »
Switched to sfml2 from git master. Works perfectly now :D

@The Terminator: I don't care for a single "memory leak" in example code that demonstrates a crashkill for x11 ;)
First get it to work. Then make it pretty!

Osbios

  • Newbie
  • *
  • Posts: 23
    • View Profile
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #4 on: April 17, 2013, 07:05:12 pm »
Oh well...
turns out XInitThreads() just enables a gigantic mutex over all functions of xlib.
The "background" loading now blocks everything I want to do in the main thread...

Laurent do you have any experience with the xcb lib? This seems to be the async modern version to replace xlib. (They even have a API clone of xlib)
Is there any chance of getting real async loading and rendering of OpenGL objects? :(

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32498
    • View Profile
    • SFML's website
    • Email
Laurent Gomila - SFML developer

Osbios

  • Newbie
  • *
  • Posts: 23
    • View Profile
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #6 on: April 17, 2013, 10:00:19 pm »
I see, GLX is the devels egg.

So for x11 at last I stuck with waiting an unspecific time before using a OpenGL Object that I create/change in a load thread. Or else Mr. Overlord Mutex will block all xlib calls. Or in case of xlib/xcb I guess it "only" blocks all GLX calls.

I'm curious if drivers compile shaders in separate threads anyway. Have to do some testing...

My hope for a clean async load with instant event trigger after completion is dead. *riphope* :(

Thx for the help

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32498
    • View Profile
    • SFML's website
    • Email
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #7 on: April 17, 2013, 10:06:03 pm »
Can you provide a complete and minimal code that I can test? You should be able to achieve what you want.
Laurent Gomila - SFML developer

Osbios

  • Newbie
  • *
  • Posts: 23
    • View Profile
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #8 on: April 18, 2013, 06:10:46 pm »
Here is a small example. Every frame that needs more then 20ms will cout a message with the frame time.
I have a athlon 2 x4, and I guess you run something faster. Maybe you want to lower that value or use a more complex shader.

The calls that block my main thread are the creation of the context and the two glCompileShader.

I even tried to put a 1-second pause between each gl call in the loadThread to see if gl calls only block if previous gl data is not ready. But glCompileShader is still blocking the main thread.

The first frame lag I get is just from the context creation, then I wait 2 seconds in the loadThread and load the shaders. The first glCompileShader is causeing a ~120ms stall for me, the second one ~140ms!

Note that I don't actually use the shader. Maybe there are still drivers around that only compile on first use.

I use a 7870 Radeon with Driver Version 8.96.7-120312a-135598C-ATI (I have no clue to what catalyst Version this maps, basically just the default ATI/AMD Blob that comes with ubuntu 12.04 lte)

Code: [Select]
#include <GL/glew.h> //glew must be included befor gl.h (SFML/Window.hpp is including gl.h)

#include <SFML/Window.hpp>

#include <iostream>
#include <fstream>

#include <X11/Xlib.h>

#include <math.h>

using namespace std;

int resolutionX = 600;
int resolutionY = 600;

sf::Window *mainWindow;

void wait(float sec)
{
    sf::Clock clock;
    sf::Time time;
    while (true)
    {
        time = clock.getElapsedTime();
        if (time.asSeconds() >= sec) break;
    }
}

std::string fileToString(const std::string &fileName)
{
    std::ifstream fileStream(fileName.c_str(), std::ios::in | std::ios::binary | std::ios::ate);
    if (fileStream.is_open())
    {
        std::ifstream::pos_type fileSize = fileStream.tellg();
        fileStream.seekg(0, std::ios::beg);
        char data[static_cast<int>(fileSize)];
        fileStream.read(&data[0], fileSize);
        return std::string(&data[0], fileSize);
    }
    else
    {
        return "";
    }
}

void processEvent(sf::Event e)
{
    switch(e.type)
{
    case sf::Event::Closed:
{
mainWindow->close();
break;
}
case sf::Event::KeyPressed:
{
switch(e.key.code)
{
case sf::Keyboard::Escape:
{
mainWindow->close();
break;
}
}
break;
}
case sf::Event::Resized:
{
resolutionX = e.size.width;
resolutionY = e.size.height;
glViewport(0, 0, resolutionX, resolutionY);
break;
}
}
}

bool shaderIsReady = false;

GLuint shaderProgramId = 0;
GLuint vertexShaderId = 0;
GLuint fragmentShaderId = 0;

void threadLoadShader()
{
    sf::Context threadContext3;  //the context creation blocks the main thread... but thats ok because it only happens once at start
    wait(2);

    shaderProgramId = glCreateProgram();
    vertexShaderId = glCreateShader(GL_VERTEX_SHADER);
    const char *pCVertexString = fileToString("vertexShader.txt").c_str();
    glShaderSource(vertexShaderId, 1, reinterpret_cast<const GLchar**>(&pCVertexString), NULL);
    glCompileShader(vertexShaderId);

    fragmentShaderId = glCreateShader(GL_FRAGMENT_SHADER);
    const char *pCFragmentString = fileToString("fragmentShader.txt").c_str();
    glShaderSource(fragmentShaderId, 1, reinterpret_cast<const GLchar**>(&pCFragmentString), NULL);
    glCompileShader(fragmentShaderId);

    glAttachShader(shaderProgramId, vertexShaderId);
    glAttachShader(shaderProgramId, fragmentShaderId);
    glLinkProgram(shaderProgramId);

    shaderIsReady = true;
}

int main()
{
    XInitThreads();
    mainWindow = new sf::Window(sf::VideoMode(resolutionX, resolutionY), "SFML OpenGL async glsl load Test");

    GLenum err = glewInit();
if (GLEW_OK != err)
{
cout << "GLEW Error:" << glewGetErrorString(err) << endl;
return -1;
}

    sf::Thread thread(&threadLoadShader);
thread.launch();

    sf::Event event;

    sf::Clock clock;
    float runtimeSeconds = 0;
    float runtimeSecondsOld = 0;
    float frameSeconds = 0;
    while (true)
    {
        runtimeSecondsOld = runtimeSeconds;
        runtimeSeconds = clock.getElapsedTime().asSeconds();
        frameSeconds = runtimeSeconds - runtimeSecondsOld;
        if (frameSeconds > 0.02f)
            cout << "framelag: " << frameSeconds*1000 << "ms" << endl;

        while (mainWindow->pollEvent(event))
        {
            processEvent(event);
        }
        if(!mainWindow->isOpen()) break;

        glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
        glClear(GL_COLOR_BUFFER_BIT);

        glMatrixMode(GL_PROJECTION);
        glLoadIdentity();
        glOrtho(-1.0f, 1.0f, -1.0f, 1.0f, -1.0f, 1.0f);
        glMatrixMode(GL_MODELVIEW);
        glLoadIdentity();
        float rot = runtimeSeconds * 40.0f;
        glRotatef(rot, 0.f, 0.f, 1.f);

        glColor3f(0.8f, 0.8f, 0.8f);
        glLineWidth(4);
        glBegin(GL_LINES);
        for(float f = -2; f < 2; f += 0.1f)
        {
            glVertex2f( f, -2.0f);
            glVertex2f( f,  2.0f);
        }
        for(float f = -2; f < 2; f += 0.1f)
        {
            glVertex2f( -2.0f, f);
            glVertex2f(  2.0f, f);
        }
        glEnd();
        glLineWidth(1);

        mainWindow->display();
    }
    thread.terminate();
    return EXIT_SUCCESS;
}

Vertex Shader
Code: [Select]
#version 330

layout(location = 0) in vec2 position;
void main()
{
gl_Position = vec4(position.x, position.y, 0.0, 1.0);
}

Fragment Shader
Code: [Select]
#version 330

out vec4 outputColor;
void main()
{
   outputColor = vec4(1.0f, 1.0f, 1.0f, 1.0f);
}

Or as a complete cmake project:
https://mega.co.nz/#!cVtUUbII!X2MRaSE3SJxNOWz7cDfBWD1SMGkMwFfdTf2hUzCgv5k

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32498
    • View Profile
    • SFML's website
    • Email
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #9 on: April 18, 2013, 10:43:40 pm »
I experience the same problem as you, and I'm not sure I can explain it. It may really be caused by the Xlib or by the graphics driver, so you should try the same program on Windows or OS X.
Laurent Gomila - SFML developer

Osbios

  • Newbie
  • *
  • Posts: 23
    • View Profile
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #10 on: April 18, 2013, 11:38:47 pm »
Same phenomena with win7.
Context creation stalls and for each glCompileShader the main thread is also stalled for ~192ms.

Edit: If I put pauses betwin the gl calls in win7, glCompileShader dosn't cause stalls anymore
« Last Edit: April 19, 2013, 12:01:06 am by Osbios »

Osbios

  • Newbie
  • *
  • Posts: 23
    • View Profile
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #11 on: April 19, 2013, 05:37:13 pm »
OK, got way better results with a more up to date driver. Now using the fglrx-experimental-12 package from Ubuntu 12.04. (That would be Catalyst 12.11 Beta)

I wonder how much I have to thank Valve for this improvements? ;)

But still there is a rest of lag that I can't get rid of. I get frame lags for 10~30ms... With a lot of glFinish in the loadThread I can get the max. lag time down. But still chances for lags over 16ms. And this are noticeable on a 60Hz screen. I don't even render anything right now...

for now I blame the driver and run some more tests.

Osbios

  • Newbie
  • *
  • Posts: 23
    • View Profile
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #12 on: April 19, 2013, 11:24:33 pm »
And I found more:

Is there something like XInitThreads on windows that I have to run before using gl calls from threads?

Because, like in x11 without XInitThreads, I get random errors. My shader often won't compile. It just uses fixed function if I call glUseProgram on it. I didn't notice first because I removed error checks to simplify my tests.

When it happens:

Code: [Select]
GLint compileStatus;
gGetShaderiv(fragmentShaderId, GL_COMPILE_STATUS, &compileStatus)

will set compileStatus to GL_FALSE

trying glGetShaderInfoLog or glGetProgramInfoLog in the thread then will segfaults the program.


Using win7 with Catalyst 13.3 Beta3.

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32498
    • View Profile
    • SFML's website
    • Email
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #13 on: April 20, 2013, 08:50:40 am »
Quote
Is there something like XInitThreads on windows that I have to run before using gl calls from threads?
Nope.
Laurent Gomila - SFML developer

Osbios

  • Newbie
  • *
  • Posts: 23
    • View Profile
Re: Second OpenGL context for loading stuff in seperate thread
« Reply #14 on: April 20, 2013, 01:57:43 pm »
Finaly!

Now it does what I want!

The Windows Versions now works OK after I found the part where I shoot my self in the food.
And without any frame lags what so ever! :D

On Linux I still have the small frame lags. But I'm 99.5% sure it is the driver.

Thx again for the help!