SFML community forums

General => SFML projects => Topic started by: lolz123 on March 17, 2014, 06:29:56 pm

Title: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on March 17, 2014, 06:29:56 pm
Hello all,

I originally wrote a 3D game engine with SDL + OpenGL + Bullet + Ogg + Vorbis + OpenAL + LUA. This original version had a lot of features, but was not very user-friendly. So I decided to re-write it, this time using SFML  ;D

The new version is especially different in that all game logic can be executed in parallel: there is a thread pool (using C++11 threading) that runs game entities all at once. It double-buffers the game logic and has special buffer-shifting smart pointers to make working in such an engine relatively simple.

The aim is to use only the most modern C++ as well as only the most modern OpenGL. The engine never uses raw new/delete or primitive arrays and only uses OpenGL 4+ features.

Here is an example of voxel terrain (it can do overhangs) with massive amounts of grass rendering.

(http://i1218.photobucket.com/albums/dd401/222464/postEffects.png)

Here is the engine's real-time global illumination in action:

(http://i1218.photobucket.com/albums/dd401/222464/GI.png)

Finally, here is a link to the bitbucket repository:

https://bitbucket.org/CireNeikual/deferred3d-parallel-game-engine/ (https://bitbucket.org/CireNeikual/deferred3d-parallel-game-engine/)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: eXpl0it3r on March 17, 2014, 07:07:29 pm
Looks awesome! :)

Will be looking forward to see more of it.
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Nexus on March 17, 2014, 07:49:59 pm
Looks really nice! Great to see bigger projects that build upon SFML for 3D!

Also, it seems like you have spent a lot of effort on the code! It looks mostly good, here are some random suggestions:
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: G. on March 17, 2014, 07:52:45 pm
Do you have a devblog?
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on March 17, 2014, 08:18:07 pm
@ Nexus: Thank you very much for the tips! I was hoping that you would come by and help me straighten it out  ;D

@ G.: Unfortunately not. I should probably make one  :P
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Grimshaw on March 17, 2014, 09:03:22 pm
Man you just keep bringing awesome stuff to the community! Probably one of the best contributors in the whole community.

Thanks!
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Nexus on March 17, 2014, 09:34:39 pm
Ah, I first didn't realize you were the creator of Let There Be Light/GLLight2D and all the AI-related simulations/games (NEAT Visualizer, NNBrain, Alife, Hero Must Die). I agree with Grimshaw, your contributions (http://en.sfml-dev.org/forums/index.php?action=profile;area=showposts;sa=topics;u=2614) are indeed amazing! :)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: eXpl0it3r on March 17, 2014, 10:10:18 pm
  • Inherit privately from Uncopyable.
  • Uncopyable should not be polymorphic; it introduces a VTable in every class that inherits it, although you'll hardly ever delete the classes through that base class, because of the last point.
Is that even needed with C++11? Shouldn't one just use = delete instead or is that more of a design choice?
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: DJuego on March 17, 2014, 11:55:20 pm
Man you just keep bringing awesome stuff to the community! Probably one of the best contributors in the whole community.

Ah, I first didn't realize you were the creator of Let There Be Light/GLLight2D and all the AI-related simulations/games (NEAT Visualizer, NNBrain, Alife, Hero Must Die). I agree with Grimshaw, your contributions (http://en.sfml-dev.org/forums/index.php?action=profile;area=showposts;sa=topics;u=2614) are indeed amazing! :)

True!. A fantastic contributor with -inspired contributions very inspiring-.  A pioneer in exotic domains for SFML :-*

I only am sorry (selfishly) that he lose interest/abandon so quickly his projects. :-X. Let's hope that he decides to found a project with mileage. And so, he will enter in the club of Nexus (Thor), SFGUI (Tank+binary), Ceylo(sfeMovie)... or Laurent (SFML)! :P

DJuego
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Lee R on March 18, 2014, 08:35:21 pm
Looks like you have a potential deadlock in your thread pool implementation.

mutex mut_worker;
mutex mut_pool;

thread1:
ThreadPool::destroy() {
    std::lock_guard<std::mutex> lock(mut_pool);
    // mut_pool is now locked.

thread2:
ThreadPool::WorkerThread::run(...) {
    std::unique_lock<std::mutex> lock(mut_worker);
    _conditionVariable.wait(lock, ...);
    // mut_worker is now locked.
    _pPool->onWorkerAvailable(...); // takes lock on mut_pool.
    // thread2 is now blocked until thread1 unlocks mut_pool.

thread1:
    std::lock_guard<std::mutex> lock(mut_worker);
    // thread1 is now blocked until thread2 unlocks mut_worker.
 

At this point, there is a deadlock; neither thread can make progress.

I've only recently started seriously looking into multithreading so it's quite possible that I've misunderstood something, though.
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on April 03, 2014, 02:18:10 am
Cascaded shadow maps!

(http://i1218.photobucket.com/albums/dd401/222464/directionalShadows.png)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Grimshaw on April 03, 2014, 01:59:20 pm
Hey lolz, can you please elaborate on what kind of GI you had? The technique you used, efficiency?

Also, can you elaborate a bit more on your process of rendering that grass? :)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on April 03, 2014, 05:09:18 pm
Sure!

I sort of made up my own global illumination technique (I like to think I did anyways  ;D ), it is based on path tracing.
I compile the scene into a KD-tree, and then convert that tree into a OpenGL texture buffer object (TBO) so I can read it in a shader. I then path trace at a low resolution, and then blur the results with a special blur that keeps edge details intact.

Performance is not bad, I get 60fps on that simple scene, but I haven't gotten it to work at all (like, nothing shows up) on large scenes for some reason. I think my KD-tree is broken somehow  :P.

The grass isn't too special really  ;) basically I just generate it using the voxel data, and then group the grass into chunks and compile it into a VBO. The vertex shader animates it to sway a bit.
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Grimshaw on April 03, 2014, 06:04:50 pm
http://3d.benjamin-thaut.de/?p=16

Have you looked at that before? I am really leaning towards implementing that GI system in my engine. Fully working source code included, there are papers for both the original technique and the crytek improvements. The results seem to be really pleasant and are more than proven since CryEngine has GI since version 2.

What do you think? Does it seem viable? :)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on April 03, 2014, 06:52:11 pm
Yeah, I have read papers on their technique before. From what I gather, it is basically like doing Minecraft lighting (a flooding algorithm).

Cryengine didn't *really* have global illumination until 3 AFAIK, they only had some screen space effects.
The technique seems good, not as accurate as the raytraced version though. Cryengine 3 in its entirety, however, is a total mess from what I have seen  :-X Unreal Engine 4 blows it out of the water in my opinion, especially code quality-wise. Cryengine 3 was clearly not made with indie development in mind.

Unreal Engine 4 uses voxel cone tracing, it is more accurate than Light Propagation Volumes but less accurate than full-blown polygonal path tracing  ;)

I am sure my technique can be improved upon A LOT, since I know my dream game engine Brigade 3 runs in real time, and for low-res GI you only need a fraction of the capabilities it has.

Here is a link to Brigade 3, the ultimate graphics engine IMO: http://icelaglace.com/portfolio-type/brigade-3-0/ (http://icelaglace.com/portfolio-type/brigade-3-0/) (and no, it is not a scam, they released a demo for Brigade 2 (I tried it), which also looks fantastic already)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Grimshaw on April 03, 2014, 07:52:08 pm
That Brigade 3 looks so amazing :D But how do you get rid of the noise you see when moving? It blows the entire experience imo :X
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: MadMartin on April 03, 2014, 11:12:40 pm
The noise is a typical problem of path tracers, since they do random direction sampling for the GI (called Monte Carlo sampling). More samples would reduce the noise, but increase computation effort.
But in the video I think it's the codec that gets f**cked up by the noise. In realtime it is certainly less visible (my own [smaller ;) ] pathtracer has the same problems: the output of the program looks great, videos are more difficult).
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Lolilolight on April 05, 2014, 10:42:19 am
It looks like to be very amazing, I wonder how you did for the lighting.
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Peteck on April 05, 2014, 12:43:10 pm
It looks like to be very amazing, I wonder how you did for the lighting.

Well thats the biggest benefit of deferred shading. You can read more about here:

http://en.wikipedia.org/wiki/Deferred_shading (http://en.wikipedia.org/wiki/Deferred_shading)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: StormWingDelta on April 05, 2014, 05:11:36 pm
hmm now if only there was a C# version of this. :)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on April 11, 2014, 06:39:07 pm
Hello again!

I have been working on d3d a lot, mostly adding new features, but also improving some existing functionality.

Here is a new screenshot:

(http://i1218.photobucket.com/albums/dd401/222464/houseInFields.png)

Also, virtual creatures make a return, and I integrated SFGUI (great lib so far  ;D)!

(http://i1218.photobucket.com/albums/dd401/222464/vcplussfgui.png)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Grimshaw on April 11, 2014, 06:48:14 pm
You really are talented with graphics :) I'd love to discuss some things about rendering with you later ^^
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on April 11, 2014, 07:19:49 pm
Thanks! If you have any questions, I will gladly answer them!
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on April 14, 2014, 02:42:44 am
Another update!

I added 3 new post-processing effects, SSAO, Depth of Field, and Light Scattering.

Here is a picture! More effects to come!

(http://i1218.photobucket.com/albums/dd401/222464/postEffects.png)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: zmertens on April 14, 2014, 05:03:36 am
Those pictures look amazing!
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Grimshaw on April 14, 2014, 03:33:22 pm
Damn, this is looking better and better! I definitely have to learn a couple of things from you!!
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on April 17, 2014, 05:38:41 am
Physically based shading with screen space reflections! Parallax occlusion mapping!

(http://i1218.photobucket.com/albums/dd401/222464/ssr.png)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Grimshaw on April 17, 2014, 11:13:06 am
Oh oh oh! I am predicting I will lose many hours studying your code when I am about to implement these things hehe :)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Lolilolight on April 22, 2014, 07:30:48 pm
Quote
Oh oh oh! I am predicting I will lose many hours studying your code when I am about to implement these things hehe

Me too.
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on July 18, 2014, 11:30:36 pm
Hello!

It's been a while since I updated this, but d3d is still under active development. I have added .3dw (3D World Studio) level loading, real-time global illumination, OpenCL, and added CMake support.

Check the original post for the repository link! It has moved!
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: IngloriousTom on July 25, 2014, 02:57:47 pm
That looks gorgeous. I would love to try it for a little project, but your wiki seems to be empty and I don't see examples, and I'm afraid to know too little about OpenGL to try it alone.
Maybe could you add examples from your screenshots ? They looks really nice and I would definitely love to alter them. Adding documentation in your wiki should help too.
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: FRex on July 25, 2014, 04:10:07 pm
Isn't the name choice a bit unfortunate and hard to google/generic? D3D is what (most gamedev) people call Direct3D from DirectX. Even the headers for 9 were d3d9.h
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on July 25, 2014, 07:47:39 pm
@IngloriousTom: Examples and tutorials will come! If you cannot wait though, I can walk you through how to use it on Skype if you want to (PM me!)

@FRex: Yeah I know, I was struggling choosing a name  ::) The engine is unique in how it defers updates and then does them all in parallel, and also has a heavy focus on deferred shading, so I called it "Deferred 3D". Perhaps I will rename it at some point  ;)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: paupav on July 26, 2014, 05:49:02 pm
Wow! Awesome job.
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: Hapax on July 27, 2014, 01:55:35 am
Wow! Your renderer looks like it can be quite beautiful! Excellent work  :)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: lolz123 on August 21, 2014, 10:51:55 pm
I already had real-time ray traced global illumination, but I recently also added a lighter option, something I call SSRGI (screen space reflections and global illumination). It's much faster than the full blown technique, but not quite as pretty. But overall I think it still looks quite nice. Here is an example of ambient vs SSRGI:

SSRGI on:

(http://i1218.photobucket.com/albums/dd401/222464/SSRGIon.png)

SSRGI off (ambient only):

(http://i1218.photobucket.com/albums/dd401/222464/houseAmbient.png)
Title: Re: d3d (Deferred 3D) - A 3D Engine Using SFML
Post by: CypherStone on August 22, 2014, 02:41:54 am
Amazing work!
Here's hoping my deferred renderer will eventually be this good.

Anyway, I noticed (going through your source code) that your using glBlitFramebuffer for your final output, have you tried rendering out using a simple quad to improve performance. I had a significant gain of FPS by using a fullscreen quad instead. Hope that helps anyway.