Re:creation - a top down action adventure about undeads [hiatus]

[Elias Daler]

Yeah, of course. std::vector is not the only container in the world.

I can't think of many data structures which store the data continuously in memory.

[Premature optimization] is very likely.

This is probably true for medium-scale games like Re:creation, but the same can be said for many parts of the ECS, maybe even the choice of an ECS design itself. I think it can't hurt to know some alternatives, even if one is not going to use them in a game. But I definitely agree that profiling makes sense.

There are several reasons for all this new ECS stuff. But yeah, this may not improve the perfomance a lot and I throw away the code without any hesitation.
It was just an interesting challenge which has taught me some things about templates, vectors and some C++11 which I've never knew before. I didn't spend much time doing this, so no regrets.

I agree that I have to do profiling. What is the best way to do this? Do I need to create some test data which I'll use or do I need to integrate this new system to be completely sure?
The one way I think I can test this is to compare how much time each system's update() function takes. It involves iteration through std::vector of related components so it may make some difference.

[lezebulon]

I've been looking at game designs things for year and I always see people praising ECS everywhere, except that I've never seen an ECS actually working well for a medium-sized game or more. Like, I get that an entity is any aggregation of components but I don't get how the logic will be simplified, because the logic would have to be working on the whole entity itself, and not each component independently anyway....

Anyway Elias, you don't need *absolute* contiguous memory for the whole iteration of all the components of a single type. You could store things in like an std::vector < std::array < ComponentType, 128 > > , so that 128 items will be contigous. From an index i, you can easily work out that the object should be placed at index (i/128, i %128), which will end up being calculated really fast. I admit this requires re-coding a whole new data structure but with the requirements you are saying, there is no way around that.

[Elias Daler]

@lezebulon, ECS worked very well for me. I was getting stuck with standard OOP way of doing things and adding new stuff to the game wasn't this easy.
The logic doesn't have to be working on the whole entity. Most of the time (especially for collision, movement and graphics which is what the game is doing most of the time, other stuff is taking less much time) you don't need the whole entity, you are fine with just a component. So, when I draw entities, I just use graphics components which tell me nothing about what entities are these. When I check collision, I don't care what entities collided, I just care that they're collided and I just call their Lua response functions.

Well, that's a good idea, thanks. This way I can be sure that the vector doesn't take too much space.
I'll see if storing components contiguously is giving me advantage first, though.

[Nexus]

Because storing components in one array is better for cache, considering I have to iterate through them in every frame.

By the way: this whole cache story is only true as long as you iterate over a single component type. And in my experience, many systems need access to multiple components. For all those cases, performance is thrown out the window again.

I agree that I have to do profiling. What is the best way to do this? Do I need to create some test data which I'll use or do I need to integrate this new system to be completely sure?

The profiling should be done on data that is somewhat representative of a typical situation in your game. Ideally, you profile the game itself, but you can already draw conclusions from smaller experiments.

Whether you measure time in code or with an external profiler is your choice.

I admit this requires re-coding a whole new data structure but with the requirements you are saying, there is no way around that.

Such a data structure already exists in the standard library, even if it doesn't let you specify the bunch size. It's called std::deque. Know your tools

Other than that, you have trade-offs like boost::stable_vector and what not. There are enough C++ libraries out there to have a look at before reinventing the wheel

[lezebulon2]

@lezebulon, ECS worked very well for me. I was getting stuck with standard OOP way of doing things and adding new stuff to the game wasn't this easy.
The logic doesn't have to be working on the whole entity. Most of the time (especially for collision, movement and graphics which is what the game is doing most of the time, other stuff is taking less much time) you don't need the whole entity, you are fine with just a component. So, when I draw entities, I just use graphics components which tell me nothing about what entities are these. When I check collision, I don't care what entities collided, I just care that they're collided and I just call their Lua response functions.

But for instance, if thé collision response is to move back the object, how do you tell the graphics component that it has to move?

Àbout needing your data to be contiguous: as I said you probably don't care if instead of iterating over 300 items in an array you do it 3 times for 100 items per array, I doubt thé cache miss world be important

[Elias Daler]

By the way: this whole cache story is only true as long as you iterate over a single component type. And in my experience, many systems need access to multiple components. For all those cases, performance is thrown out the window again.

Yup, agreed. Didn't think of it.

Such a data structure already exists in the standard library, even if it doesn't let you specify the bunch size. It's called std::deque. Know your tools

Didn't know it stored data contiguously. (Well, sort of...). My bad

But for instance, if thé collision response is to move back the object, how do you tell the graphics component that it has to move?

Events. Collisions don't happen frequently, so this is not a big perfomance issue.

Progress update
Okay, so I've created this wonderful scene:

Lots of entities, moving randomly.
I've tested this scene with old ECS system (components are stored randomly in memory) and new ECS system (components are stored contiguously in std::vector).
The results shocked me: there was no perfomance increase.
This is not a dissapointing result for me (but kind of stange... I was expecting at least some perfomance increase). In fact, I'm quite relieved that I don't have to integrate the new system into my project (this would be painful and long).
But this was not a complete waste of time. I've come up with lots of useful improvements for my old system when I was writing the new system, so I'll improve my old system with that. I'll come back to actually making the game soon. So, expect to see some cool screens in the future

P.S. Thanks for being critical, everyone. I would be stuck much longer with all that ECS stuff without a healthy dose of a good criticism.

[Nexus]

Events. Collisions don't happen frequently, so this is not a big perfomance issue.

When I experimented a bit with ECS- and event/message-based systems, I initially had an event system capable of abstracting component inter-dependencies completely. That is, by introducing intermediate event types, a lot of classes wouldn't need to know each other. The event thus acted as an interface.

Everything looked great, but as my design became more complex, I found myself adding a lot of different event classes, most of which were merely adapters to their corresponding components. The need to duplicate data as well as the additional complexity in the data flow led me to rethink this idea. I concluded that code becomes far more understandable if as much functionality as possible is handled in systems, and a system would only affect the components it depends on (as a contract), not components on the other side of the world through events. There are still events for certain features, but it's no longer the main communication approach.

[Elias Daler]

@Nexus, so, if CollisionSystem needs to change some entity's speed because of collision, it would just use MovementComponent of the entity which collided directly instead of sending a message to MovementSystem? Am I understanding this right?

[Nexus]

That's one possibility, however this design couples the collision detection with collision response. The collision system needs to know all the consequences of colliding, which is not ideal.

The systems affected by collision are already dependent on collision classes, as they react to collision events. So they could as well check the collision component directly. Systems will tend to depend on multiple component rather than one component and multiple events, so the cache aspect might be a bit worse, trading off code flow simplicity (and event's aren't free either). You can also work with both approaches and see what works better.

Sometimes you do need event queues, for delayed processing for example. What I wanted to say, is that in my opinion, the main idea of ECS is to handle communication between components in systems, and handling communication through events is somewhat contradictory to this idea. So I wouldn't use events exclusively, it also makes it quite difficult to understand what happens when. But as usual, combining multiple approaches can lead to the best results.

[Elias Daler]

@Nexus, yep, that's kinda what I'm doing at the moment. Sometimes events can lead to non-readable code.
Do you use some scripting for collision response, for example?

[Nexus]

No, I only used Lua to load configuration settings so far. It has been more an experimental project I've worked on a while ago, when I had more time... Not a complete game.

[Elias Daler]

So, how are you doing collision response then? There a lot of unique ways and object may react to collision based on its type. I wonder how it can be done without scripting.

[Nexus]

Why would you need scripting? The logic can as well be implemented in C++, it's just a question of flexibility.

I used aurora::DoubleDispatcher for collision response.

[Elias Daler]

So, do you create free functions for each type of collision response and then bind it to Entity type?
I use scripting because it lets me quickly change the collision response logic quickly. I also use it for interaction response (what happens with entity if player interacts with it by pressing "interact" button)

[Grimshaw]

Great job Elias.. very impressed with this project! It creeps me out how similar your ECS looks like to my previous approach. So much is exactly the same to what I used to have. Some parts I still have it, others not so much.

Either way, I think you are doing most things correctly, so just keep going! I don't really agree with some opinions here that oppose yours, but I'd have to support them a bit in saying that you should avoid complexity in favor of usability when the hit isn't very big.

Sometimes the time you save in development by doing something slightly less correct can pay off as well. Engineering time is a resource as much as memory or CPU, consider that