Re:creation - a top down action adventure about undeads [hiatus]

[Elias Daler]

Okay, I'm almost done with this meta-system. I decided to go with the static approach (it's even more static that I originally thought to make!)
Animation class:

class Animation
{
public:
    // main stuff here
    int getFrameCount() const;
    void setFrameCount(const int& fc);
    // meta
    static void registerClass();
private:
    std::string name;
    bool isLooped;
    int frameCount;
    FloatRect frame;
};

As you can see, no static MetaObject involved.

First of all, let's look at what I can do now:

• Set property by name
• Get property by name
• Serialize to JSON
• Deserialize from JSON

Let's look at the code:

// need to call this before using Meta<Animation>
Meta<Animation>::init();

// let's create a simple animation
Animation a;

// and set some properties using meta stuff
Meta<Animation>::setProperty(a, "name", std::string("walk"));
Meta<Animation>::setProperty(a, "isLooped", false);
Meta<Animation>::setProperty(a, "frameCount", 10);
Meta<Animation>::setProperty(a, "frame", FloatRect(10, 20, 32, 32));

// let's serialize it to JSON!
Json::Value root = Meta<Animation>::serialize(a);

// now we can save it to file, but we'll input it to console
std::cout << "Animation a in JSON:" << std::endl;
std::cout << root << std::endl;

Animation b;
Meta<Animation>::deserialize(b, root); // get info from JSON

std::cout << "____________________" << std::endl;
std::cout << "Let's look at Animation b now:" << std::endl;
std::cout << "Name = " << Meta<Animation>::getProperty<std::string>(b, "name") << std::endl;
std::cout << "isLooped = " << Meta<Animation>::getProperty<bool>(b, "isLooped") << std::endl;
std::cout << "frameCount = " << Meta<Animation>::getProperty<int>(b, "frameCount") << std::endl;
std::cout << "frame = " << Meta<Animation>::getProperty<FloatRect>(b, "frame") << std::endl;

Output:


Cool!
So, Meta<Animation> (it's Meta now instead of MetaObject) stores a static map of properties. I thought that classes won't need to have different meta presentations, so now I can just use Meta<Animation> to do all the stuff!

Let's look how I can add properties:

void Animation::registerClass()
{
    Meta<Animation>::addProperty("name", &Animation::name);
    Meta<Animation>::addProperty("isLooped", &Animation::isLooped);
    Meta<Animation>::addProperty("frameCount", &Animation::frameCount,
        &Animation::getFrameCount, &Animation::setFrameCount);
    Meta<Animation>::addProperty("frame", &Animation::frame, &serializeRect, &deserializeRect);
}

So, I can set custom getters and setters which will be called each time getProperty or setProperty are called for the property who set them.
I can also add serialization/deserialization which allow me to convert from Json::Value to T and from T to Json::Value.
I don't need these functions for int, float, bool, etc., because I can just do this (jsoncpp provides this functions):

int i = intValue.asInt();
bool b = boolValue.asBool();
... // etc

But I can't convert from T to Json::Value if T is custom class.
So I can provide functions like this which allow me to do serialization/deserialization for custom classes:

Json::Value serializeRect(const FloatRect& rect)
{
    Json::Value v;
    v[0] = rect.x;
    v[1] = rect.y;
    v[2] = rect.w;
    v[3] = rect.h;
    return v;
}

void deserializeRect(FloatRect& rect, const Json::Value& value)
{
    rect.x = value[0].asFloat();
    rect.y = value[1].asFloat();
    rect.w = value[2].asFloat();
    rect.h = value[3].asFloat();
}

So, this allows me to do some great stuff, for example I can save and load components from JSON files without writing additional code for serialization/deserializaition! I can also use all this info about objects to display their properties on the screen while the game is running. How cool is that?

Ask me any questions if you're interested in knowing some other details. I won't post code until it becomes better, it has grown a lot in size, so I have to make a lot of work documenting and cleaning up stuff.

[Jabberwocky]

No doubt, decent tools can make or break a game, especially content heavy games like an RPG or action adventure.  I've enjoyed watching you hammer out your approach with the editor, Elias.  And the JSON <-> C++ serialization.

One thought - you may also want to be thinking about saving and loading functionality while you're at it.  Like, the "save game" and "load game" buttons.  It's very much the same kind of serialization/deserialization type of problem.  I'm personally a big fan of having a human-readable save file format (like XML or JSON), as this can help immensely during development and debugging.  As opposed to a binary format.

So, maybe you can use the C++ <-> JSON serialization for saving and loading the game, just like you do in your editor for authoring animations and other game content.

[dabbertorres]

^

Can even take it a step further and make a "game save editor" of sorts. That can help with debugging as well. Set the game to be at a certain level of completion or whatnot, and make sure the game loads and starts/runs fine, etc.

(Though, depending on game format, may be as simple as a bunch of checkboxes!)

[dwarfman78]

your test is invalid because you print a's properties instead of b's.

[Elias Daler]

Jabberwocky, dabbertorres, I'll think about it. There's just so many stuff I don't have to save to the save file, for example, component's default values which are stored in entity definition files. I only have to save stuff which is unique for entity: it's position, current state, HP, inventory, etc.
But, there's sometimes no need to save  HP or current position which most games don't do and they're fine. So, as for now, I'll be writing game saving/loading code by hand by thinking carefully about what I have to save and what I don't.

dwarfman78, ha-ha, right, I've changed the code.
I just checked b's values in debugger and thought:"Alright, this works, output is expected, let's post! "

[Laurent]

Just two comments about your implementation:

- Why do you need to pass the direct pointer to member when you also pass the getter and setter functions, when you register the property?

- Why do you pass the serialization functions explicitely when you register a property? You should rather use a single known function (like jsonSerialize / jsonDeserialize) for which you add overloads when you need to handle new types. The compiler will then choose the right overload for you. This is similar to what standard streams do with operators << and >>.

And if you're tired of writing all this stuff, you can add a wrapper that makes it a little funnier to write:

auto meta = getMeta(a); // returns a MetaWrapper<T>, which stores the T instance and invokes Meta<T>:: functions
meta.set("frameCount", 10);
auto count = meta.get<int>("frameCount");

And... this is also mostly for fun, but if you want to make the whole system safer, and disallow calling Meta<T>::addProperty from anywhere, you can do something like this:

void Animation::registerClass(MetaBuilder<Animation>& builder)
{
    builder.add("name", &Animation::name);
    builder.add("isLooped", &Animation::isLooped);
    ...
}

By restricting construction of MetaBuilder<T> and access to Meta<T>::addProperty, you can make sure by design that:
- Meta<T> can only be populated in T::registerClass, so the whole initialization of meta stuff is kept under control
- the rest of the code only has access to Meta<T>::get/set
- ... and if you use the MetaWrapper, you don't even need a visible Meta<T> class anymore, you can keep everything hidden and only expose construction with MetaBuilder and access with MetaWrapper.

[Elias Daler]

Thanks, Laurent, there are fantastic suggestions! I've implemented everything that you've proposed.
Once I finish cleaning everything up, I'll create a github repository, so maybe someone will help improve it even further

[Elias Daler]

I've tried out more complex classes and here are the results.
Animation class stays the same, let's add GraphicsComponent class:

class GraphicsComponent {
public:
    GraphicsComponent();

    Json::Value serializeAnimations() const;
    void deserializeAnimations(const Json::Value& root);

    static void registerClass(MetaBuilder<GraphicsComponent>& builder);
private:
    std::string filename;
    std::map<std::string, Animation> animations;
    Animation specialAnimation;
};

Here's how register function looks:

void GraphicsComponent::registerClass(MetaBuilder<GraphicsComponent>& builder)
{
    builder.add("filename", &GraphicsComponent::filename);

    builder.add<decltype(animations)>("animations");

    auto& animationsProperty = builder.get<decltype(animations)>("animations");
    animationsProperty.setMemberSerializer(&GraphicsComponent::serializeAnimations);
    animationsProperty.setMemberDeserializer(&GraphicsComponent::deserializeAnimations);

    builder.add("specialAnimation", &GraphicsComponent::specialAnimation);
}

Here's how JSON file for this class looks (all I have to do to get this is call Meta<GraphicsComponent>::serialize) :

{
        "animations" :
        [
                {
                        "frame" : [ 0, 32, 32, 32 ],
                        "frameCount" : 30,
                        "isLooped" : true,
                        "name" : "idle"
                },
        ...
        ],
        "filename" : "res/hero.png",
        "specialAnimation" :
        {
                "frame" : [ 0, 64, 32, 32 ],
                "frameCount" : 20,
                "isLooped" : false,
                "name" : "megaAttack"
        }
}

Serializing/deserializing map of animations is tricky, because it's array in JSON and we want to use animation names as map keys in C++. So, let's add functions which will be called each time Meta<GraphicsComponent> tries to serialize/deserialize animations map:

Json::Value GraphicsComponent::serializeAnimations() const
{
    Json::Value animationsArray;
    for (const auto& animationPair : animations) {
        auto& animation = animationPair.second;
        animationsArray.append(Meta<Animation>::serialize(animation));
    }
    return animationsArray;
}

void GraphicsComponent::deserializeAnimations(const Json::Value& root)
{
    for (int i = 0; i < root.size(); ++i) {
        Animation a;
        Meta<Animation>::deserialize(a, root[i]);
        animations[a.getName()] = a;
    }
}

Great! What about Animation specialAnimation? If no functions are provided, Meta<T>::serialize / Meta<T>::deserialize get called. If the meta class was not explicitly created for the T (using T::registerClass function) , it will have no properties, so the first function will return Json::nullValue and the second function will return T().

As for MetaWrapper object, here's how useful it is:

auto metaGc = Meta<GraphicsComponent>::getMeta(gc);
metaGc.set("filename", std::string("res/test.png"));
int frameCount = metaGc.get<int>("frameCount");

The code is much cleaner and easier to read!

Oh, and as Laurent suggested, doing this is possible:

builder.add<int>("frameCount");
auto& frameCountProperty = builder.get<int>("frameCount");
frameCountProperty.setGetter(&Animation::getFrameCount);
frameCountProperty.setSetter(&Animation::setFrameCount);

As you can see, there's no need to provide &Animation::frameCount for this property. What happens if I don't provide setter and try to call set for this property? (Or try to get some property without getter?) Assertion tell me about it ("Forgot to add setter/getter!"), and everything is alright (if I don't use assertions, C++ silently writes to nullptr and the stack gets corrupted!)

And now I'm getting back to work on Animation tool... which will later become Entity Creation Tool!
But I'll have to rewrite a lot of stuff to store all component data in JSON instead, so this will take a while. I'll try not to stop working on the game and will draw art and think about levels/mechanics in parallel.

[Laurent]

auto metaGc = Meta<GraphicsComponent>::getMeta(gc);

The purpose of using a function was mainly to benefit from type deduction, so that you never have to write any template parameter. Just make getMeta a free function, and you can remove the "Meta<GraphicsComponent>::" part.

Oh, and as Laurent suggested, doing this is possible

I don't remember suggesting that
What I previously said is that:
- there were useless arguments in your calls (pointer to member in addition to getter/setter)
- using an overload of a single function for serialize/deserialize would be better than explicitly passing the functions when you register the property

[Elias Daler]

The purpose of using a function was mainly to benefit from type deduction, so that you never have to write any template parameter. Just make getMeta a free function, and you can remove the "Meta<GraphicsComponent>::" part.

Ohhh, yeah, I'll make that

Oh, and as Laurent suggested, doing this is possible

I don't remember suggesting that
What I previously said is that:
- there were useless arguments in your calls (pointer to member in addition to getter/setter)

Well, as I've shown you can either pass just pointer to member ot getter + setter. How is this useless? You either access the member directly or use getter + setter to do this.

- using an overload of a single function for serialize/deserialize would be better than explicitly passing the functions when you register the property

Can you please explain more? I have no idea how to do this.

And... I've encountered the first huge problem.
What I need is to have Property::serialize function which does this:
1) Calls serializer member function if it exists
2) If it doesn't exist:
     a) Returns Json::Value(T&) for these types: bool, string, float, int.
     b) Tries to call Meta<T>::serialize for other types
This is easily done with template specializations...
3) If everything else fails, returns empty Json::Value() (null value)

Here's the problem: suppose I try to serialize size_t. Template specialization for size_t doesn't exist, so Meta<size_t>::serialize gets called. But this will fail to sompile, because size_t doesn't have registerClass function which is required for all types which use Meta<T>.
Even if I call it only with "good" types, this introduces another problem: circular dependencies. Meta<T>::serialize calls Property::serialize for each property of T, some of which may call Meta<U>::serialize, etc.
This introduces circular header dependencies and other headaches... So, I feel like I'm doing this wrong. Any suggestions?
I feel there's much easier solution...

[Laurent]

Well, as I've shown you can either pass just pointer to member ot getter + setter.

I was referring to this:

Meta<Animation>::addProperty("frameCount", &Animation::frameCount,
        &Animation::getFrameCount, &Animation::setFrameCount);

If you have a getter and a setter you obviously don't need direct access to the member

You either access the member directly or use getter + setter to do this.

If now it's one or the other, it's fine.

By the way, what do you gain by allowing to set the meta-property's properties () after constructing it, rather than passing all you need directly to the add function?

Can you please explain more? I have no idea how to do this.

Sure.

Json::Value json::serialize(int x) {...}
void json::deserialize(const Json::Value& root, int& x) {...}

Json::Value json::serialize(std::string s) {...}
void json::deserialize(const Json::Value& root, std::string& s) {...}

Json::Value json::serialize(sf::FloatRect rect) {...}
void json::deserialize(const Json::Value& root, sf::FloatRect& rect) {...}

Json::Value json::serialize(sf::IntRect rect) {...}
void json::deserialize(const Json::Value& root, sf::IntRect& rect) {...}

...

auto value = json::serialize(v);

If an overload exists for the type of v, the compiler will automatically choose it. If it doesn't, the compiler will complain. The point is, the serialization code remains generic, it only has to call json::serialize and the compiler does the rest, as long as a compatible overload exists.

And... I've encountered the first huge problem.

What I explained may solve it as well. If not, I'll try to come up with a proper solution later

[Elias Daler]

I was referring to this:

Meta<Animation>::addProperty("frameCount", &Animation::frameCount,
        &Animation::getFrameCount, &Animation::setFrameCount);

If you have a getter and a setter you obviously don't need direct access to the member

Look at the code I've posted previously:

builder.add<int>("frameCount");
auto& frameCountProperty = builder.get<int>("frameCount");
frameCountProperty.setGetter(&Animation::getFrameCount);
frameCountProperty.setSetter(&Animation::setFrameCount);

Just pointers to get/set functions, no direct access!

By the way, what do you gain by allowing to set the meta-property's properties () after constructing it, rather than passing all you need directly to the add function?

That's because I was tired of having lots of constructor overloads
1) No pointers passed
2) Just pointer to member
3) Getter + Setter
4) Getter + Setter + Serializer/Deserializer
5) Pointer to member + Serializer / Deserializer
Setting particular functions makes the code easier to read and write, imo. But maybe I should come back to having 3 (or 4?) overloads, I don't know

Sure.

Json::Value json::serialize(int x) {...}
void json::deserialize(const Json::Value& root, int& x) {...}

Json::Value json::serialize(std::string s) {...}
void json::deserialize(const Json::Value& root, std::string& s) {...}

Json::Value json::serialize(sf::FloatRect rect) {...}
void json::deserialize(const Json::Value& root, sf::FloatRect& rect) {...}

Json::Value json::serialize(sf::IntRect rect) {...}
void json::deserialize(const Json::Value& root, sf::IntRect& rect) {...}

...

auto value = json::serialize(v);

Yeah, this is something I previously did, though I tried to make the function generic and failed!
You see, I just wanted to have function like this:

template <typename T>
Json::Value cast(const T& obj) {
     return Meta<T>::serialize(obj);
}

and having a bunch of template specializations for other specializations. But this introduces cyclic dependency between Meta -> Property -> cast -> Meta which I couldn't resolve. Once again, the cyclic dependency is caused by these things I have to do:
Meta<T>::serialize calls Property<U>::serialize for some property with type U, which then either calls template specialization (for simple types) or Meta<U>::serialize and so on. And I can't even forward declare all this stuff

This is what made it possible to simply create JSON in the GraphicsAnimation class. Meta<GraphicsAnimation> iterated over all the properties, it did this:
1) Called template specialization for std::string
2) Called serialization function for std::map<...> which I've provided
3) Called template function Json::cast<Animation> which then called Meta<Animation>::serialize

This worked in MSVC, but GCC fails to compile this code because of the cycle. So, this is my problem now.

[Laurent]

Would you mind sharing your code for Meta and Property, so that I can have a look?

[Jabberwocky]

If an overload exists for the type of v, the compiler will automatically choose it. If it doesn't, the compiler will complain. The point is, the serialization code remains generic, it only has to call json::serialize and the compiler does the rest, as long as a compatible overload exists.

This is pretty close to how I've implemented XML serialization in my game.

You've given a lot of fantastic advice in this thread Laurent, and it's cool seeing the SFML devs so involved in these devlog threads.

Jabberwocky, dabbertorres, I'll think about it. There's just so many stuff I don't have to save to the save file, for example, component's default values which are stored in entity definition files. I only have to save stuff which is unique for entity: it's position, current state, HP, inventory, etc.
But, there's sometimes no need to save  HP or current position which most games don't do and they're fine. So, as for now, I'll be writing game saving/loading code by hand by thinking carefully about what I have to save and what I don't.

Absolutely.  You don't save the same data to a savegame file as you would to an entity definition file.  It should only be "dynamic state" stuff, like position and inventory, as you mention.

What I am saying is that in both cases (editing, saving) you need to serialize and deserialize data to a file.  And that it's useful to be able to open and read your savegame files (in a non-binary format).  So JSON may be an ideal format for both.

[Elias Daler]

Would you mind sharing your code for Meta and Property, so that I can have a look?

Sure, here:
https://bitbucket.org/edaler/meta-stuff/src
Check out include/JsonCast.h. It includes Meta.h which causes the problem according to GCC. It compiles with MSVC, huh. Also, Property.inl and Meta.inl have some very important code
And yeah, I'm sorry that some code may be bad, this is very WIP, so I still have some stuff to improve.

You've given a lot of fantastic advice in this thread Laurent, and it's cool seeing the SFML devs so involved in these devlog threads.

Ha-ha, yeah, I'm very glad that so many awesome people read my dev logs and help me make the game!

Absolutely.  You don't save the same data to a savegame file as you would to an entity definition file.  It should only be "dynamic state" stuff, like position and inventory, as you mention.

What I am saying is that in both cases (editing, saving) you need to serialize and deserialize data to a file.  And that it's useful to be able to open and read your savegame files (in a non-binary format).  So JSON may be an ideal format for both.

Yup, it's quite good, working with it was a pleasure. Easy to read, easy to make, easy to parse.