The new graphics API in SFML 2

[Laurent]

Hi

I'm currently out of inspiration for the new graphics API. So I thought it would be a good idea to share what I've done so far, and use your feedback to complete this task

I decided to start with the lowest level. I needed something flexible enough so that people stop complaining about performances or limitations, but still easy to use; and as a bonus, something that will allow me to drop deprecated functions so that SFML can be ported to pure OpenGL (ES) 2 architectures.

The result is something very close to how OpenGL works, and it contains the 5 ingredients required by modern rendering architectures:

Textures
sf::Texture is left unchanged.

Shaders
sf::Shader is left unchanged, but it will later be upgraded to support vertex shaders.

Transforms
They will be handled by a new class, sf::Transform. Internally a transform is a 3x3 matrix, but its public API provides the usual functions: Translate, Rotate, Scale, Combine (with other matrix), TransformPoint, GetInverse, and some operator overloads.
So, transformations are now completely separated from drawables, and are much more flexible.

Geometry
Geometry is no longer hidden behind high-level classes, it is now directly exposed.
Vertices are defined with a new class, sf::Point. It's very simple, it contains 3 public members (Position, Color, TexCoords) as well as a bunch of constructors to easily construct points with 1, 2 or 3 of these attributes.
On top of that there's a sf::Mesh class (its name can change if you have better ideas), which is a thin layer on top of OpenGL vertex arrays. It is also quite simple: it combines a dynamic array of sf::Point (Resize, operator[], Append, ...) with a primitive type which defines how the points must be interpreted as geometrical primitives when they are rendered (for example it can be Points, Lines, Triangles, Quads, ...).

Renderer
All these things are put together in the sf::RenderTarget class (the same as now, the base of RenderWindow and RenderTexture), with the following functions:
- SetView(sf::View)
- SetTransform(sf::Transform)
- SetTexture(const sf::Texture*)
- SetShader(const sf::Shader*)
- SetBlendMode(sf::BlendMode)
- Draw(sf::Mesh)

Here is a typical example, a tiled map:

Code: [Select]

const int size = 32;

// load the texture containing all the tiles
sf::Texture tileset;
tileset.LoadFromFile("tileset.png");

// create the map
sf::Mesh map(sf::Quads, width * height * 4);
for (int x = 0; x < width; ++x)
    for (int y = 0; y < height; ++y)
    {
        int current = (x + y * width) * 4;

        // define the position of the 4 points of the current tile
        map[current + 0].Position = sf::Vector2f((x + 0) * size, (y + 0) * size);
        map[current + 1].Position = sf::Vector2f((x + 0) * size, (y + 1) * size);
        map[current + 2].Position = sf::Vector2f((x + 1) * size, (y + 1) * size);
        map[current + 3].Position = sf::Vector2f((x + 1) * size, (y + 0) * size);

        // define the texture coordinates of the 4 points of the current tile
        int tx = /* X index of the tile in the tileset */;
        int ty = /* Y index of the tile in the tileset */;
        map[current + 0].TexCoords = sf::Vector2i((tx + 0) * size, (ty + 0) * size);
        map[current + 1].TexCoords = sf::Vector2i((tx + 0) * size, (ty + 1) * size);
        map[current + 2].TexCoords = sf::Vector2i((tx + 1) * size, (ty + 1) * size);
        map[current + 3].TexCoords = sf::Vector2i((tx + 1) * size, (ty + 0) * size);
    }

// let's scale and translate the whole map
sf::Transform mapTransform;
mapTransform.Scale(2, 2);
mapTransform.Translate(500, 400);

...

// draw the map
window.SetTexture(&tileset);
window.SetTransform(mapTransform);
window.Draw(map);


Building the map is a little more verbose than with sprites, but the whole map is now a single 2D entity and it can be drawn with a single call (which is very fast -- no need for tricks such as manual clipping or pre-rendering to a texture).

That's it for the new low-level API. Hopefully, users who required more flexibility and/or performances will be happy with it.
But it's not enough: it's very low-level, a lot of SFML users enjoy the simplicity of the high-level classes and I don't want to force them to use these classes.

I can easily implement the old sf::Text, sf::Sprite and sf::Shape classes on top of this new API. But it would probably not be a good idea. These classes were required in the old API because they were the only way to draw things on screen. But now that we have a low-level layer that can do almost everything, these classes would exist only for convenience. Therefore, maybe we can think of something different. And we must also decide how these classes would mix with the low-level layer: do they define and use their own transform, or the one that is set with SetTransform? Should they inherit a base class, that would allow to write "window.Draw(sprite)"? Or rather define a Draw function in all of them so that we would write "sprite.Draw(window)"?

I actually have a lot more pending questions and problems, but let's see what you think about this

Don't hesitate to comment the low-level API, suggest features or modifications, propose ideas for the high-level classes, etc. But please don't focus on details (except class names), here we are talking about general concepts.

PS: sorry for those who, after reading the title, thought that the new API was already written and pushed :lol:

[Richy19]

I think you should create the text, sprite and shape ontop of the current API as you say people use these for convenience.
But if they want more performance they can use their custom classes with the sf::mesh
As far as how to draw them, I think you should keep it with App.Draw(sprite) try and keep it as similar to the current SFML as possible to not break everything

Does this mean that once this is done SFML 2 will be officially released?

[Groogy]

First I have to say... DAMN! I'm gonna have a busy week with rbSFML *cries a little then man up*

Anyway I noticed you seemed to be using the texture width/height to define the texture coordinates, at least that's the only impression I can get since you used an integer. I really hope you won't have that in the final and that was just for example. Having texture coordinates going in the range 0..1 is standard pretty much everywhere as far as I know and it actually makes it easier. Not gonna preach on it since you said not to focus on the details but that really got me scared.

Anyway I agree that something new is entirely right since we got new tools to work with.

For instance I have had several times when the sprite has in fact been a bit annoying because all I wanted to do was render an image to a specific point on the screen and nothing more fancy. No offsetting the origin, rotating, scaling like that but just a normal "blit" like in SDL. Would be cool if SFML would now support this, might even have a mesh cached away in the render target. To show what I mean:

Code: [Select]

sf::Texture background;
background.LoadFromFile( "someFile.png" );

// ...Code here and code there...

window.Draw( background, 0, 0 );
Having a sprite for the background in a let's say static view game or menu seems unnecessary. It's a tiny short cut and you know just a comfort factor.

Other than that I have a hard time figuring out anything besides what I am already used to. Sprites, Text and Shape. Though I think there are some extensions to be made on these already in place classes. For instance what about letting us clip sprites, text and shapes by a rectangle or even another shape? Or what about skew them? Giving your users the ability to fake depth by skewing the images giving some kind of perspective.(I realize this is not done with skewing but I don't know what the other transformation is called in English)

Though I am wondering if these new things should be directly in the RenderTarget and not in the Renderer? The RenderTarget's mission is not to actually render stuff but only be a place in memory that we render to, right?

Also this is a more advanced thing I would like to see is maybe some more work on shaders? I'm stealing a little from D3D10 but let's say we want several different things to manipulate the shader a little here and a little there. What if you would allow one to have "reference" objects to variables in the shader? So instead of having to pass the shader around we have references to different variables that we can change freely. (That was the D3D10 inspiration, now comes my own idea) Though this alone I guess is not a good idea but if we combine it to like batches. So say we have obj1 and obj2 which both are going to be rendered with shader1. So instead of manipulating each variable by it's own we have an object that represents a "shader state" or something which has this objects variables defined for it and updates the shader when drawn. Am I making myself clear on it? Anyway base idea is kind of how GetMatrix worked for drawable, if any values changed recalculate the matrix. The same goes for the state kind of.

Trying on some example code:

Code: [Select]

class GameObject
{
public:
        void Render( sf::RenderTarget &aTarget ) const
        {
                // What I am talking about
                aTarget.Draw( mySprite, GetShaderState() );

               // How it is currently with the information given
               sf::Shader *shader = aTarget.GetShader();
               shader->SetParameter( blabla );
               shader->SetParameter( blabla );
               shader->SetParameter( blabla );
               shader->SetParameter( blabla );
               // ...etc
        }
private:
        mutable sf::ShaderState myState;
        sf::Sprite mySprite;
};

Get my point? I think this is actually a good idea to expand on. My initial idea might not be that great but we can work on it if we think about adding light sources, shadows and the like and passing this information on to the shader. Like let's say if we somehow add the possibility to pop and push the states?

I will expand on this shader idea tomorrow when I wake up. I believe it really have potential. For instance these can be used to optimize the bandwidth between CPU and GPU.

I believe there will be many wall of texts on this thread

[Disch]

I'm just thinking out loud here.  Take all of this with a grain of salt:


The SetTransform and SetView thing seems a little awkward to me.  Views and Transforms seem to be very similar things.  Functionally, there isn't much that separates them, other than that a View does everything in the opposing direction.

sf::View could just be a specialized sf::Transform (inherited) which simply inverts all operations.

Then, you can remove SetView completely, and instead of an interface where you have only one view and only one transform active at any time, you allow a way to "stack" any number of transformations.

Older OpenGL did this with glPushMatrix and glPopMatrix.  I'm envisioning something similar.

The only way I can think of to do this though would make the interface awkward, though.  I don't think the user should be responsible for pushing/popping transformations and keeping track of how many are pushed.

It also makes moving the view more difficult because you have to pop all transformations, change the view, then push them all back.  You can't just set a new view as you could before.



---

Maybe that could all be avoided if you keep SetView and SetTransform, but allow ways for transformations to be stacked internally.  This would offer more or less the same functionality, but would keep the interface simpler.

You said sf::Transform was basically just a wrapper around a 3x3 matrix, right?  So then it could be as simple as overloading the * operator:

Code: [Select]


sf::Transform bos = target.GetTransform();
sf::Transform tos = bos * some_other_transformation;
target.SetTransform(tos);
target.Draw(whatever); // both 'bos' and 'some_other_transformation' applied
target.SetTransform(bos);  // effectively "popping" the stack

// from here, only 'bos' applied




again just thinking out loud....

[Groogy]

I think he already thought of that because he said under Transformation that he was overloading operators

[Walker]

I think having some more transformations (like Groogy was saying) would definitely be good. sf::Transform could even be opened up to a policy system to allow users to define their own transformations. Although this may not really fit with the S in SFML.

Other than sf::VertArray I don't think you can get much more descriptive than sf::Mesh Although "mesh" tends to suggest (to me) three dimensions.

It would be nice to be able to define custom blend modes with sf::BlendMode.

Overall, this looks pretty good.



Higher-level convenience classes:

You could make sf::Sprite really basic and handle texture resources internally. i.e. you could make a sprite with sf::Sprite sprite("somefile.png");

In my opinion sf::Sprite needs to have its own transform stuff - it's very easy to consider that my sprite is rotated so much. But there is always people wanting to be able to rotate about a point that is different from the sprite's "origin" so they will either need to use the lower-level system, be able to use sf::Transform with their sprites, or all of the transforms will be needed to be implemented in sf::Sprite.

I think getting rid of sf::Drawable and having the "drawable" classes as their own things sounds good.

Simple text rendering is a crucial feature but I can't think of any major improvements over the current API.

[thePyro_13]

I do hope you implement sprites and so on on top of this(or a similar simple renderable system). They provide a much higher level of simplicity than most drawing operations, and were what originally drew me to SFML.

The new API leaves room for people to squeeze performance out when they need it, just make sure you don't completely abandon the simple, or we'll just be left with FML.

Just a thought, with this new API, would it be possible to create merge-able sprites? Would make the example code even simpler, just load a tilemap like normal, than merge the lot of them. Performance and simplicity.

[pdinklag]

I'd have to agree to what some people said here.

While, of course, classes like Sprite and Text are nothing but convenience classes which aren't technically needed, they do make the "S" in "SFML". I support the idea that it should stay simple, and these convenience classes should be implemented, but those who seek performance or more detailed control should be able to use the new API. It's a win-win that way.

What I'm not sure about is the transformation thing. Firstly, I kindof have to agree to Disch, they and views have a lot in common. Furthermore, I think that maybe the Drawable class should stay mostly as it was, basically for the same reasons I mentioned above, but also because in my eyes, this new approach is a step away from object-orientation. This may be debatable, but I think that a drawable entity is the combination of a Mesh and a Transformation, and the former API nailed it.

Besides, I would maybe rename the sf::Point class to sf::Vertex. "Point" to me sounds more like a Vector2i alias than a structure for world coords, color and tex coords.

Other than that, I like that you are now able to define custom meshes! sf::Mesh fits perfectly as the name. It was the missing link in the drawable API and completes it IMO.

[Laurent]

Anyway I noticed you seemed to be using the texture width/height to define the texture coordinates, at least that's the only impression I can get since you used an integer. I really hope you won't have that in the final and that was just for example. Having texture coordinates going in the range 0..1 is standard pretty much everywhere as far as I know and it actually makes it easier. Not gonna preach on it since you said not to focus on the details but that really got me scared.

I'll keep pixel coordinates. I don't see how it would be better to write this:

Code: [Select]

Vector2f(static_cast<float>(x) / texture.GetWidth(), static_cast<float>(y) / texture.GetHeight())
rather than this

Code: [Select]

Vector2i(x, y)
You always have to divide by the texture size -- and not the public one, but the internal one which is not accessible (a texture may be padded if it has not power-of-two dimensions on old hardware). It's much easier to do the division internally and let people play with pixels.
But for those who use sf::Texture with OpenGL it's still possible to use normalized coordinates when binding the texture, don't worry.

For instance I have had several times when the sprite has in fact been a bit annoying because all I wanted to do was render an image to a specific point on the screen and nothing more fancy. No offsetting the origin, rotating, scaling like that but just a normal "blit" like in SDL. Would be cool if SFML would now support this, might even have a mesh cached away in the render target

Hmm, I'll have to think about this. I'm not sure it would be a good idea. I already hate having two APIs for doing the same job, so adding even more rendering options could just be confusing.

For instance what about letting us clip sprites, text and shapes by a rectangle or even another shape?

That's the RenderMask task planned for SFML 2.1. Let's not talk about this here

Or what about skew them?

Can be done easily in sf::Transform. Maybe I'll even add a Skew function to it.

Giving your users the ability to fake depth by skewing the images giving some kind of perspective.(I realize this is not done with skewing but I don't know what the other transformation is called in English)

It's far more complicated to fake depth. It requires rendering tricks to get the texture correctly mapped on the triangles, "skewing" (or whatever) is not enough. There was a topic about this on the forum, search "perspective" if you want ot find it.

Though I am wondering if these new things should be directly in the RenderTarget and not in the Renderer? The RenderTarget's mission is not to actually render stuff but only be a place in memory that we render to, right?

There's no renderer anymore, RenderTarget is the only place where we can render stuff.

I will expand on this shader idea tomorrow when I wake up

Hmm, this is already a little out of topic, maybe it should have its own thread on the forum?
For SFML 2 I'll implement vertex shaders, but nothing fancy on top of it.

The SetTransform and SetView thing seems a little awkward to me. Views and Transforms seem to be very similar things. Functionally, there isn't much that separates them, other than that a View does everything in the opposing direction.

I agree. However a transform is not very convenient to handle when what you want to manipulate is a view. Because it's the same thing internally shouldn't mean that it has to be the same thing from user point of view.

Then, you can remove SetView completely, and instead of an interface where you have only one view and only one transform active at any time, you allow a way to "stack" any number of transformations

Easy to do:

Code: [Select]

Transform old = window.GetTransform();
window.SetTransform(old * new);
...
window.SetTransform(old);
EDIT: oh, this is exactly what you wrote at the end of your message

It also makes moving the view more difficult because you have to pop all transformations, change the view, then push them all back. You can't just set a new view as you could before.

Yep. Separating object transforms (modelview matrix) and views (projection matrix) is mandatory, they are two different things.

Simple text rendering is a crucial feature but I can't think of any major improvements over the current API.

That's right, text is a very special thing, sf::Font is hard to work with directly so I think that sf::Text will be kept unchanged in the new API.

I do hope you implement sprites and so on on top of this(or a similar simple renderable system).

That's the point of this discussion

Just a thought, with this new API, would it be possible to create merge-able sprites? Would make the example code even simpler, just load a tilemap like normal, than merge the lot of them. Performance and simplicity.

This is something I'm investigating. Using high-level classes to define entities but then merging them to a single sf::Mesh (or whatever) looks like a good idea.

While, of course, classes like Sprite and Text are nothing but convenience classes which aren't technically needed, they do make the "S" in "SFML". I support the idea that it should stay simple, and these convenience classes should be implemented, but those who seek performance or more detailed control should be able to use the new API. It's a win-win that way.

I never meant to remove them. But I think we can do much better than just keep them like they are.

Furthermore, I think that maybe the Drawable class should stay mostly as it was, basically for the same reasons I mentioned above, but also because in my eyes, this new approach is a step away from object-orientation. This may be debatable, but I think that a drawable entity is the combination of a Mesh and a Transformation, and the former API nailed it.

I'm hesitating on this point. How would this mix with the low-level API? Imagine that you setup the position/rotation of a sprite, then call SetTransform(whatever) and draw your sprite. What transform is applied to the sprite? Only its own transform? A combination of it and the global one?

Besides, I would maybe rename the sf::Point class to sf::Vertex. "Point" to me sounds more like a Vector2i alias than a structure for world coords, color and tex coords.

I was not sure if "vertex" was meaningful enough for people not familiar with 3D rendering.
Obviously, I need feedback from people who speak english better than me here

[Contadotempo]

This question is coming from someone who knows nothing about this subject but I got curious:
If sf::Text was to be removed, what other ways would there be to display a simple text?

[Laurent]

If sf::Text was to be removed, what other ways would there be to display a simple text?

Already answered:

That's right, text is a very special thing, sf::Font is hard to work with directly so I think that sf::Text will be kept unchanged in the new API.

[Hiura]

And we must also decide how these classes would mix with the low-level layer: do they define and use their own transform, or the one that is set with SetTransform? Should they inherit a base class, that would allow to write "window.Draw(sprite)"? Or rather define a Draw function in all of them so that we would write "sprite.Draw(window)"?

To keep thing simple I would say it should be consistent between the mesh-part and the high-level-part. What I suggest is to have two different and separated hierarchies of classes and to change a little bit what you describe in your first post with Transform, Mesh and Texture like this :

First, wrap the three new Transform, Mesh and Texture classes in, I don't know, DrawableMesh class. This class can have no texture if needed.
That would be the first class hierarchy. Without any child class. (I don't see what class should inherit from DrawableMesh but maybe I missed something.)

So your example would look like :

Code: [Select]

const int size = 32;

// load the texture containing all the tiles
sf::Texture tileset;
tileset.LoadFromFile("tileset.png");

// create the map
sf::Mesh map(sf::Quads, width * height * 4);
for (int x = 0; x < width; ++x)
    for (int y = 0; y < height; ++y)
    {
         /* same */
    }

// let's scale and translate the whole map
sf::Transform mapTransform;
mapTransform.Scale(2, 2);
mapTransform.Translate(500, 400);

...

DrawableMesh dm(map, mapTransform, tileset); // (*)

// draw the map
window.Draw(dm);

(*) Here there is one downside : we have to keep four objects alive instead of three which is already huge. To simply thing, a fourth argument could be added to specify a copy/ownership policy like MovePolicy (C++11 only), CopyPolicy or ReferencePolicy. I don't know exactly how it should be done properly, though. Or maybe the mesh and the transformation could be copied and only the DrawableMesh object and the Texture object should be kept alive.

Next, the second class hierarchy would be the same as the current sf::Drawable. Only it's internal implementation would change to use the DrawableMesh class EDIT : this is wrong. SetTransform or alike should replace SetScale, ... to be consistent. (Or maybe it could be the same as it currently is.)

With this two hierarchy we would have the same line of code to render stuff :

Code: [Select]

window.Draw(drawableMesh);
window.Draw(drawable);

And the user can choose between low-level API with Meshes and so on; or use the high-level API with Sprite/Text/Shape as one can do with the current API.

From the RenderTarget's point of view it would mean two Draw functions :

Code: [Select]

void Draw(const Drawable& object);
void Draw(const DrawableMesh& object);

What do you thing of this ? Have I missed something and my whole idea fells apart ?

[luiscubal]

Or rather define a Draw function in all of them so that we would write "sprite.Draw(window)"?

I like this idea. Besides, even if SFML 2 were to remove the equivalent functionality, the effective end-result would be that people would re-implement these classes themselves.

Also, if you dislike adding junk to the core, you could add a separate library SFML-SimpleGraphics, or something like that, where these extra high-level functions would be.

sf::Point. It's very simple, it contains 3 public members (Position, Color, TexCoords) as well as a bunch of constructors to easily construct points with 1, 2 or 3 of these attributes.

What if I want to pass extra-data per-Point to the shader.

[Laurent]

What if I want to pass extra-data per-Point to the shader.

Calm down
Seriously, this is not likely to happen soon. You're already looking too far.

[Laurent]

What do you thing of this ? Have I missed something and my whole idea fells apart ?

I need more time to think about it, I'll answer later