SFML Mobile Issues

[Cornstalks]

Hi Laurent (and others). Sonkun and I have been working on an Android port of SFML, and some of the issues we're seeing are going to be issues for iOS as well, so I thought we could at least talk about them to try and figure out a good direction to take.

First, OpenGL ES does not support redering quads (only triangles, lines, and points). This requires the vertex buffers for sprites to be different. For individual sprites (where only one quad is drawn per draw call), a triangle strip can easily be used without increasing memory consumption. For "grouped" sprites (where multiple quads are drawn in a single draw call, like sf::Text), GL_TRIANGLES will have to be used. Would you prefer to continue using glDrawArrays, or would you prefer to optionally switch to glDrawElements and use indexed draw arrays? I'm assuming you'll want to stick with glDrawArrays (I think it'll save 8 bytes per quad), but I wanted to try and get your (and others) input.

I thought I had more things to bring up, but I can't think of anything else. If I do, I'll post here. I know I and others are eager to contribute to SFML, and hopefully this will help us better collaborate.

[Laurent]

Yes, I will have to get rid of quads since OpenGL ES doesn't support them at all. sf::Text could use triangle strip if glyphs were adjacent, but unfortunately they are not, so we'll have no choice but to use triangles.

[Cornstalks]

Yes, I will have to get rid of quads since OpenGL ES doesn't support them at all. sf::Text could use triangle strip if glyphs were adjacent, but unfortunately they are not, so we'll have no choice but to use triangles.

Ok, cool. I'm just using a triangle strip for sprites and triangles for sf::Text. Thanks!

[Cornstalks]

Quick follow-up question: would you like commits like this to be submitted to the SFML main repo via pull requests, too? There are some changes that are/will be made that could be integrated to SFML, but I don't know if you want these pull requests yet.

[Laurent]

I initially answered twice because I saw your github issue before this post, but now we don't have to continue a duplicated discussion, please don't post everything to both the task tracker and the forum

[Cornstalks]

I initially answered twice because I saw your github issue before this post, but now we don't have to continue a duplicated discussion, please don't post everything to both the task tracker and the forum

Heh, sorry

[Sonkun]

Note that it doesn't actually break the API. It's just an easy change which can be made now (or later) in order to anticipate the future. But I understand if more thinking has to be done, though the only concern is whether there are or whether there will be more optimized methods to draw vertices in the future. E.g: if drawing with GL_TRIANGLE_FAN is as fast as GL_QUAD and if this statement remains true with recent OpenGL versions, I don't see any reason not to implement these changes now.

[Laurent]

sf::Text would have to use GL_TRIANGLES so it will be less efficient.
GL_QUADS is not supported by any OpenGL ES version, so as soon as we get support for it we'll have to remove sf::Quads (deprecate it in SFML 2 and really remove it in SFML 3).

But yes, we can (and must) do these modifications as soon as possible -- but not now

[Cornstalks]

More questions!

This has to do with application and window lifecycles. For iOS and Android, the (simplified) application and window lifecycle is:

• Application created
• Window created and application made "active"/resumed
• Program runs...
• Window destroyed and application paused/backgrounded
• Application destroyed

It's possible for apps to do 1->2->3->4->5 or something like 1->2->3->4->2->3->4->5. The trick here with SFML is that all events are associated with a window. However, here we have some application level (not window level) events (specifically, #1 and #5). These application level events are very useful, though. Specifically, it's nice to be able to pause/background your application, go answer a text, and then resume your application right where you left it (rather than having to reload it from scratch and go through all the menus again).

If we map window open/close events to #2 and #4 (which feels the most natural), we can't get the application level events (#1 and #5) because we don't have a window from which we can poll events.

If we map window open/close events to #1 and #5, and perhaps window gain/lost focus events to #2 and #4, we can get both the application and window level events, but it then changes the semantics of the SFML window class (for example, "window.isOpen()" no longer specifically refers to the window, but to the whole application). Additionally, when a window is destroyed, OpenGL contexts might be destroyed, and "window.isOpen()" might return true despite the window (and the OpenGL contexts) being closed/destroyed, and the user might perform some invalid OpenGL operations (like trying to render). In this case, there's extra responsibility placed on the user to "do the right thing," which will require them to have a deeper understanding of how SFML works on mobile systems.

What are people's thoughts on sensibly handling (and providing access to via the SFML API) application level events and window level events? Specifically, what are the best ways to inform the user of events 1, 2, 4, and 5?

Edit: There's no "window created/opened" event in SFML, so rather than an actual event being fired for such action, I mean "window.isOpen()" would return true. Hopefully that makes sense. Let me know if it doesn't.

[Sonkun]

I think there's no need to think hard about that, SFML design isn't suited for mobile software workflow and that's all. The first goal is providing a framework which works on Android/iOS no matter what tricks are used to make it possible (and if possible without breaking the API). Later, when the API design is reconsidered and modified to take both environment into account, all these issues will be solved (and hopefully with minor changes).

I've been through all these thoughts and ended up with the following current design:

There's no sense in having an event which tells when an application is "created" or "destroyed" since it matches the beginning of the main() function and its ending. But unlike desktop applications, Android/iOS OSes might ask the application to terminate (properly) that's why sf::Event::Closed is used.

Events are the only way to keep the application in the know about lifecycle events and knowing them is mandatory for your application to work. The window creation and destruction are two of these lifecycle events and so, creating a sf::Window and create it to get these events is the only way, even if inside it only grabs or releases an Android window which is either created or destroyed.

As for the user who might perform some invalid OpenGL operations, that's not an issue since all OpenGL operations made by SFML are protected by a check to sf::Context::setActive(true) which returns false if it failed to activate the context.

[Laurent]

I think there's no need to think hard about that, SFML design isn't suited for mobile software workflow and that's all. The first goal is providing a framework which works on Android/iOS no matter what tricks are used to make it possible (and if possible without breaking the API). Later, when the API design is reconsidered and modified to take both environment into account, all these issues will be solved (and hopefully with minor changes).

Exactly

I don't think it's a problem because on such devices, window == application; you cannot have no or more than one window. Furthermore, the application kind of never ends (at least on iOS), so the relevant events to watch are rather GainedFocus and LostFocus.

I recently noticed that Android was restarting the whole application when the device was rotated, and that you had to manually save state and restore it after restart, I don't know how users will be able to handle that through SFML

There might also be a problem when application is sent to background and restored to foreground, or interrupted by a call, since you will also have to save/restore state, which might involve iOS/Android specific functions. I don't know enough about these mechanisms, but these are potential problems.

[Foaly]

Quick side question: in what repository is the android/iOS version developed right now?

[Nexus]

I recently noticed that Android was restarting the whole application when the device was rotated, and that you had to manually save state and restore it after restart, I don't know how users will be able to handle that through SFML

Really? But that would mean every game that exploits this mechanism would have to be restarted. Considering the smooth transition in some games, I think they have a different mechanism...

[Laurent]

Quick side question: in what repository is the android/iOS version developed right now?

Android: https://github.com/Sonkun/esfml
iOS: on my mac, no public branch/repo yet

Really? But that would mean every game that exploits this mechanism would have to be restarted. Considering the smooth transition in some games, I think they have a different mechanism...

You can use a fixed orientation, in which case nothing happens when you move the device. Otherwise, I don't think there's another option. But ok, I'm far from being an expert, just started two weeks ago

[Cornstalks]

I recently noticed that Android was restarting the whole application when the device was rotated, and that you had to manually save state and restore it after restart, I don't know how users will be able to handle that through SFML

The developer can decide whether or not the activity should be restarted on a screen orientation/size change. In the AndroidManifest.xml file, if you add android:configChanges="orientation|keyboardHidden|screenSize" to the activity node, it tells the OS that you will handle these configuration changes and that the OS should not handle them (which it does by default by restarting your activity). If you state that your activity will handle them, your onConfigurationChanged() function will be called.

There might also be a problem when application is sent to background and restored to foreground, or interrupted by a call, since you will also have to save/restore state, which might involve iOS/Android specific functions. I don't know enough about these mechanisms, but these are potential problems.

That's what I've been thinking about the most. I think lost/gained focus is a good way to handle it, and then when the OS wants to really destroy your application, you signal the window as being closed. The SFML user will have to enter a power-saving state when the focus is lost, waiting for a gained focus event (or close event) while using as little CPU as possible.