Welcome, Guest. Please login or register. Did you miss your activation email?

Author Topic: Unfreezing the API  (Read 8329 times)

0 Members and 1 Guest are viewing this topic.

Foaly

  • Sr. Member
  • ****
  • Posts: 453
    • View Profile
Unfreezing the API
« on: January 25, 2018, 02:48:37 pm »
Hello everybody!

I would like to talk about the ban on API changes. For years there has been the rule in place to not introduce changes that alter the public API in any way except in major version releases.

I understand the desire to have a stable, reliable API. Afterall one of SFMLs strong points is its clean and simple API. To my understanding the incentive when introducing this restriction was to make SFML more attractive for bigger projects and companies. The reasoning was that for game studios or companies it is uneconomic to have to refactor their code base every couple month when a new version with a changed API of SFML is released.

In my opinion the assumptions have failed the test of time and have actually done more damage than good. Here is why I think that. From what I can tell (and please correct me if I am wrong) not many companies/studios use SFML for producing games since the limitation was introduced. The biggest chunck of users are still hobby developers, people learning graphics development/OpenGL and small projects. There are occasionally some polished and finished games on some game distribution platform or app store. I think its absolutly fine to confront people learning, studying or simply enjoying graphics programming with new features and potentially a couple minutes/hours of work when updating to a new version. I would actually argue that those users enjoy updates/changes.
Most importantly though the assumptions there will be a new major version "soon" turned out to be false. Back when this was introduced SFML 2.0 had just been released, the new development team had just been founded, development was going strong and there was a minor release about every half year. SFML 2 had just changed more or less the entire API so it made sense to say the next API change will happen in SFML 3. By that time and judging by the development activity at the time SFML 3 was thought to be release within a couple years. We are here a couple years later now and SFML 3 is still a couple years away. Development has slowed down significantly and there is a minor release about once a year. So any advances will just continued to be halted for years?

More that any of that I think that the attitude displayed to outside developers with this is extremly toxic to progress. Be it one time contributors or people enjoying sending patches every now and then. Reading something like: "yeah thanks thats a nice idea. we could implemented it right now, but we won't. why don't you come back in like 2-3 years, then we will think about it again". It is extremly discouraging and I have seen this many times. I think we can all agree that given the pace of SFMLs current development we should make it as welcoming and easy as possible for contributors to help.

So in the good old fashion of proving a necessity could somebody from the team please provide a concrete use case for an API that is stable and without change for years and years on end.

Sorry if this comes across harsh. I am not trying to hate on anybody. Its just that its incomprehensible and mind-boggling to me to see the already slow development of SFML being crippled even further for no advantage. But this is a discussion please let me know your opinion.

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: Unfreezing the API
« Reply #1 on: January 25, 2018, 02:55:33 pm »
Thanks for your feedback, it is really appreciated.

I have the feeling that you're heavily misunderstanding an important point: we're not talking about "no change", but rather "no breaking change". Which means that, as you've probably seen in recently merged PRs, we can easily add new stuff, even change existing stuff with our deprecation system, while still preserving API/ABI compatibility. The API is not frozen in any way.

So could you give an example of an important modification that we couldn't consider because of our compatibility policy?
Laurent Gomila - SFML developer

Foaly

  • Sr. Member
  • ****
  • Posts: 453
    • View Profile
Re: Unfreezing the API
« Reply #2 on: January 25, 2018, 06:37:24 pm »
Thanks!
Yes I know in theory the distinction between breaking change and change is made, but in reality it comes down to people thinking better leave it so I don't have to deal with changing the API. Or you have to spend a unnecessary amount of time to find an implementation that fits the old API.

About the examples, I don't have a proper list of all the changes that didn't happen/were delayed, but here are some things of the top of my head:

* A proper API for setting window states (differentiating between style and state)
* Multi monitor management
* Output streams
* Angles class
* Using any modern C++
* ...

I guess the last one is my biggest pain point. There were sooo many times a better solution using a modern version of C++ could have been used.

Jonny

  • Full Member
  • ***
  • Posts: 114
    • View Profile
    • Email
Re: Unfreezing the API
« Reply #3 on: January 25, 2018, 08:10:01 pm »
I've flogged this horse plenty, but for the record (again) I agree with OP, mainly on speed of development.

I don't think OP is saying it's frozen by choice or intent of the SFML team, just that it's frozen due to slow development, If you only do a major release every 5 years (or more) and refuse to make api breaking changes except on major releases then progression is really stifled.

In my opinion adding more structure to the release schedule would help (i.e. say minor release every quarter, major every year).

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: Unfreezing the API
« Reply #4 on: January 25, 2018, 08:47:45 pm »
Quote
* A proper API for setting window states (differentiating between style and state)
* Multi monitor management
As far as I know (which is not 100% accurate) this is something that just needs more contribution, these tasks are not officially postponed to SFML 3.

Quote
* Output streams
Not sure why you mention it. It's just an addition to the API, I wouldn't say that it has to wait for SFML 3. Maybe we decided to leave it for the next major version because the interest in this feature is very low; but if someone submits a PR tomorrow I'm sure we can merge it in SFML 2.

Quote
* Angles class
That's not even decided, as far as I know, and again, does it have anything to do with SFML 3?

Quote
* Using any modern C++
That's the main reason for SFML 3, and most of the planned changes break the API so much that it is impossible to consider it for SFML 2.

Quote
I guess the last one is my biggest pain point. There were sooo many times a better solution using a modern version of C++ could have been used.
Agreed, that's also my biggest concern about SFML. We urgently need modern C++, it's a shame we're still stuck with C++03. But what we need is to actually start working on SFML 3 (*), not discuss about the possibility to break the SFML 2 API ;)

(*) before you blame me, I've actually started something but don't expect to see something too soon
Laurent Gomila - SFML developer

Foaly

  • Sr. Member
  • ****
  • Posts: 453
    • View Profile
Re: Unfreezing the API
« Reply #5 on: January 25, 2018, 09:49:32 pm »
Sorry my last post was a little rushed, because I had to leave.
My intention was not to pinpoint every feature and discuss why exactly it has failed. I wanted to show some examples where the mentioning of SFML 3, which implys not breaking the API now, led to the development dying down. It generates the vibe/feeling of lets not do things right now, but do them properly later. It discurages people from investing time.

Please don't get me wrong I am not trying to shit on anyone or blame somebody for something. I am worried about the future of SFML. It would be really sad to see it die down.
The situation from a couple years ago to now has changed. Thats why I say the rules made back then have to be reevaluated. My reasoing is since there is less contributions from the team now, we have to rely on more contributions from within the community. So we have to make it as easy as possible for people to contribute. The way the API freez is right now, its hurting SFML more than it does good for it. Maybe you can tell me some advantages that I've missed. Like I said before, I can understand were this is comming from and I don't want to just get rid of it. A stable API is a nice thing! I want to have a discussion about it. It would be fantastic if more people could join in.
I think SFML can benefit from changing/adapting some rules. Maybe we can come up with something that motivates more community members to contribute and helps to speed up the development again :)

I agree that getting the SFML 3 development started is also a big problem. But I'll leave that to the team ;) Great to hear that you started to work on some stuff. I would love to contibute!

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: Unfreezing the API
« Reply #6 on: January 25, 2018, 10:08:37 pm »
Quote
My intention was not to pinpoint every feature and discuss why exactly it has failed. I wanted to show some examples where the mentioning of SFML 3, which implys not breaking the API now, led to the development dying down.
My intention was neither to discuss these features in detail. I wanted to demonstrate that (in my opinion) things are not as stuck as you describe it, the community is working on important changes to the code and API everyday, and so far I have the feeling that we're happily accepting those changes and not "blocking" them -- or if we do, it's not for compatibility breaking reasons.

We may have mentioned SFML 3 in some of those issues / PRs, yes, but it is probably more related to the schedule than to the versioning rules.
Laurent Gomila - SFML developer

Paul

  • Jr. Member
  • **
  • Posts: 79
    • View Profile
Re: Unfreezing the API
« Reply #7 on: January 25, 2018, 11:57:28 pm »
.. not many companies/studios use SFML for producing games ..

It's not SFML failure.

1) More than 10 years game companies completely ignore 2D style and mainstream is 3D. Maybe Paradox Interactive is an exception and there is few games from Ubisoft like South Park

2) If you have 2-3 full time job programmers, they can create own 2D engine fast

3) You can't compete engines like Unity because some part of people want dev games in easy way (no matter if it means poor performance and another issues)

Then it does not matter whether the creator of Factorio uses SFML, SDL or Allegro..

eXpl0it3r

  • SFML Team
  • Hero Member
  • *****
  • Posts: 10394
    • View Profile
    • development blog
    • Email
Re: Unfreezing the API
« Reply #8 on: January 26, 2018, 12:04:22 am »
The thing I can certainly agree on is, that the SFML Team has missed many opportunities by not being active enough and not planning enough for SFML 3.

Beyond that, SFML has been doing quite well and my opinion the most important key to that was, ease of use and stability.

You can take any example you find out there, build it and run it with the latest SFML version. Maybe you find some deprecated functions, but that's it. If SFML had small API changes every release, it would make it a lot harder to notice the changes, find the alternatives and each code example you find will be forever fixed to a specific version.
You're right that there was certainly some thoughts about potentially attracting more "companies", it was in my memory never just about this, but mostly about stability, about knowing that if I use SFML 2, I use this API.

Having been involved or just observed a lot of discussion on how to make SFML's development better, I have yet to find something that is really better than what we have right now. People wish to take shortcuts, but don't consider the costs of these shortcuts.

- "Less API discussions": This is really the core of SFML. Everyone who ever used SFML will tell you how easy the API is to understand and use, and how well it's documented. We're holding us to higher standard than most libraries out there and good API doesn't just "happen", but it's meticulously crafted through... discussions.
This also directly plays into the previous point of stability. Having a stable API makes you spend more time on discussing a good API and not just something that "I need right now" or "that fits my use case". It's a painful process and unfortunately many devs have no patience for it, leaving us with a pool of bitter devs because their idea, wasn't immediately implemented, more on this later.

- "Merge things quicker": This is a point many simply stop at "if things get merged quicker, the development is faster" and don't consider any consequences this has.
I agree that master doesn't have to be super stable, that master can contain bugs, but I don't agree in the idea that if we merge untested or not fully implemented features and bugfixes down into master, that we will be able to develop quicker. When your code ends up just one big pile of spaghetti from various half-finished features and bugfixes that made things just worse, it becomes an maintainable mess.
Cool, your feature got merged already, but who is going to go in and merge all the mess that it created? Many devs have provided or demanded something only to immediately vanish afterwards.
Having bugs and features clearly separated in branches, it allows us to work on them, review them, break them and test them independently from each other and if a developer doesn't have time right now to continue, a branch can be stalled and doesn't need immediate attention due to it breaking stuff on master.

- "More planning, more roadmaps": As my opening statement said, I do agree that more should have been done earlier. At the same time, I don't really see much activity in that regards. I've started a SFML Development sub-form with many topics to discuss. The participation is been quite thin and I haven't gotten any requests for discussing a new/next topic, nor has anyone else of the SFML Team felt the need to start a new discussion.
Besides that we have the GitHub milestone showing what is more or less planned for the current release. We also have a GitHub project that shows some more of the current status of tasks.
My response to an outcry of "there's no focus", I've created a thread on what our current focus should be. I've even made it available on IRC, Twitter and Reddit, but the influx of participation is still missing.
Planning and roadmaps only help and work, if there are actually people willing to participate and move things forward and not just sit on their own topic, complaining why it's not being taken care of.

I'm sure there are some more ideas people have mentioned, but I currently can't think of anything more.


This brings me to the topic of participation.
Again I need to preface that the SFML Team is a bad example of participation and engagement overall. I do know that, but since there's no money involved, there's no way to make people move, right at the time when you want them to move.
However, there are also some archtype of people I see and who don't help the community in the long run.

- "I demand feature X": Luckily this has toned down quite a bit, but you'll see them every now and then. Someone runs into a problem with their game/application and since it would so nicely fit them, if SFML did the work and implemented what they wanted, why don't we demand this feature.
Often quite agressive in tone, with no real use case beyond "I could use it", they post on the forum. We start asking questions, ask about use cases, ask about considerations, ask about why this would fit SFML, why it couldn't be done differently.
But since the user just wants the feature, without actually doing any work or answering questions, they take the questions as attack, they get angry and if they are young, they'll go on some other forum or chat and tell them, how bad the SFML Team is and how they can't even add a simple feature and that's the end of the story.
Long story short: If you make a feature request, we expect you to invest some time as well. If you're not willing to do so, the feature request will just die, as there's a lot of other things to do with higher priority.

- "Here's a feature, plz merge": Providing implementations is already a good step, but it's really just the first step. If you wanted to implement that feature anyways and just thought to share it with upstream, then that's fine and we really appreciate it, but your expectation then can't be, that it will just be merged without any considerations or discussions. And discussions mean, you as a contributor will be involved too.
If you don't engage in discussions or check for current status when there's no obvious progress, you shouldn't be surprised when something doesn't "just happens" to get picked up and given the final touches. If our backlog was empty and that was the only thing in it, then sure, that certainly would be strange, but there's always more to do, than we have the people to do it.

- "I can't believe X hasn't been implemented yet!": Do we like that there are issues or feature requests that are multiple years old? Of course not! But sometimes it feels people are just disillusion, thinking that we have nothing else going on, than to work on SFML and fix everything.
But more importantly, by just stating your astonishment or even being sarcastic about it, doesn't help fix the problem. We're not different from you, beyond that we decided once up on time to really invest a bunch of time into SFML. Get down from your high horse and get your hands dirty by starting discussions, by suggesting implementations, by doing implementations, by testing existing implementations, by fixing existing implementations, and so and so forth.
Don't expect the SFML Team to do all the work for and don't think your snarky comments will motivate us to work on something, it in fact is more demotivational.

- "My PR isn't moving forward!": This can have many origins and it's often our fault for not moving forward on something in time. But at the same time, I also see many PRs of people, who vanish after a while, only to come back and complain about it angrily.
Sometimes it happens that a PR might just slip off everyone's mind and nobody went through the backlog in a while to check. Just make yourself heard, ask for feedback, potentially provide more information or some example code. Don't be annoying though, asking for an update every day or even crazier multiple times a day, won't get you anywhere and might get your PR locked for further comments to stop the spam.
If your PR ends up in the review and testing state, it doesn't mean, only the SFML Team can do this. If you review the code of some other PR, you're comments will help find issues and it will often makes things get merged quicker. Again, misuse of this, by simply approving every single PR, will just destroy your reputation and not help anyone.
Also testing your own PR is a good idea. Write a minimal example, make it available to everyone. If you're on Linux, maybe try a different window manager or maybe a different distro. Ask someone you know has the same issue or is excited about the same feature to help test the code.
Testing is one of the reasons many PRs stall for a long time. If nobody can tell me that the code actually does what it's supposed to do, I can't just merge it and hop for the best.

Anyways, I think my post has grown way to big and I've repeated things many times in the post itself and from previous similar discussions.


Summary

Just making the API "anything goes", doesn't guarantee more contributions, it doesn't just solve the current issues and opens new challenges in such a way, that I don't see the benefit beyond "it looks good on paper".
What I like to see is a more engaged community, which discusses important topics, writes code, reviews code and tests code. Not just their own code, but everyone's code.

What I take away for myself is:
- Take less about what can't be done and more about what needs to be done first to get things done.
- Start planning more intensely what needs to happen to get from here to SFML 3.
- Be more active and less reactive.


PS: For everyone who has missed it, the unstable branch has already work done towards SFML 3 and C++1y support.
Official FAQ: https://www.sfml-dev.org/faq.php
Official Discord Server: https://discord.gg/nr4X7Fh
——————————————————————
Dev Blog: https://dev.my-gate.net/

Hiura

  • SFML Team
  • Hero Member
  • *****
  • Posts: 4321
    • View Profile
    • Email
Re: Unfreezing the API
« Reply #9 on: January 26, 2018, 05:51:55 pm »
Thanks for sharing your thoughts, all of you. It's good we discuss this. :)

since there is less contributions from the team now, we have to rely on more contributions from within the community.

Yes, this is true. And I think everybody in the team is ready for more and more good contributions from the community.

Sharing my feelings on that, any why I've put emphasis on good: a while back we created this document as an attempt to help contributors shape their feature requests & bug reports. Some people obviously respect this document and inject good energy in this project, but I always have the feeling they are very few and instead -- as eXpl0it3r's summarised -- we get a lot of, according to my interpretation of good and bad, low quality contributions, or even noise.

Now, dealing with this is not an issue for some people. For me, however, it's something seriously frustrating, and for that I'm extremely grateful to the one(s) who invest so much of their time to triage freshly open issues.

- "More planning, more roadmaps": As my opening statement said, I do agree that more should have been done earlier. At the same time, I don't really see much activity in that regards. I've started a SFML Development sub-form with many topics to discuss. The participation is been quite thin and I haven't gotten any requests for discussing a new/next topic, nor has anyone else of the SFML Team felt the need to start a new discussion.
Besides that we have the GitHub milestone showing what is more or less planned for the current release. We also have a GitHub project that shows some more of the current status of tasks.
My response to an outcry of "there's no focus", I've created a thread on what our current focus should be. I've even made it available on IRC, Twitter and Reddit, but the influx of participation is still missing.
Planning and roadmaps only help and work, if there are actually people willing to participate and move things forward and not just sit on their own topic, complaining why it's not being taken care of.

I'm sure there are some more ideas people have mentioned, but I currently can't think of anything more.

For the record, I have a few more ideas that I'd like to discuss regarding inputs, but I'd rather wait for the scancode thing to be properly wrapped up before starting more discussion and further diluting the time of people (such as the crucially needed testers). Does mystery motivate people to test scancodes & provide the linux missing implementation? (:

PS. I also share the concerns about stability: it can only make SFML more attractive IMO.
SFML / OS X developer

FRex

  • Hero Member
  • *****
  • Posts: 1841
  • Not doing gamedev/using SFML lately
    • View Profile
    • Email
Re: Unfreezing the API
« Reply #10 on: January 26, 2018, 07:02:27 pm »
At this point I'm really unsure if I should even comment in here since I'm so indifferent.

If I need to learn some OpenGL or use WinApi or a separate library for something it's not really a problem for me, I even like to tinker around like that. I also don't feel limited by SFML and I quite like SFML 2 API and wouldn't like any huge breaking changes, I'd much prefer a rolling incremental release to keep what I know relevant and let me slowly play with more and more features (like the vertex buffers binary1248 once mentioned).

I also have no problem with SFML being used for toy projects, by hobbyists, beginners, etc. - none of the SFML developers do it for money or fame so that's not a problem or threat to SFML itself. I think that's pretty much on par with opinions at large - SFML is easy, nice, C++, SDL has more backing (i.e. Valve once advertised it when they made their Linux push), is C, has bindings to many languages, etc.

I don't think there is something wrong with being accessible to new people. I myself picked SFML up only where I was starting to learn C++ after 1 year of Delphi Pascal and learned C++ along with SFML itself (first 1.6, then 2). I also don't think SFML has any actual hard limits that'd prevent many of the normal small indie games that are made in SDL from being made in SFML instead.

That said I guess I'd also not mind there being a bit more courage in updating and experimenting with the API itself.
I'm not using SFML nor doing any gamedev lately.

Foaly

  • Sr. Member
  • ****
  • Posts: 453
    • View Profile
Re: Unfreezing the API
« Reply #11 on: January 27, 2018, 03:41:37 pm »
Puh!
What a wall of text :D I will try to respond to it all while staying short.
First of all I would like to thank everybody who is engaging in this discussion. I think it is very good we are having it!
Also another point I would like to get out of the way, because I think it was missunderstood, is that I'm definitly not interested in having big companies run SFML. I am all for hobbyiest and independent developers. The point I was trying to make was about assumptions that have turned out false and that I think adaption to the changes is needed.
Reading over all this I think I have to state again, that I do agree with most of the things that were brought up. Obviously unfinished things should not simply be merged. In order to find good solutions to a problem or a good design for an API discussions are absolutly necessary. I do enjoy a stable API and I don't want a new core part in every new minor version, but maybe we can find a way to be less pedantic about it and allow for smaller changes.
As eXpl0it3r explained the community participation comes in varying different qualitys which doesn't make work easier and obviously can not just fill in for the work of a dedicated team. You listed may reasons why development on commuity contributions stops. Thats why I say we have to make it as easy as possible for the people who are actually willing to put in work. If it's a enjoyable proccess to them, they are more likely to help again.
Maybe the plea for opening up the API was a bit of a compulsive reaction and I was trying to take a shortcut that eXpl0it3r mentioned, because it has fallen on my feet so many times. In the end what I really want to acchive is to help make SFML development quicker and make the process more enjoyable for those dedicated. Maybe there are other things to target instead, but I am sure SFML can benefit from changing come rules.

Nexus

  • SFML Team
  • Hero Member
  • *****
  • Posts: 6278
  • Thor Developer
    • View Profile
    • Bromeon
Re: Unfreezing the API
« Reply #12 on: January 28, 2018, 05:56:13 pm »
As I see it, the main occasion for breaking API changes is also a more modern C++ version. Several parts can be made simpler, more expressive, or more efficient. But except for some core part where concepts and semantics might change (e.g. how to deal with resources), a more recent C++ version would probably not have a fundamental impact on the API, in the way that someone would need to re-read all tutorials.

In that regard, considering that 3 new C++ standards (2011, 2014, 2017) have been released since C++98/03 which SFML uses, we should definitely plan a migration in the not too distant future. Maybe a good starting point is to consoldiate the feature suggestions for SFML 3, check if they are still relevant and prioritize them.

I don't think the non-breaking policy should be changed within a major version. For huge design changes that are not compatible with a deprecation policy, a new major version can be considered.
Zloxx II: action platformer
Thor Library: particle systems, animations, dot products, ...
SFML Game Development:

aggsol

  • Newbie
  • *
  • Posts: 24
  • C++
    • View Profile
    • My Github Account
Re: Unfreezing the API
« Reply #13 on: March 23, 2018, 03:25:02 pm »
So what kind of breaking changes are you talking about? I would remove anything thread related and recommend the usage of std::thread, IMHO even sf::VertexArray would go but what actual API would break when adopting C++11 and C++14 features? Examples? There is a lot that could be already used in work for SFML 3.0

* Move semantics would be added but wouldn't break anything
* Using enum class
* constexpr, auto, lambdas

eXpl0it3r

  • SFML Team
  • Hero Member
  • *****
  • Posts: 10394
    • View Profile
    • development blog
    • Email
Re: Unfreezing the API
« Reply #14 on: March 23, 2018, 04:13:53 pm »
The breaking changes aren't really in relation to newer C++ standards, but due to changes for SFML. Removing deprecated stuff. Changing some API setup. Different basic setups in SFML.

The point is that code written with SFML 2.1 should still compile with SFML 2.5, that's why we can't just go about changing or removing things
« Last Edit: March 23, 2018, 04:29:18 pm by eXpl0it3r »
Official FAQ: https://www.sfml-dev.org/faq.php
Official Discord Server: https://discord.gg/nr4X7Fh
——————————————————————
Dev Blog: https://dev.my-gate.net/

 

anything