SFML community forums

General => Feature requests => Topic started by: binary1248 on January 08, 2015, 08:16:09 pm

Title: Facility to get standard paths
Post by: binary1248 on January 08, 2015, 08:16:09 pm
Currently there is no way for the user to retrieve the paths of standard locations in a cross-platform way.

One can check the environment variables using std::getenv() but often times you wouldn't be able to use their values directly without a bit more information about the target system.

As an example: On Linux systems, it might be typical to store user data in the HOME location, on Windows, there is also a HOME path, but that is not where user data is usually stored (APPDATA is used for this).

If an SFML application is installed by a user with administrative privileges and is to be used by unprivileged users, there would be no place to store per-user data since the binary directory would be read-only.

Other examples of useful paths might be: The current working directory, a directory where temporary data can be stored, the location where fonts are installed (yes... currently if you wanted to use installed fonts you would have a lot of pre-processor in your code).
Title: Re: Facility to get standard paths
Post by: Laurent on January 08, 2015, 08:32:03 pm
Possibly useful reference:
http://doc.qt.io/qt-5/qstandardpaths.html#StandardLocation-enum
Title: Re: Facility to get standard paths
Post by: Hiura on January 08, 2015, 08:48:31 pm
Something that partially answers your needs, old but still working I guess: http://websvn.tuxfamily.org/filedetails.php?repname=hblog%2Fsvn&path=%2Fcpd%2Ftrunk%2Fcpd.hpp
Title: Re: Facility to get standard paths
Post by: Ixrec on January 08, 2015, 08:50:51 pm
Awesome, I always assumed this was out of SFML's scope.

Could we also consider the executable directory? The current working directory isn't that great for (portably and reliably) loading resources, which is what most basic SFML programs need the filesystem for.
Title: Re: Facility to get standard paths
Post by: Hiura on January 08, 2015, 08:58:42 pm
Loading resources is a complex issue: some OSes expect a specific kind of file hierarchy (e.g. OS X with bundled application, Linux with standard paths, mobiles OSes with sandboxing, ...). Then you have a different directory whereto save config, and another one to store temporary data... Before we commit to provide such functionality, as Qt does, let's make sure we understand what's at stake.  ;)
Title: Re: Facility to get standard paths
Post by: Jesper Juhl on January 08, 2015, 09:50:42 pm
Currently there is no way for the user to retrieve the paths of standard locations in a cross-platform way.
I agree. This is actually a fairly common need. I've had to implement things like this for various personal and work related, cross platform, projects multiple times - it would be nice if the SFML system module provided some of the basic stuff.

One can check the environment variables using std::getenv() but often times you wouldn't be able to use their values directly without a bit more information about the target system.
Environment variables contain a lot of useful information, but even on POSIX systems what is available and what they contain differs. As you say, it is essential to know what system you are on before using them. Some could probably be used in OS specific implementation of SFML functions though.

Other examples of useful paths might be: The current working directory,
Yes, the current working directory is useful, but is also one of the easier ones that might not necessarily need a SFML wrapper (although it might be convenient) since I believe all supported platforms (even Windows) supports the getcwd() function. But GetCurrentDirectory() might be a better choice on Windows, so yeah, maybe it ought to be wrapped.

a directory where temporary data can be stored,
Doesn't getenv("TMP") get you this on all supported platforms? I know Windows prefers TEMP, but it still sets/supports TMP as well IIRC.
But yes, I agree, a useful path for apps to know regardless of how SFML obtains it.

the location where fonts are installed (yes... currently if you wanted to use installed fonts you would have a lot of pre-processor in your code).
Certainly also useful.

Another path that may be useful to an application that we could consider providing is the path to the executable currently running. That's often useful to find resources located in a directory relative to your executable; On Linux (and Android I'd assume) you'd read the target of the /proc/self/exe symlink, on Windows you'd call GetModuleFileName(), on OS X _NSGetExecutablePath() gets the job done, FreeBSD has a sysctl for the purpose etc etc - all systems provide a way to get the path but they are all different which makes this an obvious candidate for something SFML could provide.
Title: Re: Facility to get standard paths
Post by: Laurent on January 08, 2015, 09:54:07 pm
Quote
Awesome, I always assumed this was out of SFML's scope.
As a multimedia library, SFML deals with resources, and resources needs paths. It's more and more important these days where everything is properly categorized and dispatched on OSes ("everything in the app directory" is an old pattern), especially on mobile systems where you cannot even work at all in the executable's directory.

Quote
Could we also consider the executable directory?
Why not.
Title: Re: Facility to get standard paths
Post by: Jesper Juhl on August 06, 2015, 06:47:12 pm
This topic is getting a bit old, but if there is still interest in this I could cook up a proof-of-concept implementation for (at least) Linux and Windows for this and we could then continue discussions from there??
Title: AW: Facility to get standard paths
Post by: eXpl0it3r on August 06, 2015, 07:09:41 pm
I'm still interested. ;)

With a real implementation we might actually see the pros and cons of the API more clearly.
Title: Re: Facility to get standard paths
Post by: Jesper Juhl on August 06, 2015, 07:12:09 pm
OK. I'll try to cook up something over the weekend/start of next week.
Title: Re: Facility to get standard paths
Post by: Arvamer on August 06, 2015, 10:35:29 pm


As an example: On Linux systems, it might be typical to store user data in the HOME location,

But please use $XDG_DATA_HOME or if not set, $HOME/.local/share. Every Linux gamer say this :).

Title: Re: Facility to get standard paths
Post by: SeriousITGuy on August 27, 2015, 12:42:51 pm
As I ran into an issue with this recently I also support this request, it would be more convinient to have a SFML API for the basic directory stuff. And an additional useful feature would be to get the system time in a plattform independent SFML API, as I plan on displaying the actual system time ingame (like Diablo 3 e.g. just informational).
Title: Re: Facility to get standard paths
Post by: Laurent on August 27, 2015, 01:24:39 pm
Quote
And an additional useful feature would be to get the system time in a plattform independent SFML API
No need, you can already do that with the standard library.
Title: Re: Facility to get standard paths
Post by: Hapax on August 27, 2015, 01:28:29 pm
I'd like it if SFML could handle files better too. Not sure if it's in its scope but maybe it should slightly expand its scope to allow for this because of how important it really is  ;)

an additional useful feature would be to get the system time in a plattform independent...
I did a really basic version of something like this here (https://github.com/Hapaxia/Kairos/blob/master/Kairos/BasicClock.hpp). It uses <chrono> but doesn't take into account fluxuations such as time offsets (Summer Time, for example).
Title: Re: Facility to get standard paths
Post by: Laurent on August 27, 2015, 02:44:04 pm
Let's stay focused on the topic, ie. standard paths ;)
Title: Re: Facility to get standard paths
Post by: Mario on August 28, 2015, 11:43:31 am
It's on my list once some current issues (both SFML and work code) are fixed. :)
Title: Re: Facility to get standard paths
Post by: silverweed on September 15, 2015, 12:31:31 pm
Is there already any idea how would this be implemented in the API? Global namespaced variables like sf::Path::Home? Also, would we also need some basic functions to check if a path exists/create a directory/etc in a portable way?
Title: Re: Facility to get standard paths
Post by: Mario on September 15, 2015, 03:25:34 pm
You'd most likely construct a "sf::Path" object, which would essentially be a string representation of a path providing some additional features, such as concatenation, access to attributes, etc.

Some mockup stuff, nothing decided or even set in stone yet:

// On Windows this would produce something like "C:\Users\Mario\Documents\My Awesome SFML Game"
// On Linux this would be "~/My Awesome SFML Game"
sf::Path localPath = sf::Path::Documents + "My Awesome SFML Game";

// Load a texture from below working directory
sf::Path assetPath = sf::Path::WorkingDirectory + ".." + "data";
texture.loadFromFile(assetPath + "texture.png");

// Iterate over all file system entries
for (auto &filepath : (localPath + "saves").getFiles())
{
    // ...
}
 
Title: Re: Facility to get standard paths
Post by: Nexus on September 15, 2015, 04:10:12 pm
Variables are a bad idea, they limit the initialization to static functions run at program startup. Use functions and you have the full flexibility: either compute paths on-demand, or at program startup, or on first use.

Furthermore, I think you're underestimating the complexity of a sf::Path in a boost::filesystem::path style manner. Even seemingly simple operations like reducing a path (./dir/.. to .) can hide a lot of pitfalls, see here (http://stackoverflow.com/questions/1467274/path-simplification-reduction) for example. Things like symbolic links and their platform difference add further to the complexity; in the end SFML would need a filesystem library as a backend. As long as the class doesn't provide much more than string concatenation, we can as well use strings directly.

Jesper Juhl, you said you could provide a proof-of-concept, do you still have the time and motivation? :)
Title: Re: Facility to get standard paths
Post by: Mario on September 15, 2015, 07:33:30 pm
Meh, I'm too spoiled from being able to write getters. :D

Path complexity: Haven't taken a look at boost's implementation, but similar to other SFML stuff, I think just the most common basics should be enough. It doesn't have to be a full fledged out implementation where you're able to set or access every tidbit.

Regarding path alternatives resolving to the same real path (like using "." or ".."): On Windows there are API calls to resolve all of these. I'm not sure about the other systems, but there should be something like that I guess.
Title: Re: Facility to get standard paths
Post by: dabbertorres on September 15, 2015, 09:10:28 pm
I made my own Path sort of class implementations. Path reduction was not that hard, as Mario said about API calls, both Windows and Linux have functions to get the "realpath" (as Linux calls it) from a path string.

Linux (https://github.com/dabbertorres/gfs/blob/master/src/LinuxImpl/LinuxPath.cpp#L96-L103)
Windows (https://github.com/dabbertorres/gfs/blob/master/src/WinImpl/WinPath.cpp#L99-L107)

If inspiration is needed/desired/wanted/etc, here's mine (https://github.com/dabbertorres/gfs). It's not quite complete. I think I didn't get into much permissions stuff on Windows, and I don't have an OSX machine, so that is lacking.

Here's an example of using it:
(click to show/hide)

PS: Just to make it clear, I'm not intending to step on anyone's toes, or force my project on to you guys. I'm just offering another example of how someone else did it, and a place to get guidance/etc from!
Title: Re: Facility to get standard paths
Post by: Nexus on September 15, 2015, 09:25:41 pm
Maybe asked differently: why not a string, why do we need a dedicated path class?

Keep in mind that we're discussing about getting standard paths here, not about providing filesystem functionality like resolving symlinks, checking the type of a file, etc.
Title: Re: Facility to get standard paths
Post by: Jesper Juhl on September 15, 2015, 09:32:28 pm
I haven't been able to find the time to implement anything yet (although I've implemented all of these in the past for AiX, Solaris, Linux, OS X and Windows, so I know it is possible), but originally I was thinking of something like:
string getExecPath(); // full path to executable (/proc/self/exe)
string getExecDir(); // directory containing executable
string getCWD(); // get current working directory
string getHomeDir(); // users home dir $HOME or %HOME%
string getTempDir(); // /tmp, /var/tmp or %TEMP%
string getDataDir(); // /var/app/foo or %APPDATA%
string getRootDir(); // C:/ or /
 
Title: Re: Facility to get standard paths
Post by: AlexAUT on September 15, 2015, 11:44:33 pm
On Android you would only have 3 path which are useful.

I don't know what you should do for the other paths listed in Jesper's last post, because I'm not even sure if you have read access of the root folder etc. (can't test it at the moment).

So you should definitly keep that limitation in your mind when deciding the final API



AlexAUT
Title: Re: Facility to get standard paths
Post by: silverweed on September 16, 2015, 09:36:29 am
Variables are a bad idea, they limit the initialization to static functions run at program startup. Use functions and you have the full flexibility: either compute paths on-demand, or at program startup, or on first use.
That's true, but since the idea is to get "standard" paths, how complicated should it get to initialize those paths?

I think the question which needs to be sorted out first is: should we just provide those paths (either as objects, variables or getter functions), without bothering to check if they actually exist on the machine SFML is installed on, or should we also ensure the paths we retreive from the API are always valid?

Of course from the user's viewpoint the second choice is the obvious one but, as you said before, it can get pretty complicated to implement, while the first option would be slightly less than trivial if I'm not mistaken.
Title: Re: Facility to get standard paths
Post by: Laurent on September 16, 2015, 10:02:12 am
Quote
That's true, but since the idea is to get "standard" paths, how complicated should it get to initialize those paths?
Since you most likely have to do system calls, doing them at global startup may be impossible on some OSes, like Android and iOS. We must use functions to keep maximum flexibility for implementation.

Quote
I think the question which needs to be sorted out first is: should we just provide those paths (either as objects, variables or getter functions), without bothering to check if they actually exist on the machine SFML is installed on, or should we also ensure the paths we retreive from the API are always valid?
Since we don't want to turn SFML into another filesystem library, I'd say just provide the paths.
Title: Re: Facility to get standard paths
Post by: silverweed on September 16, 2015, 11:23:15 am
Since we don't want to turn SFML into another filesystem library, I'd say just provide the paths.
Yeah, seems the best choice to me too.
Title: Re: Facility to get standard paths
Post by: Klaim on September 19, 2015, 03:32:30 am
Quote
I think the question which needs to be sorted out first is: should we just provide those paths (either as objects, variables or getter functions), without bothering to check if they actually exist on the machine SFML is installed on, or should we also ensure the paths we retreive from the API are always valid?
Since we don't want to turn SFML into another filesystem library, I'd say just provide the paths.

By the way, the unicode mess on Windows might be a major issue for providing any kind of path using just std::string. The basic issue is that if the path have unicode characters, they are provided by the Windows API as either UTF-16 in wide chars, or as ANSI something in chars with non-ansi characters replaced by '?'.
Therefore, you need to use the UTF-16 way on Windows to get something useful, then you need to decide what is the encoding of your output std::string, then convert from one to the other.
The solution boost::filesystem took is to keep the internal representation of the system (UTF-16 on windows) to avoid conversions as much as possible, but then if you as a string from it it will do a dumb conversion OR use a codecvt you provide, which migth do the right thing (or not). Unfortunately there is a strange behaviour where no conversion happen if the type used for storage (char/wchar_t) is same between the two encodings (which mean that a linux OS which is not UTF-8 might have no conversion, or I'm wrong, which might be possible for these kinds of details which I couldn't manage to properly check so far).
In my dayjob I had to work on these kind of issues recently. We chose that any std::string in the API is expected to be UTF-8. We have a path type which currently uses boost:::filesystem::path inside (it's open source by the way but I wouldn't call it a perfect example), then do the conversion when you need a string (so UTF-8) depending on the plateform we are on, so that whatever the boost representation, we always end up with UTF-8.

I don't know what is the policy of SFML regarding encoding of API-provided strings, but if there is none, providing a system paths api will requires that.