Personally I would do the following if I wanted to distribute as a .tar.gz :
Package my application so that when you extract app.tar.gz you'd get an "app/" directory with 3 subdirectories; "lib/", "bin/" and "data/".
My applications binary(s) would go in "app/bin/", SFML and other libraries my app depends on would go in "app/lib/" and other stuff like music files, graphics etc would go in "app/data/". If I had configuration files I'd probably put them in a "app/etc/" directory.
When building my app I would make sure to set my apps path to the libraries it needs to be relative to the location of the executable, using the -rpath option for the linker along with $ORIGIN, so that my app would look for its libraries in "../lib/" independently of where it is located.
This way, when the user extracts the tarball they can put the toplevel "app/" directory wherever they please. Locations like "/opt/app/", "/usr/local/app/" and "~/app/" will all work equally well - as will symlinks to the executables that the user may choose to create.
At runtime when the app needs to find its own files in "data/" & "etc/" it can just read the "/proc/self/exe" symlink to find out where it is located, strip the executable name from that path and add "../data/" and it then has a path to its data files. This works regardless of where the user moves the app to.