Providing pre-built binaries for different compilers on different platforms is a courtesy on behalf of the SFML team as I see it.
All you really need is the source. Then you can build the library yourself on your platform of choice with your toolchain of choice.
If the SFML team was to provide every conceivable combination of compiler/OS this would quickly lead to a combinatorial explosion of builds and I'd rather they spend their time on improving the library rather than doing builds that any competent developer should be able to do on her own in 5 minutes.
Just imagine having to provide builds for: Windows Vista, Windows 7, Windows 8.x, various Linux distributions, multiple OS X 10.x versions, multiple iOS and Android versions, for both 32 and 64 bits, for different architectures like x86, arm, mips and more and with different compilers like Visual C++, gcc, mingw gcc, clang, the Intel compiler and more, all multiple variants/versions. It's just not doable.
Much better, that you yourself build the source on/for the platforms that you care about with the compiler you prefer, optimized for the architectures you target etc. Once you have done it a few times it doesn't take more than a few minutes anyway.