Well, actually, the Debug packages are useful if you want to debug your own code. Because depending upon whether you build in debug mode or release mode, different versions of the runtime library are used. Mixing runtime libraries is possible but not recommended, so if you want to build your own code in debug mode, it's best to use a version of SFML which was also built in debug mode.
However, like I've said, it's possible to build a library in debug mode (i.e. using the debug runtime libraries), but with no debug information for the library itself. Just remove the /Zi flag.