Interesting. :) Looked into the Pi differences myself last year but didn't really bother to try it since it can get quite some PITA as you'd essentially have to manage the surface yourself, update/scale/move it with your window, etc.
Would be nice being able to omit X as a whole, considering it's not really used anyway. :)
Edit:
Why the "SFML_BCMHOST" define? Couldn't you hardcode it specific to the Raspberry Pi as a platform? I'm quite sure there's some CMake variable to check.
Edit 2:
According to Stack Exchange (http://raspberrypi.stackexchange.com/questions/754/how-can-i-detect-that-im-compiling-for-raspberry-pi) you should be able to use the following in CMake:
include(CheckIncludeFile
)CHECK_INCLUDE_FILE
(/opt/vc/include/bcm_host.h BCMHOST
)