mhhh, that's quite strange.
So you're saying as long as you don't call your window's Display()-Method, the window won't crash your X11?
I'm not quite sure, but as SFML is using openGL for displaying content within the window, maybe sth is wrong with your system's configuration of openGL (or your graphics card is not capable of openGL, what I doubt, though...)