That's because SFML uses double-buffering. So if you clear once instead of every frame, only one of the two internal buffers will have the chosen color, and the other will keep the default one (black, in your case). And everytime you call Display() the buffers are switched, that's why you see it flickering.
So you must call Clear() everytime.