OpenGL has the following texture wrapping modes: GL_CLAMP_TO_EDGE, GL_CLAMP_TO_BORDER, GL_MIRRORED_REPEAT, GL_REPEAT and GL_MIRROR_CLAMP_TO_EDGE.
Clamp to edge does what you are seeing, the edge pixels are extended infinitely around the texture.
Clamp to border lets you specify a single colour (like black) to show around the texture.
Repeat makes the texture tile (so it repeats).
SFML however only supports repeat and clamp to edge. It doesn't have clamp to border. (At least in SFML 3, which I'm using).
I don't know if there's an easy way to trick it besides editing texture.cpp to add the border mode in.