First of all, you set the maximum frame rate to be 60. It's not going to get faster than that. That won't make it refresh noticeably slowly, though. Under 20 fps is where it gets unacceptably slow.
Secondly, you don't assign an image or anything to the sprite. If you want just a plain colored square, perhaps you should use sf::Shape. I haven't seen this used before like what you are doing with the sprite, but maybe it works.
Also, you might want to use sf::Input for getting input. It's often easier, especially for key presses and mouse location.