700 frames per second? That's your problem, you are running at full throttle with the main-loop. Because you do that it will starve the thread processing events in the background resulting in a huge delay in the events getting registered since the event thread don't get any opportunity to run since your main thread is busy working it's ass off trying to pull events that will never be registered.
Like I said before, easy fix is setting a framerate or just simply calling sf::Sleep.