I noticed that I was getting around 50fps and tracked that down to the joystick prosessing code in the event polling. I had a joystick connected to my computer about an hour prior but it was disconnected. I found when I changed sfml to not poll the joysticks it went back to normal.