The frequency of the processor does not determine the performance, and that's not even mentioning core count. You also have to take into consideration other factors like the OS, graphics device, background processes, etc. Your computer could just be really slow or something's wrong with your game. It's unlikely the fault of SFML.
It'd be helpful to know:
- The language(s) you're using
- The compiler (and version) you're using
- The version of SFML you're using
- The compiler options
- The computer's specs (CPU model, graphics device model, OS, etc)
- Benchmark information (use SFML's timers to see the average time spent rendering, average time spent updating logic, etc)
- You're games source
These are a bit general, but based off of wild guesses you could try:
- Updating the drivers
- Stop background tasks
- Turn on compiler optimizations
- Improve your code (not be applicable if it's your computer's fault, but it may be yours)