yes
@quasius: 18 fps in a small one layer map. There'll be more layers, bigger maps and some movement.
Re bigger maps: That shouldn't make a big difference if your code is smart. Are you culling off-screen objects? If it's still slow, consider using some kind of quad-tree structure (dividing all objects in the map into "zones" so you don't even have to look at objects outside of your current "zone") instead of doing an on-screen test for every single object. If your talking about ginormous maps where memory becomes a constraint, you will need to do some kind of streaming or make the maps smaller and link them.
Again, debug is supposed to be slow. When I was working in professional game dev, I was working on an engine that effectively *didn't run at all* on debug. It sounds like you're worrying about something that really isn't a problem.
If you really think your code is running too slow, the problem likely isn't SFML. How old is your dev machine? Are you running innefficent collision detection? Did you do something stupid like pass an stl::vector by value in a critical loop? Are you misusing stl::vector by constantly inserting/removing items to/from the middle? Are you over-manipulating memory? Anyone of these things can tank your FPS, so it's really hard to answer a question like "why is it slow?"
But again, it sounds like you really may have no problem at all other than not understanding how slow debug mode is.
Edit: If you're really worried about this stuff, get a hold of and figure out vTune. It can profile your code and tell you what's eating all the clock time. You shouldn't need to worry about the 3DAPI side of things (which has it's own profiler apps) since SFML is taking care of that.