As soon as you're moving a map, apparently what you want is a Camera implementation using sf::View.
What I'm doing at the moment, I draw tiles (32x32 pixels) in chunks of 50x50.
I always draw them at the same position, but I move the sf::View around when the player moves.
That way, it's fairly simple to keep track/render everything, AND, when I know a chunk is too far away to be drawn on the screen, I unload it