If you can, you should consider it. It's not automatically ignored - the draw calls are still made - but if it's outside of the window/viewport, it'll not be shown.
For things where there isn't too much outside of the window, this shouldn't be so necessary but if it's, for example, a large map where you only show a small part of it, consider "culling" the parts of the map outside of the visible area.
Feel free to check out
Cheese Map, an automatic map drawer with automatic culling designed for SFML and for just this situation.
