Good tutorial!
I have written something similar (ResourceManager) in my library, but it doesn't provide much functionality specific to sf::Image. In your code, I like the possibility to specify multiple directories. I think an option to disable the debug outputs to std::cout would be handy, as well as the possibility to check whether image loading was successful. But you can also leave this as an exercise to the reader
By the way, a mistake I came across: The following code doesn't compile because std::map::erase() doesn't return any iterator.
it = images_.erase( it );
Instead, you could write the code below (which is okay because map iterators remain valid for all operations except removal), however I think in that case you may also exit the function.
images_.erase(it++);