From a performance point of view, the current implementation is so easily optimizable that if you try profiling your code, setting the view shouldn't even show up in the hot spot list unless your compiler did something wrong. If SFML had C++11 support, the move would make it even less significant than it already is. In this respect, your suggestion would indeed fall into the category "premature optimization".
From an interface design point of view, the current implementation follows the idea that you should really only be able to mutate an object through its mutator methods. Since the sf::View is part of the sf::RenderWindow, mutating it should only be possible through the sf::RenderWindow interface. And if you follow convention, getter methods should be const and thus only return const references if not PODs. Getting a view in order to change it in its owner doesn't convince me of a clean interface even if it saves those few lines of code, which really has to only be done once in a well designed application. As many C++ experts say regarding modern C++, pass by const reference if you want to inspect the object, pass by value if the object is a sink object (only makes sense if there is move support). The same can be said about returning values as well.