The vector's destructor will invoke the destructors of all the objects it contains.
sf::Sprite is a class, so it will have a destructor that cleans up everything. So std::vector<sf::Sprite> is fine.
A pointer to sf::Sprite is just a pointer. In terms of construction and destruction, it may as well be an int or a float or something. The vector will free the heap-allocated memory it was using to store the int/float/pointer as usual, but it has no way of knowing that it needs to go destruct the things pointed to by the pointers. So a vector of raw pointers is only okay if those are "non-owning" pointers that aren't supposed to clean up the actual sprite objects, but only point to them (of course, in modern C++, a non-owning raw pointer is the only kind of raw pointer you should ever have).
This is the reason why "smart pointers" exist. A smart pointer is essentially a class that contains a pointer, with a destructor that deletes the thing it points to. So a vector of unique_ptrs or shared_ptrs to Sprites would clean itself up just fine.
Of course, even though smart pointers will work, most of the time there won't be any reason to use pointers of any kind. Always use ordinary objects or references if you don't have a good reason to get pointers involved.
Note that there are some languages in which a "raw pointer" is able to "clean up after itself" and you don't need to worry about any of this. But as far as I know, those are all garbage collected languages. Obviously, C and C++ (normally) do not have garbage collection.