Depends on your architecture. Of course, the bruteforce approach "put 4 vertices into the vertex array instead of using a sprite" usually works (given they use a common texture), but if the objects are persistent for a longer time, it might be worthwhile to keep them in the array. I mean it doesn't hurt to use sf::VertexArray from the beginning, code will just be slightly more verbose. But if you design it cleverly, you can abstract everything away and handle vertex array manipulations in one place.
Out of interest, how many objects are we talking about?