Pushing the actual sprites would be my choice though; they're tiny and most likely don't need to be on the heap.
The header information of a vector is stored on the stack, while the elements that you push back are still allocated in free store (i.e. on the heap). But of course using a vector is way better than performing heap allocations for single objects with operator new.
There doesn't seem to be any benefit to using std::array instead of std::vector here. Both are equivalent really with vector having more flexibility if something changes.
True, std::vector is indisputably more flexible, the only reason I brought it up is that using an array performs allocation for the contained elements all at once upon construction, while using std::vector::push_back(const & value_type) calls additional copy constructors for each call to push_back(), and you take a performance hit each time vector doubles in size upon reallocation. If you know before hand that you're pushing back at least three elements and are using a vector for future flexibility, it might be worth calling the fill constructor rather than calling push_back() three times, or if you do want to call push_back() first calling std::vector::reserve().