Maybe you should create a class DrawableTransformable : public sf::Drawable, sf::Transformable and inherit your classes from it, and then make vector of std::reference_wrapper of DrawableTransformable.
You don't need to have Dude::Draw function, because RedDude::draw function will be called because it's virtual.
That's because you're trying to push_back unique_ptr<Dude> while World expects to have something convertible to sf::Drawable&. Just do this:World.push_back(*Dudes[0]);
What causes that exception? Btw, I just noticed that you don't initialize red_dude_tex.
No, you're not supposed to have anything else in base class as far as I can tell.