Alright, did some brainstorming with binary1248 and came up with one possible solution.
Diagram first:
Let's take sf::Texture as an example:
An sf::Texture object is created and you call loadFromImage() on it. sf::Texture then calls sf::Context::create_resource() to create the appropriate texture resource and gets back a proxy object (or even better: turn sf::Context into a factory that creates objects like sf::Texture).
The resource proxy holds a weak pointer to the real resource that's being owned by sf::Context. That means that when the Context is getting destroyed, all resources will be destroyed to and the proxies reference to nullptr (sf::Texture has to check for this case, of course).
If sf::Texture gets destroyed before sf::Context, then ResourceProxy's destructor will unregister the resource at sf::Context.
This can be done simply using shared_ptr/weak_ptr or an own implementation of reference counting and especially weak references to detect when a resource got invalid.
Another option (which should be considered indepedently from the crash bug, too!) is removing the shared context stuff, which would also remove the need to wait for static resources being destroyed. Instead of having a shared contexts, sf::Window should
completely own and manage one.
I don't know all the benefits, but also for having multiple windows I don't think it's worth all the trouble.