Coming from a networking background, I've personally set up a system using 'blocks' of clients. I usually set up an array of sockets in a struct (perhaps with some other header-type info about the block), and then have an array of those 'blocks' (along with some more optional header info).
The total number of clients you can support would be the number of blocks times the number of sockets in the block.
Each block is then given a thread. When you add/remove clients, check if the size is coming from/going to 0 and suspend the thread.
Make sure to set the clients to non-blocking, so you can just loop in that thread checking for each client.
This has been proven and works; however, not for SFML. I imagine the process would be similar, if not easier.
Just my 2 cents. Cheers!