This is apparently a very specific behaviour on Windows: after sending to an unreachable port with a UDP socket, reading on this same socket returns a WSAECONNRESET status. So the selector marks it as "ready" (because the operation won't block), but there's nothing to read, just an error code to retrieve.
I don't know what should be done about this. There's a flag that I could use to disable this behaviour, but I'm not sure it's a clean solution. I have to read more about it.