The real issue is that X11 refuses to resize the window if the user is still dragging it. This is how the control flow works:
This all happens in milliseconds:
X11 performs a window resize when a user drags the window borders. Then X11 puts a resized event onto the event queue. SFML puts that event onto its own event queue. The program polls for events from SFML's event queue. The program notices that the resize made the window too small; therefore, it will send a set size to SFML. SFML sends the set size parameters to X11. X11 tries to set the size of the window, but can't because the user is still holding onto the window border.
If we could force the user to not grab onto the border, the set size function would work seamlessly. Hypothetically speaking if the user lets go of the window between the time the window actually gets resized and X11 trying to set the size of the window with X11ResizeWindow, it would work fine.
Overall, the real solution is to have a min size and max size function. All other window APIs do this (godot, sdl, etc...)