The fix seems to be to change
sfml_bind_dependency(TARGET OpenGL FRIENDLY_NAME "OpenGL" SEARCH_NAMES "OpenGL" "GL")
to
sfml_bind_dependency(TARGET OpenGL FRIENDLY_NAME "OpenGL" SEARCH_NAMES "GL" "OpenGL")
in SFMLConfigDependencies.cmake.in (just swapping the "GL" and "OpenGL" strings).
With the current code, the OpenGl_LIB variable contained "/usr/lib/libOpenGL.so" and I got linking errors to GLX.
After the change, OpenGl_LIB contained "/usr/lib/libGL.so" and there were no more linking errors.
The libGL.so file is also the one the SFML links to. The "find_package(OpenGL)" call provides both OPENGL_gl_LIBRARY and OPENGL_opengl_LIBRARY values, but OPENGL_gl_LIBRARY is the one that SFML uses for target_link_libraries.