It's a little confusing.
First you mention a problem with OpenAL32.dll, which is a SFML dependency that cannot be linked statically (because of its license), there's nothing you can do about it (*). This also means that you link to sfml-audio.
Then you talk about SFMLConfig.cmake, which you do not use since you compile SFML as part of your own sources (SFMLConfig.cmake is used to find a pre-compiled SFML release, through a call to find_package). "Should this work" -> but what? You never say what you intend to do with it.
And finally you show something that "works", but you don't explain anything, all we see is that you don't link to sfml-audio anymore so you hide the OpenAL32.dll "issue" you had in first place.
(*) In fact there is, but let's keep things simple