On mac, you can completely ditch Xcode and do everything it does yourself. You'll might have to play with `install_name_tool` (I don't remember exactly to be honest) when creating the .app folder & its internal components. To ship your app, it's pretty much a matter of copying the dependencies into your app bundle.
If you created an application bundle for SFML project with the Xcode wizard, you can go to your project build phases and have a look at the script created for you to get some inspiration. You can also have a look at all the steps Xcode does to get the full picture.
Or, I guess, you could use cmake on all platforms and not worry about all that. I understand you want to learn how it works, but once you got the idea you probably want to have something easy to maintain. I'll let you judge if it worth it for you or not. ;-)