Just an update to what I've found so far. I was able to link statically with SFML on Android after I followed these steps.
I changed BUILD_SHARED_LIBS in the cmake file to be false (it is currently hard coded to true)
After building, there are -s variants of the libs in the ndk directory. e.g. sfml-system-s.a, sfml-graphics-s.a
The Android.mk file already has -d flavor of libraries, so I added a -s option with all the libraries set to link statically.
I also changed the c++_shared option to c++_static.
The activity must also be changed not to use sfml-activity as the main native library any more, but just the example library. The sfml-main module was set up so that it also has a NativeActivity entry point, which works out well. This is also why it should be set as LOCAL_WHOLE_STATIC_LIBRARIES, so that the entry point doesn't get stripped.
I might have left out some steps, but this is the general idea to do it.
There is one big issue with this. The sound libraries are still dynamic, so using the audio module will crash the app, because those libraries aren't loaded by the sfml-activity any more. Linking them statically is possible, but there is an ongoing issue about that if I remember correctly.