You can access the joystick manager without modifying any source, its not actually private, just in a namespace called priv.
For example I just did this and it works for my gamepad:
auto state = sf::priv::JoystickManager::getInstance().getState(0);
All buttons and axes are then in the state variable.
Although the manager's header and a bunch of dependent ones are located in the source directories rather than the header directories, so you may need to add some additional include paths when you build.
The most likely reason for the way its designed is to prevent excessive dependencies being pulled in. Including just joystick.hpp has little impact, it's header is pretty minimal. But including joystickmanager.hpp will also pull in dinput.h and mmsystem.h (when doing windows builds). If the end user doesn't need to use dinput and mmsystem themselves, then its best to not pull them into any cpp that wants joystick access.
This also means an sdk build has minimal headers needed, for example you don't need directinput headers available if you are just using a prebuilt sdk, because the joystickmanager is just internal to the sfml libs.