UPDATE: I created a simple OpenAL example on OS X 10.8.2 (no SFML at all), and this fade-in problem is still present (again with mono sound files but not stereo files). So the problem is in OpenAL.
Even if I force the sound source to be at the listener's position, like this:
alSourcei(sourceID, AL_SOURCE_RELATIVE, AL_TRUE);
alSource3f(sourceID, AL_POSITION, 0.0f, 0.0f, 0.0f);
alSource3f(sourceID, AL_VELOCITY, 0.0f, 0.0f, 0.0f);
...the problem remains.
Does anyone know how to avoid this problem in OpenAL?