SFML community forums

Help => Audio => Topic started by: Lauthai on February 20, 2020, 11:40:05 pm

Title: Music getPlayingOffset() more accurate?
Post by: Lauthai on February 20, 2020, 11:40:05 pm
I am working on a rhythm style game and am using sf::Music to load and play my song files.  However, when I try to use the getPlayingOffset().asMilliseconds() function, it only is producing updates to the number every 20 milliseconds. For example:

// Create and open a song file
sf::Music song;


// Test by outputting the offset.
while(1) {
    cout << song.getPlayingOffset().asMilliseconds() << endl;

I would expect that it would output something continouous like 1, 2, 3, 4, ... but instead get 0, 0, ..., 0, 20, 20 and repeat by 20s.

Is there a way to make it so that the offset is more accurately outputting to the millisecond instead of 20? Would that involve modifying a compiling a custom copy of the audio.dll?  Or would you have a better suggestion to get the current millisecond since the start of the song (i.e. would output 1000 if the song has been going for 1 second)?

EDIT: Would it be better to load the song into a sound buffer and then play using sf::Sound instead? Would the reduce the update delay?
Title: Re: Music getPlayingOffset() more accurate?
Post by: eXpl0it3r on February 24, 2020, 08:52:02 am
Probably better to keep control of a buffer, that way you know exactly where in the buffer you are.
Title: Re: Music getPlayingOffset() more accurate?
Post by: Stepland on March 22, 2020, 05:49:31 pm
I was looking for something like this for my own project so I searched how other rhythm games were handling this, turns out it's a really common problem yet I did not find much regarding SFML on that matter.

My goals for a solution were :

Here's a lengthy guide through a few methods I came up with.

The first basic (and wrong) idea all the following methods are build upon is starting an sf::Clock when you call sf::Music::play() and just reading elasped time from said sf::Clock, this has two problems :

Method #1 : The Watchdog Thread

This works better because even if we are still using an sf::Clock() to keep time elapsed since the last buffer update, sf::Clock does not drift too much from the audio clock on a timespan as short as a buffer.

Here's my code as a subclass of sf::Music :
(click to show/hide)
(click to show/hide)

Here's my test code :
(click to show/hide)

And the resulting chart :
(click to show/hide)

You can clearly see how the code waits for the first step to correct the raw offset

The downside of this method is the watchdog thread, my implementation uses a 1ms loop, it's still not unreasonably fast but the impact is noticeable when you look at CPU usage. A minimal executable simply playing back an audio file using sf::Music takes about 1.5% of the CPU on my machine, bare playback with Method #1 takes 5%.

Method #2 : overriding onGetData()

Instead of restarting the clock using a busy loop in the watchdog thread, override the buffer callback, onGetData(). Since it should be called on every buffer request from the underlying sound lib, we should get similar results while reducing CPU load right ?

Here's my code :
(click to show/hide)
(click to show/hide)

And here are the results :
(click to show/hide)

You can clearly see it's way too early, my understanding is that audio data is queued/buffered way earlier than when it is actually played. We are wayyy off (still only about 10ms early) and we have weird glitches at the beggining, but at least CPU usage is back to normal.

Method #3 : Taking Lag Into Account

Another idea I had was to make a hybrid of the first two methods :

The first step in the raw offset is monitored by a fast busy loop in a watchdog thread as previously but since we only need to check for the first step to measure lag, the thread only runs for about one second at most.

Here's my code :
(click to show/hide)
(click to show/hide)

And here's the resulting chart :
(click to show/hide)

We are clearly overshooting on our lag correction.

Method #3.5 : Take less lag into account

Interestingly, only substracting half the measured lag gives this curve :
(click to show/hide)

I then tried adjusting the lag correction to get the same results as Method #1 but I couldn't find a non-magic value based solution, while substracting half the lag seemed like a reasonable thing, I'm not going to try and adjust it even more to get the curves looking nice since that would probably just make my code overfitted to my test file , there has to be some hardware-dependant things that will break any result that's been found by twiddling the amount of lag correction.

I'm unsure about the ideal solution, I like Method 3.5 but I think the correction we are looking for is closer to Method #1
Title: Re: Music getPlayingOffset() more accurate?
Post by: Laurent on March 22, 2020, 07:21:03 pm
If I understand correctly, we're tied to the internal precision of OpenAL Soft, which depends on the sample rate and internal buffer size. The sample rate is a property of the loaded sound, so it can be changed easily; the internal buffer size can be changed with a configuration file (see https://github.com/kcat/openal-soft/blob/master/alsoftrc.sample#L74).

I don't like solutions using a separate clock, ie. not relying completely on the played bytes, but I guess there's no other reasonable choice as long as OpenAL Soft doesn't give us more precision.
Title: Re: Music getPlayingOffset() more accurate?
Post by: Stepland on March 22, 2020, 07:49:36 pm
If one were able to measure time using the audio hardware clock you could have a simplified Method #1 where you just check for the first step and then measure time from there since there should be no drift ? Or maybe other things I don't know about would come into play, I'm unsure.

Looks like it was in the works at some point for OpenAL ? http://openal.org/pipermail/openal/2017-December/000670.html
Title: Re: Music getPlayingOffset() more accurate?
Post by: Stepland on February 18, 2022, 01:29:40 am
Just found out about AL_SEC_OFFSET_LATENCY_SOFT (It's described here (https://openal-soft.org/openal-extensions/SOFT_source_latency.txt)) and figured it might help.

There might be a way to use it to make a better Method #2

I tried it but I couldn't quite make sense of the values it was returning. I'm very new to OpenAL so I'm still struggling a bit ...
Title: Re: Music getPlayingOffset() more accurate?
Post by: Stepland on February 19, 2022, 01:26:44 am
I am getting very close to a solution :

(https://i.ibb.co/VN4qDH2/image.png) (https://ibb.co/j5XrGVg)

Subtracting lag from the offset reported by OpenAL gives a very smooth result, only two thing to fix :

I already have ideas to overcome both of these. I'm getting there !
Title: Re: Music getPlayingOffset() more accurate?
Post by: Stepland on February 19, 2022, 02:36:11 am
Well, I completely overlooked that I could just do

getPlayingOffset() - latency

This is almost perfect :

(click to show/hide)
(click to show/hide)
Title: Re: Music getPlayingOffset() more accurate?
Post by: Stepland on February 28, 2022, 10:45:37 pm
Alright here's what I came up with so far !

(click to show/hide)
(click to show/hide)

I just measure the initial lag once per play call and shift by that amount.

Apparently the reinterpret_cast of a void pointer to a function pointer is something very non-standard that could very possibly break at any given time, but it worked for me ! (gcc (Ubuntu 9.3.0-17ubuntu1~20.04))

If a more knowledgeable person could step in and help us find a better way to handle this that would be awesome.

The resulting value occasionally jumps a little, but overall it's pretty clean :

(click to show/hide)
(click to show/hide)
(click to show/hide)

It even seems to work when changing the playback position or the pitch while the song plays, so I don't think I'll bother fixing this any further. It seems to suit my needs for now, and I hope it'll be enough for you as well