I was looking for something like this for my own project so I searched how other rhythm games were handling this, turns out it's a really common problem yet I did not find much regarding SFML on that matter.
My goals for a solution were :
- Having a linear output, avoiding both the step function shape and jitter
- Being at most a few ms off regardless of playback time
- Not hogging the CPU
Here's a lengthy guide through a few methods I came up with.
The first basic (and wrong) idea all the following methods are build upon is starting an sf::Clock when you call sf::Music::play() and just reading elasped time from said sf::Clock, this has two problems :
- sf::Music::play() is not blocking, since it just instructs another thread that playback has been requested, for this reason and maybe others having to do with the underlying sound lib, there will be an unpredictable delay between the time this function returns and the time playback actually starts
- I'm not sure why (audio hardware using its own clock ?) but sf::Clock will pretty quickly drift away from the time reported by sf::Music::getPlayingOffset() (as much as a few seconds per minute), and the drift factor itself is pretty unpredictable
Method #1 : The Watchdog Thread- Have a "watchdog" thread monitor changes in the value returned by sf::Music::getPlayingOffset()
- Each time a new value is detected, reset an sf::Clock
- Report offset as : last detected value + time elapsed on the sf::Clock
This works better because even if we are still using an sf::Clock() to keep time elapsed since the last buffer update, sf::Clock does not drift
too much from the audio clock on a timespan as short as a buffer.
Here's my code as a subclass of sf::Music :
#include <thread>
#include <atomic>
#include <SFML/Audio.hpp>
struct PreciseMusic : sf::Music {
PreciseMusic(std::string path);
~PreciseMusic();
sf::Time getPrecisePlayingOffset() const;
// Position update thread
void position_updater_main();
std::thread position_updater;
std::atomic<bool> stop_position_updater = false;
std::atomic<bool> should_correct = false;
std::atomic<sf::Time> last_music_position;
sf::Clock reference_clock;
};
#include "PreciseMusic.hpp"
PreciseMusic::PreciseMusic(std::string path) {
if (not this->openFromFile(path)) {
throw std::invalid_argument("Could not open "+path);
}
position_updater = std::thread(&PreciseMusic::position_updater_main, this);
}
PreciseMusic::~PreciseMusic() {
stop_position_updater.store(true);
position_updater.join();
}
sf::Time PreciseMusic::getPrecisePlayingOffset() const {
if (should_correct.load()) {
return last_music_position.load()+reference_clock.getElapsedTime();
} else {
return this->getPlayingOffset();
}
}
void PreciseMusic::position_updater_main() {
sf::Time next_music_position;
while (not stop_position_updater.load()) {
if (this->getStatus() == sf::Music::Playing) {
next_music_position = this->getPlayingOffset();
if (next_music_position != last_music_position.load()) {
last_music_position.store(next_music_position);
reference_clock.restart();
should_correct.store(true);
}
}
sf::sleep(sf::milliseconds(1));
}
}
Here's my test code :
#include <iostream>
#include <thread>
#include <chrono>
#include <SFML/Audio.hpp>
#include "PreciseMusic.hpp"
int main(int, char const **) {
PreciseMusic m{"test.ogg"};
m.play();
std::cout << "raw,corrected" << std::endl;
while (true) {
auto raw = m.getPlayingOffset();
auto corrected = m.getPrecisePlayingOffset();
std::cout << raw.asMicroseconds() << "," << corrected.asMicroseconds() << std::endl;
std::this_thread::sleep_for(std::chrono::milliseconds(1));
}
return 0;
}
And the resulting chart :
You can clearly see how the code waits for the first step to correct the raw offset
The downside of this method is the watchdog thread, my implementation uses a 1ms loop, it's still not unreasonably fast but the impact is noticeable when you look at CPU usage. A minimal executable simply playing back an audio file using sf::Music takes about 1.5% of the CPU on my machine, bare playback with Method #1 takes 5%.
Method #2 : overriding onGetData()Instead of restarting the clock using a busy loop in the watchdog thread, override the buffer callback, onGetData(). Since it should be called on every buffer request from the underlying sound lib, we should get similar results while reducing CPU load right ?
Here's my code :
#include <thread>
#include <atomic>
#include <SFML/Audio.hpp>
struct PreciseMusic : sf::Music {
PreciseMusic(std::string path);
sf::Time getPrecisePlayingOffset() const;
std::atomic<bool> should_correct = false;
std::atomic<sf::Time> last_music_position;
sf::Clock last_position_update;
protected:
bool onGetData(sf::SoundStream::Chunk& data) override;
};
#include "PreciseMusic.hpp"
PreciseMusic::PreciseMusic(std::string path) {
if (not this->openFromFile(path)) {
throw std::invalid_argument("Could not open "+path);
}
}
sf::Time PreciseMusic::getPrecisePlayingOffset() const {
if (should_correct.load()) {
return last_music_position.load()+last_position_update.getElapsedTime();
} else {
return this->getPlayingOffset();
}
}
bool PreciseMusic::onGetData(sf::SoundStream::Chunk& data) {
bool continue_playback = sf::Music::onGetData(data);
if (continue_playback) {
last_music_position.store(getPlayingOffset());
last_position_update.restart();
should_correct.store(true);
}
return continue_playback;
}
And here are the results :
You can clearly see it's way too early, my understanding is that audio data is queued/buffered way earlier than when it is actually played. We are wayyy off (still only about 10ms early) and we have weird glitches at the beggining, but at least CPU usage is back to normal.
Method #3 : Taking Lag Into AccountAnother idea I had was to make a hybrid of the first two methods :
- Measure the lag by measuring the time between :
- the first onGetData() call
- the first step in the raw offset given by getPlayingOffset()
- Report offset as previously but substract lag
The first step in the raw offset is monitored by a fast busy loop in a watchdog thread as previously but since we only need to check for the
first step to measure lag, the thread only runs for about one second at most.
Here's my code :
#include <thread>
#include <atomic>
#include <SFML/Audio.hpp>
struct PreciseMusic : sf::Music {
PreciseMusic(std::string path);
~PreciseMusic();
std::thread position_update_watchdog;
void position_update_watchdog_main();
std::atomic<bool> should_stop_watchdog = false;
sf::Time getPrecisePlayingOffset() const;
bool first_onGetData_call = true;
sf::Clock lag_measurement;
std::atomic<sf::Time> lag;
std::atomic<bool> should_correct = false;
std::atomic<sf::Time> last_music_position;
sf::Clock last_position_update;
protected:
bool onGetData(sf::SoundStream::Chunk& data) override;
};
#include "PreciseMusic.hpp"
#include <chrono>
PreciseMusic::PreciseMusic(std::string path) {
if (not this->openFromFile(path)) {
throw std::invalid_argument("Could not open "+path);
}
position_update_watchdog = std::thread{&PreciseMusic::position_update_watchdog_main, this};
}
void PreciseMusic::position_update_watchdog_main() {
sf::Time next_music_position = sf::Time::Zero;
while (not should_stop_watchdog.load()) {
if (this->getStatus() == sf::Music::Playing) {
next_music_position = this->getPlayingOffset();
if (next_music_position != last_music_position.load()) {
lag.store(lag_measurement.getElapsedTime());
should_correct.store(true);
break;
}
}
std::this_thread::sleep_for(std::chrono::milliseconds(1));
}
};
PreciseMusic::~PreciseMusic() {
should_stop_watchdog.store(true);
position_update_watchdog.join();
}
sf::Time PreciseMusic::getPrecisePlayingOffset() const {
if (should_correct.load()) {
return last_music_position.load()+last_position_update.getElapsedTime()-lag;
} else {
return this->getPlayingOffset();
}
}
bool PreciseMusic::onGetData(sf::SoundStream::Chunk& data) {
if (first_onGetData_call) {
first_onGetData_call = false;
lag_measurement.restart();
}
bool continue_playback = sf::Music::onGetData(data);
if (continue_playback) {
last_music_position.store(getPlayingOffset());
last_position_update.restart();
}
return continue_playback;
}
And here's the resulting chart :
We are clearly overshooting on our lag correction.
Method #3.5 : Take less lag into accountInterestingly, only substracting half the measured lag gives this curve :
I then tried adjusting the lag correction to get the same results as
Method #1 but I couldn't find a non-magic value based solution, while substracting half the lag seemed like a reasonable thing, I'm not going to try and adjust it even more to get the curves looking nice since that would probably just make my code overfitted to my test file , there
has to be some hardware-dependant things that will break any result that's been found by twiddling the amount of lag correction.
I'm unsure about the ideal solution, I like
Method 3.5 but I think the correction we are looking for is closer to
Method #1
Show Posts
