Topics

1

Audio / How exactly does SoundFileReaderFlac work?

« on: March 14, 2019, 06:28:41 pm »

Since I based this code off the SFML codebase, I thought that this was the right place to ask, so here it goes:

Long story short: I wanted to learn about the inner workings of the audio module, so I decided to try making a subset of it (only OGG and FLAC, decided to start with FLAC), completely independent from SFML. It is Windows only at the moment,  but that is not a concern, as I don't plan to use this on production code, just as my own personal learning tool.

The problem is that, while audio reading/decoding and playback work, the samples seem to be being read very slowly (alternatively, the playback is stremely slow, feels like the music is at least 2x slower than normal). I'm using WASAPI for playback on Windows. Here's the code:

FlacReader.h

(click to show/hide)

#pragma once
////////////////////////////////////////////////////////////
// Headers
////////////////////////////////////////////////////////////
#include <FLAC/stream_decoder.h>
#include <string>
#include <fstream>
#include <vector>
#include <memory>

struct Info
{
        uint64_t       sampleCount;  ///< Total number of samples in the file
        unsigned int channelCount; ///< Number of channels of the sound
        unsigned int sampleRate;   ///< Samples rate of the sound, in samples per second
};


////////////////////////////////////////////////////////////
/// \brief Implementation of sound file reader that handles FLAC files
///
////////////////////////////////////////////////////////////
class FlacReader
{
public:

        ////////////////////////////////////////////////////////////
        /// \brief Check if this reader can handle a file given by an input stream
        ///
        /// \param stream Source stream to check
        ///
        /// \return True if the file is supported by this reader
        ///
        ////////////////////////////////////////////////////////////
        static bool check(std::shared_ptr<std::ifstream> stream);

public:

        ////////////////////////////////////////////////////////////
        /// \brief Default constructor
        ///
        ////////////////////////////////////////////////////////////
        FlacReader();

        ////////////////////////////////////////////////////////////
        /// \brief Default constructor
        ///
        ////////////////////////////////////////////////////////////
        ~FlacReader();

        ////////////////////////////////////////////////////////////
        /// \brief Open a sound file for reading
        ///
        /// \param stream Stream to open
        /// \param info   Structure to fill with the attributes of the loaded sound
        ///
        ////////////////////////////////////////////////////////////
        bool open(std::shared_ptr<std::ifstream> stream, Info& info);

        ////////////////////////////////////////////////////////////
        /// \brief Change the current read position to the given sample offset
        ///
        /// The sample offset takes the channels into account.
        /// If you have a time offset instead, you can easily find
        /// the corresponding sample offset with the following formula:
        /// `timeInSeconds * sampleRate * channelCount`
        /// If the given offset exceeds to total number of samples,
        /// this function must jump to the end of the file.
        ///
        /// \param sampleOffset Index of the sample to jump to, relative to the beginning
        ///
        ////////////////////////////////////////////////////////////
        void seek(uint64_t sampleOffset);

        ////////////////////////////////////////////////////////////
        /// \brief Read audio samples from the open file
        ///
        /// \param samples  Pointer to the sample array to fill
        /// \param maxCount Maximum number of samples to read
        ///
        /// \return Number of samples actually read (may be less than \a maxCount)
        ///
        ////////////////////////////////////////////////////////////
        uint64_t read(int32_t* samples, uint64_t maxCount);

public:

        ////////////////////////////////////////////////////////////
        /// \brief Hold the state that is passed to the decoder callbacks
        ///
        ////////////////////////////////////////////////////////////
        struct ClientData
        {
                std::shared_ptr<std::ifstream> stream = nullptr;
                Info info = {};
                int32_t* buffer = nullptr;
                uint64_t                remaining = 0;
                std::vector<int32_t>    leftovers;
                bool                  error = false;
        };

private:

        ////////////////////////////////////////////////////////////
        /// \brief Close the open FLAC file
        ///
        ////////////////////////////////////////////////////////////
        void close();

        ////////////////////////////////////////////////////////////
        // Member data
        ////////////////////////////////////////////////////////////
        std::unique_ptr<FLAC__StreamDecoder> m_decoder;    ///< FLAC decoder
        ClientData           m_clientData; ///< Structure passed to the decoder callbacks
};
 

FlacReader.cpp

(click to show/hide)

#include "FlacReader.h"
#include <cassert>
#include <algorithm>
#include <iostream>

// https://github.com/SFML/SFML/blob/cd9b8b9a150b8c883e827ebd3e060dd7daa402e6/src/SFML/Audio/SoundFileReaderFlac.cpp

FLAC__StreamDecoderReadStatus streamRead(const FLAC__StreamDecoder*, FLAC__byte buffer[], std::size_t* bytes, void* clientData)
{
        FlacReader::ClientData* data = static_cast<FlacReader::ClientData*>(clientData);

        if (*bytes > 0) {
                data->stream->read(reinterpret_cast<char*>(buffer), *bytes);
                *bytes = data->stream->gcount();
                if (data->stream->eof()) {
                        return FLAC__STREAM_DECODER_READ_STATUS_END_OF_STREAM;
                }

                if (data->stream->fail()) {
                        return FLAC__STREAM_DECODER_READ_STATUS_ABORT;
                }

                else {
                        return FLAC__STREAM_DECODER_READ_STATUS_CONTINUE;
                }
        }
        else {
                return FLAC__STREAM_DECODER_READ_STATUS_ABORT;
        }
}

FLAC__StreamDecoderSeekStatus streamSeek(const FLAC__StreamDecoder*, FLAC__uint64 absoluteByteOffset, void* clientData)
{
        FlacReader::ClientData* data = static_cast<FlacReader::ClientData*>(clientData);
        data->stream->seekg(absoluteByteOffset);

        if (data->stream->tellg() >= 0)
                return FLAC__STREAM_DECODER_SEEK_STATUS_OK;
        else
                return FLAC__STREAM_DECODER_SEEK_STATUS_ERROR;
}

FLAC__StreamDecoderTellStatus streamTell(const FLAC__StreamDecoder*, FLAC__uint64 * absoluteByteOffset, void* clientData)
{
        FlacReader::ClientData* data = static_cast<FlacReader::ClientData*>(clientData);

        int64_t position = data->stream->tellg();
        if (position >= 0)
        {
                *absoluteByteOffset = position;
                return FLAC__STREAM_DECODER_TELL_STATUS_OK;
        }
        else
        {
                return FLAC__STREAM_DECODER_TELL_STATUS_ERROR;
        }
}

FLAC__StreamDecoderLengthStatus streamLength(const FLAC__StreamDecoder*, FLAC__uint64 * streamLength, void* clientData)
{
        FlacReader::ClientData* data = static_cast<FlacReader::ClientData*>(clientData);

        std::streampos currentPos = data->stream->tellg();
        data->stream->seekg(std::ios::end);
        int64_t count = data->stream->tellg();
        data->stream->seekg(currentPos);

        if (count >= 0)
        {
                *streamLength = count;
                return FLAC__STREAM_DECODER_LENGTH_STATUS_OK;
        }
        else
        {
                return FLAC__STREAM_DECODER_LENGTH_STATUS_ERROR;
        }
}

FLAC__bool streamEof(const FLAC__StreamDecoder*, void* clientData)
{
        FlacReader::ClientData* data = static_cast<FlacReader::ClientData*>(clientData);

        std::streampos currentPos = data->stream->tellg();
        data->stream->seekg(std::ios::end);
        int64_t size = data->stream->tellg();
        data->stream->seekg(currentPos);

        return data->stream->tellg() == size;
}

FLAC__StreamDecoderWriteStatus streamWrite(const FLAC__StreamDecoder*, const FLAC__Frame* frame, const FLAC__int32* const buffer[], void* clientData)
{
        FlacReader::ClientData* data = static_cast<FlacReader::ClientData*>(clientData);

        // Reserve memory if we're going to use the leftovers buffer
        unsigned int frameSamples = frame->header.blocksize * frame->header.channels;
        if (data->remaining < frameSamples)
                data->leftovers.reserve(static_cast<std::size_t>(frameSamples - data->remaining));

        int shift = 32 - frame->header.bits_per_sample;

        // Decode the samples
        for (unsigned i = 0; i < frame->header.blocksize; ++i)
        {
                for (unsigned int j = 0; j < frame->header.channels; ++j)
                {
                        // Decode the current sample
                        int32_t sample = buffer[j][i] << shift;

                        if (data->buffer && data->remaining > 0)
                        {
                                // If there's room in the output buffer, copy the sample there
                                *data->buffer++ = sample;
                                data->remaining--;
                        }
                        else
                        {
                                // We are either seeking (null buffer) or have decoded all the requested samples during a
                                // normal read (0 remaining), so we put the sample in a temporary buffer until next call
                                data->leftovers.push_back(sample);
                        }
                }
        }

        return FLAC__STREAM_DECODER_WRITE_STATUS_CONTINUE;
}

void streamMetadata(const FLAC__StreamDecoder*, const FLAC__StreamMetadata * meta, void* clientData)
{
        FlacReader::ClientData* data = static_cast<FlacReader::ClientData*>(clientData);

        if (meta->type == FLAC__METADATA_TYPE_STREAMINFO)
        {
                data->info.sampleCount = meta->data.stream_info.total_samples * meta->data.stream_info.channels;
                std::cout << "Sample count: " << meta->data.stream_info.total_samples * meta->data.stream_info.channels << std::endl;
                data->info.sampleRate = meta->data.stream_info.sample_rate;
                std::cout << "Sample rate: " << meta->data.stream_info.sample_rate << std::endl;
                data->info.channelCount = meta->data.stream_info.channels;
                std::cout << "Channels: " << meta->data.stream_info.channels << std::endl;
                std::cout << "Bits per sample: " << meta->data.stream_info.bits_per_sample << std::endl;
        }
}

void streamError(const FLAC__StreamDecoder*, FLAC__StreamDecoderErrorStatus, void* clientData)
{
        FlacReader::ClientData* data = static_cast<FlacReader::ClientData*>(clientData);
        data->error = true;
}

////////////////////////////////////////////////////////////
bool FlacReader::check(std::shared_ptr<std::ifstream> stream)
{
        // Create a decoder
        std::unique_ptr<FLAC__StreamDecoder> decoder(FLAC__stream_decoder_new());
        if (!decoder)
                return false;

        // Initialize the decoder with our callbacks
        ClientData data;
        data.stream = stream;
        data.error = false;
        FLAC__stream_decoder_init_stream(decoder.get(), &streamRead, &streamSeek, &streamTell, &streamLength, &streamEof, &streamWrite, nullptr, &streamError, &data);

        // Read the header
        bool valid = FLAC__stream_decoder_process_until_end_of_metadata(decoder.get()) != 0;

        // Destroy the decoder
        FLAC__stream_decoder_finish(decoder.get());
        FLAC__stream_decoder_delete(decoder.get());

        return valid && !data.error;
}


////////////////////////////////////////////////////////////
FlacReader::FlacReader() :
        m_decoder(nullptr),
        m_clientData()
{
}


////////////////////////////////////////////////////////////
FlacReader::~FlacReader()
{
        close();
}


////////////////////////////////////////////////////////////
bool FlacReader::open(std::shared_ptr<std::ifstream> stream, Info& info)
{
        // Create the decoder
        m_decoder = std::unique_ptr<FLAC__StreamDecoder>(FLAC__stream_decoder_new());
        if (!m_decoder)
        {
                //err() << "Failed to open FLAC file (failed to allocate the decoder)" << std::endl;
                return false;
        }

        // Initialize the decoder with our callbacks
        m_clientData.stream = stream;
        FLAC__stream_decoder_init_stream(m_decoder.get(), &streamRead, &streamSeek, &streamTell, &streamLength, &streamEof, &streamWrite, &streamMetadata, &streamError, &m_clientData);

        // Read the header
        if (!FLAC__stream_decoder_process_until_end_of_metadata(m_decoder.get()))
        {
                close();
                //err() << "Failed to open FLAC file (failed to read metadata)" << std::endl;
                return false;
        }

        // Retrieve the sound properties
        info = m_clientData.info; // was filled in the "metadata" callback

        return true;
}


////////////////////////////////////////////////////////////
void FlacReader::seek(uint64_t sampleOffset)
{
        assert(m_decoder);

        // Reset the callback data (the "write" callback will be called)
        m_clientData.buffer = nullptr;
        m_clientData.remaining = 0;
        m_clientData.leftovers.clear();

        // FLAC decoder expects absolute sample offset, so we take the channel count out
        if (sampleOffset < m_clientData.info.sampleCount)
        {
                // The "write" callback will populate the leftovers buffer with the first batch of samples from the
                // seek destination, and since we want that data in this typical case, we don't re-clear it afterward
                FLAC__stream_decoder_seek_absolute(m_decoder.get(), sampleOffset / m_clientData.info.channelCount);
        }
        else
        {
                // FLAC decoder can't skip straight to EOF, so we short-seek by one sample and skip the rest
                FLAC__stream_decoder_seek_absolute(m_decoder.get(), (m_clientData.info.sampleCount / m_clientData.info.channelCount) - 1);
                FLAC__stream_decoder_skip_single_frame(m_decoder.get());

                // This was re-populated during the seek, but we're skipping everything in this, so we need it emptied
                m_clientData.leftovers.clear();
        }
}


////////////////////////////////////////////////////////////
uint64_t FlacReader::read(int32_t * samples, uint64_t maxCount)
{
        assert(m_decoder);

        // If there are leftovers from previous call, use it first
        uint64_t left = m_clientData.leftovers.size();
        if (left > 0)
        {
                if (left > maxCount)
                {
                        // There are more leftovers than needed
                        std::copy(m_clientData.leftovers.begin(), m_clientData.leftovers.end(), samples);
                        std::vector<int32_t> leftovers(m_clientData.leftovers.begin() + maxCount, m_clientData.leftovers.end());
                        m_clientData.leftovers.swap(leftovers);
                        return maxCount;
                }
                else
                {
                        // We can use all the leftovers and decode new frames
                        std::copy(m_clientData.leftovers.begin(), m_clientData.leftovers.end(), samples);
                }
        }

        // Reset the data that will be used in the callback
        m_clientData.buffer = samples + left;
        m_clientData.remaining = maxCount - left;
        m_clientData.leftovers.clear();

        // Decode frames one by one until we reach the requested sample count, the end of file or an error
        while (m_clientData.remaining > 0)
        {
                // Everything happens in the "write" callback
                // This will break on any fatal error (does not include EOF)
                if (!FLAC__stream_decoder_process_single(m_decoder.get()))
                        break;

                // Break on EOF
                if (FLAC__stream_decoder_get_state(m_decoder.get()) == FLAC__STREAM_DECODER_END_OF_STREAM)
                        break;
        }

        return maxCount - m_clientData.remaining;
}


////////////////////////////////////////////////////////////
void FlacReader::close()
{
        if (m_decoder)
        {
                FLAC__stream_decoder_finish(m_decoder.get());
                FLAC__stream_decoder_delete(m_decoder.get());
                m_decoder = nullptr;
        }
}

 

And the code I use for playback (excuse the mess, I already mentioned that I don't intend to use this code anywhere besides a learning environment).

(click to show/hide)

#define WIN32_LEAN_AND_MEAN
#define NOMINMAX
#include <Windows.h>
#include <mmdeviceapi.h>
#include <Audioclient.h>
#include <avrt.h>

#include <iostream>
#include <filesystem>
#include <string>
#include <vector>
#include <iterator>
#include <FLAC/stream_decoder.h>

#include "FlacReader.h"


constexpr void COMSafeRelease(IUnknown* p) {
        if (p) {
                p->Release();
        }
}

int main()
{
        HRESULT hr = S_OK;
        IMMDeviceEnumerator* pEnumerator = nullptr;
        IMMDevice* pDevice = nullptr;
        IAudioClient3* pAudioClient = nullptr;
        IAudioRenderClient* pAudioRenderClient = nullptr;
        WAVEFORMATEX* pwfx = nullptr;
        uint32_t bufferFrameCount = 0;
        uint32_t numFramesAvailable = 0;
        uint32_t defaultPeriodInFrames = 0;
        uint32_t fundamentalPeriodInFrames = 0;
        uint32_t minPeriodInFrames = 0;
        uint32_t maxPeriodInFrames = 0;
        HANDLE hTask = nullptr;

        BYTE* pData = nullptr;

        hr = CoInitializeEx(nullptr, tagCOINIT::COINIT_MULTITHREADED);

        if (FAILED(hr)) {
                throw std::runtime_error("CoInitializeEx failed! Code: " + std::to_string(hr) + "\n");
        }

        hr = CoCreateInstance(__uuidof(MMDeviceEnumerator), nullptr, CLSCTX_ALL, __uuidof(IMMDeviceEnumerator), reinterpret_cast<void**>(&pEnumerator));

        if (FAILED(hr)) {
                throw std::runtime_error("CoCreateInstance failed! Code: " + std::to_string(hr) + "\n");
        }

        pEnumerator->GetDefaultAudioEndpoint(EDataFlow::eRender, ERole::eConsole, &pDevice);

        hr = pDevice->Activate(__uuidof(IAudioClient3), CLSCTX_ALL, nullptr, reinterpret_cast<void**>(&pAudioClient));

        if (FAILED(hr)) {
                throw std::runtime_error("pDevice->Activate() failed! Code: " + std::to_string(hr) + "\n");
        }

        hr = pAudioClient->GetMixFormat(&pwfx);

        if (FAILED(hr)) {
                throw std::runtime_error("pAudioClient->GetMixFormat() failed! Code: " + std::to_string(hr) + "\n");
        }

        WAVEFORMATEXTENSIBLE* exwfx = reinterpret_cast<WAVEFORMATEXTENSIBLE*>(pwfx);

        std::cout << exwfx->Format.nChannels << std::endl;

        // TODO: Investigate AUDIOFORMAT
        WAVEFORMATEX wave_format = {};
        wave_format.wFormatTag = WAVE_FORMAT_PCM;
        wave_format.nChannels = 2;
        wave_format.nSamplesPerSec = 44100;
        //nSamplesPerSec * nBlockAlign
        wave_format.nAvgBytesPerSec = 44100 * 2 * 16 / 8;
        wave_format.nBlockAlign = 2 * 16 / 8;
        wave_format.wBitsPerSample = 16;

        //hr = pAudioClient->GetSharedModeEnginePeriod(pwfx, &defaultPeriodInFrames, &fundamentalPeriodInFrames, &minPeriodInFrames, &maxPeriodInFrames);
        hr = pAudioClient->GetSharedModeEnginePeriod(&wave_format, &defaultPeriodInFrames, &fundamentalPeriodInFrames, &minPeriodInFrames, &maxPeriodInFrames);

        if (FAILED(hr)) {
                throw std::runtime_error("pAudioClient->GetSharedModeEnginePeriod() failed! Code: " + std::to_string(hr) + "\n");
        }

        AudioClientProperties pProperties = {};
        pProperties.cbSize = sizeof(AudioClientProperties);
        //pProperties.bIsOffload = true;
        pProperties.eCategory = _AUDIO_STREAM_CATEGORY::AudioCategory_GameMedia;

        hr = pAudioClient->SetClientProperties(&pProperties);

        if (FAILED(hr)) {
                throw std::runtime_error("pAudioClient-SetClientProperties() failed! Code: " + std::to_string(hr) + "\n");
        }

        //hr = pAudioClient->InitializeSharedAudioStream(AUDCLNT_STREAMFLAGS_EVENTCALLBACK, minPeriodInFrames, pwfx, nullptr);
        hr = pAudioClient->InitializeSharedAudioStream(AUDCLNT_STREAMFLAGS_EVENTCALLBACK, minPeriodInFrames, &wave_format, nullptr);

        if (FAILED(hr)) {
                throw std::runtime_error("pAudioClient->InitializeSharedModeAudioStream() failed! Code: " + std::to_string(hr) + "\n");
        }

        // Create an event handle and register it for
   // buffer-event notifications.
        HANDLE audioSamplesReadyEvent = audioSamplesReadyEvent = CreateEvent(nullptr, false, false, nullptr);
        if (audioSamplesReadyEvent == nullptr)
        {
                hr = E_FAIL;
                throw std::runtime_error("CreateEvent failed! Code: " + std::to_string(hr) + "\n");
        }

        hr = pAudioClient->SetEventHandle(audioSamplesReadyEvent);

        if (FAILED(hr)) {
                throw std::runtime_error("pAudioClient->SetEventHandle() failed! Code: " + std::to_string(hr) + "\n");
        }

        // Get the actual size of the two allocated buffers.
        hr = pAudioClient->GetBufferSize(&bufferFrameCount);

        if (FAILED(hr)) {
                throw std::runtime_error("pAudioClient->GetBufferSize() failed! Code: " + std::to_string(hr) + "\n");
        }

        hr = pAudioClient->GetService(__uuidof(IAudioRenderClient), reinterpret_cast<void**>(&pAudioRenderClient));

        if (FAILED(hr)) {
                throw std::runtime_error("pAudioClient->GetService() failed! Code: " + std::to_string(hr) + "\n");
        }

        FlacReader fr;
        Info info;

        fr.open(std::make_unique<std::ifstream>("audio/02 Nabeel Ansari - The Ultimate Armor (Mega Man X5).flac", std::ios::binary), info);

        // To reduce latency, load the first buffer with data
        // from the audio source before starting the stream.
        hr = pAudioRenderClient->GetBuffer(bufferFrameCount, &pData);
        if (FAILED(hr)) {
                throw std::runtime_error("pAudioClient->GetBuffer() failed! Code: " + std::to_string(hr) + "\n");
        }

        fr.read(reinterpret_cast<int32_t*>(pData), bufferFrameCount);

        hr = pAudioRenderClient->ReleaseBuffer(bufferFrameCount, _AUDCLNT_BUFFERFLAGS::AUDCLNT_BUFFERFLAGS_SILENT);

        if (FAILED(hr)) {
                throw std::runtime_error("pRenderClient->ReleaseBuffer() failed! Code: " + std::to_string(hr) + "\n");
        }

        // Ask MMCSS to temporarily boost the thread priority
        // to reduce glitches while the low-latency stream plays. (TODO: Is this still necessary in shared mode?)
        DWORD taskIndex = 0;
        hTask = AvSetMmThreadCharacteristics(TEXT("Games"), &taskIndex);
        if (hTask == nullptr)
        {
                hr = E_FAIL;
        }

        hr = pAudioClient->Start();  // Start playing.

        if (FAILED(hr)) {
                throw std::runtime_error("pRenderClient->Start() failed! Code: " + std::to_string(hr) + "\n");
        }

        size_t framesRead = 0;
        bool playing = true;

        while (playing)
        {
                // Wait for next buffer event to be signaled.
                WaitForSingleObject(audioSamplesReadyEvent, 2000);

                uint32_t numPaddingFrames = 0;
                hr = pAudioClient->GetCurrentPadding(&numPaddingFrames);
                if (FAILED(hr)) {
                        throw std::runtime_error("pAudioClient->GetCurrentPadding() failed! Code: " + std::to_string(hr) + "\n");
                }

                numFramesAvailable = bufferFrameCount - numPaddingFrames;
                if (numFramesAvailable == 0) {
                        continue;
                }

                hr = pAudioRenderClient->GetBuffer(numFramesAvailable, &pData);
                if (FAILED(hr)) {
                        throw std::runtime_error("pAudioRenderClient->GetBuffer() failed! Code: " + std::to_string(hr) + "\n");
                }

                framesRead = fr.read(reinterpret_cast<int32_t*>(pData), numFramesAvailable);
                // std::cout << framesRead << std::endl;

                hr = pAudioRenderClient->ReleaseBuffer(numFramesAvailable, 0);
                if (FAILED(hr)) {
                        throw std::runtime_error("pAudioRenderClient->ReleaseBuffer() failed! Code: " + std::to_string(hr) + "\n");
                }

                playing = (framesRead == numFramesAvailable);
        }

        do
        {
                // wait for buffer to be empty
                WaitForSingleObject(audioSamplesReadyEvent, INFINITE);

                uint32_t numPaddingFrames = 0;
                hr = pAudioClient->GetCurrentPadding(&numPaddingFrames);
                if (FAILED(hr)) {
                        throw std::runtime_error("pAudioClient->GetCurrentPadding() failed! Code: " + std::to_string(hr) + "\n");
                }

                if (numPaddingFrames == 0)
                {
                        std::cout << "Current buffer padding = 0[frames]" << std::endl;
                        break;
                }
        } while (true);

        hr = pAudioClient->Stop();
        if (FAILED(hr)) {
                throw std::runtime_error("pAudioClient->Stop() failed! Code: " + std::to_string(hr) + "\n");
        }

        //--------------------------------------------------
        if (audioSamplesReadyEvent != nullptr)
        {
                CloseHandle(audioSamplesReadyEvent);
        }

        if (hTask != nullptr)
        {
                AvRevertMmThreadCharacteristics(hTask);
        }

        COMSafeRelease(pEnumerator);
        COMSafeRelease(pDevice);
        COMSafeRelease(pAudioClient);
        COMSafeRelease(pAudioRenderClient);
        CoTaskMemFree(pwfx);

        CoUninitialize();
}
 

I think the the problem is on this section of the code:

(click to show/hide)

// Decode the samples
        for (unsigned i = 0; i < frame->header.blocksize; ++i)
        {
                for (unsigned int j = 0; j < frame->header.channels; ++j)
                {
                        // Decode the current sample
                        int32_t sample = buffer[j][i] << shift;

                        if (data->buffer && data->remaining > 0)
                        {
                                // If there's room in the output buffer, copy the sample there
                                *data->buffer++ = sample;
                                data->remaining--;
                        }
                        else
                        {
                                // We are either seeking (null buffer) or have decoded all the requested samples during a
                                // normal read (0 remaining), so we put the sample in a temporary buffer until next call
                                data->leftovers.push_back(sample);
                        }
                }
        }

However, I'm unable to determine exactly what's wrong with it. The original code uses 16bit integers, which are shorts on my system. I changed all sf::Int16 in the original code to int32_t, and modified the decoding code accordingly. As far as I'm aware anyways. The only other major changes are the use of shared and unique ptrs, and the replacement of sf::InputStream for a regular ifstream. On the WASAPI side, the hacked together mix format perfectly matches the metadata of the test file I'm using. So that's is ruled out as a problem.