Welcome, Guest. Please login or register. Did you miss your activation email?

Author Topic: Continuous Audio for Synthersys  (Read 5820 times)

0 Members and 1 Guest are viewing this topic.

user31182

  • Newbie
  • *
  • Posts: 23
    • View Profile
    • Email
Continuous Audio for Synthersys
« on: June 13, 2016, 02:02:13 pm »
Hi all,

I am trying to figure out a way to use the SFML toolkit in a real-time audio synthesis project. (As a kind of experiment, just for fun.)

SFML would be handy since I can use the RenderWindow etc to create a basic UI.

It appears to me that there are 2 methods I could use.

The first is with sf::Sound and sf::SoundBuffer.

The second is with sf::SoundStream.

Let us consider a program which plays a 440 Hz sin wave when I press a key, such as the Z key on my keyboard, otherwise silence is played.

Let us discuss the first, first.

***Soundbuffer***

I have two possible ideas here, although I don't think either will work particularly well. The first idea is to generate a buffer, say for 0.1 s of sound, fill it as the event loop is processed by checking the elapsed time and filling in that duration of sin wave or silence depending on whether the Z key is pressed. Then every time the event loop "goes over" a 0.1 s interval, play the new buffer contents and start generating a new buffer to play.

This method has a problem. Typically, there will be a delay between the thread which is playing the audio finishing playing samples, and a new call to sf::Sound.Play() to play the new buffer. In addition, the sound playing will always be delayed by 0.1 s. (Which is quite large, and results in a horrendous 0.1 s low frequency noise being added to the signal by the continual stop/starting.)

To reduce this problem, one could allocate enough memory for 30 s of audio, press the keys, and then hear the output 30 s later. Obviously the problem here is you are limited to 30 s of audio, and you can't hear what you're playing until afterwards.

The only resolution would be to detect when the playing thread has finished, and then immediately play some new buffer. This will reduce delays, but there will still be some, and I am unsure if this can be done... I think there may be a better way.

***SoundStream***

After attempting to implement the first method, and finding essentially that "it doesn't work very well", as you probably suspected, I did some more researching and found the sf::SoundStream object.

Is this the correct object to use for this task?

I have had a look at the VOIP example, but found this didn't really help much as there is a load of network stuff there which makes it difficult to decipher what is happening if one is not familiar with network programming.

After studying this example, I found a further problem which is that this example appears to work by recording sound samples on one PC and sending them to another. Hence it doesn't really matter too much if there is an associated delay. (I am not sure what this delay might be. I assume less than ~ 0.2 s, but it won't really matter for a phone conversation so long as it's not "too" large.)

Going back to sf::Soundstream, it is my understanding that I must overload the "onGetData" and "onSeek" functions in my own derived class.

What are these functions supposed to do, and how should I overload them? I assume the onGetData() function should "generate new samples", but how many? Where should I store these samples? How can I discard old samples which are no longer required? What if I want to simultaneously record a .wav file to media?

Finally, what does the "onSeek" function do? Why would I need to "seek" in an audio bitstream if I am writing something like a synthesizer?

Apologies for the long question, I tried to make it clear what I am asking.

By the way, I haven't actually asked the main question yet, which is; "is soundstream" the best method of implementing this? Is there another, better method? Perhaps I should be using another external library which is not SFML based at all?
« Last Edit: June 13, 2016, 02:08:46 pm by user31182 »

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: Continuous Audio for Synthersys
« Reply #1 on: June 13, 2016, 03:24:41 pm »
Yes, sf::SoundStream is your best option.

There's a whole tutorial that covers this class, it seems like you didn't read it first. There's also a detailed API documentation that describes the class and functions to override. After that, I'm sure you'll have more precise questions about how to implement your real-time synthesis ;)
Laurent Gomila - SFML developer

user31182

  • Newbie
  • *
  • Posts: 23
    • View Profile
    • Email
Re: Continuous Audio for Synthersys
« Reply #2 on: June 13, 2016, 04:01:51 pm »
Hi there, Yes I was not aware of this tutorial. Thanks.

First question I have is in regards to mutexes and getting data about what keys are pressed inside the onGetData function, which is where I believe I should generate my next batch of samples. (Say I generate 4410 at a time or something...?)

What's the best way to approach this?

I have a possible idea which is to have a globally accessible bool variable, which tells me whether the Z key is pressed or not. Then if I do something like...

// global
bool is_Z_pressed = false;
sf::Mutex mutex; // protect global variables?

int main()...
{
    // main loop sets the bool is_Z_pressed to true/false
    // wrap accessing the bool using sf::Lock(mutex)
    // access to this variable in a local scope so that the mutex is unlocked when sf::Lock falls out of scope?
    {
        // local scope
        sf::Lock(mutex);
        if(event.key.code == Key::Z)
            is_Z_pressed = true;
            ... etc
    }
}
 

Then in my onGetData function, do the same thing, with a sf::Lock to lock the mutex before checking if Z is pressed?

Am I thinking about this in the correct manner? Perhaps I shouldn't be using global variables?

Mario

  • SFML Team
  • Hero Member
  • *****
  • Posts: 877
    • View Profile
Re: Continuous Audio for Synthersys
« Reply #3 on: June 13, 2016, 11:13:08 pm »
You're overcomplicating it. Does it matter whether Z (or any other key) is pressed a frame earlier or later? No.

Therefore you don't need any mutex or synchronization. Just flip the boolean value and act/generate data accordingly.

user31182

  • Newbie
  • *
  • Posts: 23
    • View Profile
    • Email
Re: Continuous Audio for Synthersys
« Reply #4 on: June 14, 2016, 09:54:14 pm »
Fair point Mario.

I am generating samples in batches of 2 * 441. (2 channels, 44100 sample rate, therefore a "key-pressing resolution" of 1/100th of a second.)

I'm getting a small amount of delay when requesting a new batch of samples which is causing the audio playing to "stutter" and introduce that horrible noise I described earlier.

I have compiled in release mode with O3 optimization level.

What can I do about this?

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: Continuous Audio for Synthersys
« Reply #5 on: June 14, 2016, 09:56:55 pm »
Try with bigger batches. 10 ms of audio is not enough for SoundStream.
Laurent Gomila - SFML developer

user31182

  • Newbie
  • *
  • Posts: 23
    • View Profile
    • Email
Re: Continuous Audio for Synthersys
« Reply #6 on: June 14, 2016, 10:10:15 pm »
Hi Laurent,

What sort of size should I be using?

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: Continuous Audio for Synthersys
« Reply #7 on: June 15, 2016, 08:20:37 am »
It's hard to tell, it may be 20, 100 or 500 ms. You'd better try yourself to find the smallest size that produces good results.

The only thing I can say for sure is that sf::SoundStream sleeps for 10 ms after each update, so chunks of 10 ms are really too small. By the way, you can also modify these 10 ms in the update loop and recompile SFML, if it can help.
« Last Edit: June 15, 2016, 08:22:39 am by Laurent »
Laurent Gomila - SFML developer

user31182

  • Newbie
  • *
  • Posts: 23
    • View Profile
    • Email
Re: Continuous Audio for Synthersys
« Reply #8 on: June 15, 2016, 02:05:00 pm »
Hi Laurent thanks again, that is very interesting to know.

How can I modify the 10 ms update time to something like 1 ms?

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: Continuous Audio for Synthersys
« Reply #9 on: June 15, 2016, 02:12:34 pm »
Search for "sleep(milliseconds(10))" in SoundStream.cpp, change it to whatever you like, and recompile.
Laurent Gomila - SFML developer

user31182

  • Newbie
  • *
  • Posts: 23
    • View Profile
    • Email
Re: Continuous Audio for Synthersys
« Reply #10 on: June 15, 2016, 04:35:33 pm »
Ah okay, I had missed the fact that I didn't download the "Source Code" version.

I've edited the source code to set a delay of 500 microseconds.

I've built the code using "cmake ." "make all" "sudo make install".

Now getting the runtime error: Cannot open shared object file, for libsfml-graphics.so.2.3

It wasn't obvious to me how to build SFML from source, even when following the info on this page: http://www.sfml-dev.org/tutorials/2.3/compile-with-cmake.php

What did I do wrong?

Laurent

  • Administrator
  • Hero Member
  • *****
  • Posts: 32504
    • View Profile
    • SFML's website
    • Email
Re: Continuous Audio for Synthersys
« Reply #11 on: June 15, 2016, 05:29:36 pm »
Quote
I've edited the source code to set a delay of 500 microseconds.
Remember that this is only a hint. You work with a desktop OS, not a real-time one. Chances are that your OS won't sleep less than a few microseconds anyway.

Quote
Now getting the runtime error: Cannot open shared object file, for libsfml-graphics.so.2.3
The library loader probably needs to know where to find the file, in case you didn't install your fresh SFML to a standard path. This is Linux stuff I can't help with, you'll have to Google a little bit :P
Laurent Gomila - SFML developer

user31182

  • Newbie
  • *
  • Posts: 23
    • View Profile
    • Email
Re: Continuous Audio for Synthersys
« Reply #12 on: June 15, 2016, 07:52:03 pm »
"sudo ldconfig" fixed it

user31182

  • Newbie
  • *
  • Posts: 23
    • View Profile
    • Email
Re: Continuous Audio for Synthersys
« Reply #13 on: August 02, 2016, 02:40:24 pm »
Hi Laurent - bumping an old thread here, but I recently re-installed my linux system, and I can't remember what libraries I installed to compile SFML from source. (I think some were installed a long time ago, so I didn't need them previously.)

I thought perhaps it would be useful to other people to have a list of the `apt-get` packages on this page? http://www.sfml-dev.org/tutorials/2.0/compile-with-cmake.php

The names of the libraries required are listed under "Installing dependencies" - but it's not obvious how to install those libs, or what the package names are. (Although I realize this could change from system to system - but at least a list of package names for say, debian, would give a starting point for what to look for.

user31182

  • Newbie
  • *
  • Posts: 23
    • View Profile
    • Email
Re: Continuous Audio for Synthersys
« Reply #14 on: August 02, 2016, 02:42:00 pm »
For example "xlib" according to this page https://ubuntuforums.org/showthread.php?t=1071483 is "xorg-dev" - is that correct?

 

anything