1
General / Re: Using std::futures for creating chunks
« on: July 23, 2024, 08:43:51 am »
Without running your code I can't say for sure, but I'm guessing the bottleneck is the single shared mutex. What is probably happening is that all the worker threads finish at more or less the same time, but then have to queue as each one waits for the mutex to lock and unlock for each thread.
A possible solution would be to have each worker thread have its own mutex then the main thread can check and gather results from finished threads by locking each of those individually - that way none of the other threads are interacting with each other, just the main one.
std::future is possibly also not the magic bullet you're looking for here, and you may have to get your hands dirty with threads directly - although there is the std::execution policy in C++17 (except on Apple clang, where it's not mplemented) https://en.cppreference.com/w/cpp/algorithm/execution_policy_tag_t which allows running stl algorithms in parallel.
Overall this is more generally called (or is a variant of) the producer-consumer problem: https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem - which is worth reading up on and might help you find a more suitable solution. HTH ๐
A possible solution would be to have each worker thread have its own mutex then the main thread can check and gather results from finished threads by locking each of those individually - that way none of the other threads are interacting with each other, just the main one.
std::future is possibly also not the magic bullet you're looking for here, and you may have to get your hands dirty with threads directly - although there is the std::execution policy in C++17 (except on Apple clang, where it's not mplemented) https://en.cppreference.com/w/cpp/algorithm/execution_policy_tag_t which allows running stl algorithms in parallel.
Overall this is more generally called (or is a variant of) the producer-consumer problem: https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem - which is worth reading up on and might help you find a more suitable solution. HTH ๐