I’m sure you’ve had to write some code where you have a few hundred thousand items and need to do some work on them. Performing the jobs serially would be too long. And launching everything at once with goroutines isn’t possible because of some other limitation. You need a worker pool.
Here is the worker pool available on Go by Example:
I’m a big fan of Go by Example. But can we make that even simpler?
I just had a case where I don’t even need a return value. So what can we remove?
It turns out Go’s channels are a great solution to this problem. Instead of using them to pass values, we can use them as locks to manage the concurrency.
In this version, handling the workers and the concurrency becomes a lot simpler. Increase the buffer size if you want to speed things up. Click here to try it online.
If you have ways to continue improving this code, I would love to hear about them!
Subscribe and receive every few months an email with my latest articles and other relevant ressources: