The workerpool package provides a simple and efficient worker pool implementation in Go, enabling concurrent processing of tasks with a fixed number of workers. This package is ideal for handling workloads where you need to manage and balance multiple goroutines performing tasks in parallel.
- Concurrency Management: Easily manage a pool of workers to process tasks concurrently.
- Task Scheduling: Submit tasks to the worker pool and have them automatically distributed to available workers.
- Callback Support: Define and execute callback functions for each worker before they start processing a task.
- Result Collection: Collect results from each task processed by the workers.
To install the package, run:
go get github.com/ramadani/workerpoolHere is an example of how to use the worker pool:
package main
import (
"fmt""time""github.com/ramadani/workerpool"
)
funcmain() {
constnumJobs=10constnumWorkers=2callback:=func(workerIDint, job workerpool.Job) workerpool.Result {
fmt.Printf("Worker %d processing job %d with data: %v\n", workerID, job.ID, job.Data)
// Simulate processing timetime.Sleep(100*time.Millisecond)
return workerpool.Result{JobID: job.ID, Data: job.Data, Error: nil}
}
wp:=workerpool.New(numWorkers, numJobs, callback)
// Start the worker poolwp.Start()
// Send jobs to the jobs channelforj:=1; j<=numJobs; j++ {
wp.AddJob(workerpool.Job{ID: j, Data: fmt.Sprintf("data %d", j)})
}
// Wait for all workers to finishwp.Wait()
// Collect results (if needed)forresult:=rangewp.Results() {
ifresult.Error!=nil {
fmt.Printf("Job %d encountered error: %v\n", result.JobID, result.Error)
} else {
fmt.Printf("Job %d processed with result: %v\n", result.JobID, result.Data)
}
}
}Creates a new worker pool.
funcNew(numWorkers, numJobsint, callbackFuncCallbackFunc) *WorkerPoolnumWorkers: Number of workers in the pool.numJobs: Capacity of the job queue.callbackFunc: Function to be executed by each worker for each job.
Starts the worker pool.
func (wp*WorkerPool) Start()Adds a job to the worker pool.
func (wp*WorkerPool) AddJob(jobJob)job: Job to be added to the pool.
Waits for all workers to finish processing jobs.
func (wp*WorkerPool) Wait()Returns a channel to collect results.
func (wp*WorkerPool) Results() <-chanResultThis project is licensed under the MIT License - see the LICENSE file for details.