---

A unified interface for optimization algorithms and experiments in Python.

Documentation	▸	Homepage · User Guide · API Reference · Examples
On this page	▸	Features · Examples · Concepts · Citation

---

Hyperactive provides 31 optimization algorithms across 3 backends (GFO, Optuna, scikit-learn), accessible through a unified experiment-based interface. The library separates optimization problems from algorithms, enabling you to swap optimizers without changing your experiment code.

Designed for hyperparameter tuning, model selection, and black-box optimization. Native integrations with scikit-learn, sktime, skpro, and PyTorch allow tuning ML models with minimal setup. Define your objective, specify a search space, and run.

Installation

pip install hyperactive

Optional dependencies

pip install hyperactive[sklearn-integration] # scikit-learn integration
pip install hyperactive[sktime-integration] # sktime/skpro integration
pip install hyperactive[all_extras] # Everything including Optuna

Key Features

31 Optimization Algorithms Local, global, population-based, and model-based methods across 3 backends (GFO, Optuna, sklearn).	Experiment Abstraction Clean separation between what to optimize (experiments) and how to optimize (algorithms).	Flexible Search Spaces Discrete, continuous, and mixed parameter types. Define spaces with NumPy arrays or lists.
ML Framework Integrations Native support for scikit-learn, sktime, skpro, and PyTorch with minimal code changes.	Multiple Backends GFO algorithms, Optuna samplers, and sklearn search methods through one unified API.	Stable & Tested 5+ years of development, comprehensive test coverage, and active maintenance since 2019.

Quick Start

importnumpyasnpfromhyperactive.opt.gfoimportHillClimbing# Define objective function (maximize)defobjective(params):
x, y=params["x"], params["y"]
return-(x**2+y**2) # Negative paraboloid, optimum at (0, 0)# Define search spacesearch_space= {
"x": np.arange(-5, 5, 0.1),
"y": np.arange(-5, 5, 0.1),
}
# Run optimizationoptimizer=HillClimbing(
search_space=search_space,
n_iter=100,
experiment=objective,
)
best_params=optimizer.solve()
print(f"Best params: {best_params}")

Output:

Best params: {'x': 0.0, 'y': 0.0}

Core Concepts

Hyperactive separates what you optimize from how you optimize. Define your experiment (objective function) and search space once, then swap optimizers freely without changing your code. The unified interface abstracts away backend differences, letting you focus on your optimization problem.

flowchart TB
subgraph USER["Your Code"]
direction LR
F["def objective(params):<br/> return score"]
SP["search_space = {<br/> 'x': np.arange(...),<br/> 'y': [1, 2, 3]<br/>}"]
end
subgraph HYPER["Hyperactive"]
direction TB
OPT["Optimizer"]
subgraph BACKENDS["Backends"]
GFO["GFO<br/>21 algorithms"]
OPTUNA["Optuna<br/>8 algorithms"]
SKL["sklearn<br/>2 algorithms"]
MORE["...<br/>more to come"]
end
OPT --> GFO
OPT --> OPTUNA
OPT --> SKL
OPT --> MORE
end
subgraph OUT["Output"]
BEST["best_params"]
end
F --> OPT
SP --> OPT
HYPER --> OUT

Loading

Optimizer: Implements the search strategy (Hill Climbing, Bayesian, Particle Swarm, etc.).

Search Space: Defines valid parameter combinations as NumPy arrays or lists.

Experiment: Your objective function or a built-in experiment (SklearnCvExperiment, etc.).

Best Parameters: The optimizer returns the parameters that maximize the objective.

Examples

Scikit-learn Hyperparameter Tuning

fromsklearn.svmimportSVCfromsklearn.datasetsimportload_irisfromsklearn.model_selectionimporttrain_test_splitfromhyperactive.integrations.sklearnimportOptCVfromhyperactive.opt.gfoimportHillClimbing# Load dataX, y=load_iris(return_X_y=True)
X_train, X_test, y_train, y_test=train_test_split(X, y, test_size=0.2)
# Define search space and optimizersearch_space= {"kernel": ["linear", "rbf"], "C": [1, 10, 100]}
optimizer=HillClimbing(search_space=search_space, n_iter=20)
# Create tuned estimatortuned_svc=OptCV(SVC(), optimizer)
tuned_svc.fit(X_train, y_train)
print(f"Best params: {tuned_svc.best_params_}")
print(f"Test accuracy: {tuned_svc.score(X_test, y_test):.3f}")

Bayesian Optimization

importnumpyasnpfromhyperactive.opt.gfoimportBayesianOptimizerdefackley(params):
x, y=params["x"], params["y"]
return-(
-20*np.exp(-0.2*np.sqrt(0.5* (x**2+y**2)))
-np.exp(0.5* (np.cos(2*np.pi*x) +np.cos(2*np.pi*y)))
+np.e+20
)
search_space= {
"x": np.arange(-5, 5, 0.01),
"y": np.arange(-5, 5, 0.01),
}
optimizer=BayesianOptimizer(
search_space=search_space,
n_iter=50,
experiment=ackley,
)
best_params=optimizer.solve()

Particle Swarm Optimization

importnumpyasnpfromhyperactive.opt.gfoimportParticleSwarmOptimizerdefrastrigin(params):
A=10values= [params[f"x{i}"] foriinrange(5)]
return-sum(v**2-A*np.cos(2*np.pi*v) +Aforvinvalues)
search_space= {f"x{i}": np.arange(-5.12, 5.12, 0.1) foriinrange(5)}
optimizer=ParticleSwarmOptimizer(
search_space=search_space,
n_iter=500,
experiment=rastrigin,
population_size=20,
)
best_params=optimizer.solve()

Experiment Abstraction with SklearnCvExperiment

importnumpyasnpfromsklearn.svmimportSVCfromsklearn.datasetsimportload_irisfromsklearn.metricsimportaccuracy_scorefromsklearn.model_selectionimportKFoldfromhyperactive.experiment.integrationsimportSklearnCvExperimentfromhyperactive.opt.gfoimportHillClimbingX, y=load_iris(return_X_y=True)
# Create reusable experimentsklearn_exp=SklearnCvExperiment(
estimator=SVC(),
scoring=accuracy_score,
cv=KFold(n_splits=3, shuffle=True),
X=X,
y=y,
)
search_space= {
"C": np.logspace(-2, 2, num=10),
"kernel": ["linear", "rbf"],
}
optimizer=HillClimbing(
search_space=search_space,
n_iter=100,
experiment=sklearn_exp,
)
best_params=optimizer.solve()

Optuna Backend (TPE)

importnumpyasnpfromhyperactive.opt.optunaimportTPEOptimizerdefobjective(params):
x, y=params["x"], params["y"]
return-(x**2+y**2)
search_space= {
"x": np.arange(-5, 5, 0.1),
"y": np.arange(-5, 5, 0.1),
}
optimizer=TPEOptimizer(
search_space=search_space,
n_iter=100,
experiment=objective,
)
best_params=optimizer.solve()

Time Series Forecasting with sktime

fromsktime.forecasting.naiveimportNaiveForecasterfromsktime.datasetsimportload_airlinefromhyperactive.integrations.sktimeimportForecastingOptCVfromhyperactive.opt.gfoimportRandomSearchy=load_airline()
search_space= {
"strategy": ["last", "mean", "drift"],
"sp": [1, 12],
}
optimizer=RandomSearch(search_space=search_space, n_iter=10)
tuned_forecaster=ForecastingOptCV(NaiveForecaster(), optimizer)
tuned_forecaster.fit(y)
print(f"Best params: {tuned_forecaster.best_params_}")

PyTorch Neural Network Tuning

importnumpyasnpimporttorchimporttorch.nnasnnfromtorch.utils.dataimportDataLoader, TensorDatasetfromhyperactive.opt.gfoimportBayesianOptimizer# Example dataX_train=torch.randn(1000, 10)
y_train=torch.randint(0, 2, (1000,))
deftrain_model(params):
learning_rate=params["learning_rate"]
batch_size=params["batch_size"]
hidden_size=params["hidden_size"]
model=nn.Sequential(
nn.Linear(10, hidden_size),
nn.ReLU(),
nn.Linear(hidden_size, 2),
)
optimizer=torch.optim.Adam(model.parameters(), lr=learning_rate)
criterion=nn.CrossEntropyLoss()
loader=DataLoader(TensorDataset(X_train, y_train), batch_size=batch_size)
model.train()
forepochinrange(10):
forX_batch, y_batchinloader:
optimizer.zero_grad()
loss=criterion(model(X_batch), y_batch)
loss.backward()
optimizer.step()
# Return validation accuracymodel.eval()
withtorch.no_grad():
predictions=model(X_train).argmax(dim=1)
accuracy= (predictions==y_train).float().mean().item()
returnaccuracysearch_space= {
"learning_rate": np.logspace(-5, -1, 20),
"batch_size": [16, 32, 64, 128],
"hidden_size": [64, 128, 256, 512],
}
optimizer=BayesianOptimizer(
search_space=search_space,
n_iter=30,
experiment=train_model,
)
best_params=optimizer.solve()

Ecosystem

This library is part of a suite of optimization and machine learning tools. For updates on these packages, follow on GitHub.

Package	Description
Hyperactive	Hyperparameter optimization framework with experiment abstraction and ML integrations
Gradient-Free-Optimizers	Core optimization algorithms for black-box function optimization
Surfaces	Test functions and benchmark surfaces for optimization algorithm evaluation

Documentation

Resource	Description
User Guide	Comprehensive tutorials and explanations
API Reference	Complete API documentation
Examples	Jupyter notebooks with use cases
FAQ	Common questions and troubleshooting

Contributing

Contributions welcome! See CONTRIBUTING.md for guidelines.

• Bug reports: GitHub Issues
• Feature requests: GitHub Discussions
• Questions: Discord

Citation

If you use this software in your research, please cite:

@software{hyperactive2019,
author = {Simon Blanke},
title = {Hyperactive: A hyperparameter optimization and meta-learning toolbox},
year = {2019},
url = {https://github.com/SimonBlanke/Hyperactive},
}

License

MIT License - Free for commercial and academic use.