Ray provides a simple, universal API for building distributed applications.
Ray is packaged with the following libraries for accelerating machine learning workloads:
- Tune: Scalable Hyperparameter Tuning
- RLlib: Scalable Reinforcement Learning
- RaySGD: Distributed Training Wrappers
- Ray Serve: Scalable and Programmable Serving
- Datasets: Distributed Arrow on Ray (preview)
There are also many community integrations with Ray, including Dask, MARS, Modin, Horovod, Hugging Face, Scikit-learn, and others. Check out the full list of Ray distributed libraries here.
Install Ray with: pip install ray. For nightly wheels, see the
Installation page.
Execute Python functions in parallel.
importrayray.init()
@ray.remotedeff(x):
returnx*xfutures= [f.remote(i) foriinrange(4)]
print(ray.get(futures))To use Ray's actor model:
importrayray.init()
@ray.remoteclassCounter(object):
def__init__(self):
self.n=0defincrement(self):
self.n+=1defread(self):
returnself.ncounters= [Counter.remote() foriinrange(4)]
[c.increment.remote() forcincounters]
futures= [c.read.remote() forcincounters]
print(ray.get(futures))Ray programs can run on a single machine, and can also seamlessly scale to large clusters. To execute the above Ray script in the cloud, just download this configuration file, and run:
ray submit [CLUSTER.YAML] example.py --start
Read more about launching clusters.
Tune is a library for hyperparameter tuning at any scale.
- Launch a multi-node distributed hyperparameter sweep in less than 10 lines of code.
- Supports any deep learning framework, including PyTorch, PyTorch Lightning, TensorFlow, and Keras.
- Visualize results with TensorBoard.
- Choose among scalable SOTA algorithms such as Population Based Training (PBT), Vizier's Median Stopping Rule, HyperBand/ASHA.
- Tune integrates with many optimization libraries such as Facebook Ax, HyperOpt, and Bayesian Optimization and enables you to scale them transparently.
To run this example, you will need to install the following:
$ pip install "ray[tune]"This example runs a parallel grid search to optimize an example objective function.
fromrayimporttunedefobjective(step, alpha, beta):
return (0.1+alpha*step/100)**(-1) +beta*0.1deftraining_function(config):
# Hyperparametersalpha, beta=config["alpha"], config["beta"]
forstepinrange(10):
# Iterative training function - can be any arbitrary training procedure.intermediate_score=objective(step, alpha, beta)
# Feed the score back back to Tune.tune.report(mean_loss=intermediate_score)
analysis=tune.run(
training_function,
config={
"alpha": tune.grid_search([0.001, 0.01, 0.1]),
"beta": tune.choice([1, 2, 3])
})
print("Best config: ", analysis.get_best_config(metric="mean_loss", mode="min"))
# Get a dataframe for analyzing trial results.df=analysis.results_dfIf TensorBoard is installed, automatically visualize all trial results:
tensorboard --logdir ~/ray_results
RLlib is an open-source library for reinforcement learning built on top of Ray that offers both high scalability and a unified API for a variety of applications.
pip install tensorflow # or tensorflow-gpu
pip install "ray[rllib]"importgymfromgym.spacesimportDiscrete, BoxfromrayimporttuneclassSimpleCorridor(gym.Env):
def__init__(self, config):
self.end_pos=config["corridor_length"]
self.cur_pos=0self.action_space=Discrete(2)
self.observation_space=Box(0.0, self.end_pos, shape=(1, ))
defreset(self):
self.cur_pos=0return [self.cur_pos]
defstep(self, action):
ifaction==0andself.cur_pos>0:
self.cur_pos-=1elifaction==1:
self.cur_pos+=1done=self.cur_pos>=self.end_posreturn [self.cur_pos], 1ifdoneelse0, done, {}
tune.run(
"PPO",
config={
"env": SimpleCorridor,
"num_workers": 4,
"env_config": {"corridor_length": 5}})
Ray Serve is a scalable model-serving library built on Ray. It is:
- Framework Agnostic: Use the same toolkit to serve everything from deep learning models built with frameworks like PyTorch or Tensorflow & Keras to Scikit-Learn models or arbitrary business logic.
- Python First: Configure your model serving declaratively in pure Python, without needing YAMLs or JSON configs.
- Performance Oriented: Turn on batching, pipelining, and GPU acceleration to increase the throughput of your model.
- Composition Native: Allow you to create "model pipelines" by composing multiple models together to drive a single prediction.
- Horizontally Scalable: Serve can linearly scale as you add more machines. Enable your ML-powered service to handle growing traffic.
To run this example, you will need to install the following:
$ pip install scikit-learn
$ pip install "ray[serve]"This example runs serves a scikit-learn gradient boosting classifier.
fromrayimportserveimportpickleimportrequestsfromsklearn.datasetsimportload_irisfromsklearn.ensembleimportGradientBoostingClassifier# Train modeliris_dataset=load_iris()
model=GradientBoostingClassifier()
model.fit(iris_dataset["data"], iris_dataset["target"])
# Define Ray Serve model,classBoostingModel:
def__init__(self):
self.model=modelself.label_list=iris_dataset["target_names"].tolist()
def__call__(self, flask_request):
payload=flask_request.json["vector"]
print("Worker: received flask request with data", payload)
prediction=self.model.predict([payload])[0]
human_name=self.label_list[prediction]
return {"result": human_name}
# Deploy modelclient=serve.start()
client.create_backend("iris:v1", BoostingModel)
client.create_endpoint("iris_classifier", backend="iris:v1", route="/iris")
# Query it!sample_request_input= {"vector": [1.2, 1.0, 1.1, 0.9]}
response=requests.get("http://localhost:8000/iris", json=sample_request_input)
print(response.text)
# Result:# {# "result": "versicolor"# }- Documentation
- Tutorial
- Blog
- Ray 1.0 Architecture whitepaper(new)
- Ray Design Patterns(new)
- RLlib paper
- RLlib flow paper
- Tune paper
Older documents:
- Forum: For discussions about development, questions about usage, and feature requests.
- GitHub Issues: For reporting bugs.
- Twitter: Follow updates on Twitter.
- Meetup Group: Join our meetup group.
- StackOverflow: For questions about how to use Ray.