ONNX Runtime Ruby

🔥 ONNX Runtime - the high performance scoring engine for ML models - for Ruby

Check out an example

For transformer models, check out Informers

Installation

Add this line to your application’s Gemfile:

gem"onnxruntime"

On Mac x86-64, also run brew install onnxruntime to install the shared library

Getting Started

Load a model and make predictions

model=OnnxRuntime::Model.new("model.onnx")model.predict({x: [1,2,3]})

Download pre-trained models from the ONNX Model Zoo

Get inputs

model.inputs

Get outputs

model.outputs

Get metadata

model.metadata

Load a model from a string or other IO object

io=StringIO.new("...")model=OnnxRuntime::Model.new(io)

Get specific outputs

model.predict({x: [1,2,3]},output_names: ["label"])

Session Options

OnnxRuntime::Model.new(path_or_io,enable_cpu_mem_arena: true,enable_mem_pattern: true,enable_profiling: false,execution_mode: :sequential,# :sequential or :parallelfree_dimension_overrides_by_denotation: nil,free_dimension_overrides_by_name: nil,graph_optimization_level: nil,# :none, :basic, :extended, or :allinter_op_num_threads: nil,intra_op_num_threads: nil,log_severity_level: 2,log_verbosity_level: 0,logid: nil,optimized_model_filepath: nil,profile_file_prefix: "onnxruntime_profile_",session_config_entries: nil)

Run Options

model.predict(input_feed,output_names: nil,log_severity_level: 2,log_verbosity_level: 0,logid: nil,terminate: false,output_type: :ruby# :ruby or :numo)

Inference Session API

You can also use the Inference Session API, which follows the Python API.

session=OnnxRuntime::InferenceSession.new("model.onnx")session.run(nil,{x: [1,2,3]})

The Python example models are included as well.

OnnxRuntime::Datasets.example("sigmoid.onnx")

GPU Support

Linux and Windows

Download the appropriate GPU release and set:

OnnxRuntime.ffi_lib="path/to/lib/libonnxruntime.so"# onnxruntime.dll for Windows

and use:

model=OnnxRuntime::Model.new("model.onnx",providers: ["CUDAExecutionProvider"])

Mac

Use:

model=OnnxRuntime::Model.new("model.onnx",providers: ["CoreMLExecutionProvider"])

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

• Report bugs
• Fix bugs and submit pull requests
• Write, clarify, or fix documentation
• Suggest or add new features

To get started with development and testing:

git clone https://github.com/ankane/onnxruntime-ruby.git
cd onnxruntime-ruby
bundle install
bundle exec rake vendor:all
bundle exec rake test