Python bindings for ggml

ℹ️ Note this fork of the original project is now archived. The original project is being kept up to date and integrates all changes that were added to this fork.

Python bindings for the ggml tensor library for machine learning.

Installation

Requirements

• Python 3.10+
• C compiler (gcc, clang, msvc, etc)

You can install ggml-py using pip:

pip install ggml-py

This will compile ggml using cmake which requires a c compiler installed on your system. To build ggml with specific features (ie. OpenBLAS, GPU Support, etc) you can pass specific cmake options through the cmake.args pip install configuration setting. For example to install ggml-py with cuBLAS support you can run:

pip install --upgrade pip
pip install ggml-py --config-settings=cmake.args='-DGGML_CUDA=ON'

Options

Option	Description	Default
GGML_CUDA	Enable cuBLAS support	OFF
GGML_CLBLAST	Enable CLBlast support	OFF
GGML_OPENBLAS	Enable OpenBLAS support	OFF
GGML_METAL	Enable Metal support	OFF
GGML_RPC	Enable RPC support	OFF

Usage

importggmlimportctypes# Allocate a new context with 16 MB of memoryparams=ggml.ggml_init_params(mem_size=16*1024*1024, mem_buffer=None)
ctx=ggml.ggml_init(params)
# Instantiate tensorsx=ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
a=ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
b=ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
# Use ggml operations to build a computational graphx2=ggml.ggml_mul(ctx, x, x)
f=ggml.ggml_add(ctx, ggml.ggml_mul(ctx, a, x2), b)
gf=ggml.ggml_new_graph(ctx)
ggml.ggml_build_forward_expand(gf, f)
# Set the input valuesggml.ggml_set_f32(x, 2.0)
ggml.ggml_set_f32(a, 3.0)
ggml.ggml_set_f32(b, 4.0)
# Compute the graphggml.ggml_graph_compute_with_ctx(ctx, gf, 1)
# Get the output valueoutput=ggml.ggml_get_f32_1d(f, 0)
assertoutput==16.0# Free the contextggml.ggml_free(ctx)

Troubleshooting

If you are having trouble installing ggml-py or activating specific features please try to install it with the --verbose and --no-cache-dir flags to get more information about any issues:

pip install ggml-py --verbose --no-cache-dir --force-reinstall --upgrade

License

This project is licensed under the terms of the MIT license.