gliner-api

Easily configurable API & frontend providing simple access to dynamic NER models

Features

• FastAPI backend for serving GLiNER models (NER).
• Gradio frontend (optional) for interactive use.
• Prometheus metrics endpoint (/metrics).
• Configurable via YAML, CLI, or environment variables.
• Docker and Docker Compose support.
• ONNX inference support (including quantized models).
• API key authentication (optional).
• Custom metrics port and enable/disable option for Prometheus metrics.

---

Documentation

For detailed documentation, see DeepWiki (⚠️ AI-generated)

---

Live Demo

You can try the live demo of the GLiNER API container in it's Huggingface Space: GLiNER API Demo.

It uses a minimally changed image to make it work in the Huggingface Space environment.

---

Usage

Run with Docker

You can either build the container yourself or use a prebuilt image from GitHub Container Registry.

Run prebuilt container (recommended)

docker run \
-p 8080:8080 \
-p 9090:9090 \
-v $(pwd)/config.yaml:/app/config.yaml \
-v $HOME/.cache/huggingface:/app/huggingface \
ghcr.io/freinold/gliner-api:latest

Mounting volumes:

• -v $(pwd)/config.yaml:/app/config.yaml mounts your config file (edit as needed)
• -v $HOME/.cache/huggingface:/app/huggingface mounts your Huggingface cache for faster model loading

Build and run locally

docker build \
-f Dockerfile \
--build-arg IMAGE_CREATED="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--build-arg IMAGE_REVISION="$(git rev-parse HEAD)" \
--build-arg IMAGE_VERSION="$(git describe --tags --always)" \
-t gliner-api .
docker run --rm \
-p 8080:8080 \
-p 9090:9090 \
-v $(pwd)/example_configs/general.yaml:/app/config.yaml \
-v $HOME/.cache/huggingface:/app/huggingface \
gliner-api

PowerShell version

docker build `-f Dockerfile `--build-arg IMAGE_CREATED="$(Get-Date-Format 'yyyy-MM-ddTHH:mm:ssZ')"`--build-arg IMAGE_REVISION="$(git rev-parse HEAD)"`--build-arg IMAGE_VERSION="$(git describe --tags --always)"`-t gliner-api .
docker run --rm `-p 8080:8080`-p 9090:9090`-v "$PWD/example_configs/general.yaml:/app/config.yaml"`-v "$HOME/.cache/huggingface:/app/huggingface"`
gliner-api

---

Run with Docker Compose

Edit compose.yaml to select the config you want (see example_configs).

Then run:

docker compose -f compose.yaml up

---

Run the app directly

Be sure to check the installation instructions first.

uv run main.py [OPTIONS]

Or with FastAPI CLI:

fastapi run main.py --host localhost

Run options

uv run main.py --help

Option	Description	Default
--use-case / --name	Use case for the GLiNER model (application/domain)	general
--model-id	Huggingface model ID (browse models)	knowledgator/gliner-x-base
--onnx-enabled	Use ONNX for inference	False
--onnx-model-path	Path to ONNX model file	model.onnx
--default-entities	Default entities to detect	['person', 'organization', 'location', 'date']
--default-threshold	Default detection threshold	0.5
--api-key	API key for authentication (if set, required in requests)	null
--host	Host address	"" (bind to all interfaces)
--port	Port	8080
--metrics-enabled	Enable Prometheus metrics endpoint	True
--metrics-port	Port for Prometheus metrics endpoint	9090
--frontend-enabled	Enable Gradio frontend	True

---

API & Frontend Endpoints

Description	Path	Demo Link
Gradio Frontend (if enabled)	/	Frontend
API Docs (Swagger)	/docs	Swagger UI
API Docs (ReDoc)	/redoc	ReDoc
Prometheus Metrics	/metrics	(no public demo link; available on metrics port if enabled)

---

Example Request

curl -X POST "http://localhost:8080/api/invoke" -H "Content-Type: application/json" -d '{"text": "Steve Jobs founded Apple in Cupertino."}'

---

Installation

Prerequisites:

• Python 3.13.9
• uv (for dependency management)

Install from source:

git clone https://github.com/freinold/gliner-api.git
cd gliner-api
uv sync

---

Configuration

You can configure the app via:

• config.yaml (default, see example_configs/)
• CLI options (see above)
• Environment variables (prefix: GLINER_API_)

Example configs:

• example_configs/general.yaml (default NER)
• example_configs/pii.yaml (PII detection)
• example_configs/medical.yaml (medical NER)
• example_configs/general_onnx.yaml (ONNX inference)
• example_configs/general_onnx_quantized.yaml (quantized ONNX)

---

Used Frameworks & Libraries

• FastAPI (API backend)
• Gradio (optional frontend)
• Uvicorn (ASGI server)
• Prometheus Client (metrics)
• Huggingface Hub (model loading)
• PyTorch (CPU based inference)
• ONNX (optional, for ONNX models)
• uv (dependency management)

---

License

See LICENSE.

---