session-affinity

Here are 2 public repositories matching this topic...

shared-goals / hermes-custom-header-plugin

Hermes Agent plugin for fail-closed, session-derived custom request headers across named local and OpenAI-compatible providers.

plugin llm local-inference custom-headers olla session-affinity openai-compatible hermes-agent

Updated Jul 29, 2026
Python

rajeev-chaurasia / agent-serve

Star

One OpenAI-compatible endpoint in front of three vLLM replicas. Pins each conversation to the same GPU so its KV cache stays warm, sends cheap work to the small tier, and tracks token budgets per agent.

python portfolio prometheus load-balancing fastapi opentelemetry kv-cache rendezvous-hashing llmops llm-serving vllm session-affinity inference-gateway

Updated Aug 10, 2026
Python

Improve this page

Add a description, image, and links to the session-affinity topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the session-affinity topic, visit your repo's landing page and select "manage topics."

Learn more