A comprehensive Python SDK for HoneyHive, providing LLM observability, evaluation, and tracing capabilities with OpenTelemetry integration.
- OpenTelemetry Integration - Full OTEL compliance with custom span processor and exporter
- Automatic Session Management - Seamless session creation and management
- Decorator Support - Easy-to-use
@trace(unified sync/async),@atrace, and@trace_classdecorators - Context Managers -
start_spanandenrich_spanfor manual span management - HTTP Instrumentation - Automatic HTTP request tracing
- Baggage Support - Context propagation across service boundaries
- Experiment Harness Integration - Automatic experiment tracking with MLflow, Weights & Biases, and Comet support
- Real-time API Integration - Direct integration with HoneyHive backend services
- Comprehensive Testing - Full test suite with 203 passing tests
Choose Your Instrumentor Type:
HoneyHive supports both OpenInference (lightweight) and OpenLLMetry (enhanced metrics) instrumentors.
Option A: OpenInference (Recommended for Beginners)
# Install with OpenAI integration (most common)
pip install honeyhive[openinference-openai]
# Install with Anthropic integration
pip install honeyhive[openinference-anthropic]
# Install with Google AI integration
pip install honeyhive[openinference-google-ai]
# Install with multiple providers
pip install honeyhive[openinference-openai,openinference-anthropic,openinference-google-ai]
# Install all OpenInference integrations
pip install honeyhive[all-openinference]Option B: OpenLLMetry (Enhanced Metrics)
# Install with OpenAI integration (enhanced metrics)
pip install honeyhive[traceloop-openai]
# Install with Anthropic integration
pip install honeyhive[traceloop-anthropic]
# Install with Google AI integration
pip install honeyhive[traceloop-google-ai]
# Install with multiple providers
pip install honeyhive[traceloop-openai,traceloop-anthropic,traceloop-google-ai]
# Install all OpenLLMetry integrations
pip install honeyhive[all-traceloop]Option C: Mix Both Types
# Strategic mixing based on your needs
pip install honeyhive[traceloop-openai,openinference-anthropic]Basic Installation (manual instrumentor setup required):
pip install honeyhive📋 Including in Your Project
For detailed guidance on including HoneyHive in your pyproject.toml, see our pyproject.toml Integration Guide.
fromhoneyhiveimportHoneyHiveTracer, trace# Initialize tracertracer=HoneyHiveTracer.init(
api_key="your-api-key",
source="production"
)
# Use unified decorator for automatic tracing (works with both sync and async)@trace(event_type="demo", event_name="my_function")defmy_function():
return"Hello, World!"@trace(event_type="demo", event_name="my_async_function")asyncdefmy_async_function():
awaitasyncio.sleep(0.1)
return"Hello, Async World!"# Manual span managementwithtracer.start_span("custom-operation"):
# Your code herepass# With HTTP tracing enabled (new simplified API)tracer=HoneyHiveTracer.init(
api_key="your-api-key",
source="production",
disable_http_tracing=False# project derived from API key
)The HoneyHiveTracer.init() method is the recommended way to initialize the tracer:
fromhoneyhiveimportHoneyHiveTracer# Standard initializationtracer=HoneyHiveTracer.init(
api_key="your-api-key",
source="production"# project derived from API key
)
# With custom server URL for self-hosted deploymentstracer=HoneyHiveTracer.init(
api_key="your-api-key",
source="production",
server_url="https://custom-server.com"# project derived from API key
)fromhoneyhiveimportHoneyHiveTracerfromopeninference.instrumentation.openaiimportOpenAIInstrumentor# All features are available in the init methodtracer=HoneyHiveTracer.init(
api_key="your-api-key",
source="production",
test_mode=True, # Test mode supportinstrumentors=[OpenAIInstrumentor()], # Auto-integrationdisable_http_tracing=True# Performance control
)✅ The init method now supports ALL constructor features!
fromhoneyhiveimportHoneyHiveTracerfromopeninference.instrumentation.openaiimportOpenAIInstrumentor# Initialize tracer with OpenInference instrumentor (recommended pattern)tracer=HoneyHiveTracer.init(
api_key="your-api-key",
source="production",
instrumentors=[OpenAIInstrumentor()] # Auto-integration
)
# OpenInference automatically traces OpenAI callsimportopenairesponse=openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hello!"}]
)v1.0+ Recommended Pattern: Instance Methods
fromhoneyhiveimportHoneyHiveTracer# Initialize tracertracer=HoneyHiveTracer.init(
api_key="your-api-key",
)
# Use instance methods for enrichment (PRIMARY - Recommended)@tracer.trace(event_type="tool")defmy_function(input_data):
result=process_data(input_data)
# ✅ Instance method (PRIMARY pattern in v1.0+)tracer.enrich_span(
metadata={"input": input_data, "result": result},
metrics={"processing_time_ms": 150}
)
returnresult# Enrich session with user propertiestracer.enrich_session(
user_properties={"user_id": "user-123", "plan": "premium"}
)Legacy Pattern: Free Functions (Backward Compatibility)
For backward compatibility, the free function pattern from v0.2.x still works:
fromhoneyhiveimporttrace, enrich_span, enrich_session# Free functions with automatic tracer discovery (LEGACY)@trace(event_type="tool")defmy_function(input_data):
result=process_data(input_data)
# Free function with auto-discovery (backward compatible)enrich_span(
metadata={"input": input_data, "result": result},
metrics={"processing_time_ms": 150}
)
returnresult# Enrich session via free functionenrich_session(user_properties={"user_id": "user-123"})Why Instance Methods?
- ✅ Explicit tracer reference (no auto-discovery overhead)
- ✅ Better multi-instance support (multiple tracers in same process)
- ✅ Clearer code (explicit is better than implicit)
- ✅ Future-proof (primary pattern going forward)
src/honeyhive/
├── api/ # API client implementations
│ ├── client.py # Main API client
│ ├── configurations.py # Configuration management
│ ├── datapoints.py # Data point operations
│ ├── datasets.py # Dataset operations
│ ├── events.py # Event management
│ ├── evaluations.py # Evaluation operations
│ ├── metrics.py # Metrics operations
│ ├── projects.py # Project management
│ ├── session.py # Session operations
│ └── tools.py # Tool operations
├── tracer/ # OpenTelemetry integration
│ ├── otel_tracer.py # Main tracer implementation
│ ├── span_processor.py # Custom span processor
│ ├── span_exporter.py # Custom span exporter
│ ├── decorators.py # Tracing decorators
│ └── http_instrumentation.py # HTTP request tracing
├── evaluation/ # Evaluation framework
│ └── evaluators.py # Evaluation decorators
├── models/ # Pydantic models
│ └── generated.py # Auto-generated from OpenAPI
└── utils/ # Utility functions
├── config.py # Configuration management
├── connection_pool.py # HTTP connection pooling
├── retry.py # Retry mechanisms
└── logger.py # Logging utilities
- Singleton Pattern - Single tracer instance per application
- Environment Configuration - Flexible configuration via environment variables
- Graceful Degradation - Fallback mechanisms for missing dependencies
- Test Isolation - Comprehensive test suite with proper isolation
- OpenTelemetry Compliance - Full OTEL standard compliance
| Variable | Description | Default |
|---|---|---|
HH_API_KEY | HoneyHive API key | Required |
HH_API_URL | API base URL | https://api.dp1.us.honeyhive.ai |
HH_SOURCE | Source environment | production |
HH_DISABLE_TRACING | Disable tracing completely | false |
HH_DISABLE_HTTP_TRACING | Disable HTTP request tracing | false |
HH_TEST_MODE | Enable test mode | false |
HH_DEBUG_MODE | Enable debug mode | false |
HH_VERBOSE | Enable verbose API logging | false |
HH_OTLP_ENABLED | Enable OTLP export | true |
| Variable | Description | Default |
|---|---|---|
HH_EXPERIMENT_ID | Unique experiment identifier | None |
HH_EXPERIMENT_NAME | Human-readable experiment name | None |
HH_EXPERIMENT_VARIANT | Experiment variant/treatment | None |
HH_EXPERIMENT_GROUP | Experiment group/cohort | None |
HH_EXPERIMENT_METADATA | JSON experiment metadata | None |
| Variable | Description | Default |
|---|---|---|
HH_MAX_CONNECTIONS | Maximum HTTP connections | 100 |
HH_MAX_KEEPALIVE_CONNECTIONS | Keepalive connections | 20 |
HH_KEEPALIVE_EXPIRY | Keepalive expiry (seconds) | 30.0 |
HH_POOL_TIMEOUT | Connection pool timeout | 30.0 |
HH_RATE_LIMIT_CALLS | Rate limit calls per window | 1000 |
HH_RATE_LIMIT_WINDOW | Rate limit window (seconds) | 60.0 |
HH_HTTP_PROXY | HTTP proxy URL | None |
HH_HTTPS_PROXY | HTTPS proxy URL | None |
HH_NO_PROXY | Proxy bypass list | None |
HH_VERIFY_SSL | SSL verification | true |
Want to contribute to HoneyHive? See CONTRIBUTING.md for development setup and guidelines.