Langtrace Python SDK

Open Source & Open Telemetry(OTEL) Observability for LLM Applications

---

📚 Table of Contents

• ✨ Features
• 🚀 Quick Start
• 🔗 Integrations
• 🌐 Getting Started
• ⚙️ Configuration
• 🔧 Advanced Features
• 📐 Examples
• 🏠 Self Hosting
• 🤝 Contributing
• 🔒 Security
• ❓ FAQ
• 📜 License

Langtrace is an open source observability software which lets you capture, debug and analyze traces and metrics from all your applications that leverages LLM APIs, Vector Databases and LLM based Frameworks.

✨ Features

• 📊 Open Telemetry Support: Built on OTEL standards for comprehensive tracing
• 🔄 Real-time Monitoring: Track LLM API calls, vector operations, and framework usage
• 🎯 Performance Insights: Analyze latency, costs, and usage patterns
• 🔍 Debug Tools: Trace and debug your LLM application workflows
• 📈 Analytics: Get detailed metrics and visualizations
• 🛠️ Framework Support: Extensive integration with popular LLM frameworks
• 🔌 Vector DB Integration: Support for major vector databases
• 🎨 Flexible Configuration: Customizable tracing and monitoring options

🚀 Quick Start

pip install langtrace-python-sdk

fromlangtrace_python_sdkimportlangtracelangtrace.init(api_key='<your_api_key>') # Get your API key at langtrace.ai

🔗 Supported Integrations

Langtrace automatically captures traces from the following vendors:

LLM Providers

Provider	TypeScript SDK	Python SDK
OpenAI	✅	✅
Anthropic	✅	✅
Azure OpenAI	✅	✅
Cohere	✅	✅
Groq	✅	✅
Perplexity	✅	✅
Gemini	❌	✅
Mistral	❌	✅
AWS Bedrock	✅	✅
Ollama	❌	✅
Cerebras	❌	✅

Frameworks

Framework	TypeScript SDK	Python SDK
Langchain	❌	✅
LlamaIndex	✅	✅
Langgraph	❌	✅
LiteLLM	❌	✅
DSPy	❌	✅
CrewAI	❌	✅
VertexAI	✅	✅
EmbedChain	❌	✅
Autogen	❌	✅
HiveAgent	❌	✅
Inspect AI	❌	✅
Graphlit	❌	✅
Phidata	❌	✅
Arch	❌	✅

Vector Databases

Database	TypeScript SDK	Python SDK
Pinecone	✅	✅
ChromaDB	✅	✅
QDrant	✅	✅
Weaviate	✅	✅
PGVector	✅	✅ (SQLAlchemy)
MongoDB	❌	✅
Milvus	❌	✅

🌐 Getting Started

Langtrace Cloud ☁️

1. Sign up by going to this link.
2. Create a new Project after signing up. Projects are containers for storing traces and metrics generated by your application. If you have only one application, creating 1 project will do.
3. Generate an API key by going inside the project.
4. In your application, install the Langtrace SDK and initialize it with the API key you generated in the step 3.
5. The code for installing and setting up the SDK is shown below

Framework Quick Starts

FastAPI

fromfastapiimportFastAPIfromlangtrace_python_sdkimportlangtracefromopenaiimportOpenAIlangtrace.init()
app=FastAPI()
client=OpenAI()
@app.get("/")defroot():
client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Say this is a test"}],
stream=False,
)
return {"Hello": "World"}

Django

# settings.pyfromlangtrace_python_sdkimportlangtracelangtrace.init()
# views.pyfromdjango.httpimportJsonResponsefromopenaiimportOpenAIclient=OpenAI()
defchat_view(request):
response=client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": request.GET.get('message', '')}]
)
returnJsonResponse({"response": response.choices[0].message.content})

Flask

fromflaskimportFlaskfromlangtrace_python_sdkimportlangtracefromopenaiimportOpenAIapp=Flask(__name__)
langtrace.init()
client=OpenAI()
@app.route('/')defchat():
response=client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Hello!"}]
)
return {"response": response.choices[0].message.content}

LangChain

fromlangtrace_python_sdkimportlangtracefromlangchain.chat_modelsimportChatOpenAIfromlangchain.promptsimportChatPromptTemplatelangtrace.init()
# LangChain operations are automatically tracedchat=ChatOpenAI()
prompt=ChatPromptTemplate.from_messages([
("system", "You are a helpful assistant."),
("user", "{input}")
])
chain=prompt|chatresponse=chain.invoke({"input": "Hello!"})

LlamaIndex

fromlangtrace_python_sdkimportlangtracefromllama_indeximportVectorStoreIndex, SimpleDirectoryReaderlangtrace.init()
# Document loading and indexing are automatically traceddocuments=SimpleDirectoryReader('data').load_data()
index=VectorStoreIndex.from_documents(documents)
# Queries are traced with metadataquery_engine=index.as_query_engine()
response=query_engine.query("What's in the documents?")

DSPy

fromlangtrace_python_sdkimportlangtraceimportdspyfromdspy.telepromptimportBootstrapFewShotlangtrace.init()
# DSPy operations are automatically tracedlm=dspy.OpenAI(model="gpt-4")
dspy.settings.configure(lm=lm)
classSimpleQA(dspy.Signature):
"""Answer questions with short responses."""question=dspy.InputField()
answer=dspy.OutputField(desc="short answer")
compiler=BootstrapFewShot(metric=dspy.metrics.Answer())
program=compiler.compile(SimpleQA)

CrewAI

fromlangtrace_python_sdkimportlangtracefromcrewaiimportAgent, Task, Crewlangtrace.init()
# Agents and tasks are automatically tracedresearcher=Agent(
role="Researcher",
goal="Research and analyze data",
backstory="Expert data researcher",
allow_delegation=False
)
task=Task(
description="Analyze market trends",
agent=researcher
)
crew=Crew(
agents=[researcher],
tasks=[task]
)
result=crew.kickoff()

For more detailed examples and framework-specific features, visit our documentation.

⚙️ Configuration

Initialize Options

The SDK can be initialized with various configuration options to customize its behavior:

langtrace.init(
api_key: Optional[str] =None, # API key for authenticationbatch: bool=True, # Enable/disable batch processingwrite_spans_to_console: bool=False, # Console loggingcustom_remote_exporter: Optional[Any] =None, # Custom exporterapi_host: Optional[str] =None, # Custom API hostdisable_instrumentations: Optional[Dict] =None, # Disable specific integrationsservice_name: Optional[str] =None, # Custom service namedisable_logging: bool=False, # Disable all loggingheaders: Dict[str, str] = {}, # Custom headers
)

Configuration Details

Parameter	Type	Default Value	Description
api_key	str	LANGTRACE_API_KEY or None	The API key for authentication. Can be set via environment variable
batch	bool	True	Whether to batch spans before sending them to reduce API calls
write_spans_to_console	bool	False	Enable console logging for debugging purposes
custom_remote_exporter	Optional[Exporter]	None	Custom exporter for sending traces to your own backend
api_host	Optional[str]	https://langtrace.ai/	Custom API endpoint for self-hosted deployments
disable_instrumentations	Optional[Dict]	None	Disable specific vendor instrumentations (e.g., {'only': ['openai']})
service_name	Optional[str]	None	Custom service name for trace identification
disable_logging	bool	False	Disable SDK logging completely
headers	Dict[str, str]	{}	Custom headers for API requests

Environment Variables

Configure Langtrace behavior using these environment variables:

Variable	Description	Default	Impact
LANGTRACE_API_KEY	Primary authentication method	Required*	Required if not passed to init()
TRACE_PROMPT_COMPLETION_DATA	Control prompt/completion tracing	true	Set to 'false' to opt out of prompt/completion data collection
TRACE_DSPY_CHECKPOINT	Control DSPy checkpoint tracing	true	Set to 'false' to disable checkpoint tracing
LANGTRACE_ERROR_REPORTING	Control error reporting	true	Set to 'false' to disable Sentry error reporting
LANGTRACE_API_HOST	Custom API endpoint	https://langtrace.ai/	Override default API endpoint for self-hosted deployments

Performance Note: Setting TRACE_DSPY_CHECKPOINT=false is recommended in production environments as checkpoint tracing involves state serialization which can impact latency.

Security Note: When TRACE_PROMPT_COMPLETION_DATA=false, no prompt or completion data will be collected, ensuring sensitive information remains private.

🔧 Advanced Features

Root Span Decorator

Use the root span decorator to create custom trace hierarchies:

fromlangtrace_python_sdkimportlangtrace@langtrace.with_langtrace_root_span(name="custom_operation")defmy_function():
# Your code herepass

Additional Attributes

Inject custom attributes into your traces:

# Using decorator@langtrace.with_additional_attributes({"custom_key": "custom_value"})defmy_function():
pass# Using context managerwithlangtrace.inject_additional_attributes({"custom_key": "custom_value"}):
# Your code herepass

Prompt Registry

Register and manage prompts for better traceability:

fromlangtrace_python_sdkimportlangtrace# Register a prompt templatelangtrace.register_prompt("greeting", "Hello, {name}!")
# Use registered promptresponse=client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": langtrace.get_prompt("greeting", name="Alice")}]
)

User Feedback System

Collect and analyze user feedback:

fromlangtrace_python_sdkimportlangtrace# Record user feedback for a tracelangtrace.record_feedback(
trace_id="your_trace_id",
rating=5,
feedback_text="Great response!",
metadata={"user_id": "123"}
)

DSPy Checkpointing

Manage DSPy checkpoints for workflow tracking:

fromlangtrace_python_sdkimportlangtrace# Enable checkpoint tracing (disabled by default in production)langtrace.init(
api_key="your_api_key",
dspy_checkpoint_tracing=True
)

Vector Database Operations

Track vector database operations:

fromlangtrace_python_sdkimportlangtrace# Vector operations are automatically tracedwithlangtrace.inject_additional_attributes({"operation_type": "similarity_search"}):
results=vector_db.similarity_search("query", k=5)

For more detailed examples and use cases, visit our documentation.

📐 Examples

🏠 Langtrace Self Hosted

Get started with self-hosted Langtrace:

fromlangtrace_python_sdkimportlangtracelangtrace.init(write_spans_to_console=True) # For console logging# ORlangtrace.init(custom_remote_exporter=<your_exporter>, batch=<TrueorFalse>) # For custom exporter

🤝 Contributing

We welcome contributions! To get started:

1. Fork this repository and start developing
2. Join our Discord workspace
3. Run examples:

   # In run_example.py, set ENABLED_EXAMPLES flag to True for desired examplepythonsrc/run_example.py

4. Run tests:

   pipinstall'.[test]'&&pipinstall'.[dev]'pytest-v

🔒 Security

To report security vulnerabilities, email us at security@scale3labs.com. You can read more on security here.

❓ Frequently Asked Questions

📜 License

Langtrace Python SDK is licensed under the Apache 2.0 License. You can read about this license here.