Add LangChain integration to main package with auto_instrument() support by clutchski · Pull Request #1320 · braintrustdata/braintrust-sdk-javascript

Matt Perpick (clutchski) · 2026-01-29T22:07:02Z

Summary

Move LangChain wrapper from integrations/langchain-py into the main braintrust package
Enable auto-instrumentation via braintrust.auto_instrument()
Add setup_langchain() for manual setup with global callback handler

Test plan

nox -s "test_langchain(0.3.27)" passes (335 tests)
make fixup passes
Verify examples work: python py/examples/langchain/auto.py

🤖 Generated with Claude Code

Move LangChain wrapper from integrations/langchain-py into the main braintrust package, enabling auto-instrumentation via braintrust.auto_instrument(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The deprecation wrapper can be added after the new braintrust package is released. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Olmo Maldonado (ibolmo) · 2026-01-30T22:10:39Z

+ """
+ span = current_span()
+ if span == NOOP_SPAN:
+ init_logger(project=project_name, api_key=api_key, project_id=project_id)


do we know what happens if init logger is initialized up front without a project name etc.? I have vague recollection that it could make traces show up in project log instead of in the an ongoing eval.

if you want to add to this repo
importasynciofrombraintrustimportEvalAsync, Score, init_dataset, init_loggerfrombraintrust_langchainimportBraintrustCallbackHandler, set_global_handlerfromlangchain_core.messagesimportHumanMessage, SystemMessagefromlangchain_openaiimportChatOpenAIproject_name="test-braintrust-converted"logger=init_logger(project=project_name) set_global_handler(BraintrustCallbackHandler(logger=logger)) chat_model=ChatOpenAI(model="gpt-4o-mini", temperature=0) asyncdeftoxicity_classifier(inputs: dict) ->dict: instructions= ( "Please review the user query below and determine if it contains any form of toxic behavior, ""such as insults, threats, or highly negative comments. Respond with 'Toxic' if it does ""and 'Not toxic' if it doesn't." ) messages= [ SystemMessage(content=instructions), HumanMessage(content=inputs["text"]), ] result=awaitchat_model.ainvoke(messages) return {"class": result.content} examples= [ { "input": {"text": "Shut up, idiot"}, "expected": "Toxic", }, { "input": {"text": "You're a wonderful person"}, "expected": "Not toxic", }, { "input": {"text": "This is the worst thing ever"}, "expected": "Toxic", }, { "input": {"text": "I had a great day today"}, "expected": "Not toxic", }, { "input": {"text": "Nobody likes you"}, "expected": "Toxic", }, { "input": {"text": "This is unacceptable. I want to speak to the manager."}, "expected": "Not toxic", }, ] dataset=init_dataset(project=project_name, name="Toxic Queries") iflen(list(dataset.fetch())) ==0: forexampleinexamples: dataset.insert(**example) dataset.summarize() defcorrect(input, output, expected): returnScore( name="Correct", score=1ifoutput["class"] ==expectedelse0, ) asyncdefrun_evaluation(): awaitEvalAsync( project_name, data=dataset, task=toxicity_classifier, scores=[correct], experiment_name="gpt-4o-mini, baseline", metadata={"description": "Testing the baseline system."}, max_concurrency=4, ) if__name__=="__main__": asyncio.run(run_evaluation())

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:25Z

+# langchain requires Python >= 3.10
+# Note: langchain ecosystem packages have tight version coupling, so we pin
+# entire sets of compatible versions rather than testing "latest"
+LANGCHAIN_VERSIONS = ("0.3.27",)


there's a 1.x now too

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:51Z

+def test_langchain(session, version):
+ """Test LangChain integration."""
+ # langchain requires Python >= 3.10
+ if sys.version_info < (3, 10):


we don't support 3.9 anymore

Olmo Maldonado (ibolmo) · 2026-01-30T22:13:12Z

 # langsmith is needed for the wrapper module but not in VENDOR_PACKAGES
 session.install("langsmith")
+ # langchain dependencies for the langchain wrapper (pinned compatible versions)
+ session.install("langchain==0.3.27", "langchain-openai==0.3.35", "langchain-anthropic==0.3.22", "langgraph>=0.2.1,<0.4.0", "tenacity")


we should probably test 1.x stuff as well. there shouldn't be any breaking changes between 0.x and 1.x but good to have the coverage now

Olmo Maldonado (ibolmo) · 2026-01-30T22:15:13Z

+from .context import clear_global_handler, set_global_handler

-__all__ = ["BraintrustCallbackHandler", "set_global_handler"]
+__all__ = ["BraintrustCallbackHandler", "set_global_handler", "clear_global_handler"]


not sure we needed this change. should we just kill the source in the repo? the published pypi may be enough. perhaps we can save a tag or branch if we need to provide patch fixes.

Olmo Maldonado (ibolmo)

I would update and run the langchain.py golden tests https://github.com/braintrustdata/braintrust-sdk/blob/main/internal/golden/langchain.py

I also have a few (separate local repo) I'll try to add to the examples here

- Add LATEST to LANGCHAIN_VERSIONS for testing against newest releases - Remove redundant version pinning and explicit transitive deps (tenacity, pydantic) - Remove conditional skip for langgraph - it's now a required test dependency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Abhijeet Prasad (AbhiPrasad) · 2026-02-25T16:33:06Z

Going to take this PR over and apply it to https://github.com/braintrustdata/braintrust-sdk-python

Abhijeet Prasad (AbhiPrasad) · 2026-03-05T17:00:29Z

Gonna close - braintrustdata/braintrust-sdk-python#27 is tracking this to do it in the Python SDK.

Matt Perpick (clutchski) · 2026-01-29T22:07:02Z

Summary

Move LangChain wrapper from integrations/langchain-py into the main braintrust package
Enable auto-instrumentation via braintrust.auto_instrument()
Add setup_langchain() for manual setup with global callback handler

Test plan

nox -s "test_langchain(0.3.27)" passes (335 tests)
make fixup passes
Verify examples work: python py/examples/langchain/auto.py

🤖 Generated with Claude Code

Move LangChain wrapper from integrations/langchain-py into the main braintrust package, enabling auto-instrumentation via braintrust.auto_instrument(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The deprecation wrapper can be added after the new braintrust package is released. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Olmo Maldonado (ibolmo) · 2026-01-30T22:10:39Z

+ """
+ span = current_span()
+ if span == NOOP_SPAN:
+ init_logger(project=project_name, api_key=api_key, project_id=project_id)


do we know what happens if init logger is initialized up front without a project name etc.? I have vague recollection that it could make traces show up in project log instead of in the an ongoing eval.

if you want to add to this repo
importasynciofrombraintrustimportEvalAsync, Score, init_dataset, init_loggerfrombraintrust_langchainimportBraintrustCallbackHandler, set_global_handlerfromlangchain_core.messagesimportHumanMessage, SystemMessagefromlangchain_openaiimportChatOpenAIproject_name="test-braintrust-converted"logger=init_logger(project=project_name) set_global_handler(BraintrustCallbackHandler(logger=logger)) chat_model=ChatOpenAI(model="gpt-4o-mini", temperature=0) asyncdeftoxicity_classifier(inputs: dict) ->dict: instructions= ( "Please review the user query below and determine if it contains any form of toxic behavior, ""such as insults, threats, or highly negative comments. Respond with 'Toxic' if it does ""and 'Not toxic' if it doesn't." ) messages= [ SystemMessage(content=instructions), HumanMessage(content=inputs["text"]), ] result=awaitchat_model.ainvoke(messages) return {"class": result.content} examples= [ { "input": {"text": "Shut up, idiot"}, "expected": "Toxic", }, { "input": {"text": "You're a wonderful person"}, "expected": "Not toxic", }, { "input": {"text": "This is the worst thing ever"}, "expected": "Toxic", }, { "input": {"text": "I had a great day today"}, "expected": "Not toxic", }, { "input": {"text": "Nobody likes you"}, "expected": "Toxic", }, { "input": {"text": "This is unacceptable. I want to speak to the manager."}, "expected": "Not toxic", }, ] dataset=init_dataset(project=project_name, name="Toxic Queries") iflen(list(dataset.fetch())) ==0: forexampleinexamples: dataset.insert(**example) dataset.summarize() defcorrect(input, output, expected): returnScore( name="Correct", score=1ifoutput["class"] ==expectedelse0, ) asyncdefrun_evaluation(): awaitEvalAsync( project_name, data=dataset, task=toxicity_classifier, scores=[correct], experiment_name="gpt-4o-mini, baseline", metadata={"description": "Testing the baseline system."}, max_concurrency=4, ) if__name__=="__main__": asyncio.run(run_evaluation())

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:25Z

+# langchain requires Python >= 3.10
+# Note: langchain ecosystem packages have tight version coupling, so we pin
+# entire sets of compatible versions rather than testing "latest"
+LANGCHAIN_VERSIONS = ("0.3.27",)


there's a 1.x now too

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:51Z

+def test_langchain(session, version):
+ """Test LangChain integration."""
+ # langchain requires Python >= 3.10
+ if sys.version_info < (3, 10):


we don't support 3.9 anymore

Olmo Maldonado (ibolmo) · 2026-01-30T22:13:12Z

 # langsmith is needed for the wrapper module but not in VENDOR_PACKAGES
 session.install("langsmith")
+ # langchain dependencies for the langchain wrapper (pinned compatible versions)
+ session.install("langchain==0.3.27", "langchain-openai==0.3.35", "langchain-anthropic==0.3.22", "langgraph>=0.2.1,<0.4.0", "tenacity")


we should probably test 1.x stuff as well. there shouldn't be any breaking changes between 0.x and 1.x but good to have the coverage now

Olmo Maldonado (ibolmo) · 2026-01-30T22:15:13Z

+from .context import clear_global_handler, set_global_handler

-__all__ = ["BraintrustCallbackHandler", "set_global_handler"]
+__all__ = ["BraintrustCallbackHandler", "set_global_handler", "clear_global_handler"]


not sure we needed this change. should we just kill the source in the repo? the published pypi may be enough. perhaps we can save a tag or branch if we need to provide patch fixes.

Olmo Maldonado (ibolmo)

I would update and run the langchain.py golden tests https://github.com/braintrustdata/braintrust-sdk/blob/main/internal/golden/langchain.py

I also have a few (separate local repo) I'll try to add to the examples here

- Add LATEST to LANGCHAIN_VERSIONS for testing against newest releases - Remove redundant version pinning and explicit transitive deps (tenacity, pydantic) - Remove conditional skip for langgraph - it's now a required test dependency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Abhijeet Prasad (AbhiPrasad) · 2026-02-25T16:33:06Z

Going to take this PR over and apply it to https://github.com/braintrustdata/braintrust-sdk-python

Abhijeet Prasad (AbhiPrasad) · 2026-03-05T17:00:29Z

Gonna close - braintrustdata/braintrust-sdk-python#27 is tracking this to do it in the Python SDK.

Matt Perpick (clutchski) · 2026-01-29T22:07:02Z

Summary

Move LangChain wrapper from integrations/langchain-py into the main braintrust package
Enable auto-instrumentation via braintrust.auto_instrument()
Add setup_langchain() for manual setup with global callback handler

Test plan

nox -s "test_langchain(0.3.27)" passes (335 tests)
make fixup passes
Verify examples work: python py/examples/langchain/auto.py

🤖 Generated with Claude Code

Move LangChain wrapper from integrations/langchain-py into the main braintrust package, enabling auto-instrumentation via braintrust.auto_instrument(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The deprecation wrapper can be added after the new braintrust package is released. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Olmo Maldonado (ibolmo) · 2026-01-30T22:10:39Z

+ """
+ span = current_span()
+ if span == NOOP_SPAN:
+ init_logger(project=project_name, api_key=api_key, project_id=project_id)


do we know what happens if init logger is initialized up front without a project name etc.? I have vague recollection that it could make traces show up in project log instead of in the an ongoing eval.

if you want to add to this repo
importasynciofrombraintrustimportEvalAsync, Score, init_dataset, init_loggerfrombraintrust_langchainimportBraintrustCallbackHandler, set_global_handlerfromlangchain_core.messagesimportHumanMessage, SystemMessagefromlangchain_openaiimportChatOpenAIproject_name="test-braintrust-converted"logger=init_logger(project=project_name) set_global_handler(BraintrustCallbackHandler(logger=logger)) chat_model=ChatOpenAI(model="gpt-4o-mini", temperature=0) asyncdeftoxicity_classifier(inputs: dict) ->dict: instructions= ( "Please review the user query below and determine if it contains any form of toxic behavior, ""such as insults, threats, or highly negative comments. Respond with 'Toxic' if it does ""and 'Not toxic' if it doesn't." ) messages= [ SystemMessage(content=instructions), HumanMessage(content=inputs["text"]), ] result=awaitchat_model.ainvoke(messages) return {"class": result.content} examples= [ { "input": {"text": "Shut up, idiot"}, "expected": "Toxic", }, { "input": {"text": "You're a wonderful person"}, "expected": "Not toxic", }, { "input": {"text": "This is the worst thing ever"}, "expected": "Toxic", }, { "input": {"text": "I had a great day today"}, "expected": "Not toxic", }, { "input": {"text": "Nobody likes you"}, "expected": "Toxic", }, { "input": {"text": "This is unacceptable. I want to speak to the manager."}, "expected": "Not toxic", }, ] dataset=init_dataset(project=project_name, name="Toxic Queries") iflen(list(dataset.fetch())) ==0: forexampleinexamples: dataset.insert(**example) dataset.summarize() defcorrect(input, output, expected): returnScore( name="Correct", score=1ifoutput["class"] ==expectedelse0, ) asyncdefrun_evaluation(): awaitEvalAsync( project_name, data=dataset, task=toxicity_classifier, scores=[correct], experiment_name="gpt-4o-mini, baseline", metadata={"description": "Testing the baseline system."}, max_concurrency=4, ) if__name__=="__main__": asyncio.run(run_evaluation())

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:25Z

+# langchain requires Python >= 3.10
+# Note: langchain ecosystem packages have tight version coupling, so we pin
+# entire sets of compatible versions rather than testing "latest"
+LANGCHAIN_VERSIONS = ("0.3.27",)


there's a 1.x now too

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:51Z

+def test_langchain(session, version):
+ """Test LangChain integration."""
+ # langchain requires Python >= 3.10
+ if sys.version_info < (3, 10):


we don't support 3.9 anymore

Olmo Maldonado (ibolmo) · 2026-01-30T22:13:12Z

 # langsmith is needed for the wrapper module but not in VENDOR_PACKAGES
 session.install("langsmith")
+ # langchain dependencies for the langchain wrapper (pinned compatible versions)
+ session.install("langchain==0.3.27", "langchain-openai==0.3.35", "langchain-anthropic==0.3.22", "langgraph>=0.2.1,<0.4.0", "tenacity")


we should probably test 1.x stuff as well. there shouldn't be any breaking changes between 0.x and 1.x but good to have the coverage now

Olmo Maldonado (ibolmo) · 2026-01-30T22:15:13Z

+from .context import clear_global_handler, set_global_handler

-__all__ = ["BraintrustCallbackHandler", "set_global_handler"]
+__all__ = ["BraintrustCallbackHandler", "set_global_handler", "clear_global_handler"]


not sure we needed this change. should we just kill the source in the repo? the published pypi may be enough. perhaps we can save a tag or branch if we need to provide patch fixes.

Olmo Maldonado (ibolmo)

I would update and run the langchain.py golden tests https://github.com/braintrustdata/braintrust-sdk/blob/main/internal/golden/langchain.py

I also have a few (separate local repo) I'll try to add to the examples here

- Add LATEST to LANGCHAIN_VERSIONS for testing against newest releases - Remove redundant version pinning and explicit transitive deps (tenacity, pydantic) - Remove conditional skip for langgraph - it's now a required test dependency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Abhijeet Prasad (AbhiPrasad) · 2026-02-25T16:33:06Z

Going to take this PR over and apply it to https://github.com/braintrustdata/braintrust-sdk-python

Abhijeet Prasad (AbhiPrasad) · 2026-03-05T17:00:29Z

Gonna close - braintrustdata/braintrust-sdk-python#27 is tracking this to do it in the Python SDK.

Matt Perpick (clutchski) · 2026-01-29T22:07:02Z

Summary

Move LangChain wrapper from integrations/langchain-py into the main braintrust package
Enable auto-instrumentation via braintrust.auto_instrument()
Add setup_langchain() for manual setup with global callback handler

Test plan

nox -s "test_langchain(0.3.27)" passes (335 tests)
make fixup passes
Verify examples work: python py/examples/langchain/auto.py

🤖 Generated with Claude Code

Move LangChain wrapper from integrations/langchain-py into the main braintrust package, enabling auto-instrumentation via braintrust.auto_instrument(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The deprecation wrapper can be added after the new braintrust package is released. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Olmo Maldonado (ibolmo) · 2026-01-30T22:10:39Z

+ """
+ span = current_span()
+ if span == NOOP_SPAN:
+ init_logger(project=project_name, api_key=api_key, project_id=project_id)


do we know what happens if init logger is initialized up front without a project name etc.? I have vague recollection that it could make traces show up in project log instead of in the an ongoing eval.

if you want to add to this repo
importasynciofrombraintrustimportEvalAsync, Score, init_dataset, init_loggerfrombraintrust_langchainimportBraintrustCallbackHandler, set_global_handlerfromlangchain_core.messagesimportHumanMessage, SystemMessagefromlangchain_openaiimportChatOpenAIproject_name="test-braintrust-converted"logger=init_logger(project=project_name) set_global_handler(BraintrustCallbackHandler(logger=logger)) chat_model=ChatOpenAI(model="gpt-4o-mini", temperature=0) asyncdeftoxicity_classifier(inputs: dict) ->dict: instructions= ( "Please review the user query below and determine if it contains any form of toxic behavior, ""such as insults, threats, or highly negative comments. Respond with 'Toxic' if it does ""and 'Not toxic' if it doesn't." ) messages= [ SystemMessage(content=instructions), HumanMessage(content=inputs["text"]), ] result=awaitchat_model.ainvoke(messages) return {"class": result.content} examples= [ { "input": {"text": "Shut up, idiot"}, "expected": "Toxic", }, { "input": {"text": "You're a wonderful person"}, "expected": "Not toxic", }, { "input": {"text": "This is the worst thing ever"}, "expected": "Toxic", }, { "input": {"text": "I had a great day today"}, "expected": "Not toxic", }, { "input": {"text": "Nobody likes you"}, "expected": "Toxic", }, { "input": {"text": "This is unacceptable. I want to speak to the manager."}, "expected": "Not toxic", }, ] dataset=init_dataset(project=project_name, name="Toxic Queries") iflen(list(dataset.fetch())) ==0: forexampleinexamples: dataset.insert(**example) dataset.summarize() defcorrect(input, output, expected): returnScore( name="Correct", score=1ifoutput["class"] ==expectedelse0, ) asyncdefrun_evaluation(): awaitEvalAsync( project_name, data=dataset, task=toxicity_classifier, scores=[correct], experiment_name="gpt-4o-mini, baseline", metadata={"description": "Testing the baseline system."}, max_concurrency=4, ) if__name__=="__main__": asyncio.run(run_evaluation())

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:25Z

+# langchain requires Python >= 3.10
+# Note: langchain ecosystem packages have tight version coupling, so we pin
+# entire sets of compatible versions rather than testing "latest"
+LANGCHAIN_VERSIONS = ("0.3.27",)


there's a 1.x now too

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:51Z

+def test_langchain(session, version):
+ """Test LangChain integration."""
+ # langchain requires Python >= 3.10
+ if sys.version_info < (3, 10):


we don't support 3.9 anymore

Olmo Maldonado (ibolmo) · 2026-01-30T22:13:12Z

 # langsmith is needed for the wrapper module but not in VENDOR_PACKAGES
 session.install("langsmith")
+ # langchain dependencies for the langchain wrapper (pinned compatible versions)
+ session.install("langchain==0.3.27", "langchain-openai==0.3.35", "langchain-anthropic==0.3.22", "langgraph>=0.2.1,<0.4.0", "tenacity")


we should probably test 1.x stuff as well. there shouldn't be any breaking changes between 0.x and 1.x but good to have the coverage now

Olmo Maldonado (ibolmo) · 2026-01-30T22:15:13Z

+from .context import clear_global_handler, set_global_handler

-__all__ = ["BraintrustCallbackHandler", "set_global_handler"]
+__all__ = ["BraintrustCallbackHandler", "set_global_handler", "clear_global_handler"]


not sure we needed this change. should we just kill the source in the repo? the published pypi may be enough. perhaps we can save a tag or branch if we need to provide patch fixes.

Olmo Maldonado (ibolmo)

I would update and run the langchain.py golden tests https://github.com/braintrustdata/braintrust-sdk/blob/main/internal/golden/langchain.py

I also have a few (separate local repo) I'll try to add to the examples here

- Add LATEST to LANGCHAIN_VERSIONS for testing against newest releases - Remove redundant version pinning and explicit transitive deps (tenacity, pydantic) - Remove conditional skip for langgraph - it's now a required test dependency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Abhijeet Prasad (AbhiPrasad) · 2026-02-25T16:33:06Z

Going to take this PR over and apply it to https://github.com/braintrustdata/braintrust-sdk-python

Abhijeet Prasad (AbhiPrasad) · 2026-03-05T17:00:29Z

Gonna close - braintrustdata/braintrust-sdk-python#27 is tracking this to do it in the Python SDK.

Matt Perpick (clutchski) · 2026-01-29T22:07:02Z

Summary

Move LangChain wrapper from integrations/langchain-py into the main braintrust package
Enable auto-instrumentation via braintrust.auto_instrument()
Add setup_langchain() for manual setup with global callback handler

Test plan

nox -s "test_langchain(0.3.27)" passes (335 tests)
make fixup passes
Verify examples work: python py/examples/langchain/auto.py

🤖 Generated with Claude Code

Move LangChain wrapper from integrations/langchain-py into the main braintrust package, enabling auto-instrumentation via braintrust.auto_instrument(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The deprecation wrapper can be added after the new braintrust package is released. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Olmo Maldonado (ibolmo) · 2026-01-30T22:10:39Z

+ """
+ span = current_span()
+ if span == NOOP_SPAN:
+ init_logger(project=project_name, api_key=api_key, project_id=project_id)


do we know what happens if init logger is initialized up front without a project name etc.? I have vague recollection that it could make traces show up in project log instead of in the an ongoing eval.

if you want to add to this repo
importasynciofrombraintrustimportEvalAsync, Score, init_dataset, init_loggerfrombraintrust_langchainimportBraintrustCallbackHandler, set_global_handlerfromlangchain_core.messagesimportHumanMessage, SystemMessagefromlangchain_openaiimportChatOpenAIproject_name="test-braintrust-converted"logger=init_logger(project=project_name) set_global_handler(BraintrustCallbackHandler(logger=logger)) chat_model=ChatOpenAI(model="gpt-4o-mini", temperature=0) asyncdeftoxicity_classifier(inputs: dict) ->dict: instructions= ( "Please review the user query below and determine if it contains any form of toxic behavior, ""such as insults, threats, or highly negative comments. Respond with 'Toxic' if it does ""and 'Not toxic' if it doesn't." ) messages= [ SystemMessage(content=instructions), HumanMessage(content=inputs["text"]), ] result=awaitchat_model.ainvoke(messages) return {"class": result.content} examples= [ { "input": {"text": "Shut up, idiot"}, "expected": "Toxic", }, { "input": {"text": "You're a wonderful person"}, "expected": "Not toxic", }, { "input": {"text": "This is the worst thing ever"}, "expected": "Toxic", }, { "input": {"text": "I had a great day today"}, "expected": "Not toxic", }, { "input": {"text": "Nobody likes you"}, "expected": "Toxic", }, { "input": {"text": "This is unacceptable. I want to speak to the manager."}, "expected": "Not toxic", }, ] dataset=init_dataset(project=project_name, name="Toxic Queries") iflen(list(dataset.fetch())) ==0: forexampleinexamples: dataset.insert(**example) dataset.summarize() defcorrect(input, output, expected): returnScore( name="Correct", score=1ifoutput["class"] ==expectedelse0, ) asyncdefrun_evaluation(): awaitEvalAsync( project_name, data=dataset, task=toxicity_classifier, scores=[correct], experiment_name="gpt-4o-mini, baseline", metadata={"description": "Testing the baseline system."}, max_concurrency=4, ) if__name__=="__main__": asyncio.run(run_evaluation())

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:25Z

+# langchain requires Python >= 3.10
+# Note: langchain ecosystem packages have tight version coupling, so we pin
+# entire sets of compatible versions rather than testing "latest"
+LANGCHAIN_VERSIONS = ("0.3.27",)


there's a 1.x now too

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:51Z

+def test_langchain(session, version):
+ """Test LangChain integration."""
+ # langchain requires Python >= 3.10
+ if sys.version_info < (3, 10):


we don't support 3.9 anymore

Olmo Maldonado (ibolmo) · 2026-01-30T22:13:12Z

 # langsmith is needed for the wrapper module but not in VENDOR_PACKAGES
 session.install("langsmith")
+ # langchain dependencies for the langchain wrapper (pinned compatible versions)
+ session.install("langchain==0.3.27", "langchain-openai==0.3.35", "langchain-anthropic==0.3.22", "langgraph>=0.2.1,<0.4.0", "tenacity")


we should probably test 1.x stuff as well. there shouldn't be any breaking changes between 0.x and 1.x but good to have the coverage now

Olmo Maldonado (ibolmo) · 2026-01-30T22:15:13Z

+from .context import clear_global_handler, set_global_handler

-__all__ = ["BraintrustCallbackHandler", "set_global_handler"]
+__all__ = ["BraintrustCallbackHandler", "set_global_handler", "clear_global_handler"]


not sure we needed this change. should we just kill the source in the repo? the published pypi may be enough. perhaps we can save a tag or branch if we need to provide patch fixes.

Olmo Maldonado (ibolmo)

I would update and run the langchain.py golden tests https://github.com/braintrustdata/braintrust-sdk/blob/main/internal/golden/langchain.py

I also have a few (separate local repo) I'll try to add to the examples here

- Add LATEST to LANGCHAIN_VERSIONS for testing against newest releases - Remove redundant version pinning and explicit transitive deps (tenacity, pydantic) - Remove conditional skip for langgraph - it's now a required test dependency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Abhijeet Prasad (AbhiPrasad) · 2026-02-25T16:33:06Z

Going to take this PR over and apply it to https://github.com/braintrustdata/braintrust-sdk-python

Abhijeet Prasad (AbhiPrasad) · 2026-03-05T17:00:29Z

Gonna close - braintrustdata/braintrust-sdk-python#27 is tracking this to do it in the Python SDK.

Matt Perpick (clutchski) · 2026-01-29T22:07:02Z

Summary

Move LangChain wrapper from integrations/langchain-py into the main braintrust package
Enable auto-instrumentation via braintrust.auto_instrument()
Add setup_langchain() for manual setup with global callback handler

Test plan

nox -s "test_langchain(0.3.27)" passes (335 tests)
make fixup passes
Verify examples work: python py/examples/langchain/auto.py

🤖 Generated with Claude Code

Move LangChain wrapper from integrations/langchain-py into the main braintrust package, enabling auto-instrumentation via braintrust.auto_instrument(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The deprecation wrapper can be added after the new braintrust package is released. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Olmo Maldonado (ibolmo) · 2026-01-30T22:10:39Z

+ """
+ span = current_span()
+ if span == NOOP_SPAN:
+ init_logger(project=project_name, api_key=api_key, project_id=project_id)


do we know what happens if init logger is initialized up front without a project name etc.? I have vague recollection that it could make traces show up in project log instead of in the an ongoing eval.

if you want to add to this repo
importasynciofrombraintrustimportEvalAsync, Score, init_dataset, init_loggerfrombraintrust_langchainimportBraintrustCallbackHandler, set_global_handlerfromlangchain_core.messagesimportHumanMessage, SystemMessagefromlangchain_openaiimportChatOpenAIproject_name="test-braintrust-converted"logger=init_logger(project=project_name) set_global_handler(BraintrustCallbackHandler(logger=logger)) chat_model=ChatOpenAI(model="gpt-4o-mini", temperature=0) asyncdeftoxicity_classifier(inputs: dict) ->dict: instructions= ( "Please review the user query below and determine if it contains any form of toxic behavior, ""such as insults, threats, or highly negative comments. Respond with 'Toxic' if it does ""and 'Not toxic' if it doesn't." ) messages= [ SystemMessage(content=instructions), HumanMessage(content=inputs["text"]), ] result=awaitchat_model.ainvoke(messages) return {"class": result.content} examples= [ { "input": {"text": "Shut up, idiot"}, "expected": "Toxic", }, { "input": {"text": "You're a wonderful person"}, "expected": "Not toxic", }, { "input": {"text": "This is the worst thing ever"}, "expected": "Toxic", }, { "input": {"text": "I had a great day today"}, "expected": "Not toxic", }, { "input": {"text": "Nobody likes you"}, "expected": "Toxic", }, { "input": {"text": "This is unacceptable. I want to speak to the manager."}, "expected": "Not toxic", }, ] dataset=init_dataset(project=project_name, name="Toxic Queries") iflen(list(dataset.fetch())) ==0: forexampleinexamples: dataset.insert(**example) dataset.summarize() defcorrect(input, output, expected): returnScore( name="Correct", score=1ifoutput["class"] ==expectedelse0, ) asyncdefrun_evaluation(): awaitEvalAsync( project_name, data=dataset, task=toxicity_classifier, scores=[correct], experiment_name="gpt-4o-mini, baseline", metadata={"description": "Testing the baseline system."}, max_concurrency=4, ) if__name__=="__main__": asyncio.run(run_evaluation())

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:25Z

+# langchain requires Python >= 3.10
+# Note: langchain ecosystem packages have tight version coupling, so we pin
+# entire sets of compatible versions rather than testing "latest"
+LANGCHAIN_VERSIONS = ("0.3.27",)


there's a 1.x now too

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:51Z

+def test_langchain(session, version):
+ """Test LangChain integration."""
+ # langchain requires Python >= 3.10
+ if sys.version_info < (3, 10):


we don't support 3.9 anymore

Olmo Maldonado (ibolmo) · 2026-01-30T22:13:12Z

 # langsmith is needed for the wrapper module but not in VENDOR_PACKAGES
 session.install("langsmith")
+ # langchain dependencies for the langchain wrapper (pinned compatible versions)
+ session.install("langchain==0.3.27", "langchain-openai==0.3.35", "langchain-anthropic==0.3.22", "langgraph>=0.2.1,<0.4.0", "tenacity")


we should probably test 1.x stuff as well. there shouldn't be any breaking changes between 0.x and 1.x but good to have the coverage now

Olmo Maldonado (ibolmo) · 2026-01-30T22:15:13Z

+from .context import clear_global_handler, set_global_handler

-__all__ = ["BraintrustCallbackHandler", "set_global_handler"]
+__all__ = ["BraintrustCallbackHandler", "set_global_handler", "clear_global_handler"]


not sure we needed this change. should we just kill the source in the repo? the published pypi may be enough. perhaps we can save a tag or branch if we need to provide patch fixes.

Olmo Maldonado (ibolmo)

I would update and run the langchain.py golden tests https://github.com/braintrustdata/braintrust-sdk/blob/main/internal/golden/langchain.py

I also have a few (separate local repo) I'll try to add to the examples here

- Add LATEST to LANGCHAIN_VERSIONS for testing against newest releases - Remove redundant version pinning and explicit transitive deps (tenacity, pydantic) - Remove conditional skip for langgraph - it's now a required test dependency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Abhijeet Prasad (AbhiPrasad) · 2026-02-25T16:33:06Z

Going to take this PR over and apply it to https://github.com/braintrustdata/braintrust-sdk-python

Abhijeet Prasad (AbhiPrasad) · 2026-03-05T17:00:29Z

Gonna close - braintrustdata/braintrust-sdk-python#27 is tracking this to do it in the Python SDK.

Matt Perpick (clutchski) · 2026-01-29T22:07:02Z

Summary

Move LangChain wrapper from integrations/langchain-py into the main braintrust package
Enable auto-instrumentation via braintrust.auto_instrument()
Add setup_langchain() for manual setup with global callback handler

Test plan

nox -s "test_langchain(0.3.27)" passes (335 tests)
make fixup passes
Verify examples work: python py/examples/langchain/auto.py

🤖 Generated with Claude Code

Move LangChain wrapper from integrations/langchain-py into the main braintrust package, enabling auto-instrumentation via braintrust.auto_instrument(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The deprecation wrapper can be added after the new braintrust package is released. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Olmo Maldonado (ibolmo) · 2026-01-30T22:10:39Z

+ """
+ span = current_span()
+ if span == NOOP_SPAN:
+ init_logger(project=project_name, api_key=api_key, project_id=project_id)


do we know what happens if init logger is initialized up front without a project name etc.? I have vague recollection that it could make traces show up in project log instead of in the an ongoing eval.

if you want to add to this repo
importasynciofrombraintrustimportEvalAsync, Score, init_dataset, init_loggerfrombraintrust_langchainimportBraintrustCallbackHandler, set_global_handlerfromlangchain_core.messagesimportHumanMessage, SystemMessagefromlangchain_openaiimportChatOpenAIproject_name="test-braintrust-converted"logger=init_logger(project=project_name) set_global_handler(BraintrustCallbackHandler(logger=logger)) chat_model=ChatOpenAI(model="gpt-4o-mini", temperature=0) asyncdeftoxicity_classifier(inputs: dict) ->dict: instructions= ( "Please review the user query below and determine if it contains any form of toxic behavior, ""such as insults, threats, or highly negative comments. Respond with 'Toxic' if it does ""and 'Not toxic' if it doesn't." ) messages= [ SystemMessage(content=instructions), HumanMessage(content=inputs["text"]), ] result=awaitchat_model.ainvoke(messages) return {"class": result.content} examples= [ { "input": {"text": "Shut up, idiot"}, "expected": "Toxic", }, { "input": {"text": "You're a wonderful person"}, "expected": "Not toxic", }, { "input": {"text": "This is the worst thing ever"}, "expected": "Toxic", }, { "input": {"text": "I had a great day today"}, "expected": "Not toxic", }, { "input": {"text": "Nobody likes you"}, "expected": "Toxic", }, { "input": {"text": "This is unacceptable. I want to speak to the manager."}, "expected": "Not toxic", }, ] dataset=init_dataset(project=project_name, name="Toxic Queries") iflen(list(dataset.fetch())) ==0: forexampleinexamples: dataset.insert(**example) dataset.summarize() defcorrect(input, output, expected): returnScore( name="Correct", score=1ifoutput["class"] ==expectedelse0, ) asyncdefrun_evaluation(): awaitEvalAsync( project_name, data=dataset, task=toxicity_classifier, scores=[correct], experiment_name="gpt-4o-mini, baseline", metadata={"description": "Testing the baseline system."}, max_concurrency=4, ) if__name__=="__main__": asyncio.run(run_evaluation())

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:25Z

+# langchain requires Python >= 3.10
+# Note: langchain ecosystem packages have tight version coupling, so we pin
+# entire sets of compatible versions rather than testing "latest"
+LANGCHAIN_VERSIONS = ("0.3.27",)


there's a 1.x now too

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:51Z

+def test_langchain(session, version):
+ """Test LangChain integration."""
+ # langchain requires Python >= 3.10
+ if sys.version_info < (3, 10):


we don't support 3.9 anymore

Olmo Maldonado (ibolmo) · 2026-01-30T22:13:12Z

 # langsmith is needed for the wrapper module but not in VENDOR_PACKAGES
 session.install("langsmith")
+ # langchain dependencies for the langchain wrapper (pinned compatible versions)
+ session.install("langchain==0.3.27", "langchain-openai==0.3.35", "langchain-anthropic==0.3.22", "langgraph>=0.2.1,<0.4.0", "tenacity")


we should probably test 1.x stuff as well. there shouldn't be any breaking changes between 0.x and 1.x but good to have the coverage now

Olmo Maldonado (ibolmo) · 2026-01-30T22:15:13Z

+from .context import clear_global_handler, set_global_handler

-__all__ = ["BraintrustCallbackHandler", "set_global_handler"]
+__all__ = ["BraintrustCallbackHandler", "set_global_handler", "clear_global_handler"]


not sure we needed this change. should we just kill the source in the repo? the published pypi may be enough. perhaps we can save a tag or branch if we need to provide patch fixes.

Olmo Maldonado (ibolmo)

I would update and run the langchain.py golden tests https://github.com/braintrustdata/braintrust-sdk/blob/main/internal/golden/langchain.py

I also have a few (separate local repo) I'll try to add to the examples here

- Add LATEST to LANGCHAIN_VERSIONS for testing against newest releases - Remove redundant version pinning and explicit transitive deps (tenacity, pydantic) - Remove conditional skip for langgraph - it's now a required test dependency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Abhijeet Prasad (AbhiPrasad) · 2026-02-25T16:33:06Z

Going to take this PR over and apply it to https://github.com/braintrustdata/braintrust-sdk-python

Abhijeet Prasad (AbhiPrasad) · 2026-03-05T17:00:29Z

Gonna close - braintrustdata/braintrust-sdk-python#27 is tracking this to do it in the Python SDK.

Matt Perpick (clutchski) · 2026-01-29T22:07:02Z

Summary

Move LangChain wrapper from integrations/langchain-py into the main braintrust package
Enable auto-instrumentation via braintrust.auto_instrument()
Add setup_langchain() for manual setup with global callback handler

Test plan

nox -s "test_langchain(0.3.27)" passes (335 tests)
make fixup passes
Verify examples work: python py/examples/langchain/auto.py

🤖 Generated with Claude Code

Move LangChain wrapper from integrations/langchain-py into the main braintrust package, enabling auto-instrumentation via braintrust.auto_instrument(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The deprecation wrapper can be added after the new braintrust package is released. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Olmo Maldonado (ibolmo) · 2026-01-30T22:10:39Z

+ """
+ span = current_span()
+ if span == NOOP_SPAN:
+ init_logger(project=project_name, api_key=api_key, project_id=project_id)


do we know what happens if init logger is initialized up front without a project name etc.? I have vague recollection that it could make traces show up in project log instead of in the an ongoing eval.

if you want to add to this repo
importasynciofrombraintrustimportEvalAsync, Score, init_dataset, init_loggerfrombraintrust_langchainimportBraintrustCallbackHandler, set_global_handlerfromlangchain_core.messagesimportHumanMessage, SystemMessagefromlangchain_openaiimportChatOpenAIproject_name="test-braintrust-converted"logger=init_logger(project=project_name) set_global_handler(BraintrustCallbackHandler(logger=logger)) chat_model=ChatOpenAI(model="gpt-4o-mini", temperature=0) asyncdeftoxicity_classifier(inputs: dict) ->dict: instructions= ( "Please review the user query below and determine if it contains any form of toxic behavior, ""such as insults, threats, or highly negative comments. Respond with 'Toxic' if it does ""and 'Not toxic' if it doesn't." ) messages= [ SystemMessage(content=instructions), HumanMessage(content=inputs["text"]), ] result=awaitchat_model.ainvoke(messages) return {"class": result.content} examples= [ { "input": {"text": "Shut up, idiot"}, "expected": "Toxic", }, { "input": {"text": "You're a wonderful person"}, "expected": "Not toxic", }, { "input": {"text": "This is the worst thing ever"}, "expected": "Toxic", }, { "input": {"text": "I had a great day today"}, "expected": "Not toxic", }, { "input": {"text": "Nobody likes you"}, "expected": "Toxic", }, { "input": {"text": "This is unacceptable. I want to speak to the manager."}, "expected": "Not toxic", }, ] dataset=init_dataset(project=project_name, name="Toxic Queries") iflen(list(dataset.fetch())) ==0: forexampleinexamples: dataset.insert(**example) dataset.summarize() defcorrect(input, output, expected): returnScore( name="Correct", score=1ifoutput["class"] ==expectedelse0, ) asyncdefrun_evaluation(): awaitEvalAsync( project_name, data=dataset, task=toxicity_classifier, scores=[correct], experiment_name="gpt-4o-mini, baseline", metadata={"description": "Testing the baseline system."}, max_concurrency=4, ) if__name__=="__main__": asyncio.run(run_evaluation())

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:25Z

+# langchain requires Python >= 3.10
+# Note: langchain ecosystem packages have tight version coupling, so we pin
+# entire sets of compatible versions rather than testing "latest"
+LANGCHAIN_VERSIONS = ("0.3.27",)


there's a 1.x now too

Olmo Maldonado (ibolmo) · 2026-01-30T22:12:51Z

+def test_langchain(session, version):
+ """Test LangChain integration."""
+ # langchain requires Python >= 3.10
+ if sys.version_info < (3, 10):


we don't support 3.9 anymore

Olmo Maldonado (ibolmo) · 2026-01-30T22:13:12Z

 # langsmith is needed for the wrapper module but not in VENDOR_PACKAGES
 session.install("langsmith")
+ # langchain dependencies for the langchain wrapper (pinned compatible versions)
+ session.install("langchain==0.3.27", "langchain-openai==0.3.35", "langchain-anthropic==0.3.22", "langgraph>=0.2.1,<0.4.0", "tenacity")


we should probably test 1.x stuff as well. there shouldn't be any breaking changes between 0.x and 1.x but good to have the coverage now

Olmo Maldonado (ibolmo) · 2026-01-30T22:15:13Z

+from .context import clear_global_handler, set_global_handler

-__all__ = ["BraintrustCallbackHandler", "set_global_handler"]
+__all__ = ["BraintrustCallbackHandler", "set_global_handler", "clear_global_handler"]


not sure we needed this change. should we just kill the source in the repo? the published pypi may be enough. perhaps we can save a tag or branch if we need to provide patch fixes.

Olmo Maldonado (ibolmo)

I would update and run the langchain.py golden tests https://github.com/braintrustdata/braintrust-sdk/blob/main/internal/golden/langchain.py

I also have a few (separate local repo) I'll try to add to the examples here

- Add LATEST to LANGCHAIN_VERSIONS for testing against newest releases - Remove redundant version pinning and explicit transitive deps (tenacity, pydantic) - Remove conditional skip for langgraph - it's now a required test dependency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Abhijeet Prasad (AbhiPrasad) · 2026-02-25T16:33:06Z

Going to take this PR over and apply it to https://github.com/braintrustdata/braintrust-sdk-python

Abhijeet Prasad (AbhiPrasad) · 2026-03-05T17:00:29Z

Gonna close - braintrustdata/braintrust-sdk-python#27 is tracking this to do it in the Python SDK.

Matt Perpick (clutchski)and others added 2 commits January 29, 2026 17:01

Add LangChain integration to main package with auto_instrument() support
d9879e4
Move LangChain wrapper from integrations/langchain-py into the main braintrust package, enabling auto-instrumentation via braintrust.auto_instrument(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Keep braintrust-langchain working standalone
d3fcaee
The deprecation wrapper can be added after the new braintrust package is released. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Olmo Maldonado (ibolmo) reviewed Jan 30, 2026
View reviewed changes

Olmo Maldonado (ibolmo) approved these changes Jan 30, 2026
View reviewed changes

Abhijeet Prasad (AbhiPrasad) self-assigned this Feb 25, 2026

Abhijeet Prasad (AbhiPrasad) mentioned this pull request Mar 5, 2026
Migrate braintrust-langchain into main braintrust package. braintrustdata/braintrust-sdk-python#27
Closed

Abhijeet Prasad (AbhiPrasad) closed this Mar 5, 2026

Abhijeet Prasad (AbhiPrasad) deleted the matt/lc-auto branch March 5, 2026 17:00