OpenAI integration by colin-sentry · Pull Request #2791 · getsentry/sentry-python

colin-sentry · 2024-03-06T21:45:20Z

OpenAI integration for performance monitoring.

This commit instruments two OpenAI functions (for now) with monkeypatches, and creates span objects automatically.

General Notes

Thank you for contributing to sentry-python!

Please add tests to validate your changes, and lint your code using tox -e linters.

Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.

For maintainers

Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.

Before running sensitive test suites, please carefully check the PR. Then, apply the Trigger: tests using secrets label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.

indragiek

looks great! will defer to @Zylphrex and others for final review

antonpirker

Yay! An OpenAI integration! Great work! Requested some changes to have the data in all the right places.

One important general question:

Is it possible that this will send personal data of any kind?
If yes, please guard it like we do here: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/flask.py#L186

antonpirker

Wow, that was a fast update!

I took the liberty do add a small fix to make the test pass. Please pull your openai branch.

Some little changes we will still have to make, but then we are good to go!

antonpirker

Yes, looks great now! Thanks for all the hard work!
You can merge this now!

* ref: Improve scrub_dict typing (#2768) This change improves the typing of the scrub_dict method. Previously, the scrub_dict method's type hints indicated that only dict[str, Any] was accepted as the parameter. However, the method is actually implemented to accept any object, since it checks the types of the parameters at runtime. Therefore, object is a more appropriate type hint for the parameter. #2753 depends on this change for mypy to pass * Propagate sentry-trace and baggage to huey tasks (#2792) This PR enables passing `sentry-trace` and `baggage` headers to background tasks using the Huey task queue. This allows easily correlating what happens inside a background task with whatever transaction (e.g. a user request in a Django application) queued the task in the first place. Periodic tasks do not get these headers, because otherwise each execution of the periodic task would be tied to the same parent trace (the long-running worker process). --- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * OpenAI integration (#2791) * OpenAI integration * Fix linting errors * Fix CI * Fix lint * Fix more CI issues * Run tests on version pinned OpenAI too * Fix pydantic issue in test * Import type in TYPE_CHECKING gate * PR feedback fixes * Fix tiktoken test variant * PII gate the request and response * Rename set_data tags * Move doc location * Add "exclude prompts" flag as optional * Change prompts to be excluded by default * Set flag in tests * Fix tiktoken tox.ini extra dash * Change strip PII semantics * More test coverage for PII * notiktoken --------- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * Add a method for normalizing data passed to set_data (#2800) * Discard open spans after 10 minutes (#2801) OTel spans that are handled in the Sentry span processor can never be finished/closed. This leads to a memory leak. This change makes sure that open spans will be removed from memory after 10 minutes to prevent memory usage from growing constantly. Fixes#2722 --------- Co-authored-by: Daniel Szoke <szokeasaurusrex@users.noreply.github.com> * ref: Event Type (#2753) Implements type hinting for Event via a TypedDict. This commit mainly adjusts type hints; however, there are also some minor code changes to make the code type-safe following the new changes. Some items in the Event could have their types expanded by being defined as TypedDicts themselves. These items have been indicated with TODO comments. FixesGH-2357 * Fix mypy in `client.py` * Fix functools import * Fix CI config problem ... by running `python scripts/split-tox-gh-actions/split-tox-gh-actions.py` --------- Co-authored-by: Christian Schneider <christian@cnschn.com> Co-authored-by: Anton Pirker <anton.pirker@sentry.io> Co-authored-by: colin-sentry <161344340+colin-sentry@users.noreply.github.com>

Upstream in SDKs (getsentry/sentry-python#2791) we added strings like "ai.prompt_tokens.used" This adds them as known data fields, otherwise they are removed as PII since most configurations consider any superstring of "token" to be a password.

colin-sentry · 2024-03-06T21:45:20Z

OpenAI integration for performance monitoring.

This commit instruments two OpenAI functions (for now) with monkeypatches, and creates span objects automatically.

General Notes

Thank you for contributing to sentry-python!

Please add tests to validate your changes, and lint your code using tox -e linters.

Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.

For maintainers

Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.

Before running sensitive test suites, please carefully check the PR. Then, apply the Trigger: tests using secrets label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.

indragiek

looks great! will defer to @Zylphrex and others for final review

antonpirker

Yay! An OpenAI integration! Great work! Requested some changes to have the data in all the right places.

One important general question:

Is it possible that this will send personal data of any kind?
If yes, please guard it like we do here: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/flask.py#L186

antonpirker

Wow, that was a fast update!

I took the liberty do add a small fix to make the test pass. Please pull your openai branch.

Some little changes we will still have to make, but then we are good to go!

antonpirker

Yes, looks great now! Thanks for all the hard work!
You can merge this now!

* ref: Improve scrub_dict typing (#2768) This change improves the typing of the scrub_dict method. Previously, the scrub_dict method's type hints indicated that only dict[str, Any] was accepted as the parameter. However, the method is actually implemented to accept any object, since it checks the types of the parameters at runtime. Therefore, object is a more appropriate type hint for the parameter. #2753 depends on this change for mypy to pass * Propagate sentry-trace and baggage to huey tasks (#2792) This PR enables passing `sentry-trace` and `baggage` headers to background tasks using the Huey task queue. This allows easily correlating what happens inside a background task with whatever transaction (e.g. a user request in a Django application) queued the task in the first place. Periodic tasks do not get these headers, because otherwise each execution of the periodic task would be tied to the same parent trace (the long-running worker process). --- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * OpenAI integration (#2791) * OpenAI integration * Fix linting errors * Fix CI * Fix lint * Fix more CI issues * Run tests on version pinned OpenAI too * Fix pydantic issue in test * Import type in TYPE_CHECKING gate * PR feedback fixes * Fix tiktoken test variant * PII gate the request and response * Rename set_data tags * Move doc location * Add "exclude prompts" flag as optional * Change prompts to be excluded by default * Set flag in tests * Fix tiktoken tox.ini extra dash * Change strip PII semantics * More test coverage for PII * notiktoken --------- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * Add a method for normalizing data passed to set_data (#2800) * Discard open spans after 10 minutes (#2801) OTel spans that are handled in the Sentry span processor can never be finished/closed. This leads to a memory leak. This change makes sure that open spans will be removed from memory after 10 minutes to prevent memory usage from growing constantly. Fixes#2722 --------- Co-authored-by: Daniel Szoke <szokeasaurusrex@users.noreply.github.com> * ref: Event Type (#2753) Implements type hinting for Event via a TypedDict. This commit mainly adjusts type hints; however, there are also some minor code changes to make the code type-safe following the new changes. Some items in the Event could have their types expanded by being defined as TypedDicts themselves. These items have been indicated with TODO comments. FixesGH-2357 * Fix mypy in `client.py` * Fix functools import * Fix CI config problem ... by running `python scripts/split-tox-gh-actions/split-tox-gh-actions.py` --------- Co-authored-by: Christian Schneider <christian@cnschn.com> Co-authored-by: Anton Pirker <anton.pirker@sentry.io> Co-authored-by: colin-sentry <161344340+colin-sentry@users.noreply.github.com>

Upstream in SDKs (getsentry/sentry-python#2791) we added strings like "ai.prompt_tokens.used" This adds them as known data fields, otherwise they are removed as PII since most configurations consider any superstring of "token" to be a password.

colin-sentry · 2024-03-06T21:45:20Z

OpenAI integration for performance monitoring.

This commit instruments two OpenAI functions (for now) with monkeypatches, and creates span objects automatically.

General Notes

Thank you for contributing to sentry-python!

Please add tests to validate your changes, and lint your code using tox -e linters.

Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.

For maintainers

Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.

Before running sensitive test suites, please carefully check the PR. Then, apply the Trigger: tests using secrets label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.

indragiek

looks great! will defer to @Zylphrex and others for final review

antonpirker

Yay! An OpenAI integration! Great work! Requested some changes to have the data in all the right places.

One important general question:

Is it possible that this will send personal data of any kind?
If yes, please guard it like we do here: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/flask.py#L186

antonpirker

Wow, that was a fast update!

I took the liberty do add a small fix to make the test pass. Please pull your openai branch.

Some little changes we will still have to make, but then we are good to go!

antonpirker

Yes, looks great now! Thanks for all the hard work!
You can merge this now!

* ref: Improve scrub_dict typing (#2768) This change improves the typing of the scrub_dict method. Previously, the scrub_dict method's type hints indicated that only dict[str, Any] was accepted as the parameter. However, the method is actually implemented to accept any object, since it checks the types of the parameters at runtime. Therefore, object is a more appropriate type hint for the parameter. #2753 depends on this change for mypy to pass * Propagate sentry-trace and baggage to huey tasks (#2792) This PR enables passing `sentry-trace` and `baggage` headers to background tasks using the Huey task queue. This allows easily correlating what happens inside a background task with whatever transaction (e.g. a user request in a Django application) queued the task in the first place. Periodic tasks do not get these headers, because otherwise each execution of the periodic task would be tied to the same parent trace (the long-running worker process). --- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * OpenAI integration (#2791) * OpenAI integration * Fix linting errors * Fix CI * Fix lint * Fix more CI issues * Run tests on version pinned OpenAI too * Fix pydantic issue in test * Import type in TYPE_CHECKING gate * PR feedback fixes * Fix tiktoken test variant * PII gate the request and response * Rename set_data tags * Move doc location * Add "exclude prompts" flag as optional * Change prompts to be excluded by default * Set flag in tests * Fix tiktoken tox.ini extra dash * Change strip PII semantics * More test coverage for PII * notiktoken --------- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * Add a method for normalizing data passed to set_data (#2800) * Discard open spans after 10 minutes (#2801) OTel spans that are handled in the Sentry span processor can never be finished/closed. This leads to a memory leak. This change makes sure that open spans will be removed from memory after 10 minutes to prevent memory usage from growing constantly. Fixes#2722 --------- Co-authored-by: Daniel Szoke <szokeasaurusrex@users.noreply.github.com> * ref: Event Type (#2753) Implements type hinting for Event via a TypedDict. This commit mainly adjusts type hints; however, there are also some minor code changes to make the code type-safe following the new changes. Some items in the Event could have their types expanded by being defined as TypedDicts themselves. These items have been indicated with TODO comments. FixesGH-2357 * Fix mypy in `client.py` * Fix functools import * Fix CI config problem ... by running `python scripts/split-tox-gh-actions/split-tox-gh-actions.py` --------- Co-authored-by: Christian Schneider <christian@cnschn.com> Co-authored-by: Anton Pirker <anton.pirker@sentry.io> Co-authored-by: colin-sentry <161344340+colin-sentry@users.noreply.github.com>

Upstream in SDKs (getsentry/sentry-python#2791) we added strings like "ai.prompt_tokens.used" This adds them as known data fields, otherwise they are removed as PII since most configurations consider any superstring of "token" to be a password.

colin-sentry · 2024-03-06T21:45:20Z

OpenAI integration for performance monitoring.

This commit instruments two OpenAI functions (for now) with monkeypatches, and creates span objects automatically.

General Notes

Thank you for contributing to sentry-python!

Please add tests to validate your changes, and lint your code using tox -e linters.

Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.

For maintainers

Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.

Before running sensitive test suites, please carefully check the PR. Then, apply the Trigger: tests using secrets label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.

indragiek

looks great! will defer to @Zylphrex and others for final review

antonpirker

Yay! An OpenAI integration! Great work! Requested some changes to have the data in all the right places.

One important general question:

Is it possible that this will send personal data of any kind?
If yes, please guard it like we do here: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/flask.py#L186

antonpirker

Wow, that was a fast update!

I took the liberty do add a small fix to make the test pass. Please pull your openai branch.

Some little changes we will still have to make, but then we are good to go!

antonpirker

Yes, looks great now! Thanks for all the hard work!
You can merge this now!

* ref: Improve scrub_dict typing (#2768) This change improves the typing of the scrub_dict method. Previously, the scrub_dict method's type hints indicated that only dict[str, Any] was accepted as the parameter. However, the method is actually implemented to accept any object, since it checks the types of the parameters at runtime. Therefore, object is a more appropriate type hint for the parameter. #2753 depends on this change for mypy to pass * Propagate sentry-trace and baggage to huey tasks (#2792) This PR enables passing `sentry-trace` and `baggage` headers to background tasks using the Huey task queue. This allows easily correlating what happens inside a background task with whatever transaction (e.g. a user request in a Django application) queued the task in the first place. Periodic tasks do not get these headers, because otherwise each execution of the periodic task would be tied to the same parent trace (the long-running worker process). --- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * OpenAI integration (#2791) * OpenAI integration * Fix linting errors * Fix CI * Fix lint * Fix more CI issues * Run tests on version pinned OpenAI too * Fix pydantic issue in test * Import type in TYPE_CHECKING gate * PR feedback fixes * Fix tiktoken test variant * PII gate the request and response * Rename set_data tags * Move doc location * Add "exclude prompts" flag as optional * Change prompts to be excluded by default * Set flag in tests * Fix tiktoken tox.ini extra dash * Change strip PII semantics * More test coverage for PII * notiktoken --------- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * Add a method for normalizing data passed to set_data (#2800) * Discard open spans after 10 minutes (#2801) OTel spans that are handled in the Sentry span processor can never be finished/closed. This leads to a memory leak. This change makes sure that open spans will be removed from memory after 10 minutes to prevent memory usage from growing constantly. Fixes#2722 --------- Co-authored-by: Daniel Szoke <szokeasaurusrex@users.noreply.github.com> * ref: Event Type (#2753) Implements type hinting for Event via a TypedDict. This commit mainly adjusts type hints; however, there are also some minor code changes to make the code type-safe following the new changes. Some items in the Event could have their types expanded by being defined as TypedDicts themselves. These items have been indicated with TODO comments. FixesGH-2357 * Fix mypy in `client.py` * Fix functools import * Fix CI config problem ... by running `python scripts/split-tox-gh-actions/split-tox-gh-actions.py` --------- Co-authored-by: Christian Schneider <christian@cnschn.com> Co-authored-by: Anton Pirker <anton.pirker@sentry.io> Co-authored-by: colin-sentry <161344340+colin-sentry@users.noreply.github.com>

Upstream in SDKs (getsentry/sentry-python#2791) we added strings like "ai.prompt_tokens.used" This adds them as known data fields, otherwise they are removed as PII since most configurations consider any superstring of "token" to be a password.

colin-sentry · 2024-03-06T21:45:20Z

OpenAI integration for performance monitoring.

This commit instruments two OpenAI functions (for now) with monkeypatches, and creates span objects automatically.

General Notes

Thank you for contributing to sentry-python!

Please add tests to validate your changes, and lint your code using tox -e linters.

Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.

For maintainers

Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.

Before running sensitive test suites, please carefully check the PR. Then, apply the Trigger: tests using secrets label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.

indragiek

looks great! will defer to @Zylphrex and others for final review

antonpirker

Yay! An OpenAI integration! Great work! Requested some changes to have the data in all the right places.

One important general question:

Is it possible that this will send personal data of any kind?
If yes, please guard it like we do here: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/flask.py#L186

antonpirker

Wow, that was a fast update!

I took the liberty do add a small fix to make the test pass. Please pull your openai branch.

Some little changes we will still have to make, but then we are good to go!

antonpirker

Yes, looks great now! Thanks for all the hard work!
You can merge this now!

* ref: Improve scrub_dict typing (#2768) This change improves the typing of the scrub_dict method. Previously, the scrub_dict method's type hints indicated that only dict[str, Any] was accepted as the parameter. However, the method is actually implemented to accept any object, since it checks the types of the parameters at runtime. Therefore, object is a more appropriate type hint for the parameter. #2753 depends on this change for mypy to pass * Propagate sentry-trace and baggage to huey tasks (#2792) This PR enables passing `sentry-trace` and `baggage` headers to background tasks using the Huey task queue. This allows easily correlating what happens inside a background task with whatever transaction (e.g. a user request in a Django application) queued the task in the first place. Periodic tasks do not get these headers, because otherwise each execution of the periodic task would be tied to the same parent trace (the long-running worker process). --- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * OpenAI integration (#2791) * OpenAI integration * Fix linting errors * Fix CI * Fix lint * Fix more CI issues * Run tests on version pinned OpenAI too * Fix pydantic issue in test * Import type in TYPE_CHECKING gate * PR feedback fixes * Fix tiktoken test variant * PII gate the request and response * Rename set_data tags * Move doc location * Add "exclude prompts" flag as optional * Change prompts to be excluded by default * Set flag in tests * Fix tiktoken tox.ini extra dash * Change strip PII semantics * More test coverage for PII * notiktoken --------- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * Add a method for normalizing data passed to set_data (#2800) * Discard open spans after 10 minutes (#2801) OTel spans that are handled in the Sentry span processor can never be finished/closed. This leads to a memory leak. This change makes sure that open spans will be removed from memory after 10 minutes to prevent memory usage from growing constantly. Fixes#2722 --------- Co-authored-by: Daniel Szoke <szokeasaurusrex@users.noreply.github.com> * ref: Event Type (#2753) Implements type hinting for Event via a TypedDict. This commit mainly adjusts type hints; however, there are also some minor code changes to make the code type-safe following the new changes. Some items in the Event could have their types expanded by being defined as TypedDicts themselves. These items have been indicated with TODO comments. FixesGH-2357 * Fix mypy in `client.py` * Fix functools import * Fix CI config problem ... by running `python scripts/split-tox-gh-actions/split-tox-gh-actions.py` --------- Co-authored-by: Christian Schneider <christian@cnschn.com> Co-authored-by: Anton Pirker <anton.pirker@sentry.io> Co-authored-by: colin-sentry <161344340+colin-sentry@users.noreply.github.com>

Upstream in SDKs (getsentry/sentry-python#2791) we added strings like "ai.prompt_tokens.used" This adds them as known data fields, otherwise they are removed as PII since most configurations consider any superstring of "token" to be a password.

colin-sentry · 2024-03-06T21:45:20Z

OpenAI integration for performance monitoring.

This commit instruments two OpenAI functions (for now) with monkeypatches, and creates span objects automatically.

General Notes

Thank you for contributing to sentry-python!

Please add tests to validate your changes, and lint your code using tox -e linters.

Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.

For maintainers

Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.

Before running sensitive test suites, please carefully check the PR. Then, apply the Trigger: tests using secrets label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.

indragiek

looks great! will defer to @Zylphrex and others for final review

antonpirker

Yay! An OpenAI integration! Great work! Requested some changes to have the data in all the right places.

One important general question:

Is it possible that this will send personal data of any kind?
If yes, please guard it like we do here: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/flask.py#L186

antonpirker

Wow, that was a fast update!

I took the liberty do add a small fix to make the test pass. Please pull your openai branch.

Some little changes we will still have to make, but then we are good to go!

antonpirker

Yes, looks great now! Thanks for all the hard work!
You can merge this now!

* ref: Improve scrub_dict typing (#2768) This change improves the typing of the scrub_dict method. Previously, the scrub_dict method's type hints indicated that only dict[str, Any] was accepted as the parameter. However, the method is actually implemented to accept any object, since it checks the types of the parameters at runtime. Therefore, object is a more appropriate type hint for the parameter. #2753 depends on this change for mypy to pass * Propagate sentry-trace and baggage to huey tasks (#2792) This PR enables passing `sentry-trace` and `baggage` headers to background tasks using the Huey task queue. This allows easily correlating what happens inside a background task with whatever transaction (e.g. a user request in a Django application) queued the task in the first place. Periodic tasks do not get these headers, because otherwise each execution of the periodic task would be tied to the same parent trace (the long-running worker process). --- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * OpenAI integration (#2791) * OpenAI integration * Fix linting errors * Fix CI * Fix lint * Fix more CI issues * Run tests on version pinned OpenAI too * Fix pydantic issue in test * Import type in TYPE_CHECKING gate * PR feedback fixes * Fix tiktoken test variant * PII gate the request and response * Rename set_data tags * Move doc location * Add "exclude prompts" flag as optional * Change prompts to be excluded by default * Set flag in tests * Fix tiktoken tox.ini extra dash * Change strip PII semantics * More test coverage for PII * notiktoken --------- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * Add a method for normalizing data passed to set_data (#2800) * Discard open spans after 10 minutes (#2801) OTel spans that are handled in the Sentry span processor can never be finished/closed. This leads to a memory leak. This change makes sure that open spans will be removed from memory after 10 minutes to prevent memory usage from growing constantly. Fixes#2722 --------- Co-authored-by: Daniel Szoke <szokeasaurusrex@users.noreply.github.com> * ref: Event Type (#2753) Implements type hinting for Event via a TypedDict. This commit mainly adjusts type hints; however, there are also some minor code changes to make the code type-safe following the new changes. Some items in the Event could have their types expanded by being defined as TypedDicts themselves. These items have been indicated with TODO comments. FixesGH-2357 * Fix mypy in `client.py` * Fix functools import * Fix CI config problem ... by running `python scripts/split-tox-gh-actions/split-tox-gh-actions.py` --------- Co-authored-by: Christian Schneider <christian@cnschn.com> Co-authored-by: Anton Pirker <anton.pirker@sentry.io> Co-authored-by: colin-sentry <161344340+colin-sentry@users.noreply.github.com>

Upstream in SDKs (getsentry/sentry-python#2791) we added strings like "ai.prompt_tokens.used" This adds them as known data fields, otherwise they are removed as PII since most configurations consider any superstring of "token" to be a password.

colin-sentry · 2024-03-06T21:45:20Z

OpenAI integration for performance monitoring.

This commit instruments two OpenAI functions (for now) with monkeypatches, and creates span objects automatically.

General Notes

Thank you for contributing to sentry-python!

Please add tests to validate your changes, and lint your code using tox -e linters.

Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.

For maintainers

Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.

Before running sensitive test suites, please carefully check the PR. Then, apply the Trigger: tests using secrets label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.

indragiek

looks great! will defer to @Zylphrex and others for final review

antonpirker

Yay! An OpenAI integration! Great work! Requested some changes to have the data in all the right places.

One important general question:

Is it possible that this will send personal data of any kind?
If yes, please guard it like we do here: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/flask.py#L186

antonpirker

Wow, that was a fast update!

I took the liberty do add a small fix to make the test pass. Please pull your openai branch.

Some little changes we will still have to make, but then we are good to go!

antonpirker

Yes, looks great now! Thanks for all the hard work!
You can merge this now!

* ref: Improve scrub_dict typing (#2768) This change improves the typing of the scrub_dict method. Previously, the scrub_dict method's type hints indicated that only dict[str, Any] was accepted as the parameter. However, the method is actually implemented to accept any object, since it checks the types of the parameters at runtime. Therefore, object is a more appropriate type hint for the parameter. #2753 depends on this change for mypy to pass * Propagate sentry-trace and baggage to huey tasks (#2792) This PR enables passing `sentry-trace` and `baggage` headers to background tasks using the Huey task queue. This allows easily correlating what happens inside a background task with whatever transaction (e.g. a user request in a Django application) queued the task in the first place. Periodic tasks do not get these headers, because otherwise each execution of the periodic task would be tied to the same parent trace (the long-running worker process). --- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * OpenAI integration (#2791) * OpenAI integration * Fix linting errors * Fix CI * Fix lint * Fix more CI issues * Run tests on version pinned OpenAI too * Fix pydantic issue in test * Import type in TYPE_CHECKING gate * PR feedback fixes * Fix tiktoken test variant * PII gate the request and response * Rename set_data tags * Move doc location * Add "exclude prompts" flag as optional * Change prompts to be excluded by default * Set flag in tests * Fix tiktoken tox.ini extra dash * Change strip PII semantics * More test coverage for PII * notiktoken --------- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * Add a method for normalizing data passed to set_data (#2800) * Discard open spans after 10 minutes (#2801) OTel spans that are handled in the Sentry span processor can never be finished/closed. This leads to a memory leak. This change makes sure that open spans will be removed from memory after 10 minutes to prevent memory usage from growing constantly. Fixes#2722 --------- Co-authored-by: Daniel Szoke <szokeasaurusrex@users.noreply.github.com> * ref: Event Type (#2753) Implements type hinting for Event via a TypedDict. This commit mainly adjusts type hints; however, there are also some minor code changes to make the code type-safe following the new changes. Some items in the Event could have their types expanded by being defined as TypedDicts themselves. These items have been indicated with TODO comments. FixesGH-2357 * Fix mypy in `client.py` * Fix functools import * Fix CI config problem ... by running `python scripts/split-tox-gh-actions/split-tox-gh-actions.py` --------- Co-authored-by: Christian Schneider <christian@cnschn.com> Co-authored-by: Anton Pirker <anton.pirker@sentry.io> Co-authored-by: colin-sentry <161344340+colin-sentry@users.noreply.github.com>

Upstream in SDKs (getsentry/sentry-python#2791) we added strings like "ai.prompt_tokens.used" This adds them as known data fields, otherwise they are removed as PII since most configurations consider any superstring of "token" to be a password.

colin-sentry · 2024-03-06T21:45:20Z

OpenAI integration for performance monitoring.

This commit instruments two OpenAI functions (for now) with monkeypatches, and creates span objects automatically.

General Notes

Thank you for contributing to sentry-python!

Please add tests to validate your changes, and lint your code using tox -e linters.

Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.

For maintainers

Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.

Before running sensitive test suites, please carefully check the PR. Then, apply the Trigger: tests using secrets label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.

indragiek

looks great! will defer to @Zylphrex and others for final review

antonpirker

Yay! An OpenAI integration! Great work! Requested some changes to have the data in all the right places.

One important general question:

Is it possible that this will send personal data of any kind?
If yes, please guard it like we do here: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/flask.py#L186

antonpirker

Wow, that was a fast update!

I took the liberty do add a small fix to make the test pass. Please pull your openai branch.

Some little changes we will still have to make, but then we are good to go!

antonpirker

Yes, looks great now! Thanks for all the hard work!
You can merge this now!

* ref: Improve scrub_dict typing (#2768) This change improves the typing of the scrub_dict method. Previously, the scrub_dict method's type hints indicated that only dict[str, Any] was accepted as the parameter. However, the method is actually implemented to accept any object, since it checks the types of the parameters at runtime. Therefore, object is a more appropriate type hint for the parameter. #2753 depends on this change for mypy to pass * Propagate sentry-trace and baggage to huey tasks (#2792) This PR enables passing `sentry-trace` and `baggage` headers to background tasks using the Huey task queue. This allows easily correlating what happens inside a background task with whatever transaction (e.g. a user request in a Django application) queued the task in the first place. Periodic tasks do not get these headers, because otherwise each execution of the periodic task would be tied to the same parent trace (the long-running worker process). --- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * OpenAI integration (#2791) * OpenAI integration * Fix linting errors * Fix CI * Fix lint * Fix more CI issues * Run tests on version pinned OpenAI too * Fix pydantic issue in test * Import type in TYPE_CHECKING gate * PR feedback fixes * Fix tiktoken test variant * PII gate the request and response * Rename set_data tags * Move doc location * Add "exclude prompts" flag as optional * Change prompts to be excluded by default * Set flag in tests * Fix tiktoken tox.ini extra dash * Change strip PII semantics * More test coverage for PII * notiktoken --------- Co-authored-by: Anton Pirker <anton.pirker@sentry.io> * Add a method for normalizing data passed to set_data (#2800) * Discard open spans after 10 minutes (#2801) OTel spans that are handled in the Sentry span processor can never be finished/closed. This leads to a memory leak. This change makes sure that open spans will be removed from memory after 10 minutes to prevent memory usage from growing constantly. Fixes#2722 --------- Co-authored-by: Daniel Szoke <szokeasaurusrex@users.noreply.github.com> * ref: Event Type (#2753) Implements type hinting for Event via a TypedDict. This commit mainly adjusts type hints; however, there are also some minor code changes to make the code type-safe following the new changes. Some items in the Event could have their types expanded by being defined as TypedDicts themselves. These items have been indicated with TODO comments. FixesGH-2357 * Fix mypy in `client.py` * Fix functools import * Fix CI config problem ... by running `python scripts/split-tox-gh-actions/split-tox-gh-actions.py` --------- Co-authored-by: Christian Schneider <christian@cnschn.com> Co-authored-by: Anton Pirker <anton.pirker@sentry.io> Co-authored-by: colin-sentry <161344340+colin-sentry@users.noreply.github.com>

Upstream in SDKs (getsentry/sentry-python#2791) we added strings like "ai.prompt_tokens.used" This adds them as known data fields, otherwise they are removed as PII since most configurations consider any superstring of "token" to be a password.

colin-sentry force-pushed the openai branch from 409fd88 to 5c7026cCompare March 6, 2024 22:01

Zylphrex reviewed Mar 6, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py
Comment threadsentry_sdk/integrations/openai.py Outdated
Comment threadsentry_sdk/integrations/openai.py Outdated

Zylphrex requested review from antonpirker, sentrivana and szokeasaurusrex March 6, 2024 22:31

indragiek reviewed Mar 7, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py Outdated

antonpirker suggested changes Mar 7, 2024
View reviewed changes

colin-sentry requested a review from antonpirker March 7, 2024 16:03

colin-sentry mentioned this pull request Mar 7, 2024
Update span-data-conventions.mdx getsentry/develop#1183
Merged

colin-sentry force-pushed the openai branch from d9a6557 to cbf4425Compare March 7, 2024 18:01

colin-sentry added 14 commits March 7, 2024 13:01

OpenAI integration
6d79ebf

Fix linting errors
edf396c

Fix CI
05ecefa

Fix lint
a3ccbcd

Fix more CI issues
6003902

Run tests on version pinned OpenAI too
09013f2

Fix pydantic issue in test
2f9667a

Import type in TYPE_CHECKING gate
ec84c04

PR feedback fixes
e45420e

Fix tiktoken test variant
0e319f1

PII gate the request and response
3acc9ab

Rename set_data tags
65c311d

Move doc location
4b33a4e

Add "exclude prompts" flag as optional
56d2679

colin-sentry force-pushed the openai branch from cbf4425 to 56d2679Compare March 7, 2024 18:01

Change prompts to be excluded by default
72d4b5a

colin-sentry mentioned this pull request Mar 7, 2024
Add OpenAI docs getsentry/sentry-docs#9376
Merged
3 tasks

colin-sentry added 2 commits March 7, 2024 13:16

Set flag in tests
b206e4c

Fix tiktoken tox.ini extra dash
4bc4310

antonpirker suggested changes Mar 8, 2024
View reviewed changes

colin-sentry force-pushed the openai branch from 26bc2aa to f7b8c05Compare March 8, 2024 15:16

Change strip PII semantics
c6f5cd2

colin-sentry force-pushed the openai branch from f7b8c05 to c6f5cd2Compare March 8, 2024 15:25

colin-sentry added 2 commits March 8, 2024 10:34

More test coverage for PII
d1eae09

notiktoken
c96e0e5

antonpirker approved these changes Mar 11, 2024
View reviewed changes

antonpirker mentioned this pull request Mar 11, 2024
feat(ai): LLM monitoring init #2514
Closed

antonpirkerand others added 2 commits March 11, 2024 11:30

Merge branch 'master' into openai
0278e2c

Merge branch 'master' into openai
452b03b

colin-sentry enabled auto-merge (squash) March 11, 2024 13:55

colin-sentry merged commit ff0a94b into masterMar 11, 2024

colin-sentry deleted the openai branch March 11, 2024 14:06

colin-sentry mentioned this pull request Mar 11, 2024
feat(ai): Add support for AI token span data getsentry/relay#3250
Merged

colin-sentry force-pushed the openai branch from 409fd88 to 5c7026cCompare March 6, 2024 22:01

Zylphrex reviewed Mar 6, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py
Comment threadsentry_sdk/integrations/openai.py Outdated
Comment threadsentry_sdk/integrations/openai.py Outdated

Zylphrex requested review from antonpirker, sentrivana and szokeasaurusrex March 6, 2024 22:31

indragiek reviewed Mar 7, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py Outdated

antonpirker suggested changes Mar 7, 2024
View reviewed changes

colin-sentry requested a review from antonpirker March 7, 2024 16:03

colin-sentry mentioned this pull request Mar 7, 2024
Update span-data-conventions.mdx getsentry/develop#1183
Merged

colin-sentry force-pushed the openai branch from d9a6557 to cbf4425Compare March 7, 2024 18:01

colin-sentry added 14 commits March 7, 2024 13:01

OpenAI integration
6d79ebf

Fix linting errors
edf396c

Fix CI
05ecefa

Fix lint
a3ccbcd

Fix more CI issues
6003902

Run tests on version pinned OpenAI too
09013f2

Fix pydantic issue in test
2f9667a

Import type in TYPE_CHECKING gate
ec84c04

PR feedback fixes
e45420e

Fix tiktoken test variant
0e319f1

PII gate the request and response
3acc9ab

Rename set_data tags
65c311d

Move doc location
4b33a4e

Add "exclude prompts" flag as optional
56d2679

colin-sentry force-pushed the openai branch from cbf4425 to 56d2679Compare March 7, 2024 18:01

Change prompts to be excluded by default
72d4b5a

colin-sentry mentioned this pull request Mar 7, 2024
Add OpenAI docs getsentry/sentry-docs#9376
Merged
3 tasks

colin-sentry added 2 commits March 7, 2024 13:16

Set flag in tests
b206e4c

Fix tiktoken tox.ini extra dash
4bc4310

antonpirker suggested changes Mar 8, 2024
View reviewed changes

colin-sentry force-pushed the openai branch from 26bc2aa to f7b8c05Compare March 8, 2024 15:16

Change strip PII semantics
c6f5cd2

colin-sentry force-pushed the openai branch from f7b8c05 to c6f5cd2Compare March 8, 2024 15:25

colin-sentry added 2 commits March 8, 2024 10:34

More test coverage for PII
d1eae09

notiktoken
c96e0e5

antonpirker approved these changes Mar 11, 2024
View reviewed changes

antonpirker mentioned this pull request Mar 11, 2024
feat(ai): LLM monitoring init #2514
Closed

antonpirkerand others added 2 commits March 11, 2024 11:30

Merge branch 'master' into openai
0278e2c

Merge branch 'master' into openai
452b03b

colin-sentry enabled auto-merge (squash) March 11, 2024 13:55

colin-sentry merged commit ff0a94b into masterMar 11, 2024

colin-sentry deleted the openai branch March 11, 2024 14:06

colin-sentry mentioned this pull request Mar 11, 2024
feat(ai): Add support for AI token span data getsentry/relay#3250
Merged

colin-sentry force-pushed the openai branch from 409fd88 to 5c7026cCompare March 6, 2024 22:01

Zylphrex reviewed Mar 6, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py
Comment threadsentry_sdk/integrations/openai.py Outdated
Comment threadsentry_sdk/integrations/openai.py Outdated

Zylphrex requested review from antonpirker, sentrivana and szokeasaurusrex March 6, 2024 22:31

indragiek reviewed Mar 7, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py Outdated

antonpirker suggested changes Mar 7, 2024
View reviewed changes

colin-sentry requested a review from antonpirker March 7, 2024 16:03

colin-sentry mentioned this pull request Mar 7, 2024
Update span-data-conventions.mdx getsentry/develop#1183
Merged

colin-sentry force-pushed the openai branch from d9a6557 to cbf4425Compare March 7, 2024 18:01

colin-sentry added 14 commits March 7, 2024 13:01

OpenAI integration
6d79ebf

Fix linting errors
edf396c

Fix CI
05ecefa

Fix lint
a3ccbcd

Fix more CI issues
6003902

Run tests on version pinned OpenAI too
09013f2

Fix pydantic issue in test
2f9667a

Import type in TYPE_CHECKING gate
ec84c04

PR feedback fixes
e45420e

Fix tiktoken test variant
0e319f1

PII gate the request and response
3acc9ab

Rename set_data tags
65c311d

Move doc location
4b33a4e

Add "exclude prompts" flag as optional
56d2679

colin-sentry force-pushed the openai branch from cbf4425 to 56d2679Compare March 7, 2024 18:01

Change prompts to be excluded by default
72d4b5a

colin-sentry mentioned this pull request Mar 7, 2024
Add OpenAI docs getsentry/sentry-docs#9376
Merged
3 tasks

colin-sentry added 2 commits March 7, 2024 13:16

Set flag in tests
b206e4c

Fix tiktoken tox.ini extra dash
4bc4310

antonpirker suggested changes Mar 8, 2024
View reviewed changes

colin-sentry force-pushed the openai branch from 26bc2aa to f7b8c05Compare March 8, 2024 15:16

Change strip PII semantics
c6f5cd2

colin-sentry force-pushed the openai branch from f7b8c05 to c6f5cd2Compare March 8, 2024 15:25

colin-sentry added 2 commits March 8, 2024 10:34

More test coverage for PII
d1eae09

notiktoken
c96e0e5

antonpirker approved these changes Mar 11, 2024
View reviewed changes

antonpirker mentioned this pull request Mar 11, 2024
feat(ai): LLM monitoring init #2514
Closed

antonpirkerand others added 2 commits March 11, 2024 11:30

Merge branch 'master' into openai
0278e2c

Merge branch 'master' into openai
452b03b

colin-sentry enabled auto-merge (squash) March 11, 2024 13:55

colin-sentry merged commit ff0a94b into masterMar 11, 2024

colin-sentry deleted the openai branch March 11, 2024 14:06

colin-sentry mentioned this pull request Mar 11, 2024
feat(ai): Add support for AI token span data getsentry/relay#3250
Merged

colin-sentry force-pushed the openai branch from 409fd88 to 5c7026cCompare March 6, 2024 22:01

Zylphrex reviewed Mar 6, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py
Comment threadsentry_sdk/integrations/openai.py Outdated
Comment threadsentry_sdk/integrations/openai.py Outdated

Zylphrex requested review from antonpirker, sentrivana and szokeasaurusrex March 6, 2024 22:31

indragiek reviewed Mar 7, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py Outdated

antonpirker suggested changes Mar 7, 2024
View reviewed changes

colin-sentry requested a review from antonpirker March 7, 2024 16:03

colin-sentry mentioned this pull request Mar 7, 2024
Update span-data-conventions.mdx getsentry/develop#1183
Merged

colin-sentry force-pushed the openai branch from d9a6557 to cbf4425Compare March 7, 2024 18:01

colin-sentry added 14 commits March 7, 2024 13:01

OpenAI integration
6d79ebf

Fix linting errors
edf396c

Fix CI
05ecefa

Fix lint
a3ccbcd

Fix more CI issues
6003902

Run tests on version pinned OpenAI too
09013f2

Fix pydantic issue in test
2f9667a

Import type in TYPE_CHECKING gate
ec84c04

PR feedback fixes
e45420e

Fix tiktoken test variant
0e319f1

PII gate the request and response
3acc9ab

Rename set_data tags
65c311d

Move doc location
4b33a4e

Add "exclude prompts" flag as optional
56d2679

colin-sentry force-pushed the openai branch from cbf4425 to 56d2679Compare March 7, 2024 18:01

Change prompts to be excluded by default
72d4b5a

colin-sentry mentioned this pull request Mar 7, 2024
Add OpenAI docs getsentry/sentry-docs#9376
Merged
3 tasks

colin-sentry added 2 commits March 7, 2024 13:16

Set flag in tests
b206e4c

Fix tiktoken tox.ini extra dash
4bc4310

antonpirker suggested changes Mar 8, 2024
View reviewed changes

colin-sentry force-pushed the openai branch from 26bc2aa to f7b8c05Compare March 8, 2024 15:16

Change strip PII semantics
c6f5cd2

colin-sentry force-pushed the openai branch from f7b8c05 to c6f5cd2Compare March 8, 2024 15:25

colin-sentry added 2 commits March 8, 2024 10:34

More test coverage for PII
d1eae09

notiktoken
c96e0e5

antonpirker approved these changes Mar 11, 2024
View reviewed changes

antonpirker mentioned this pull request Mar 11, 2024
feat(ai): LLM monitoring init #2514
Closed

antonpirkerand others added 2 commits March 11, 2024 11:30

Merge branch 'master' into openai
0278e2c

Merge branch 'master' into openai
452b03b

colin-sentry enabled auto-merge (squash) March 11, 2024 13:55

colin-sentry merged commit ff0a94b into masterMar 11, 2024

colin-sentry deleted the openai branch March 11, 2024 14:06

colin-sentry mentioned this pull request Mar 11, 2024
feat(ai): Add support for AI token span data getsentry/relay#3250
Merged

colin-sentry force-pushed the openai branch from 409fd88 to 5c7026cCompare March 6, 2024 22:01

Zylphrex reviewed Mar 6, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py
Comment threadsentry_sdk/integrations/openai.py Outdated
Comment threadsentry_sdk/integrations/openai.py Outdated

Zylphrex requested review from antonpirker, sentrivana and szokeasaurusrex March 6, 2024 22:31

indragiek reviewed Mar 7, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py Outdated

antonpirker suggested changes Mar 7, 2024
View reviewed changes

colin-sentry requested a review from antonpirker March 7, 2024 16:03

colin-sentry mentioned this pull request Mar 7, 2024
Update span-data-conventions.mdx getsentry/develop#1183
Merged

colin-sentry force-pushed the openai branch from d9a6557 to cbf4425Compare March 7, 2024 18:01

colin-sentry added 14 commits March 7, 2024 13:01

OpenAI integration
6d79ebf

Fix linting errors
edf396c

Fix CI
05ecefa

Fix lint
a3ccbcd

Fix more CI issues
6003902

Run tests on version pinned OpenAI too
09013f2

Fix pydantic issue in test
2f9667a

Import type in TYPE_CHECKING gate
ec84c04

PR feedback fixes
e45420e

Fix tiktoken test variant
0e319f1

PII gate the request and response
3acc9ab

Rename set_data tags
65c311d

Move doc location
4b33a4e

Add "exclude prompts" flag as optional
56d2679

colin-sentry force-pushed the openai branch from cbf4425 to 56d2679Compare March 7, 2024 18:01

Change prompts to be excluded by default
72d4b5a

colin-sentry mentioned this pull request Mar 7, 2024
Add OpenAI docs getsentry/sentry-docs#9376
Merged
3 tasks

colin-sentry added 2 commits March 7, 2024 13:16

Set flag in tests
b206e4c

Fix tiktoken tox.ini extra dash
4bc4310

antonpirker suggested changes Mar 8, 2024
View reviewed changes

colin-sentry force-pushed the openai branch from 26bc2aa to f7b8c05Compare March 8, 2024 15:16

Change strip PII semantics
c6f5cd2

colin-sentry force-pushed the openai branch from f7b8c05 to c6f5cd2Compare March 8, 2024 15:25

colin-sentry added 2 commits March 8, 2024 10:34

More test coverage for PII
d1eae09

notiktoken
c96e0e5

antonpirker approved these changes Mar 11, 2024
View reviewed changes

antonpirker mentioned this pull request Mar 11, 2024
feat(ai): LLM monitoring init #2514
Closed

antonpirkerand others added 2 commits March 11, 2024 11:30

Merge branch 'master' into openai
0278e2c

Merge branch 'master' into openai
452b03b

colin-sentry enabled auto-merge (squash) March 11, 2024 13:55

colin-sentry merged commit ff0a94b into masterMar 11, 2024

colin-sentry deleted the openai branch March 11, 2024 14:06

colin-sentry mentioned this pull request Mar 11, 2024
feat(ai): Add support for AI token span data getsentry/relay#3250
Merged

colin-sentry force-pushed the openai branch from 409fd88 to 5c7026cCompare March 6, 2024 22:01

Zylphrex reviewed Mar 6, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py
Comment threadsentry_sdk/integrations/openai.py Outdated
Comment threadsentry_sdk/integrations/openai.py Outdated

Zylphrex requested review from antonpirker, sentrivana and szokeasaurusrex March 6, 2024 22:31

indragiek reviewed Mar 7, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py Outdated

antonpirker suggested changes Mar 7, 2024
View reviewed changes

colin-sentry requested a review from antonpirker March 7, 2024 16:03

colin-sentry mentioned this pull request Mar 7, 2024
Update span-data-conventions.mdx getsentry/develop#1183
Merged

colin-sentry force-pushed the openai branch from d9a6557 to cbf4425Compare March 7, 2024 18:01

colin-sentry added 14 commits March 7, 2024 13:01

OpenAI integration
6d79ebf

Fix linting errors
edf396c

Fix CI
05ecefa

Fix lint
a3ccbcd

Fix more CI issues
6003902

Run tests on version pinned OpenAI too
09013f2

Fix pydantic issue in test
2f9667a

Import type in TYPE_CHECKING gate
ec84c04

PR feedback fixes
e45420e

Fix tiktoken test variant
0e319f1

PII gate the request and response
3acc9ab

Rename set_data tags
65c311d

Move doc location
4b33a4e

Add "exclude prompts" flag as optional
56d2679

colin-sentry force-pushed the openai branch from cbf4425 to 56d2679Compare March 7, 2024 18:01

Change prompts to be excluded by default
72d4b5a

colin-sentry mentioned this pull request Mar 7, 2024
Add OpenAI docs getsentry/sentry-docs#9376
Merged
3 tasks

colin-sentry added 2 commits March 7, 2024 13:16

Set flag in tests
b206e4c

Fix tiktoken tox.ini extra dash
4bc4310

antonpirker suggested changes Mar 8, 2024
View reviewed changes

colin-sentry force-pushed the openai branch from 26bc2aa to f7b8c05Compare March 8, 2024 15:16

Change strip PII semantics
c6f5cd2

colin-sentry force-pushed the openai branch from f7b8c05 to c6f5cd2Compare March 8, 2024 15:25

colin-sentry added 2 commits March 8, 2024 10:34

More test coverage for PII
d1eae09

notiktoken
c96e0e5

antonpirker approved these changes Mar 11, 2024
View reviewed changes

antonpirker mentioned this pull request Mar 11, 2024
feat(ai): LLM monitoring init #2514
Closed

antonpirkerand others added 2 commits March 11, 2024 11:30

Merge branch 'master' into openai
0278e2c

Merge branch 'master' into openai
452b03b

colin-sentry enabled auto-merge (squash) March 11, 2024 13:55

colin-sentry merged commit ff0a94b into masterMar 11, 2024

colin-sentry deleted the openai branch March 11, 2024 14:06

colin-sentry mentioned this pull request Mar 11, 2024
feat(ai): Add support for AI token span data getsentry/relay#3250
Merged

colin-sentry force-pushed the openai branch from 409fd88 to 5c7026cCompare March 6, 2024 22:01

Zylphrex reviewed Mar 6, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py
Comment threadsentry_sdk/integrations/openai.py Outdated
Comment threadsentry_sdk/integrations/openai.py Outdated

Zylphrex requested review from antonpirker, sentrivana and szokeasaurusrex March 6, 2024 22:31

indragiek reviewed Mar 7, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py Outdated

antonpirker suggested changes Mar 7, 2024
View reviewed changes

colin-sentry requested a review from antonpirker March 7, 2024 16:03

colin-sentry mentioned this pull request Mar 7, 2024
Update span-data-conventions.mdx getsentry/develop#1183
Merged

colin-sentry force-pushed the openai branch from d9a6557 to cbf4425Compare March 7, 2024 18:01

colin-sentry added 14 commits March 7, 2024 13:01

OpenAI integration
6d79ebf

Fix linting errors
edf396c

Fix CI
05ecefa

Fix lint
a3ccbcd

Fix more CI issues
6003902

Run tests on version pinned OpenAI too
09013f2

Fix pydantic issue in test
2f9667a

Import type in TYPE_CHECKING gate
ec84c04

PR feedback fixes
e45420e

Fix tiktoken test variant
0e319f1

PII gate the request and response
3acc9ab

Rename set_data tags
65c311d

Move doc location
4b33a4e

Add "exclude prompts" flag as optional
56d2679

colin-sentry force-pushed the openai branch from cbf4425 to 56d2679Compare March 7, 2024 18:01

Change prompts to be excluded by default
72d4b5a

colin-sentry mentioned this pull request Mar 7, 2024
Add OpenAI docs getsentry/sentry-docs#9376
Merged
3 tasks

colin-sentry added 2 commits March 7, 2024 13:16

Set flag in tests
b206e4c

Fix tiktoken tox.ini extra dash
4bc4310

antonpirker suggested changes Mar 8, 2024
View reviewed changes

colin-sentry force-pushed the openai branch from 26bc2aa to f7b8c05Compare March 8, 2024 15:16

Change strip PII semantics
c6f5cd2

colin-sentry force-pushed the openai branch from f7b8c05 to c6f5cd2Compare March 8, 2024 15:25

colin-sentry added 2 commits March 8, 2024 10:34

More test coverage for PII
d1eae09

notiktoken
c96e0e5

antonpirker approved these changes Mar 11, 2024
View reviewed changes

antonpirker mentioned this pull request Mar 11, 2024
feat(ai): LLM monitoring init #2514
Closed

antonpirkerand others added 2 commits March 11, 2024 11:30

Merge branch 'master' into openai
0278e2c

Merge branch 'master' into openai
452b03b

colin-sentry enabled auto-merge (squash) March 11, 2024 13:55

colin-sentry merged commit ff0a94b into masterMar 11, 2024

colin-sentry deleted the openai branch March 11, 2024 14:06

colin-sentry mentioned this pull request Mar 11, 2024
feat(ai): Add support for AI token span data getsentry/relay#3250
Merged

colin-sentry force-pushed the openai branch from 409fd88 to 5c7026cCompare March 6, 2024 22:01

Zylphrex reviewed Mar 6, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py
Comment threadsentry_sdk/integrations/openai.py Outdated
Comment threadsentry_sdk/integrations/openai.py Outdated

Zylphrex requested review from antonpirker, sentrivana and szokeasaurusrex March 6, 2024 22:31

indragiek reviewed Mar 7, 2024
View reviewed changes

Comment threadsentry_sdk/integrations/openai.py Outdated

antonpirker suggested changes Mar 7, 2024
View reviewed changes

colin-sentry requested a review from antonpirker March 7, 2024 16:03

colin-sentry mentioned this pull request Mar 7, 2024
Update span-data-conventions.mdx getsentry/develop#1183
Merged

colin-sentry force-pushed the openai branch from d9a6557 to cbf4425Compare March 7, 2024 18:01

colin-sentry added 14 commits March 7, 2024 13:01

OpenAI integration
6d79ebf

Fix linting errors
edf396c

Fix CI
05ecefa

Fix lint
a3ccbcd

Fix more CI issues
6003902

Run tests on version pinned OpenAI too
09013f2

Fix pydantic issue in test
2f9667a

Import type in TYPE_CHECKING gate
ec84c04

PR feedback fixes
e45420e

Fix tiktoken test variant
0e319f1

PII gate the request and response
3acc9ab

Rename set_data tags
65c311d

Move doc location
4b33a4e

Add "exclude prompts" flag as optional
56d2679

colin-sentry force-pushed the openai branch from cbf4425 to 56d2679Compare March 7, 2024 18:01

Change prompts to be excluded by default
72d4b5a

colin-sentry mentioned this pull request Mar 7, 2024
Add OpenAI docs getsentry/sentry-docs#9376
Merged
3 tasks

colin-sentry added 2 commits March 7, 2024 13:16

Set flag in tests
b206e4c

Fix tiktoken tox.ini extra dash
4bc4310

antonpirker suggested changes Mar 8, 2024
View reviewed changes

colin-sentry force-pushed the openai branch from 26bc2aa to f7b8c05Compare March 8, 2024 15:16

Change strip PII semantics
c6f5cd2

colin-sentry force-pushed the openai branch from f7b8c05 to c6f5cd2Compare March 8, 2024 15:25

colin-sentry added 2 commits March 8, 2024 10:34

More test coverage for PII
d1eae09

notiktoken
c96e0e5

antonpirker approved these changes Mar 11, 2024
View reviewed changes

antonpirker mentioned this pull request Mar 11, 2024
feat(ai): LLM monitoring init #2514
Closed

antonpirkerand others added 2 commits March 11, 2024 11:30

Merge branch 'master' into openai
0278e2c

Merge branch 'master' into openai
452b03b

colin-sentry enabled auto-merge (squash) March 11, 2024 13:55

colin-sentry merged commit ff0a94b into masterMar 11, 2024

colin-sentry deleted the openai branch March 11, 2024 14:06

colin-sentry mentioned this pull request Mar 11, 2024
feat(ai): Add support for AI token span data getsentry/relay#3250
Merged

Uh oh!

Conversation

colin-sentry commented Mar 6, 2024

General Notes

For maintainers

Uh oh!

Uh oh!

Uh oh!

Uh oh!

indragiek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

antonpirker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

antonpirker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

antonpirker left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

colin-sentry commented Mar 6, 2024

General Notes

For maintainers

Uh oh!

Uh oh!

Uh oh!

Uh oh!

indragiek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

antonpirker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

antonpirker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!