Hi, not sure if this is the right place or it should rather go to ddtrace or somewhere else:
Reproduction repo with CDK: https://github.com/maxfriedrich/datadog-cloud-payload-tagging-repro
The cloud request/response payload tagging feature causes a big slowdown for us for any DynamoDB query, even when "disabled" with the environment variable value "$.*" (0.xx seconds to > 10 seconds query), with the following setup:
- Docker Lambda based on the Lambda Python 3.11 (or 3.13) base image
- Installing the Datadog Lambda extension into the container
- Installing datadog-lambda and ddtrace with into the container
- Running the function with
datadog_lambda.handler
(this function outgrew the zip size limit, this is why it's built like this -- is this a supported setup?)
We can work around it by setting the environment variable values to "", fully disabling the feature, not just redacting all fields.
The problem only occurs with a Docker based Lambda function, the handler from my reproduction is fine in the Python runtime and the Datadog extension and Lambda layers.
Output from my test script:
Testing slow function: arn:aws:lambda:eu-central-1:...:function:DdTaggingReproStack-Function1904AA941-QLjJ2nwy96AVExecution time: 11.545 secondsExecution time: 10.865 secondsExecution time: 11.279 secondsExecution time: 11.604 secondsExecution time: 11.659 secondsTesting fast function: arn:aws:lambda:eu-central-1:...:function:DdTaggingReproStack-Function28EA0674A-36P7xO9x9CbGExecution time: 0.235 secondsExecution time: 0.188 secondsExecution time: 0.168 secondsExecution time: 0.194 secondsExecution time: 0.206 seconds
Do you have any ideas what could cause this? Feel free to move the issue if it belongs somewhere else. Thank you very much!
Hi, not sure if this is the right place or it should rather go to ddtrace or somewhere else:
Reproduction repo with CDK: https://github.com/maxfriedrich/datadog-cloud-payload-tagging-repro
The cloud request/response payload tagging feature causes a big slowdown for us for any DynamoDB query, even when "disabled" with the environment variable value
"$.*"(0.xx seconds to > 10 seconds query), with the following setup:datadog_lambda.handler(this function outgrew the zip size limit, this is why it's built like this -- is this a supported setup?)
We can work around it by setting the environment variable values to
"", fully disabling the feature, not just redacting all fields.The problem only occurs with a Docker based Lambda function, the handler from my reproduction is fine in the Python runtime and the Datadog extension and Lambda layers.
Output from my test script:
Do you have any ideas what could cause this? Feel free to move the issue if it belongs somewhere else. Thank you very much!