Skip to content

[Storage][Logging]Let users opt in logging_request_body - #19710

Merged
Xiaoxi Fu (xiafu-msft) merged 8 commits into
Azure:mainfrom
xiafu-msft:logging
Oct 15, 2021
Merged

Xiaoxi Fu (xiafu-msft) merged 8 commits into
Azure:mainfrom
xiafu-msft:logging

Conversation

@xiafu-msft

@xiafu-msft Xiaoxi Fu (xiafu-msft) commented Jul 7, 2021

Copy link
Copy Markdown
Contributor

Resolves #19419

@ghost ghost added the Storage Storage Service (Queues, Blobs, Files) label Jul 7, 2021
if isinstance(http_request.body, types.GeneratorType):
_LOGGER.debug("File upload")
else:
if self.logging_request_body or options.pop("logging_request_body", False):

ghost Jul 15, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't touch on_response because logging response body is handled pretty well.
For download blob, since we are doing streaming download, no actually body will be logged. For other responses logging response body is helpful, since the response body could contains raw error message or some xml elements we want to check.
Let me know if you think we should disable logging for any kinds of response body .

ghost Jul 15, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ghost Jul 15, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

besides some may want to disable body logging due to PII being present inside.

ghost left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add test coverage ?

Comment on lines +225 to +226
# We don't want to log the binary data of a file upload.
_LOGGER.debug("Hidden body, please use logging_body to show body")

ghost Jul 22, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather delete this else branch. It's going to produce lot of entries that are not very useful.
Instead consider adding logger.debug in ctor/ctors if logging is enabled but body logging is not. so that there's some hint about that option but is not rendered on each request.

ghost Oct 15, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll leave this like this for now, and put it in the TODO so it will be picked up later.

Comment on lines 247 to 254
_LOGGER.debug("File attachments: %s", filename)
elif response.http_response.headers.get("content-type", "").endswith("octet-stream"):
elif resp_content_type.endswith("octet-stream"):
_LOGGER.debug("Body contains binary data.")
elif response.http_response.headers.get("content-type", "").startswith("image"):
elif resp_content_type.startswith("image"):
_LOGGER.debug("Body contains image data.")

ghost Jul 22, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could consider dumbing binary payloads in encoded (base64 for example) form. (not blocking this PR).

@tasherif-msft

ghost commented Oct 11, 2021

Copy link
Copy Markdown
Contributor

Is there a reason we've stalled on this? Xiaoxi Fu (@xiafu-msft) Kamil Sobol (@kasobol-msft)

@xiafu-msft

ghost commented Oct 15, 2021

Copy link
Copy Markdown
Contributor Author

Is there a reason we've stalled on this? Xiaoxi Fu (@xiafu-msft) Kamil Sobol (@kasobol-msft)

just I forgot it

@xiafu-msft
Xiaoxi Fu (xiafu-msft) merged commit fee8f87 into Azure:main Oct 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Storage Storage Service (Queues, Blobs, Files)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable customers to configure debug logging

3 participants