log output

[bpostlethwaite]

Expanding on #1 (comment)

For logging to STDERR printing single lines is easier to integrate into tools like Stackdriver which is already integrated into GKE. Printing single line JSON blobs with error information and consistent keys is even better and seems like the approach taken by newer tools. It is also much more easily digestible by monitoring endpoints.

{error: 'error message', level: 'debug', requestID: 'someRequestIDIfApplicable'}... etc.

I can help with this stuff.

[scjody]

@etpinard Here's some documentation on logging: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry

Unfortunately this is for the "on-wire" format used by Stackdriver, and I can't find information on what fluentd (the logging agent used in GKE) needs exactly, but it's pretty close. The big thing is that most fields aren't actually required (since fluentd will add them), and it's probably best not to include these (because I bet bad things will happen if they're wrong).

I recommend:

• 1 log message per http request once that request completes, giving as much information as possible in the HttpRequest format of that documentation:

{
 "httpRequest": {
 "requestMethod": "POST",
 ....
 }
}

(non-applicable fields can be omitted)

• For errors, log a message with "ERROR" severity:

{
 "severity": "ERROR",
 "textPayload": "Some error message"
}

If you're sending an error back to the user like "invalid or malformed request syntax (no 'data' and no 'layout' in figure)", I don't think it's useful to log this as a separate error in the logs. Just send a httpRequest entry with 400 status - that's enough to let anyone viewing the logs know that a 400 error happened and it was handled correctly. Logged errors are for things we don't want to show to the user, like stack traces or other internal details.