The goal of this document is to summarize the differences between various structured logging libraries, in order to encourage a discussion so that a consensus can be reached for a common format.
Please use the GitHub issues for discussion, and submit GitHub Pull requests to add/fix the information in the tables.
Behold the following logging command that you might find in an application. There is a "message" string, and a "payload" object:
logger.info("Hello World", {"animal": "cat", "numLegs": 4})
^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Message Payload object
Some libraries merge the payload object with the root object, so the event will look like this:
{
"timestamp": "2018-06-18T23:16:45.000Z",
"message": "Hello World",
"animal": "cat",
"numLegs": 4
}While other libraries nest the payload object beneath some key, with a result like this:
{
"timestamp": "2018-06-18T23:16:45.000Z",
"message": "Hello World",
"data": {
"animal": "cat",
"numLegs": 4
}
}| Library | Message JSON Key | Payload Object JSON Key |
|---|---|---|
| bunyan (JavaScript) | msg | Merged with root |
| katip (Haskell) | msg | data |
| logrus (Go) | msg | Merged with root |
| logstash | message | Merged with root |
| ougai (Ruby) | msg | Merged with root |
| pygogo (Python) | message | Merged with root |
| roarr (JavaScript) | message | context |
| semantic logger (Ruby) | message | payload |
| serilog (C#) | @m / @mt | Merged with root |
| slog (Rust) | msg | Merged with root |
| structlog (Python) | msg | HELP NEEDED |
All libraries automatically add a timestamp to all events.
| Library | Timestamp JSON Key | Timestamp format |
|---|---|---|
| bunyan (JavaScript) | time | ISO 8601 |
| katip (Haskell) | at | ISO 8601 |
| logrus (Go) | time | ISO 8601 |
| logstash | timestamp / @timestamp | ISO 8601 / varies |
| ougai (Ruby) | time | ISO 8601 |
| pygogo (Python) | time | ISO 8601 |
| roarr (JavaScript) | time | Epoch milliseconds |
| semantic logger (Ruby) | timestamp | ISO 8601 |
| serilog (C#) | Timestamp / @t | ISO 8601 |
| slog (Rust) | ts | ISO 8601 |
| structlog (Python) | timestamp | ISO 8601 |
All libraries require that that all events are tagged with a level, chosen from a pre-defined list of available levels.
| Library | Log Level JSON Key |
|---|---|
| bunyan (JavaScript) | level |
| katip (Haskell) | sev |
| logrus (Go) | level |
| logstash | HELP NEEDED |
| ougai (Ruby) | level |
| pygogo (Python) | level |
| roarr (JavaScript) | context.logLevel |
| semantic logger (Ruby) | level |
| serilog (C#) | Level / @l |
| slog (Rust) | level |
| structlog (Python) | level |
| Library | VERBOSE | TRACE | DEBUG | INFO | NOTICE | WARNING | ERROR | CRITICAL | ALERT | FATAL | PANIC | EMERGENCY |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bunyan (JavaScript) | N/A | 10 | 20 | 30 | N/A | 40 | 50 | N/A | N/A | 60 | N/A | N/A |
| katip (Haskell) | N/A | N/A | Debug | Info | Notice | Warning | Error | Critical | Alert | N/A | N/A | Emergency |
| logrus (Go) | N/A | N/A | debug | info | N/A | warning | error | N/A | N/A | fatal | panic | N/A |
| logstash | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED |
| ougai (Ruby) | N/A | 10 | 20 | 30 | N/A | 40 | 50 | N/A | N/A | 60 | N/A | N/A |
| pygogo (Python) | N/A | N/A | DEBUG | INFO | N/A | WARNING | ERROR | CRITICAL | N/A | N/A | N/A | N/A |
| roarr (JavaScript) | N/A | 10 | 20 | 30 | N/A | 40 | 50 | N/A | N/A | 60 | N/A | N/A |
| semantic logger (Ruby) | N/A | trace | debug | info | N/A | warn | error | N/A | N/A | fatal | N/A | N/A |
| serilog (C#) | Verbose | N/A | Debug | Information | N/A | Warning | Error | N/A | N/A | Fatal | N/A | N/A |
| slog (Rust) | N/A | TRACE | DEBUG | INFO | N/A | WARN | ERROR | CRITICAL | N/A | N/A | N/A | N/A |
| structlog (Python) | N/A | N/A | debug | info | N/A | warning | error | critical | N/A | N/A | N/A | N/A |
Some libraries automatically add additional fields to all events (such as process ID and thread ID) and/or allow you to configure a few pre-determined global fields suchs the current environment and application.
| Library | Environment | Application | Namespace | Package | Machine Hostname | Process ID | Thread ID |
|---|---|---|---|---|---|---|---|
| bunyan (JavaScript) | N/A | N/A | N/A | N/A | hostname | pid | N/A |
| katip (Haskell) | env | app (Array of strings) | ns (Array of strings) | N/A | host | pid | thread |
| logrus (Go) | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| logstash | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED | HELP NEEDED |
| ougai (Ruby) | N/A | app | N/A | N/A | hostname | pid | N/A |
| pygogo (Python) | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| roarr (JavaScript) | N/A | context.application | context.namespace | context.package | context.hostname | N/A | N/A |
| semantic logger (Ruby) | environment | application | N/A | N/A | host | pid | thread |
| serilog (C#) | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| slog (Rust) | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| structlog (Python) | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
Logging of HTTP requests is very common. Let's figure out a common format for HTTP method, host, url, headers, remote IP, remote IP geo-lookup, response status code, response size, etc...
!!! TODO !!!