Logging Best Practices with Example

Posted on:July 29, 2023 at 03:00 PM

logging-best-practices-with-example Photo by NEOM on Unsplash

This is part one of the logging best practices. For part 2: Logging Abstraction in NodeJS Using Winston, Typescript

Firstly we should use a logging library (like log4j, JCL, Logback, Winston …) and not reinvent the wheel and we should log at the appropriate level.

Table of Contents

Open Table of Contents

Logging Levels

These logging levels are not mandatory but they are traditional and logging levels may vary by application requirements.

Trace

Debug

Info

Notice

Warning

Error

Fatal

Best Practices

Use timestamps at logs and consider these timestamps are in format UTC and standard ISO8601 date format.

Log messages should be comprehensible and include context.

For easier parsing of the automation tools payload may be separated

— Created user records | { count: 5, total: 8 }

Automatic parsing of log messages is essential in most situations we should write log messages considering this.

We should not log Debug, Trace logs at the Production, and logging at the debug level in production should do nothing.

We may include stack-trace while logging exceptions

Thread name, id may be included in logs if we are logging a multithreaded application.

We should not log sensitive information for example API keys, credentials, and passwords as clear text. We can use placeholders:

While writing log messages we should include the location of the unit that logs to the log message so we can easily find the source of error in the large codebase. We can use macros like FILE, and __filename which are defined by the programming environment.

We should implement abstraction for the logging framework and use the logging framework behind an abstraction (framework independent)

Log messages should be consistent. For example, if we are using customer_id for stating the id of the customer, we should use exactly customer_id in all of the other messages.

Log messages should be deleted or rotated after a specific duration(e.g. last 1 month) or file size. If we are logging into files, we may compress log files for reducing disk usage.

Send log messages to a single service in a distributed environment. (SoC, Fluentd, Sentry) In the distributed environment we may want to store our log messages in a single location and in a structured format(like JSON) so we can automatically parse log messages and do some analysis of them.

Log messages may be written to multiple locations simultaneously like console, file, and remote service(these locations may be named Sinks, Transports)

Structured Logging

Structured logging means logging with a specific structure like JSON, or XML. Automation tools can parse structured logs easily.

Example unstructured log

Example structured log

{
  "level": "error",
  "timestamp": "2023-01-01T00",
  "message": "This is error message",
  "context": {
    "abc": 5
  }
}

For example, we can use the jq command line tool to parse from JSON log message:

echo '{"fruit":{"name":"apple","color":"green"}}' | jq '.[] | select(.color=="green")'

Outputs:

{
  "name": "apple",
  "color": "green"
}

Key Points

Example

Thanks for reading. For the example: Logging Abstraction in NodeJS Using Winston, Typescript

References