Logging Packages in Golang

Support for logging in golang is fragmented. What logging packages should you use?

Nick Galbreath · May 21, 2018

Out of the 35 packages in Awesome Go, a majority are obsolete, duplicative or fail to address what problem they are trying to solve. Of the remaining (and some not on the list), they fall into four categories.

Basics
General purpose
Performance focused
Composable

The Basics

The basics are just that - there do the job and are common or installed by default. However none of these support context logging or structured logging. They only support a printf style log message.

First is good ‘ol stdlib/log. It was probably designed for building and debugging golang itself, and not a complete application logging framework. It’s just enough to get started. There is no support for structured logging, contexts, or even levels. That’s said it’s easy to build on top of this to do more. See spf/jwalterweatherman and hashicorp/logutils for inspiration.

At other end of the spectrum is golang/glog. It’s a implementation of the C++ log package that google uses. It’s supports leveled logging and few other features. While it’s on github, its frozen. No updates are accepted. It’s assume this for google’s internal use and compatibility. Besides logging, it also provides configuration if you use stdlib/flag. This may good or bad depending on your use case.

General Purpose

General purpose (and every other logger listed here), support a variety of output and export formats, and provide context and structured logging support.

First is sirupsen/log. It’s client interface duplicates the methods of stdlib log, so you can drop in and replace your existing logging and everything will just work.

// import "log"
import github.com/sirupsen/log

sirupus has a lot of output handlers and formats. It’s likely it can export the log into whatever format or consumer you wish.

The downside is that IMHO is messy. The interface is huge, and leaks things (e.g. Entry) logging customers don’t need. It has two output mechanisms: the regular handlers and hooks, but unclear why both are needed. The main package imports a few secondary packages for use in console logging, even if you aren’t use console logging.

An alternative is apex/log. It’s inspired by logrus but has a simpler interface and completely separates out the handler implementations. The console handers are especially snazzy and not loaded in by default. It has additional features for error logging and supports pkg/errors which you are probably already using (or should be).

Both use a WithFields method, along with a Fields object, to provide structured and context logging:

log.WithFields(log.Fields{
  "event": event,
  "topic": topic,
  "key": key,
}).Fatal("Failed to send event")

While type-safe, it seems clunky to use in practice and forces creating a throwaway object. Both are “relatively slow.” It won’t matter a bit for console and light duty applications.

High Performance

The packages uber/zap and rs/zerolog are designed for absolute performance. In particular they strive for no memory allocations. They both provide structured, leveled and context logging. They output JSON to stdout and expect someone else to consume and import it. For performance reasons, they both avoid using reflection and instead use explicit typed fields:

// zap uses a vararg style -- message first
logger.Info("failed to fetch URL",
  // Structured context as strongly typed Field values.
  zap.String("url", url),
  zap.Int("attempt", 3),
  zap.Duration("backoff", time.Second),
)

// zerolog uses a fluent style -- message last
log.Debug().
  Str("Scale", "833 cents").
  Float64("Interval", 833.09).
  Msg("Fibonacci is everywhere")

Zap also has a helper to skip this step. It’s easier to use, but slightly slower:

sugar.Infow("failed to fetch URL",
  "url", url,
  "attempt", 3,
  "backoff", time.Second,
)

They are both large packages (as everything is customized for performance) and do not provide interfaces. As mentioned they only output JSON (they are some console packages for debugging) and it’s expected someone else is importing these or converting to whatever the destination needs. But if you need the speed, these are the loggers use.

Composable Kits

Somewhere in the middle are logging packages that are based from composable functions.

The first is inconshreveable/log15. The godoc reference is worth reading, not just to see how log15 works, but also on the practice of logging. Especially good are sections on context and lazy logging.

The interface is mostly simple and uses a vararg, message-first style (like uber/zap) for structured and context logging:

requestlogger := log.New("path", r.URL.Path)
requestlogger.Debug("db txn commit", "duration", txnTimer.Finish())

The only problem being the Logger interface also exposes it’s Handler which seems un-necessary. Included by default is a console handler which loads in three external packages (mostly dealing with colorization). FWIW, log15 seems always rank last in performance testing.

Inspired by log15 is go-kit/kit/log. It works off a single logging interface:

Log(keyval ...interface{}) error

Using this interface, more complicated loggers for levels and context are made while preserving this interface. While it is very clever, the end result for leveled loggers is unusual compared to other systems:

import (
    "github.com/go-kit/kit/log"
    "github.com/go-kit/kit/log/level"
)
logger := log.NewLogfmtLogger(os.Stderr)
logger.Log("foo", "bar") // as normal, no level
level.Debug(logger).Log("request_id", reqID, "foo", "bar")

Everything can return an error which also seems unusual for application logging. The idea is that the universal interface could be used for any type of logging, including transaction processing, so errors need to be returned.

It includes by default two handlers (output formats), one for JSON and another in logfmt, which loads in 2 or 3 external packages.

For performance, go-kit isn’t as fast as zerolog or zap, but faster than just about everything else.

The following design documents are worth reading even if you chose not to use go-kit/kit/log:

Summary slides: go-kit/log package
More slides: The Hunt For a Logger Interface
Why return an error? GitHub Issue164

Conclusion

What package to use? These all work, but it all depends on what problem are you trying to solve.

For high volume (think: 100s or 1000s logs per second) or constrained devices (think Raspberry Pi): zap or zerolog.
To quickly replace all your crappy stdlib log calls to have a better output: sirupus, maybe apex
To exported logs to an existing system: logrus or apex/log
For pretty console applications: apex/log

While log15 and go-kit are excellent loggers, I couldn’t really find a sweet spot for them.

There is nothing wrong with keeping it simple and using stdlib. Consider this from Peter Bourgon in Go: Best Practices for Product Environments:

We played around with several logging frameworks, providing things like leveled logging, debug, output routing, special formatting, and so on. In the end, we settled on plain package log. It works because we only log actionable information.

Log on…