↓Skip to main content

Deep Dive in the Upcoming Go Error Inspection Changes

·14 mins
Table of Contents

The team behind the language started working on a document titled “Go 2 Draft Designs”. The draft document is part of the design process for Go v2.0. In this article we will zoom in at the proposal for error inspection in Go 2.

What’s worth noting is that this is still part of a draft proposal. Still, from where we are standing at the moment, it’s certain that these changes will be added to the language starting Go v1.13 and v2.0.

Before we go on, it’s good to note that this article is aimed at folks that consider themselves new to the language. Not just with time spent with the language, but also in terms of size of projects/codebases. So if that sounds like you - keep on reading, I promise it will be helpful to understand the pros and cons of errors in Go.

If you have experience with Go and error handling you might have a good idea about the topic. Still, you may find something of value in the article. Let me know in the comments section below!

Go 2 Draft Designs #

The intent behind the draft documents is to create a discussion cornerstone for the community. They touch three important topics: generics, error handling and error value semantics.

The reason for discussing these three topics is… well, us – the community. The Go team got the community’s feedback on the shortcomings of the language via the annual Go user survey. And there is no surprise on the prioritised areas for v2.0: package management, errors and generics.

Package management was a big thing, but that area is already addressed with the introduction of modules.

In fact, here’s a more detailed breakdown of the biggest challenges that people face with Go today:

Image blatantly linked from Go’s blog Image blatantly linked from Go’s blog

If you would like to learn more, the Go team published a blog post with the results and other insights.

As we said earlier, in this article we will focus on the error inspection proposal. The error handling and formatting are quite interesting as well, but those are topics for other articles.

Shortcomings of the current errors inspection approach #

Before we go on to see what are the problems with the current situation in the ecosystem, let’s get a better understanding of errors first. Looking at the topic from my perspective, as a beginner in Go, it’s challenging to understand the shortcomings here.

Go promotes the idea that errors are values. When something is a value, it means that we can assign it to a variable. This implies that we can work with errors via ordinary programming. No need for special exception flows or rescue blocks. For example, this is an error that’s a value:

r, err := os.Open(src)
if err != nil {
	return err
}
defer r.Close()

The the os.Open function returns a file for reading or an error. Assigning the error to a variable and checking for its presence (err != nil) is possible because errors are values. In contrast with other languages, when a it’s not possible to open a file, we have to catch/handle an exception.

This flexibility that Go unlocks is great. But also it changes they way we perceive and use errors as values in our programs. A side-effect of this is that equality checks and type assertion on errors has proved to be tricky and have limitations.

If you read the Problem Overview document on error values by Russ Cox, he describes four ways of testing for specific errors:

  1. Test for equality with sentinel errors, like io.EOF
  2. Check for an error implementation type using a type assertion or type switch
  3. Ad-hoc checks, like os.IsNotExist, check for a specific kind of error
  4. Substring searches in the error text reported by err.Error()

Let’s take this a step further.

Error wrapping #

The people in the Go ecosystem have created various packages that aid error inspection. For example, one of the most popular packages that serves this purpose is pkg/errors. Another one is Uber’s multierr. Others include spacemonkeygo/errors, hashicorp/errwrap and upspin.io/error.

If you would inspect each of these you would notice different patterns for error wrapping. Error wrapping is a technique where we wrap one error value in another error value, of a different type. The goal is to add more information to it. In 2016 Dave Cheney wrote an article on the topic, titled “Don’t just check errors, handle them gracefully”.

The basic idea is that any error that happens deeper in the call stack will be unavailable for inspection on the surface. We have to annotate errors down the stack, so we can inspect and handle the proper error at a certain level in the stack.

Let’s look at an example: imagine we want to open a file that doesn’t exist.

package main

import (
	"fmt"
	"os"
)

func main() {
	f, err := os.Open("notes.txt")
	if err != nil {
		fmt.Print(fmt.Errorf("%+v", err))
	}
	defer f.Close()
}

If we run this small program, we will get the following output:

$ go run test.go
open notes.txt: no such file or directory

There is no hint of the type of the error or where this error originated from, such as function name or line number. To be able to get more information to user, we would have to add annotation to our error:

package main

import (
	"fmt"
	"os"
)

func main() {
	f, err := os.Open("notes.txt")
	if err != nil {
		fmt.Print(fmt.Errorf("Error opening file: %+v", err))
	}
	defer f.Close()
}

The output:

$ go run test.go
Error opening file: open notes.txt: no such file or directory%

A better way to approach this is to use error wrapping.

Here’s a very simple example: our type will allow attaching of a timestamp to the error. It will also print the timestamp as part of the error message. The code:

package main

import (
    "fmt"
    "os"
    "time"
)

type ErrorWithTime struct {
    err error // the wrapped error
    t time.Time // the time when the error happened
}

func (e *ErrorWithTime) Error() string {
    return fmt.Sprintf("%v @ %s", e.err, e.t)
}

func wrap(err error) *ErrorWithTime {
    return &ErrorWithTime{err, time.Now()}
}

func openFile(filename string) (*os.File, error) {
    f, err := os.Open(filename)
    if err != nil {
        return nil, wrap(err)
    }

    return f, nil
}

func main() {
    file, err := openFile("notes.txt")
    if err != nil {
        fmt.Print(err)
    }
    defer file.Close()
}

The ErrorWithTime struct has two attributes: the wrapped error err and the time of the error occurrence t. The Error() function will include the timestamp of the occurrence alongside with the error itself. The wrap function is the one doing the magic. It takes an error as an argument and returns the error wrapped in a ErrorWithTime, with the exception timestamp attached (set to time.Now()).

The main function does not care about the type of the error. It checks for its presence and prints (if present). If we run the program, this will be the output:

$ go run test.go
open notes.txt: no such file or directory @ 2019-04-05 21:40:06.055665 +0200 CEST m=+0.000287159%

I admit - the timestamp of the error occurrence might not be that useful in this context. What is important is that it paints the idea how we can attach more information to errors by wrapping them. If you read Dave’s article I linked above, you will see the shortcomings of this approach and the argument for unwrapping.

Now, having this in mind, let’s move back to the error inspection proposal and see what the authors are proposing.

Go 2 Error Inspection #

While wrapping works, it comes at a certain cost, which the draft design document addresses:

If a sentinel error is wrapped, then a program cannot check for the sentinel by a simple equality comparison. And if an error of some type T is wrapped (presumably in an error of a different type), then type-asserting the result to T will fail. If we encourage wrapping, we must also support alternatives to the two main techniques that a program can use to act on errors, equality checks and type assertions.

So, let’s see how the authors of the proposal are going to address the two most important aspects of error wrapping: equality comparison and type assertion.

Unwrapping #

In the draft document the authors acknowledge the need for an unwrapping mechanism. The idea behind that to be able to do any comparisons or assertions, we have to be able to unwrap errors. This is in fact a simple functionality of the custom error type, but it’s of exceptional importance.

That’s why they introduce the Wrapper interface:

package errors

// A Wrapper is an error implementation
// wrapping context around another error.
type Wrapper interface {
	// Unwrap returns the next error in the error chain.
	// If there is no next error, Unwrap returns nil.
	Unwrap() error
}

The Unwrap function here does not have a body because we’re looking at an interface definition. This interface is quite important though. All types that will implement Wrapper will have inspection of wrapped errors.

Let’s see a contrived example of a custom error type that implements the Wrapper interface:

// ErrorWithTime is an error with a timestamp
type ErrorWithTime struct {
	err error     // the wrapped error
	t   time.Time // the time when the error happened
}

For the ErrorWithTime to implement the Wrapper interface, it needs to have a Unwrap function. The function will return the error that the custom error type contains:

func (e *ErrorWithTime) Unwrap() error {
	return e.err
}

That’s all. This will allow any of the next aspects, that we will discuss further, to function well with the ErrorWithTime type.

Type assertion using As #

Now, let’s combine two ideas: opening a file that does not exist and returning an error that contains a time of occurrence. Let’s reintroduce the ErrorWithTime type and its related functions:

type ErrorWithTime struct {
	err error     // the wrapped error
	t   time.Time // the time when the error happened
}

// Implements the Error interface
func (e *ErrorWithTime) Error() string {
	return fmt.Sprintf("Error: %s, ocurred at: %s\n", e.err, e.t)
}

// Implements the Wrapper interface
func (e *ErrorWithTime) Unwrap() error {
	return e.err
}

To reiterate, the Unwrap function is the one that implements the Wrapper interface. And the Error function implements the Error interface. This means that we can unwrap any error value of our ErrorWithTime type.

Now, let’s create a simple function that can open a file and return an error (if any):

func openFile(path string) error {
	_, err := os.Open(path)
	if err != nil {
		return &ErrorWithTime{
			err: err,
			t:   time.Now(),
		}
	}
	return nil
}

This function is a tad useless because it only returns a potential error, but it does the trick for our example. If the os.Open call returns an error we will wrap it in a ErrorWithTime and attach the current time to it. Then, we’ll return it.

Now, let’s see the main function:

func main() {
	err := openFile("non-existent-file")
	if err != nil {
		var timeError *ErrorWithTime
		if xerrors.As(err, &timeError) {
			fmt.Println("Failed at: ", timeError.t)
		}

                var pathError *os.PathError
                if xerrors.As(err, &pathError) {
                        fmt.Println("Failed at path:", pathError.Path)
                }
	}
}

Here’s how the new As function helps by taking care of the type assertion. It receives the error from the openFile function and asserts the error’s type to *ErrorWithTime. If the assertion is successful, it will use its t attribute to display the time when the error happened.

The same goes for asserting the error type to *os.PathError. Again, the *ErrorWithTime implements the Wrapper interface. This allows the xerrors.As function to unwrap the error and check for the type of the error under wraps. Let’s run the main function:

$ go run as.go
Failed at:  2019-04-21 18:11:01.922332 +0200 CEST m=+0.000303480
Failed at path: non-existent

Note that the error type returned from the openFile function is *ErrorWithTime. But, once it’s casted to a *os.PathError we cannot access the t attribute. This is because *ErrorWithTime implements t – not *os.PathError.

Let’s test that with code:

func main() {
	err := openFile("non-existent-file")
	if err != nil {
                var pathError *os.PathError
                if xerrors.As(err, &pathError) {
                        fmt.Println("Failed at path:", pathError.Path)
                        fmt.Println("Failed at:", pathError.t)
                }
	}
}

This fails with the error:

./test.go:44:44: pathError.t undefined (type *os.PathError has no field or method t)

Value checking with Is #

While As allows us to take a value and assert its type, Golang has errors which are resistive to type assertions. They are special and have a special name - sentinel errors. Their name descends from the practice in computer programming of using a specific value to signify that no further processing is possible.

When a caller handles a sentinel error, they have to compare the returned error value to a predeclared value using the equality operator. Here’s an example (adapted from A Tour of Go) that handles a io.EOF error:

package main

import (
	"io"
	"strings"
)

func main() {
	r := strings.NewReader("Hello, Reader!")

	b := make([]byte, 8)
	for {
		_, err := r.Read(b)
		if err == io.EOF {
			break
		}
	}
}

The example above creates a strings.Reader and consumes its output 8 bytes at a time. When the reading hits a io.EOF (I/O End Of File) error, it will break the for loop and exit the program.

So, how does the new Is function help us in such cases? Well, if we would use the xerrors package in the same example, it would look like this:

package main

import (
	"io"
	"strings"

	"golang.org/x/xerrors"
)

func main() {
	r := strings.NewReader("Hello, Reader!")

	b := make([]byte, 8)
	for {
		_, err := r.Read(b)
		if xerrors.Is(err, io.EOF) {
			break
		}
	}
}

The change is simple on the surface, but considerable under the hood. The Is function takes an error value and a sentinel error as arguments. If the error value implements Wrapper, it will unwrap its chain of error until it reaches (or not) the sentinel error. Hence, if it finds the sentinel error it will return true, otherwise false.

Let’s expand the above example with our ErrorWithTime type and see the Is function in action:

package main

import (
	"fmt"
	"io"
	"strings"
	"time"

	"golang.org/x/xerrors"
)

type ErrorWithTime struct {
	err error     // the wrapped error
	t   time.Time // the time when the error happened
}

func (e *ErrorWithTime) Error() string {
	return fmt.Sprintf("Error: %s, ocurred at: %s\n", e.err, e.t)
}

func (e *ErrorWithTime) Unwrap() error {
	return e.err
}

func readReader(r io.Reader) error {
	buffer := make([]byte, 8)
	for {
		_, err := r.Read(buffer)
		if err != nil {
			return &ErrorWithTime{
				err: err,
				t:   time.Now(),
			}
		}
	}
}

func main() {
	r := strings.NewReader("Hello, Reader!")

	err := readReader(r)

	if xerrors.Is(err, io.EOF) {
		fmt.Println(err)
	}
}

When run, this will produce:

$ go run eof.go
Error: EOF, ocurred at: 2019-04-21 11:32:41.842139 +0200 CEST m=+0.000228985

The readReader function takes a io.Reader as argument and read it until the reading returns an error. When it returns the error, it will wrap it in an &ErrorWithTime and returned to the caller (main). The caller then uses xerrors.Is to check if the error returned is actually a io.EOF under wraps. If it is, it will print the error.

This works because ErrorWithType implements Wrapper. This allows the caller to print the error message from the error value, while still being able to detect the error under wraps.

Putting it in practice with xerrors #

So you might be wondering, how you can start using the Is and As functions in your current Go programs. Well, as you have noticed in the examples we used the xerrors package. This was in fact announced by Marcel van Lohuizen at dotGo 2019 in Paris - you can check his talk here. (If you’re curious how the event was in general, I published a report on it here.)

The xerrors package is built to support transitioning to the Go 2 proposal for error values. Most of the functions and types from xerrors will be incorporated into the standard library’s errors package in Go 1.13. The idea is that by using this package you can make your Go 1.12 code be compatible with 1.13 (and later version 2).

You can read its documentation and check out the examples included.

It is also worth mentioning that replacing equality checks with xerrors.Is and type assertions with xerrors.As will not change the meaning of existing programs that do not wrap errors and it will future-proof programs against wrapping.

As with any language changes, there will be situations where the new functions will not do the trick. For example, sometimes callers want to perform many checks against the same error. One such case would be when the caller would compare the error value against more than one sentinel value. In such cases we can still use Is and As. The drawback is in the way these functions walk through the chain of wrapped errors. This means they would walk up the chain multiple times, which is wasteful.

In any case, improving the situation with error inspection in Go is a good step forward. I am used to more conventional error handling. Still, I think that this is not a step in the wrong direction when it comes to error inspection. Also, I recommend reading the discussion section of the error inspection draft. It lays out some good guidelines on how to define and use your error types.

What is your opinion on the proposal? Do you think it will simplify your life as a Go developer? Or do you maybe prefer a different approach? Let me know in the comments below!

↑