Testing in Go: HTTP Servers

Table of Contents

Go’s a great hammer for a lot of nails, one of the areas where I find it fitting is building HTTP servers. The net/http package of Go’s standard library makes it easy to attach HTTP handlers to any Go program. What I find delightful is that Go’s standard library, also has packages that make testing HTTP servers as easy as building them.

Nowadays, it’s widely accepted that test coverage is important and useful. Having the quick feedback loop when changing your code is valuable, hence we invest time in testing our code. When combined with methodologies like continious integration and continious delivery, a good test suite becomes an invaluable part of the software project.

Knowing the value of a good test suite, what approach should we - developers using Go to build HTTP servers - take to testing our HTTP servers?

Here’s everything you need to know to test your Go HTTP servers well.

Ordering some pizzas #

As with any other article in this series on testing in Go, let’s create an example implementation that we can use as a test subject.

Here’s a small implementation of a pizza restaurant API with three endpoints:

List all pizzas on the menu: GET /pizzas
Make a simple pizza order: POST /orders
List all orders in the system: GET /orders

Here’s the code:

package main

import (
	"encoding/json"
	"fmt"
	"log"
	"net/http"
)

type Pizza struct {
	ID    int    `json:"id"`
	Name  string `json:"name"`
	Price int    `json:"price"`
}

type Pizzas []Pizza

func (ps Pizzas) FindByID(ID int) (Pizza, error) {
	for _, pizza := range ps {
		if pizza.ID == ID {
			return pizza, nil
		}
	}

	return Pizza{}, fmt.Errorf("Couldn't find pizza with ID: %d", ID)
}

type Order struct {
	PizzaID  int `json:"pizza_id"`
	Quantity int `json:"quantity"`
	Total    int `json:"total"`
}

type Orders []Order

type pizzasHandler struct {
	pizzas *Pizzas
}

func (ph pizzasHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json")

	switch r.Method {
	case http.MethodGet:
		if len(*ph.pizzas) == 0 {
			http.Error(w, "Error: No pizzas found", http.StatusNotFound)
			return
		}

		json.NewEncoder(w).Encode(ph.pizzas)
	default:
		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
	}
}

type ordersHandler struct {
	pizzas *Pizzas
	orders *Orders
}

func (oh ordersHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json")

	switch r.Method {
	case http.MethodPost:
		var o Order

		if len(*oh.pizzas) == 0 {
			http.Error(w, "Error: No pizzas found", http.StatusNotFound)
			return
		}

		err := json.NewDecoder(r.Body).Decode(&o)
		if err != nil {
			http.Error(w, "Can't decode body", http.StatusBadRequest)
			return
		}

		p, err := oh.pizzas.FindByID(o.PizzaID)
		if err != nil {
			http.Error(w, fmt.Sprintf("Error: %s", err), http.StatusBadRequest)
			return
		}

		o.Total = p.Price * o.Quantity
		*oh.orders = append(*oh.orders, o)
		json.NewEncoder(w).Encode(o)
	case http.MethodGet:
		json.NewEncoder(w).Encode(oh.orders)
	default:
		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
	}
}

func main() {
	var orders Orders
	pizzas := Pizzas{
		Pizza{
			ID:    1,
			Name:  "Pepperoni",
			Price: 12,
		},
		Pizza{
			ID:    2,
			Name:  "Capricciosa",
			Price: 11,
		},
		Pizza{
			ID:    3,
			Name:  "Margherita",
			Price: 10,
		},
	}

	mux := http.NewServeMux()
	mux.Handle("/pizzas", pizzasHandler{&pizzas})
	mux.Handle("/orders", ordersHandler{&pizzas, &orders})

	log.Fatal(http.ListenAndServe(":8080", mux))
}

The orders and the pizzas are kept in-memory – the example is simple on purpose and adding any storage solution will greatly complicate the example. The body of the POST /orders endpoint expects a JSON body, with a pizza_id and quantity.

This server has a bunch of downsides and is absolutely not ready for production usage. It never checks the Accept header of the request, the errors responses are lacking additional details and proper envelopes. Our little HTTP server also never validates the data the client sends – if the quantity is missing from the request body of the POST /orders it will create a new order with a total of $0:

$ curl -X POST localhost:8080/orders -d '{"pizza_id":1,"quantity":0}' | jq
{
  "pizza_id": 1,
  "quantity": 0,
  "total": 0
}

Lastly, it allows us to order only one type of pizza per order. What a weird pizza shop, isn’t it? Still, as an example for this article it will do just fine.

Let’s run it locally and send a few requests using cURL. To see all of the available pizzas on the menu:

$ curl -X GET localhost:8080/pizzas | jq
[
  {
    "id": 1,
    "name": "Pepperoni",
    "price": 12
  },
  {
    "id": 2,
    "name": "Capricciosa",
    "price": 11
  },
  {
    "id": 3,
    "name": "Margherita",
    "price": 10
  }

To see the (empty) list of orders:

$ curl -X GET localhost:8080/orders
null

To create an order of four Pepperoni pizzas:

$ curl -X POST localhost:8080/orders -d '{"pizza_id":1,"quantity":4}' | jq
{
  "pizza_id": 1,
  "quantity": 4,
  "total": 48
}

From the response it is clear that the total is $48. To see the new order in the list of orders:

$ curl -X GET localhost:8080/orders | jq
[
  {
    "pizza_id": 1,
    "quantity": 4,
    "total": 48
  }
]

Great! Our simple HTTP server is working as it should. So, how do we test it?

Testing the `pizzasHandler` #

As mentioned in the first section of this article, Go ships with everything we need to test our HTTP server in a single package called net/http/httptest (docs). Simply put, this package provides utilities for HTTP testing.

Let’s create a main_test.go file where we will add our tests. First, we will test our pizzasHandler:

func TestPizzasHandler(t *testing.T) {
	tt := []struct {
		name       string
		method     string
		input      *Pizzas
		want       string
		statusCode int
	}{
		{
			name:       "without pizzas",
			method:     http.MethodGet,
			input:      &Pizzas{},
			want:       "Error: No pizzas found",
			statusCode: http.StatusNotFound,
		},
		{
			name:   "with pizzas",
			method: http.MethodGet,
			input: &Pizzas{
				Pizza{
					ID:    1,
					Name:  "Foo",
					Price: 10,
				},
			},
			want:       `[{"id":1,"name":"Foo","price":10}]`,
			statusCode: http.StatusOK,
		},
		{
			name:       "with bad method",
			method:     http.MethodPost,
			input:      &Pizzas{},
			want:       "Method not allowed",
			statusCode: http.StatusMethodNotAllowed,
		},
	}

	for _, tc := range tt {
		t.Run(tc.name, func(t *testing.T) {
			request := httptest.NewRequest(tc.method, "/orders", nil)
			responseRecorder := httptest.NewRecorder()

			pizzasHandler{tc.input}.ServeHTTP(responseRecorder, request)

			if responseRecorder.Code != tc.statusCode {
				t.Errorf("Want status '%d', got '%d'", tc.statusCode, responseRecorder.Code)
			}

			if strings.TrimSpace(responseRecorder.Body.String()) != tc.want {
				t.Errorf("Want '%s', got '%s'", tc.want, responseRecorder.Body)
			}
		})
	}
}

In this test we take the common table-driven approach to testing the handler. We provide three cases: without any pizzas on the menu, with some pizzas on the menu and with a bad HTTP method being used.

Then, for each of the test cases we run a subtest, which creates a new request and a responseRecorder.

The httptest.NewRequest function returns a new http.Request, which we can then use to invoke the handler with. The returned http.Request represents an incoming request to the handler. By creating this struct within the test we do not have to rely on booting up an actual HTTP server and sending an actual HTTP request to it. The http.Request struct gives us the ability to simulate a real request just by passing the struct to the handler under test.

The httptest.NewRequest function is essential to unit testing HTTP handlers here. Yet, it does only half of the job - “sending” a request to the handler. What about the other half - the response?

Writing the response is done using the httptest.ResponseRecorder – it represents an actual implementation of http.ResponseWriter (the second argument of pizzasHandler’s ServeHTTP method). It records its mutations and allows us to make any assertions on it later in the test.

The httptest.ResponseRecorder and http.Request duo are all we need to sucessfully test any HTTP handler in Go.

If we run the above test, we’ll see the following output:

$ go test -v -run PizzasHandler
=== RUN   TestPizzasHandler
=== RUN   TestPizzasHandler/without_pizzas
=== RUN   TestPizzasHandler/with_pizzas
=== RUN   TestPizzasHandler/with_bad_method
--- PASS: TestPizzasHandler (0.00s)
    --- PASS: TestPizzasHandler/without_pizzas (0.00s)
    --- PASS: TestPizzasHandler/with_pizzas (0.00s)
    --- PASS: TestPizzasHandler/with_bad_method (0.00s)
PASS
ok  	_/Users/Ilija/Documents/go-playground/testing-in-go-http-servers	0.023s

(In case you’re following along, I recommend breaking the tests by changing any of the expected values and seeing how these tests fail.)

Testing the pizzasHandler is straightforward - we have only one GET endpoint. How about we test our ordersHandler, which is a tad more complicated?

Testing the `ordersHandler` #

The ordersHandler is more complicated relative to the pizzaHandler because it contains two endpoints, the POST one creates a new order while the GET one returns the list of all orders. This means that when testing, we will have to cover more cases.

Here’s a test:

func TestOrdersHandler(t *testing.T) {
	tt := []struct {
		name       string
		method     string
		pizzas     *Pizzas
		orders     *Orders
		body       string
		want       string
		statusCode int
	}{
		{
			name:   "with a pizza ID and quantity",
			method: http.MethodPost,
			pizzas: &Pizzas{
				Pizza{
					ID:    1,
					Name:  "Margherita",
					Price: 8,
				},
			},
			orders:     &Orders{},
			body:       `{"pizza_id":1,"quantity":1}`,
			want:       `{"pizza_id":1,"quantity":1,"total":8}`,
			statusCode: http.StatusOK,
		},
		{
			name:   "with a pizza and no quantity",
			method: http.MethodPost,
			pizzas: &Pizzas{
				Pizza{
					ID:    1,
					Name:  "Margherita",
					Price: 8,
				},
			},
			orders:     &Orders{},
			body:       `{"pizza_id":1}`,
			want:       `{"pizza_id":1,"quantity":0,"total":0}`,
			statusCode: http.StatusOK,
		},
		{
			name:       "with no pizzas on menu",
			method:     http.MethodPost,
			pizzas:     &Pizzas{},
			orders:     &Orders{},
			body:       `{"pizza_id":1,"quantity":1}`,
			want:       "Error: No pizzas found",
			statusCode: http.StatusNotFound,
		},
		{
			name:       "with GET method and no orders in memory",
			method:     http.MethodGet,
			pizzas:     &Pizzas{},
			orders:     &Orders{},
			body:       "",
			want:       "[]",
			statusCode: http.StatusOK,
		},
		{
			name:   "with GET method and with orders in memory",
			method: http.MethodGet,
			pizzas: &Pizzas{},
			orders: &Orders{
				Order{
					Quantity: 10,
					PizzaID:  1,
					Total:    100,
				},
			},
			body:       "",
			want:       `[{"pizza_id":1,"quantity":10,"total":100}]`,
			statusCode: http.StatusOK,
		},
		{
			name:       "with bad HTTP method",
			method:     http.MethodDelete,
			pizzas:     &Pizzas{},
			orders:     &Orders{},
			body:       "",
			want:       "Method not allowed",
			statusCode: http.StatusMethodNotAllowed,
		},
	}

	for _, tc := range tt {
		t.Run(tc.name, func(t *testing.T) {
			request := httptest.NewRequest(tc.method, "/orders", strings.NewReader(tc.body))
			responseRecorder := httptest.NewRecorder()

			handler := ordersHandler{tc.pizzas, tc.orders}
			handler.ServeHTTP(responseRecorder, request)

			if responseRecorder.Code != tc.statusCode {
				t.Errorf("Want status '%d', got '%d'", tc.statusCode, responseRecorder.Code)
			}

			if strings.TrimSpace(responseRecorder.Body.String()) != tc.want {
				t.Errorf("Want '%s', got '%s'", tc.want, responseRecorder.Body)
			}
		})
	}
}

The tests are similar to the previous, but here each of the requests that we create using the httptest.NewRequest function also take a body as the third parameter. The body has to be of type io.Reader, which is an interface that wraps the basic Read method.

In cases where you need to wrap a simple string as an io.Reader one function that I’ve found fitting is string.NewReader. As you might notice if you visit its documentation, it is just a plain wrapper around the string as a read-only io.Reader.

In the rest of the test we - once again - create a responseRecorder and then pass it along with the request to the ordersHandler’s ServeHTTP function. Right after we compare the Body of the responseRecorder and the expected body from the test case.

If we run the test, we will see the following output:

$ go test -v -run OrdersHandler
=== RUN   TestOrdersHandler
=== RUN   TestOrdersHandler/with_a_pizza_ID_and_quantity
=== RUN   TestOrdersHandler/with_a_pizza_and_no_quantity
=== RUN   TestOrdersHandler/with_no_pizzas_on_menu
=== RUN   TestOrdersHandler/with_GET_method_and_no_orders_in_memory
=== RUN   TestOrdersHandler/with_GET_method_and_with_orders_in_memory
=== RUN   TestOrdersHandler/with_bad_HTTP_method
--- PASS: TestOrdersHandler (0.00s)
    --- PASS: TestOrdersHandler/with_a_pizza_ID_and_quantity (0.00s)
    --- PASS: TestOrdersHandler/with_a_pizza_and_no_quantity (0.00s)
    --- PASS: TestOrdersHandler/with_no_pizzas_on_menu (0.00s)
    --- PASS: TestOrdersHandler/with_GET_method_and_no_orders_in_memory (0.00s)
    --- PASS: TestOrdersHandler/with_GET_method_and_with_orders_in_memory (0.00s)
    --- PASS: TestOrdersHandler/with_bad_HTTP_method (0.00s)
PASS
ok  	_/Users/Ilija/Documents/go-playground/testing-in-go-http-servers	0.011s

(In case you’re following along, I recommend breaking the tests by changing any of the expected values and seeing how these tests fail.)

As you can see, the testing package in combination with the net/http/httptest package, using the httptest.NewRequest and httptest.Recorder functions provide flexibility and enough power to test any HTTP server.

To undersand all of the possibilities, I recommend checking out the net/http/httptest package documentation.

P.S. You can read more about the jq tool on its homepage.

Ordering some pizzas #

Testing the pizzasHandler #

Testing the ordersHandler #

Testing the `pizzasHandler` #

Testing the `ordersHandler` #