7

Microservices in Go using the Go kit

 3 years ago
source link: https://eltonminetto.dev/en/post/2021-02-06-microservices-gokit/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Microservices in Go using the Go kit

February 16, 2021 · Elton Minetto  | 

In one of the chapters of the book Microservice Patterns: With examples in Java the author mentions the “Microservice chassis” pattern:

Build your microservices using a microservice chassis framework, which handles cross-cutting concerns, such as exception tracking, logging, health checks, externalized configuration, and distributed tracing.

He goes further and gives some examples of frameworks that implement these concepts in Java and Go:

After some research I chose the Go kit as it is one of the most popular, it is being updated at a constant speed and I liked the architecture that it proposes.

Architecture#

Service#

service

Services are where all of the business logic is implemented. In Go kit, services are typically modeled as interfaces, and implementations of those interfaces contain the business logic. Go kit services should strive to abide the Clean Architecture or the Hexagonal Architecture. That is, the business logic should not know of transport-domain concepts: your service shouldn’t know anything about HTTP headers, or gRPC error codes.

Endpoint#

endpoint

An endpoint is like an action/handler on a controller; it’s where safety and antifragile logic lives. If you implement two transports (HTTP and gRPC), you might have two methods of sending requests to the same endpoint.

Transport#

transport

The transport domain is bound to concrete transports like HTTP or gRPC. In a world where microservices may support one or more transports, this is very powerful; you can support a legacy HTTP API and a newer RPC service, all in a single microservice.

Example#

Let’s create an example of a microservice using this architecture. The directory structure looks like this:

example

Service#

The service layer code in this example is very simple:

package user

import (
	"auth/security"
	"context"
	"errors"
)

type Service interface {
	ValidateUser(ctx context.Context, mail, password string) (string, error)
	ValidateToken(ctx context.Context, token string) (string, error)
}

var (
	ErrInvalidUser  = errors.New("Invalid user")
	ErrInvalidToken = errors.New("Invalid token")
)

type service struct{}

func NewService() *service {
	return &service{}
}

func (s *service) ValidateUser(ctx context.Context, email, password string) (string, error) {
	//@TODO create validation rules, using databases or something else
	if email == "[email protected]" && password != "1234567" {
		return "nil", ErrInvalidUser
	}
	token, err := security.NewToken(email)
	if err != nil {
		return "", err
	}
	return token, nil
}

func (s *service) ValidateToken(ctx context.Context, token string) (string, error) {
	t, err := security.ParseToken(token)
	if err != nil {
		return "", ErrInvalidToken
	}
	tData, err := security.GetClaims(t)
	if err != nil {
		return "", ErrInvalidToken
	}
	return tData["email"].(string), nil
}

As the Go kit documentation recommends, the first step is to create an interface for our service, which will be implemented with our business logic. Soon, this decision to create an interface will prove useful when we include logging and monitoring metrics in the application.

Because it only has business rules, the service layer test is also very simple:

package user_test

import (
	"auth/user"
	"context"
	"testing"

	"github.com/stretchr/testify/assert"
)

func TestValidateUser(t *testing.T) {
	service := user.NewService()
	t.Run("invalid user", func(t *testing.T) {
		_, err := service.ValidateUser(context.Background(), "[email protected]", "invalid")
		assert.NotNil(t, err)
		assert.Equal(t, "Invalid user", err.Error())
	})
	t.Run("valid user", func(t *testing.T) {
		token, err := service.ValidateUser(context.Background(), "[email protected]", "1234567")
		assert.Nil(t, err)
		assert.NotEmpty(t, token)
	})
}

Endpoint#

We will now expose our functions to the outside world. In this example the two functions will be able to be accessed externally, so we will create two endpoints. But this is not always true. Depending on the scenario you can expose only a few functions and keep the others accessible only within the service layer.

package user

import (
	"context"

	"github.com/go-kit/kit/endpoint"
)

//definition of endpoint input and output structures 
type validateUserRequest struct {
	Email    string `json:"email"`
	Password string `json:"password"`
}

type validateUserResponse struct {
	Token string `json:"token,omitempty"`
	Err   string `json:"err,omitempty"` // errors don't JSON-marshal, so we use a string
}

//the endpoint will receive a request, convert to the desired
//format, invoke the service and return the response structure 
func makeValidateUserEndpoint(svc Service) endpoint.Endpoint {
	return func(ctx context.Context, request interface{}) (interface{}, error) {
		req := request.(validateUserRequest)
		token, err := svc.ValidateUser(ctx, req.Email, req.Password)
		if err != nil {
			return validateUserResponse{"", err.Error()}, err
		}
		return validateUserResponse{token, ""}, err
	}
}

//definition of endpoint input and output structures 
type validateTokenRequest struct {
	Token string `json:"token"`
}

type validateTokenResponse struct {
	Email string `json:"email,omitempty"`
	Err   string `json:"err,omitempty"`
}

//the endpoint will receive a request, convert to the desired
//format, invoke the service and return the response structure 
func makeValidateTokenEndpoint(svc Service) endpoint.Endpoint {
	return func(ctx context.Context, request interface{}) (interface{}, error) {
		req := request.(validateTokenRequest)
		email, err := svc.ValidateToken(ctx, req.Token)
		if err != nil {
			return validateTokenResponse{"", err.Error()}, err
		}
		return validateTokenResponse{email, ""}, err
	}
}

The role of the endpoint is to receive a request, convert it to the expected struct, invoke the service layer, and return another struct. The endpoint layer does not know anything about the upper layer, because it makes no difference whether the endpoint is being invoked via HTTP, gRPC, or another form of transport.

Because of its simplicity, testing this layer is equally easy to implement:

package user

import (
	"context"
	"testing"
)

func TestMakeValidateUserEndpoint(t *testing.T) {
	s := NewService()
	endpoint := makeValidateUserEndpoint(s)
	t.Run("valid user", func(t *testing.T) {
		req := validateUserRequest{
			Email:    "[email protected]",
			Password: "1234567",
		}
		_, err := endpoint(context.Background(), req)
		if err != nil {
			t.Errorf("expected %v received %v", nil, err)
		}
	})
	t.Run("invalid user", func(t *testing.T) {
		req := validateUserRequest{
			Email:    "[email protected]",
			Password: "123456",
		}
		_, err := endpoint(context.Background(), req)
		if err == nil {
			t.Errorf("expected %v received %v", ErrInvalidUser, err)
		}
	})
}

This test could be improved by replacing the use of the service with a mock that implements the same Service interface, making the tests more efficient.

Transport#

In this layer, we can have several implementations like HTTP, gRPC, AMPQ, NATS, etc. In this example, we are going to expose our endpoints in the form of an HTTP API. So, we will create the file transpor_http.go:

package user

import (
	"context"
	"encoding/json"
	"net/http"

	"github.com/go-kit/kit/log"
	httptransport "github.com/go-kit/kit/transport/http"
	"github.com/gorilla/mux"
)

func NewHttpServer(svc Service, logger log.Logger) *mux.Router {
	//options provided by the Go kit to facilitate error control 
	options := []httptransport.ServerOption{
		httptransport.ServerErrorLogger(logger),
		httptransport.ServerErrorEncoder(encodeErrorResponse),
	}
	//definition of a handler 
	validateUserHandler := httptransport.NewServer(
		makeValidateUserEndpoint(svc), //use the endpoint
		decodeValidateUserRequest, //converts the parameters received via the request body into the struct expected by the endpoint 
		encodeResponse, //converts the struct returned by the endpoint to a json response 
		options...,
	)

	validateTokenHandler := httptransport.NewServer(
		makeValidateTokenEndpoint(svc),
		decodeValidateTokenRequest,
		encodeResponse,
		options...,
	)
	r := mux.NewRouter() //I'm using Gorilla Mux, but it could be any other library, or even the stdlib 
	r.Methods("POST").Path("/v1/auth").Handler(validateUserHandler)
	r.Methods("POST").Path("/v1/validate-token").Handler(validateTokenHandler)
	return r
}

func encodeErrorResponse(_ context.Context, err error, w http.ResponseWriter) {
	if err == nil {
		panic("encodeError with nil error")
	}
	w.Header().Set("Content-Type", "application/json; charset=utf-8")
	w.WriteHeader(codeFrom(err))
	json.NewEncoder(w).Encode(map[string]interface{}{
		"error": err.Error(),
	})
}

func codeFrom(err error) int {
	switch err {
	case ErrInvalidUser:
		return http.StatusNotFound
	case ErrInvalidToken:
		return http.StatusUnauthorized
	default:
		return http.StatusInternalServerError
	}
}

//converts the parameters received via the request body into the struct expected by the endpoint 
func decodeValidateUserRequest(ctx context.Context, r *http.Request) (interface{}, error) {
	var request validateUserRequest
	if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
		return nil, err
	}
	return request, nil
}

//converts the parameters received via the request body into the struct expected by the endpoint 
func decodeValidateTokenRequest(ctx context.Context, r *http.Request) (interface{}, error) {
	var request validateTokenRequest
	if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
		return nil, err
	}
	return request, nil
}

//converts the struct returned by the endpoint to a json response
func encodeResponse(ctx context.Context, w http.ResponseWriter, response interface{}) error {
	return json.NewEncoder(w).Encode(response)
}

The code looks like a series of settings, indicating which endpoint will be used at each API address. I tried to describe the behavior in the code comments. And the test of this layer looked like this:

package user

import (
	"net/http"
	"net/http/httptest"
	"os"
	"strings"
	"testing"

	"github.com/go-kit/kit/log"
)

func TestHTTP(t *testing.T) {
	var logger log.Logger
	logger = log.NewLogfmtLogger(os.Stderr)
	logger = log.With(logger, "listen", "8081", "caller", log.DefaultCaller)
	s := NewService()
	r := NewHttpServer(s, logger)
	srv := httptest.NewServer(r)

	for _, testcase := range []struct {
		method, url, body string
		want              int
	}{
		{"POST", srv.URL + "/v1/auth", `{"email": "[email protected]", "password":"1234567"}`, http.StatusOK},
		{"GET", srv.URL + "/v1/auth", `{"email": "[email protected]", "password":"1234567"}`, http.StatusMethodNotAllowed},
		{"POST", srv.URL + "/v1/auth", `{"email": "[email protected]", "password":"invalid"}`, http.StatusNotFound},
		{"POST", srv.URL + "/v1/validate-token", `{"token": "invalid"}`, http.StatusUnauthorized},
	} {
		req, _ := http.NewRequest(testcase.method, testcase.url, strings.NewReader(testcase.body))
		resp, _ := http.DefaultClient.Do(req)
		if testcase.want != resp.StatusCode {
			t.Errorf("%s %s %s: want %d have %d", testcase.method, testcase.url, testcase.body, testcase.want, resp.StatusCode)
		}

	}
}

Just like testing the endpoint layer, we could improve this test using a mock of the service.

Main#

In the main.go file we are going to use all the layers:

package main

import (
	"auth/user"
	"net/http"
	"os"

	"github.com/go-kit/kit/log"
)

func main() {

	var logger log.Logger
	logger = log.NewLogfmtLogger(os.Stderr)
	logger = log.With(logger, "listen", "8081", "caller", log.DefaultCaller)

	svc := user.NewLoggingMiddleware(logger, user.NewService())
	r := user.NewHttpServer(svc, logger)
	logger.Log("msg", "HTTP", "addr", "8081")
	logger.Log("err", http.ListenAndServe(":8081", r))
}

Here we can see another advantage in having created an interface for our service. The user.NewHttpServer function expects as a first parameter something that implements the Service interface. The user.NewLoggingMiddleware function creates a struct that implements this interface and has our original service inside it. The code for the logging.go file looks like this:

package user

import (
	"context"
	"time"

	"github.com/go-kit/kit/log"
)

func NewLoggingMiddleware(logger log.Logger, next Service) logmw {
	return logmw{logger, next}
}

type logmw struct {
	logger log.Logger
	Service
}

func (mw logmw) ValidateUser(ctx context.Context, email, password string) (token string, err error) {
	defer func(begin time.Time) {
		_ = mw.logger.Log(
			"method", "validateUser",
			"input", email,
			"err", err,
			"took", time.Since(begin),
		)
	}(time.Now())

	token, err = mw.Service.ValidateUser(ctx, email, password)
	return
}

func (mw logmw) ValidateToken(ctx context.Context, token string) (email string, err error) {
	defer func(begin time.Time) {
		_ = mw.logger.Log(
			"method", "validateToken",
			"input", token,
			"err", err,
			"took", time.Since(begin),
		)
	}(time.Now())

	email, err = mw.Service.ValidateToken(ctx, token)
	return
}

It implements all the functions of the interface, adding the functionality of logging each function call, before invoking the code of the real service. The same can be used to implement metrics, limit access to API, etc. In the official tutorial, we have some examples of this.

If our microservice needs to deliver the logic in more formats, such as gRPC or NATS, we would only need to implement these codes in the transport layer indicating which endpoints will be used. This gives a lot of flexibility for the growth of functionalities without increasing complexity.

In this post, I focused more on the architecture provided by the Go kit, but in the official documentation, you can see the other chassis features that it provides as authentication, circuit breaker, log, metrics, rate limit, service discovery, tracing, etc.

I liked the architecture and features it provides and I believe it can be useful to create services in a fast, clean and efficient way.

The codes for this example are in this repository.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK