Distributed System Patterns in Golang

I’ve been having a lot of fun learning Golang. The language is incredibly simple but also comes with some powerful language features.

Let’s take a look at some common distributed system patterns and try implementing them in Go!

The “Token Ring” Pattern

Token ring is pretty simple. Pass a message along between nodes in a circle.

We can use Golang’s go routines to create the nodes. Each node has an inbox and an outbox to pass the message along to the next node.

package main

import (
	"fmt"
	"time"
)

func createNode(id int, in chan string, out chan string) {
	for {
		msg := <-in
		fmt.Printf("TOKEN_RING[%d]: %s\n", id, msg)
		time.Sleep(500 * time.Millisecond)
		out <- msg
	}
}

func main() {
	numWorkers := 10

	ringChannels := make([]chan string, numWorkers)
	for i := range ringChannels {
		ringChannels[i] = make(chan string)
	}

	for i := range ringChannels {
		nextChannelIdx := (i + 1) % len(ringChannels)
		go createNode(i, ringChannels[i], ringChannels[nextChannelIdx])
	}

	ringChannels[0] <- "Hello Bob, Pass the Message Along!"

	select {}
}

Pretty simple, right? select {} is used at the end so the main function doesn’t immediately exit.

TOKEN_RING[0]: Hello Bob, Pass the Message Along!
TOKEN_RING[1]: Hello Bob, Pass the Message Along!
TOKEN_RING[2]: Hello Bob, Pass the Message Along!
TOKEN_RING[3]: Hello Bob, Pass the Message Along!
TOKEN_RING[4]: Hello Bob, Pass the Message Along!
TOKEN_RING[5]: Hello Bob, Pass the Message Along!
TOKEN_RING[6]: Hello Bob, Pass the Message Along!
TOKEN_RING[7]: Hello Bob, Pass the Message Along!
TOKEN_RING[8]: Hello Bob, Pass the Message Along!

Producer & Consumer

Similar to its close cousin “Pub/Sub” this pattern is also used for scaling systems. To keep things simple, imagine we have a video uploading app where multiple consumers will pick up video to process.

This is basically “Background Jobs” in many frameworks but let’s build it from scratch because we’re crazy!

package main

import (
	"fmt"
	"time"
)

type BackgroundJob struct {
	name string
}

func producer(ch chan BackgroundJob) {
	for {
		time.Sleep(time.Second)
		ch <- BackgroundJob{"Video"}
	}
}

func consumer(id int, ch chan BackgroundJob) {
	for {
		msg := <-ch
		fmt.Printf("[Worker %d] Processing %s\n", id, msg)
	}
}

func main() {
	queue := make(chan BackgroundJob)
	go producer(queue)

	for i := range 10 {
		go consumer(i, queue)
	}

	select {}
}

go run main.go
[Worker 9] Processing {Video}
[Worker 0] Processing {Video}
[Worker 1] Processing {Video}
[Worker 2] Processing {Video}
[Worker 3] Processing {Video}
[Worker 4] Processing {Video}
[Worker 5] Processing {Video}