PROGRAMMING JOURNEY

Building a microservice architecture

A hands-on introduction to microservices in Java

Loose coupling has been a programmer goal for decades, and its latest evolution is called microservices. With microservices each services’ code is decoupled so not only can they be build independently, but also deployed independently. This requires some fancy technology tricks and quite a big mental shift.

When exploring topics like this I always found it immensely helpful to do so through a practical exercise. Not having to understand the theory, but simply following a step by step guide, while the concepts seep into my brain. Therefore I have designed this programming journey. I do not explain any of the theory behind microservices. Starting from scratch this post teaches how to setup services, and through exercises we learn how the services communicate and how the system grows.

The technologies we use are:

• Heroku — to avoid having to learn Docker and Kubernetes.
• RabbitMQ — in my experience the easiest message queue to get started with. It’s exchange functionality allows us to quickly build a sophisticated architecture.

Note: To use RabbitMQ in Heroku you need to input a credit card in Heroku. You will not be charged for anything described in this blog post.

Rapid and rivers

Following a talk by Fred George I call this the rapid-river architecture. The idea is that we have one big central message queue, called the Rapid. All messages that enter the system go through the rapid. Branching off the rapid we have several smaller message queues, called rivers.

The fundamental property of this architecture is: services on different rivers can see the same message, but only one service in a specific river sees each message. Say we have a rapid, and two rivers A and B. There is one service on B called b and there are two services on the river A, let’s call them a1 and a2. If they all listen for the same event m then a1 and a2 will take turns getting them, but b will always get it.

Within our services we can define which event to listen for, and on which river. Think of the river as a way to group services that do the same thing. If we have two algorithms for path finding we could have them in two services on the same river.

Setting up the Rapid

Locally

Install RabbitMQ locally: https://www.rabbitmq.com/download.html

On Heroku

Create a directory:

mkdir rapid
cd rapid
git init

Create an environment on Heroku to host the Rapid:

heroku create

Create the Rapid:

heroku addons:create cloudamqp:lemur

Get the configuration variables for the Rapid:

heroku config

Copy the CLOUDAMQP_APIKEY and the CLOUDAMQP_URL and paste them somewhere.

How to set up a new service?

Get the base from GitHub:

git clone https://github.com/thedrlambda/java-service-base.git

Rename it to the service name:

mv java-service-base [service name]

How to run it locally?

To build the project use:

mvn clean install

Then run the app with either:

java -jar target/java-getting-started-1.0.jar

Or:

heroku local web

Use ctrl+c to exit it.

How to deploy it to Heroku?

Create an environment on heroku:

heroku create

Redirect git push to point to heroku:

git remote remove origin
git remote rename heroku origin

Add the config values for the Rapid:

heroku config:set CLOUDAMQP_APIKEY=XXXX
heroku config:set CLOUDAMQP_URL=XXXX

First deployment to Heroku:

git push -u origin master

Subsequent deployments to Heroku:

git push

To view console output from app running on Heroku use:

heroku logs --tail

The journey

In the beginning just hard code values everywhere and focus on getting the correct message flow. Then focus on deploying everything to Heroku. Then add actual Postgres databases. Then add a front-end. Then add actual authentication and user management. Then make your own Amazon.

Whenever you make changes see how many of the services you have keep running.

Product service

A “fetch-product”, or “fetch-product-page” event contains a session id, a user id, and a product id, separated by commas.

When the product service receives a “fetch-product”, or a “fetch-product-page” event it should send a “display” event containing the session id, the string “product”, the product id, and a product name.

Gateway service

When the gateway service receives a “display” event it prints the body.

Initially (for testing) it sends a “fetch-product-page” event. Running it should cause it to display something like:

SESSION_ID,product,PRODUCT_ID,Coffee

User service

When the user service receives a “fetch-product-page” event, it sends a “fetch-stock” event with the session id, the user’s preferred location, and the product id.

Inventory service

When the inventory service receives a “fetch-stock” event, it should post a “display” event containing the session id, the string “stock”, the product id, and the stock of the product in the location.

Running the gateway now should cause it to display something like:

SESSION_ID,product,PRODUCT_ID,Coffee
SESSION_ID,stock,PRODUCT_ID,5

Recommendation service

When the recommendation service receives a “fetch-product-page” event it should:

1. Store the product id with this users item from a cache table.
2. Update the cache table with the new product id for this user id.
3. Lookup the three most commonly linked product ids to this product id.
4. Send a “display” event with the session id, the string “recommendation”, and the three product ids.
5. Send three “fetch-product”, one for each product id.

Running the gateway now should cause it to display something like:

SESSION_ID,product,PRODUCT_ID,Coffee
SESSION_ID,stock,PRODUCT_ID,5
SESSION_ID,recommendation,PRODUCT_ID1,PRODUCT_ID2,PRODUCT_ID3
SESSION_ID,product,PRODUCT_ID1,Coffee
SESSION_ID,product,PRODUCT_ID2,Coffee
SESSION_ID,product,PRODUCT_ID3,Coffee

Congratulations

You have now built a microservice system! You have seen how we can make changes to different parts of the system without stopping or even effecting the other running services. You have seen how to add a new service that hooks into the same event stream as another line of services uses. Finally you have used what I call a push based architecture, where instead of pulling information to us, we push it to the next service.

If you want to learn more about push based architecture I discuss it in my refactoring book, along with other ways to loosen coupling in code: