Linkerd 1.x + Consul Service Mesh

Linkerd is one of the most popular open source service mesh solutions. Linkerd is designed to handle all aspects of routing your traffic between containers in your cluster. In addition to basic load balancing Linkerd also provides many other powerful features such as circuit breaking, dynamic routing which allows you to setup blue/green deploys, and rich tracing and instrumentation.

When Linkerd 1.x is used with ECS it is deployed as a daemon that runs on each EC2 instance in your cluster. Additionally Consul daemons are deployed to watch for new containerized services that ECS schedules onto each instance, and a Consul server is provisioned which serves as a central store for information about the services that are available for Linkerd to send traffic to.

The architecture looks like this:

linkerd

Each EC2 instance runs consul-registrator which tells Consul about any Docker containers that ECS schedules onto the instance. Additionally each instance runs a linkerd daemon which communicates with Consul via the consul-agent daemon running on the node. When container A wants to talk to container B it uses the linkerd daemon running on its local instance as an HTTP proxy. That local Linkerd daemon accepts the proxy connection and refers to Consul to find an instance of container B on one of the other hosts. Then the linkerd agent sends the request to the chosen instance of container B and returns the response to container A.

Traffic stays inside the VPC, as Linkerd makes the request using the private IP address of the host that has the destination container. This also means that there is very low latency communication directly from the host running container A to the host running container B.

This architecture may look complicated for only two containers, but it scales extremely well as you run many containers per host. The three daemons used in this deployment are very lightweight and because of this you can easily run 10-20 other application containers per host all sharing the same Linkerd daemon to route their requests.

Deploy an EC2 cluster

Use these templates:
Launch an EC2 cluster with Linkerd 1.x support Launch Download
Add the linkerd 1.x, consul-registrator, and consul-agent daemons Launch Download

When deploying the daemon template it will ask for your IP address to authorize access to the Linkerd and Consul admin dashboards. The Consul admin dashboard URL will be available in the CloudFormation outputs. To access the Linkerd dashboard look at the list of EC2 instances that were launched by the cluster template, and point your browser to port 9990 on any of them.

Run example services

Use these templates:
Deploy the sample “helloworld” service. Launch Download
Deploy a load testing service that will send traffic to “helloworld” Launch Download