How Perry Street Software Implemented Resilient Deployment Strategies with Amazon ECS

This post was coauthored by Ben Duffield and Eric Silverberg at Perry Street Software, with contributions from Adam Tucker, Piotr Wald, and Cristian Constantinescu of PSS

Introduction

You just finished deploying that important change you spent weeks preparing, when you see this email subject in your inbox: Alarm: HTTPCode_Target_5XX_Count.

Ugh.

The code you have just deployed is causing a critical error in your application server. As dev ops professionals, we are told to just cut back over to the original code, and indeed this very sensible advice. But buried in that advice are two discrete, time-consuming steps: first, removing malfunctioning code from an Application Load Balancer, and second, adding previously functioning code back into the Application Load Balancer. Both steps take precious seconds or minutes, during which your customers are unable to use your application, and, in the worst scenarios, data is being lost or corrupted.

If you think about it, code deployments are like …