Launch a task with durable storage, using AWS Copilot

About ​

AWS Copilot is the official command line tool for Amazon ECS. It helps you to describe the container application that you would like to deploy. Then Copilot turns your higher level description into a production ready CloudFormation template that it deploys on your behalf.

Amazon Elastic File System provides durable serverless file storage over the network. An Elastic File System can be shared between multiple tasks and applications, and it automatically grows and shrinks as you store additional files or delete files.

In this pattern you will use AWS Copilot to deploy an NGINX web server that runs in AWS Fargate. The web server will serve web content out of a shared filesystem powered by Amazon Elastic File System.

Architecture ​

The following diagram shows the architecture of this pattern:

1. The application deployment consists of two NGINX web server containers that run as tasks in AWS Fargate. Traffic can be sent to the containers using an Application Load Balancer.
2. Amazon Elastic Container Service orchestrates attaching a durable storage volume to both containers, at the path /usr/share/nginx/html.
3. Both containers now see and share the same index.html file. Changes to the file are automatically propagated to both containers.
4. We can use the Amazon ECS Exec feature to open a secure shell to a running container and change the contents of index.html, then see the changes propagate to all tasks.

Dependencies ​

This pattern requires:

• An AWS account, and local credentials for connecting to that account.
• AWS Copilot CLI

Start a new application ​

Kick off a new application in your terminal using the following command:

copilot init

AWS Copilot will now go through a wizard flow to ask you information about what type of application you wish to create. Use the following choices:

Would you like to use one of your existing applications? (Y/n) n

Ok, let's create a new application then.
  What would you like to name your application? [? for help] efs-application

Which workload type best represents your architecture?  [Use arrows to move, type to filter, ? for more help]
    Request-Driven Web Service  (App Runner)
  > Load Balanced Web Service   (Internet to ECS on Fargate)
    Backend Service             (ECS on Fargate)
    Worker Service              (Events to SQS to ECS on Fargate)
    Static Site                 (Internet to CDN to S3 bucket)
    Scheduled Job               (Scheduled event to State Machine to Fargate)

 What do you want to name this service? [? for help] efs-service

We will not be building a local Dockerfile into a container image. Instead we will use a prebuilt image that has been publishing on a public registry.

Which Dockerfile would you like to use for efs-nginx?  [Use arrows to move, type to filter, ? for more help]
    Enter custom path for your Dockerfile
  > Use an existing image instead

We will deploy the public NGINX image from Elastic Container Registry: public.ecr.aws/nginx/nginx:latest

 What's the location ([registry/]repository[:tag|@digest]) of the image to use? [? for help] public.ecr.aws/nginx/nginx:latest

The application expects traffic on port 80.

Which port(s) do you want customer traffic sent to? [? for help] (80)

AWS Copilot will now create supporting infrastructure for publishing your application and write a manifest file. This will take approximately 30 seconds:

- Creating the infrastructure for stack efs-application-infrastructure-roles                    [create complete]  [33.6s]
  - A StackSet admin role assumed by CloudFormation to manage regional stacks                   [create complete]  [14.6s]
  - An IAM role assumed by the admin role to create ECR repositories, KMS keys, and S3 buckets  [create complete]  [14.7s]
✔ The directory copilot will hold service manifests for application efs-application.

✔ Wrote the manifest for service efs-service at copilot/efs-service/manifest.yml
Your manifest contains configurations like your container size and port.

Last but not least AWS Copilot will ask if you are ready to deploy your application to a test environment.

Would you like to deploy a test environment? [? for help] (y/N)

Don't enter yes or no quite yet. Let's make a few changes to the manifest first before deploying.

Configuring the Elastic File System ​

The application manifest is not yet deploying an Elastic File System.

Open up the file copilot/efs-service/manifest.yml. We need to modify this file to add an Elastic File System.

Use the following manifest.yml file as a template:

# The manifest for the "efs-service" service.
# Read the full specification for the "Load Balanced Web Service" type at:
#  https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/

# Your service name will be used in naming your resources like log groups, ECS services, etc.
name: efs-service
type: Load Balanced Web Service

# Distribute traffic to your service.
http:
  path: '/'
  healthcheck:
    success_codes: '200,403'

# Configuration for your containers and service.
image:
  location: public.ecr.aws/nginx/nginx:latest
  # Port exposed through your container to route traffic to it.
  port: 80

cpu: 256       # Number of CPU units for the task.
memory: 512    # Amount of memory in MiB used by the task.
count: 2       # Number of tasks that should be running in your service.
exec: true     # Enable running commands in your container.
network:
  connect: true # Enable Service Connect for intra-environment traffic between services.

storage:
  volumes:
    web-content:
      efs: true
      path: /usr/share/nginx/html
      read_only: false

Important things to note:

• count: 2 - This tells Copilot to deploy two copies of the container. This will allow us to ensure that both tasks are seeing the same shared filesystem.
• exec: true - This tells Copilot to turn on the Amazon ECS exec feature, so that you can easily open a shell inside of a running container
• The storage section tells Copilot to create an Elastic File System and mount it to the path /usr/share/nginx/html inside each container

Now deploy the changes by going back to the terminal running AWS Copilot and entering y to deploy the test environment.

Hydrate volume ​

Once the application deploys you can open it's URL in your browser, but all you will see is a 403 Forbidden error. This is because the EFS filesystem starts out empty. Let's fix that.

Use the following command to open a shell to a task from the service:

copilot svc exec

Now you can create a file for the web server to respond with using the following commands:

cd /usr/share/nginx/html
echo "Hello world" > index.html

Now you can refresh the URL of the service and see that both NGINX web servers are serving the same index.html. You can also test out changing the file contents to see the changes propagate to both tasks.

Tear it down ​

When you are done experimenting tear down the stack with the following commands:

copilot app delete

See Also ​

If you'd like to explore other paths check out the following:

• CloudFormation: A sample YAML template that deploys this same Elastic File System infrastructure
• AWS Cloud Development Kit: A TypeScript app that uses the Cloud Development Kit SDK to deploy this same infrastructure