Amazon API Gateway ingress for AWS Fargate

CloudFormation templates to setup an AWS Fargate task with serverless API Gateway ingress

Nathan Peck

Senior Developer Advocate at AWS

About

AWS Fargate provides serverless capacity to run your container images. Amazon Elastic Container Service launches and orchestrates containers that run in Fargate.

AWS Cloud Map is a cloud resource discovery service. Cloud Map provides a way to lookup a list of your dynamically changing resources, such as containers.

Amazon API Gateway is a serverless ingress for your web traffic. It has no minimum fee or hourly charge. Instead you pay for the API calls you receive and the amount of data transferred out.

In this pattern you will deploy API Gateway as an ingress in front of an AWS Fargate hosted container image. The API Gateway will discover instances of the running container using AWS Cloud Map.

Architecture

The following diagram shows the architecture that will be deployed:

An API Gateway receives inbound traffic from the public internet.
The API Gateway uses AWS Cloud Map to look up the private IP addresses of tasks that are part of an AWS Fargate deployed service.
API Gateway uses a VPC Link to open connections to the private IP addresses inside of the VPC.
The AWS Fargate hosted tasks receive inbound traffic over a connection opened from the API Gateway VPC Link.
The API Gateway proxies the container’s response back to the client on the public internet.

💡 Tip: API Gateway pricing has no minimum fees or upfront commitments. Instead you pay per API call you receive, and for the amount of outgoing data. This makes API Gateway less expensive than Application Load Balancer for many low traffic applications.

On the other hand if your server side application receives a very large number of small API calls from a large number of connected clients, then you may find the Application Load Balancer pattern for AWS Fargate to be more cost efficient. Application Load Balancer has a constant hourly charge that gives it a higher baseline cost, but the ALB can handle a large number of requests at a lower per request added cost.

Dependencies

This pattern uses AWS SAM CLI for deploying CloudFormation stacks on your AWS account. You should follow the appropriate steps for installing SAM CLI.

Define the network

This pattern can be deployed on top of either of the following VPC patterns:

Which one you choose depends on your goals for this deployment. You can choose the low cost VPC to start with and upgrade to the large sized VPC later on if you have additional private services, or private database servers you wish to deploy in the VPC.

Download the vpc.yml file from your chosen pattern, but do not deploy it yet. Deployment will be done later in the process.

Define the cluster

The following AWS CloudFormation template creates a simple Amazon ECS cluster for usage with AWS Fargate. It also creates an AWS Cloud Map namespace for keeping track of tasks in the cluster.

File: cluster.yml Language: yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58


AWSTemplateFormatVersion: '2010-09-09'
Description: Empty ECS cluster that has no EC2 instances. It is designed
             to be used with AWS Fargate serverless capacity
Parameters:
  VpcId:
    Type: String
    Description: The VPC that the service is running inside of

Resources:
  # Cluster that keeps track of container deployments
  ECSCluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterSettings:
        - Name: containerInsights
          Value: enabled

  # This is a role which is used within Fargate to allow the Fargate agent
  # to download images, and upload logs.
  ECSTaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: [ecs-tasks.amazonaws.com]
            Action: ['sts:AssumeRole']
            Condition:
              ArnLike:
                aws:SourceArn: !Sub arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:*
              StringEquals:
                aws:SourceAccount: !Ref AWS::AccountId
      Path: /

      # This role enables basic features of ECS. See reference:
      # https://docs.aws.amazon.com/AmazonECS/latest/developerguide/security-iam-awsmanpol.html#security-iam-awsmanpol-AmazonECSTaskExecutionRolePolicy
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

  # This namespace will keep track of the tasks in the cluster
  ServiceDiscoveryNamespace:
    Type: AWS::ServiceDiscovery::PrivateDnsNamespace
    Properties:
      Name: internal
      Description: Internal, private service discovery namespace
      Vpc: !Ref VpcId

Outputs:
  ClusterName:
    Description: The ECS cluster into which to launch resources
    Value: !Ref ECSCluster
  ECSTaskExecutionRole:
    Description: The role used to start up a task
    Value: !Ref ECSTaskExecutionRole
  ServiceDiscoveryNamespaceId:
    Description: The shared service discovery namespace for all services in the cluster
    Value: !Ref ServiceDiscoveryNamespace

Note the AWS::ServiceDiscovery::PrivateDnsNamespace. This is an AWS Cloud Map powered service discovery namespace that will be used to keep track of the tasks running in AWS Fargate.

Define the service

The following AWS CloudFormation template defines a basic NGINX task that runs in AWS Fargate, orchestrated by Amazon ECS. Amazon ECS also registers the running tasks into AWS Cloud Map for service discovery.

File: service.yml Language: yml

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139


AWSTemplateFormatVersion: '2010-09-09'
Description: An example service that deploys in AWS VPC networking mode
             on AWS Fargate. Service runs with networking in public
             subnets and public IP addresses

Parameters:
  VpcId:
    Type: String
    Description: The VPC that the service is running inside of
  PublicSubnetIds:
    Type: List<AWS::EC2::Subnet::Id>
    Description: List of public subnet ID's to put the load balancer and tasks in
  ClusterName:
    Type: String
    Description: The name of the ECS cluster into which to launch capacity.
  ServiceDiscoveryNamespaceId:
    Type: String
    Description: The ID of a CloudMap namespace into which the service will be registered
  ECSTaskExecutionRole:
    Type: String
    Description: The role used to start up an ECS task
  ServiceName:
    Type: String
    Default: web
    Description: A name for the service
  ImageUrl:
    Type: String
    Default: public.ecr.aws/docker/library/nginx:latest
    Description: The url of a docker image that contains the application process that
                 will handle the traffic for this service
  ContainerCpu:
    Type: Number
    Default: 256
    Description: How much CPU to give the container. 1024 is 1 CPU
  ContainerMemory:
    Type: Number
    Default: 512
    Description: How much memory in megabytes to give the container
  ContainerPort:
    Type: Number
    Default: 80
    Description: What port that the application expects traffic on
  DesiredCount:
    Type: Number
    Default: 2
    Description: How many copies of the service task to run

Resources:

  # The task definition. This is a simple metadata description of what
  # container to run, and what resource requirements it has.
  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: !Ref ServiceName
      Cpu: !Ref ContainerCpu
      Memory: !Ref ContainerMemory
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      ExecutionRoleArn: !Ref ECSTaskExecutionRole
      ContainerDefinitions:
        - Name: !Ref ServiceName
          Cpu: !Ref ContainerCpu
          Memory: !Ref ContainerMemory
          Image: !Ref ImageUrl
          PortMappings:
            - ContainerPort: !Ref ContainerPort
              HostPort: !Ref ContainerPort
          HealthCheck:
            Command:
              - "CMD-SHELL"
              - "curl -f http://localhost/ || exit 1"
            Interval: 5
            Retries: 2
            Timeout: 3
          LogConfiguration:
            LogDriver: 'awslogs'
            Options:
              mode: non-blocking
              max-buffer-size: 25m
              awslogs-group: !Ref LogGroup
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: !Ref ServiceName

  # The service. The service is a resource which allows you to run multiple
  # copies of a type of task, and gather up their logs and metrics, as well
  # as monitor the number of running tasks and replace any that have crashed
  Service:
    Type: AWS::ECS::Service
    Properties:
      ServiceName: !Ref ServiceName
      Cluster: !Ref ClusterName
      LaunchType: FARGATE
      NetworkConfiguration:
        AwsvpcConfiguration:
          AssignPublicIp: ENABLED
          SecurityGroups:
            - !Ref ServiceSecurityGroup
          Subnets: !Ref PublicSubnetIds
      DeploymentConfiguration:
        MaximumPercent: 200
        MinimumHealthyPercent: 75
      DesiredCount: !Ref DesiredCount
      TaskDefinition: !Ref TaskDefinition
      ServiceRegistries:
        - RegistryArn: !GetAtt ServiceDiscoveryService.Arn
          Port: !Ref ContainerPort

  # Security group that limits network access
  # to the task
  ServiceSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for service
      VpcId: !Ref VpcId

  # Keeps track of the list of tasks for the service
  ServiceDiscoveryService:
    Type: AWS::ServiceDiscovery::Service
    Properties:
      Name: !Ref ServiceName
      DnsConfig:
        NamespaceId: !Ref ServiceDiscoveryNamespaceId
        DnsRecords:
          - TTL: 0
            Type: SRV

  # This log group stores the stdout logs from this service's containers
  LogGroup:
    Type: AWS::Logs::LogGroup

Outputs:
  ServiceSecurityGroup:
    Description: The security group of the service
    Value: !Ref ServiceSecurityGroup
  ServiceDiscoveryServiceArn:
    Description: ARN of the CloudMap service
    Value: !GetAtt ServiceDiscoveryService.Arn

Important things to note:

The AWS::ServiceDiscovery::Service must use a DNS record type of SRV. SRV records keep track of both the IP address as well as the port. This is required for API Gateway to be able to locate the task and send it traffic on the right port.
Container image must have the curl command installed in order to evaluate the Container Health Check commands.

Define the API Gateway

The following AWS CloudFormation template defines an API Gateway that can access a VPC. It uses AWS Cloud Map to locate targets to send traffic to.

File: api-gateway.yml Language: yml

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102


AWSTemplateFormatVersion: '2010-09-09'
Description: Deploy an API Gateway ingress for an AWS Fargate hosted service

Parameters:
  VpcId:
    Type: String
    Description: The VPC that the service is running inside of
  PublicSubnetIds:
    Type: List<AWS::EC2::Subnet::Id>
    Description: List of public subnet ID's to put the API gateway link in
  ServiceSecurityGroup:
    Type: String
    Description: The service's security group
  ServiceDiscoveryServiceArn:
    Type: String
    Description: The ARN of the service's Cloud Map service

Resources:

  # The API Gateway itself
  ApiGateway:
    Type: AWS::ApiGatewayV2::Api
    Properties:
      Name: fargate-api-gateway
      ProtocolType: HTTP

  # This allows the AWS managed API Gateway to
  # communicate to resources inside of your own VPC
  ApiGatewayVpcLink:
    Type: AWS::ApiGatewayV2::VpcLink
    Properties:
      Name: fargate-vpc-link
      SecurityGroupIds:
        - !Ref ApiGatewaySecurityGroup
      SubnetIds: !Ref PublicSubnetIds

  # Setup the integration between the API Gateway and the VPC Link
  ApiGatewayVpcLinkIntegration:
    Type: AWS::ApiGatewayV2::Integration
    Properties:
      ApiId: !Ref ApiGateway
      ConnectionId: !Ref ApiGatewayVpcLink
      ConnectionType: VPC_LINK
      IntegrationType: HTTP_PROXY
      IntegrationUri: !Ref ServiceDiscoveryServiceArn
      PayloadFormatVersion: 1.0
      IntegrationMethod: ANY

  # An API gateway stage (production)
  ApiGatewayStage:
    Type: AWS::ApiGatewayV2::Stage
    Properties:
      ApiId: !Ref ApiGateway
      StageName: "$default"
      AutoDeploy: true
      DefaultRouteSettings:
        DetailedMetricsEnabled: true
      AccessLogSettings:
        DestinationArn: !GetAtt LogGroup.Arn
        Format: >-
          {"requestId":"$context.requestId", "ip": "$context.identity.sourceIp",
          "caller":"$context.identity.caller",
          "user":"$context.identity.user","requestTime":"$context.requestTime",
          "routeKey":"$context.routeKey",
          "status":"$context.status"}

  # The route that sends traffic to the integration
  ApiGatewayRoute:
    Type: AWS::ApiGatewayV2::Route
    Properties:
      ApiId: !Ref ApiGateway
      RouteKey: "$default"
      Target: !Sub "integrations/${ApiGatewayVpcLinkIntegration}"

  # This log group stores the access logs from the API Gateway
  LogGroup:
    Type: AWS::Logs::LogGroup

  # This security group is used by the VPC link, so that
  # you can control which resources in the VPC the VPC link is
  # allowed to communicate with.
  ApiGatewaySecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for API gateway
      VpcId: !Ref VpcId

  # Configure the security group of the service to accept
  # inbound traffic originating from the security group of
  # the API gateway's VPC link
  ServiceIngressFromApiGateway:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      Description: Allow API Gateway to communicate to the service
      GroupId: !Ref ServiceSecurityGroup
      IpProtocol: -1
      SourceSecurityGroupId: !Ref ApiGatewaySecurityGroup

Outputs:
  ApiGatewayUri:
    Description: The URI at which you can send traffic to your Fargate service
    Value: !GetAtt ApiGateway.ApiEndpoint

Things to look for in this template:

AWS::ApiGatewayV2::VpcLink and AWS::ApiGatewayV2::Integration - This is how the API Gateway is able to communicate privately to tasks inside of the VPC
ServiceIngressFromApiGateway - The containerized service’s security group must accept inbound traffic from the security group used by the API Gateway’s VPC link.
AWS::ApiGatewayV2::Stage - Note the AccessLogSettings which can be used to store access logs into an AWS CloudWatch log group.

Deploy it all

At this point you should have the following CloudFormation templates:

vpc.yml - Defines the virtual private network that everything will be deployed inside of
cluster.yml - Defines the Amazon ECS cluster and AWS Cloud Map namespace
service.yml - Defines how to deploy the container image using Amazon ECS and AWS Fargate, and register it into AWS Cloud Map.
api-gateway.yml - Defines the API Gateway and how it attaches to the VPC and sends traffic to the container service.

Use the following parent stack to deploy all four templates at once:

File: parent.yml Language: yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52


AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Parent stack that deploys VPC, Amazon ECS cluster for AWS Fargate,
             a serverless Amazon ECS service deployment that hosts
             the task containers on AWS Fargate, and an API Gateway that forwards
             traffic to the deployed containers.

Resources:

  # The networking configuration. This creates an isolated
  # network specific to this particular environment
  VpcStack:
    Type: AWS::Serverless::Application
    Properties:
      Location: vpc.yml

  # This stack contains the Amazon ECS cluster itself
  ClusterStack:
    Type: AWS::Serverless::Application
    Properties:
      Location: cluster.yml
      Parameters:
        VpcId: !GetAtt VpcStack.Outputs.VpcId

  # This stack contains the container deployment
  ServiceStack:
    Type: AWS::Serverless::Application
    Properties:
      Location: service.yml
      Parameters:
        VpcId: !GetAtt VpcStack.Outputs.VpcId
        PublicSubnetIds: !GetAtt VpcStack.Outputs.PublicSubnetIds
        ClusterName: !GetAtt ClusterStack.Outputs.ClusterName
        ECSTaskExecutionRole: !GetAtt ClusterStack.Outputs.ECSTaskExecutionRole
        ServiceDiscoveryNamespaceId: !GetAtt ClusterStack.Outputs.ServiceDiscoveryNamespaceId

  # API Gateway Ingress
  ApiGatewayStack:
    Type: AWS::Serverless::Application
    Properties:
      Location: api-gateway.yml
      Parameters:
        VpcId: !GetAtt VpcStack.Outputs.VpcId
        PublicSubnetIds: !GetAtt VpcStack.Outputs.PublicSubnetIds
        ServiceSecurityGroup: !GetAtt ServiceStack.Outputs.ServiceSecurityGroup
        ServiceDiscoveryServiceArn: !GetAtt ServiceStack.Outputs.ServiceDiscoveryServiceArn

# The public facing URI of the deployed service
Outputs:
  ApiGatewayUri:
    Description: The URI at which you can send traffic to your Fargate service
    Value: !GetAtt ApiGatewayStack.Outputs.ApiGatewayUri

You can use the following command to deploy this parent stack and it’s child stacks:

sam deploy \
  --template-file parent.yml \
  --stack-name api-gateway-fargate \
  --resolve-s3 \
  --capabilities CAPABILITY_IAM

Test it out

After you deploy the templates, you can locate the public facing URI of the API Gateway by visiting the API Gateway console. Or you can get the URI from the outputs of the CloudFormation stack using the following command:

sam list stack-outputs \
  --stack-name api-gateway-fargate

Load this URL up in your web browser and verify that you see a “Welcome to nginx!” page. You can now replace the NGINX container image with your own custom container image that listens for traffic on port 80.

Tear it down

You can tear down the entire stack with the following command:

sam delete --stack-name api-gateway-fargate

Table of Contents

About

Development Tool

Development Tool

Type

Amazon API Gateway ingress for AWS Fargate

About

Architecture

Dependencies

Define the network

Define the cluster

Define the service

Define the API Gateway

Deploy it all

Test it out

Tear it down