Deploy a CloudWatch dashboard for an Amazon ECS service

Nathan Peck profile picture
Nathan Peck
Senior Developer Advocate at AWS

About

Amazon ECS collects telemetry and generates loggable events for your service. This information is displayed in the default Amazon ECS web console views. However, you may wish to generate your own custom CloudWatch dashboard that has the specific metrics you are interested in.

This pattern shows how you can use CloudFormation to define and create a custom dashboard for observing the performance and events of your ECS deployments.

Container Insights

This pattern assumes that you have already enabled Container Insights on your ECS cluster.

TIP

There is no charge for using Amazon ECS, however the Container Insights feature does come with an additional cost based on the amount of data stored in CloudWatch, and an additional cost for querying that data using CloudWatch Log Insights. A task with one container generates about 1 MB of telemetry data per day. If there is more than one container per task, or you have frequent task turnover you may generate even more telemetry data. Queries will also cost more based on the amount of telemetry data processed by the query. See Amazon CloudWatch pricing for more info.

In order to activate Container Insights for a cluster, you can use the command line:

Language: sh
aws ecs update-cluster-settings \
  --cluster cluster_name_or_arn \
  --settings name=containerInsights,value=enabled \
  --region us-east-1

Or you can enable Container Insights when creating an ECS cluster with CloudFormation:

Language: yml
MyCluster:
  Type: AWS::ECS::Cluster
  Properties:
    ClusterName: production
    Configuration:
      containerInsights: enabled

Once Container Insights has been enabled you will start to get high cardinality telemetry data about your tasks, streamed into CloudWatch Logs and CloudWatch Metrics.

Dashboard Template

The following template demonstrates how to setup a custom CloudWatch dashboard for a single ECS service.

File: cloudwatch-dashboard-ecs.ymlLanguage: yml
AWSTemplateFormatVersion: '2010-09-09'
Description: This template deploys an automatically generated CloudWatch
             dashboard for the referenced ECS service.
Transform:
  - AWS::LanguageExtensions

Parameters:
  ServiceArn:
    Type: String
    Description: The ARN of the service that we want to generate the dashboard from.

Resources:
  # A CloudWatch log group for persisting the deployment events
  ServiceEventLog:
    Type: AWS::Logs::LogGroup

  # Create the EventBridge rule that captures deployment events into the CloudWatch log group
  CaptureServiceDeploymentEvents:
    Type: AWS::Events::Rule
    Properties:
      Description:
        Fn::Sub:
          - 'Capture service deployment events from the ECS service ${ServiceName}'
          # Scary but working way to get service's human name from it's ARN
          - ServiceName: !Select [2, !Split ['/', !Select [5, !Split [':', !Ref ServiceArn]]]]
      EventPattern:
        source:
          - aws.ecs
        detail-type:
          - "ECS Deployment State Change"
          - "ECS Service Action"
        resources:
          - !Ref ServiceArn
      # Where to send the events
      Targets:
        - Arn: !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${ServiceEventLog}
          Id: 'CloudWatchLogGroup'

  # Create a log group resource policy that allows EventBridge to put logs into
  # the log group
  LogGroupForEventsPolicy:
    Type: AWS::Logs::ResourcePolicy
    Properties:
      PolicyName: EventBridgeToCWLogsPolicy
      PolicyDocument: !Sub
      - >
        {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Sid": "EventBridgetoCWLogsPolicy",
              "Effect": "Allow",
              "Principal": {
                "Service": [
                  "delivery.logs.amazonaws.com",
                  "events.amazonaws.com"
                ]
              },
              "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
              ],
              "Resource": [
                "${LogArn}"
              ]
            }
          ]
        }
      - { LogArn: !GetAtt ServiceEventLog.Arn, RuleArn: !GetAtt CaptureServiceDeploymentEvents.Arn}

  # This resource creates the widgets that will live in the CloudWatch
  # dashboard, by pulling stats from Container Insights
  ServiceDashboard:
    Type: AWS::CloudWatch::Dashboard
    Properties:
      DashboardName:
        Fn::Sub:
          - "${ServiceName}-dashboard"
          - ServiceName: !Select [2, !Split ['/', !Select [5, !Split [':', !Ref ServiceArn]]]]
      DashboardBody:
        Fn::ToJsonString:
          widgets:
            # A table that shows recent deployment events
            - type: log
              x: 0
              y: 0
              width: 24
              height: 4
              properties:
                region: !Ref AWS::Region
                title: Service Deployments
                query: !Sub "SOURCE '${ServiceEventLog}' | fields @timestamp, detail.eventName, detail.deploymentId | sort @timestamp desc | limit 500"
                view: table

            # A graph showing the running task count over time
            - type: metric
              x: 0
              y: 13
              width: 24
              height: 6
              properties:
                metrics:
                - - "ECS/ContainerInsights"
                  - "RunningTaskCount"
                  - "ServiceName"
                  # Extract service name from service ARN
                  - !Select [2, !Split ['/', !Select [5, !Split [':', !Ref ServiceArn]]]]
                  - "ClusterName"
                  # Extract cluster name from service ARN
                  - !Select [1, !Split ['/', !Select [5, !Split [':', !Ref ServiceArn]]]]
                - [ ".", "DesiredTaskCount", ".", ".", ".", "." ]
                view: timeSeries
                stacked: false
                region: !Ref AWS::Region
                stat: Sum
                period: 60
                yAxis:
                  left:
                    min: 0
                title: "DesiredTaskCount, RunningTaskCount"

            # A graph showing CPU and Memory utilization over time
            - type: metric
              x: 0
              y: 13
              width: 24
              height: 6
              properties:
                metrics:
                - - "ECS/ContainerInsights"
                  - "CpuUtilized"
                  - "ServiceName"
                  # Extract service name from service ARN
                  - !Select [2, !Split ['/', !Select [5, !Split [':', !Ref ServiceArn]]]]
                  - "ClusterName"
                  # Extract cluster name from service ARN
                  - !Select [1, !Split ['/', !Select [5, !Split [':', !Ref ServiceArn]]]]
                - [ ".", "MemoryUtilized", ".", ".", ".", "." ]
                view: timeSeries
                stacked: false
                region: !Ref AWS::Region
                stat: Sum
                period: 60
                yAxis:
                  left:
                    min: 0
                title: "CPU Utilization, Memory Utilization"

This template only requires a single input variable:

  • ServiceArn - The ECS service's ARN (Amazon Resource Name) to track in the dashboard. It should look something like this: arn:aws:ecs:us-west-2:123456789012:service/sample-webapp

You can deploy this template using the AWS CloudFormation console, or using the AWS CLI:

Language: sh
aws cloudformation deploy \
  --template-file cloudwatch-dashboard-ecs.yml \
  --stack-name cloudwatch-dashboard-ecs \
  --capabilities CAPABILITY_IAM \
  --parameter-overrides \
     ServiceArn=arn:aws:ecs:us-east-2:209640446841:service/capacity-provider-environment-BaseStack-18PANC6K9E7D8-ECSCluster-NNBNpIh5AkZO/nginx-on-fargate
capacity-provider-environment-BaseStack-18PANC6K9E7D8-ECSCluster-NNBNpIh5AkZO

See Also