Mastering Blue/Green Deployments in AWS ECS: A Developer’s Guide

Blue/green deployment is a powerful technique for minimizing downtime and risk during application updates. AWS Elastic Container Service (ECS) offers native support for this deployment strategy through its built-in blue/green deployment type. I’ll walk you through implementing blue/green deployments in AWS ECS, with special attention to a common configuration pitfall that many developers run into.

Understanding Blue/Green Deployments in ECS

Blue/green deployment is a release methodology where you maintain two identical production environments:

• Blue environment: Your currently active production environment
• Green environment: The new version of your application that you want to deploy

The main advantage here is that you can test the green environment before routing any production traffic to it. Once you’re confident in the new version, you can gradually shift traffic from blue to green, and if anything goes wrong, you can quickly revert back to the blue environment.

How Native Blue/Green Deployments Work in AWS ECS

AWS ECS implements blue/green deployments natively through its deployment configuration. When you set up a blue/green deployment:

1. ECS creates a new task set (green) alongside your existing task set (blue)
2. The new tasks are registered with a target group
3. ECS shifts traffic from the blue target group to the green target group according to your deployment configuration
4. After successful deployment, the old task set is terminated

For more details, check out the AWS documentation on blue/green deployments.

The Target Group Configuration Requirement

One critical aspect of blue/green deployments in ECS that often trips people up is the configuration of target groups with listener rules. If you don’t specify a TestListenerRule in the loadbalancers section of your ECS service, you must ensure that both blue and green target groups are associated with the same listener rule.

If you miss this step, you’ll run into this error:

“Service deployment rolled back because of invalid networking configuration. Both targetGroup and alternateTargetGroup must be associated with the productionListenerRule or testListenerRule. Please check your configuration and make the necessary adjustments”

The Solution: Multiple Target Groups in a Listener Rule

The solution is straightforward - add multiple target groups to your listener rule, giving the blue target group a weight of 100 and the green target group a weight of 0. This tells ECS that both target groups are associated with the listener rule, but initially, all traffic should go to the blue target group.

During deployment, ECS will automatically adjust these weights to shift traffic from blue to green according to your deployment configuration.

Example: Blue and Green Target Groups

Here’s how you might define blue and green target groups for HTTP traffic:

# Blue target group for HTTP traffic
BlueTargetGroupHttp:
  Type: AWS::ElasticLoadBalancingV2::TargetGroup
  Properties:
    HealthCheckIntervalSeconds: 30
    HealthCheckPath: /health
    HealthCheckProtocol: HTTP
    HealthCheckTimeoutSeconds: 5
    HealthyThresholdCount: 2
    TargetType: ip
    Name: !Sub ${ProjectName}-${EnvironmentName}-blue-http
    Port: !Ref 'ContainerPort'
    Protocol: HTTP
    UnhealthyThresholdCount: 2
    VpcId:
      Fn::ImportValue: !Sub ${ProjectName}:${EnvironmentName}:VpcId

# Green target group for HTTP traffic
GreenTargetGroupHttp:
  Type: AWS::ElasticLoadBalancingV2::TargetGroup
  Properties:
    HealthCheckIntervalSeconds: 30
    HealthCheckPath: /health
    HealthCheckProtocol: HTTP
    HealthCheckTimeoutSeconds: 5
    HealthyThresholdCount: 2
    TargetType: ip
    Name: !Sub ${ProjectName}-${EnvironmentName}-green-http
    Port: !Ref 'ContainerPort'
    Protocol: HTTP
    UnhealthyThresholdCount: 2
    VpcId:
      Fn::ImportValue: !Sub ${ProjectName}:${EnvironmentName}:VpcId

Example: Load Balancer Rule with Multiple Target Groups

Here’s how to configure a listener rule with multiple target groups for blue/green deployment:

LoadBalancerHttpRule:
  Type: AWS::ElasticLoadBalancingV2::ListenerRule
  Properties:
    Actions:
      - Type: forward
        ForwardConfig:
          TargetGroups:
            - TargetGroupArn: !Ref BlueTargetGroupHttp
              Weight: 100
            - TargetGroupArn: !Ref GreenTargetGroupHttp
              Weight: 0
    ListenerArn:
      Fn::ImportValue: !Sub ${ProjectName}:${EnvironmentName}:PublicHttpListener

In this setup:

• The BlueTargetGroupHttp has a weight of 100, so it initially gets 100% of the traffic
• The GreenTargetGroupHttp has a weight of 0, so it initially gets no traffic
• During deployment, ECS automatically adjusts these weights to shift traffic from blue to green

Configuring the ECS Service for Blue/Green Deployment

To set up blue/green deployment in your ECS service, you need to configure the deployment controller and strategy:

Service:
  Type: AWS::ECS::Service
  Properties:
    # Other service properties...
    DeploymentController:
      Type: ECS
    DeploymentConfiguration:
      Strategy: BLUE_GREEN
      BakeTimeInMinutes: 5
    LoadBalancers:
      - ContainerName: !Ref 'ServiceName'
        ContainerPort: !Ref 'ContainerPort'
        TargetGroupArn: !Ref 'BlueTargetGroupHttp'
        AdvancedConfiguration:
          AlternateTargetGroupArn: !Ref 'GreenTargetGroupHttp'
          ProductionListenerRule: !Ref 'LoadBalancerHttpRule'

The BakeTimeInMinutes parameter specifies how long ECS should wait after the green deployment is complete before terminating the blue deployment. This gives you time to verify that the new deployment is working correctly before fully committing to it.

Practical Tips for Developers

When working with blue/green deployments in ECS, here are some things I’ve learned the hard way:

1. Health checks matter: Make sure your application has solid health check endpoints that actually reflect whether it’s ready to serve traffic.

2. Keep an eye on both environments: During the transition, watch both blue and green environments to catch any issues early.

3. Test your rollback process: Before relying on blue/green deployments in production, make sure you can successfully roll back to the blue environment if needed.

4. Think about connection draining: If your application maintains long-lived connections, configure appropriate connection draining settings to avoid disrupting active sessions.

Conclusion

Blue/green deployments in AWS ECS give you a robust way to update your applications with minimal risk and downtime. The key thing to remember is that you need to associate both blue and green target groups with your listener rule to avoid configuration errors.

By setting up your listener rule with multiple target groups (blue at weight 100, green at weight 0), ECS can handle the traffic shift during deployment automatically.