By using cloud platforms, we can take advantage of different resource configurations and compute capacities. However, deploying containerized applications on cloud platforms is proving to be quite challenging, especially for new users who have no expertise on how to use that platform. As each platform may provide specific APIs, orchestrating the deployment of a containerized application can become a hassle.
Docker Compose is a very popular tool used to manage containerized applications deployed on Docker hosts. Its popularity is maybe due to the simplicity on how to define an application and its components in a Compose file and the compact commands to manage its deployment.
Since cloud platforms for containers have emerged, being able to deploy a Compose application on them is a most-wanted feature by many developers that use Docker Compose for their local development.
In this blog post, we discuss how to use Docker Compose to deploy containerized applications to Amazon ECS. We aim to show how the transition from deploying to a local Docker environment to deploying to Amazon ECS is effortless, the application being managed in the same way for both environments.
Requirements
In order to exercise the examples in this blogpost, the following tools need to be installed locally:
- Windows and MacOS: install Docker Desktop
- Linux: install Docker Engine and Compose CLI
- To deploy to Amazon ECS: an AWS account
For deploying a Compose file to Amazon ECS, we rely on the new Docker Compose implementation embedded into the Docker CLI binary. Therefore, we are going to run docker compose commands instead of docker-compose. For local deployments, both implementations of Docker Compose should work. If you find a missing feature that you use, report it on the issue tracker.
Throughout this blogpost, we discuss how to:
- Build and ship a Compose Application. We exercise how to run an application defined in a Compose file locally and how to build and ship its images to Docker Hub to make them accessible from anywhere.
- Create an ECS context to target Amazon ECS.
- Run the Compose application on Amazon ECS.
- Build and Ship a Compose application
Let us take an example application with the following structure:
$ tree myproject/ |
The content of the files can be found here. The Compose file define only 2 services as follows:
$ cat compose.yaml |
Deploying this file locally on a Docker engine is quite straightforward:
$ docker compose up -d |
Check the application is running locally:
$ docker ps |
Query the frontend:
$ curl localhost:80 |
To remove the application:
$ docker compose down |
In order to deploy this application on ECS, we need to have the images for the application frontend and backend stored in a public image registry such as Docker Hub. This enables the images to be pulled from anywhere.
To upload the images to Docker Hub, we can set the image names in the compose file as follows:
$ cat compose.yamlservices: |
Build the images with Docker Compose:
$ docker compose build |
In the build output we can notice the image has been named and tagged according to the image field from the Compose file.
Before pushing the images to Docker Hub, check to be logged in:
$ docker login |
Push the images:
$ docker compose push |
The images should be stored now in Docker Hub.
- Create an ECS Docker Context
To make Docker Compose target the Amazon ECS platform, we need first to create a Docker context of the ECS type. A docker context is a mechanism that allows redirecting commands to different Docker hosts or cloud platforms.
We assume at this point that we have AWS credentials set up in the local environment for authenticating with the ECS platform.
To create an ECS context run the following command:
$ docker context create ecs myecscontext |
Based on the familiarity with the AWS credentials setup and the AWS tools use, we are prompted to choose between 3 context setups. To skip the details of AWS credential setup, we choose the option of using environment variables.
$ docker context create ecs myecscontext |
This requires to have the AWS_ACCESS_KEY and AWS_SECRET_KEY set in the local environment when running Docker commands that target Amazon ECS.
The current context in use is marked by * in the output of context listing:
$ docker context lsNAME TYPE DESCRIPTION DOCKER ENDPOINT default * moby Current DOCKER_HOST based configuration unix:///var/run/docker.sockmyecscontext ecs credentials read from environment |
To make all subsequent commands target Amazon ECS, make the newly created ECS context the one in use by running:
$ docker context use myecscontext |
- Run the Compose application on Amazon ECS
An alternative to having it as the context in use is to set the context flag for all commands targeting ECS.
WARNING: Check in advance the cost that the ECS deployment may incur for 2 ECS services, load balancing (ALB), cloud map (DNS resolution) etc.
For the following commands, we keep ECS context as the current context in use. Before running commands on ECS, make sure the Amazon account credentials grant access to manage resources for the application as detailed in the documentation.
We can now run a command to check we can successfully access ECS.
$ AWS_ACCESS_KEY="*****" AWS_SECRET_KEY="******" docker compose ls |
Export the AWS credentials to avoid setting them for every command.
$ export AWS_ACCESS_KEY="*****" |
The deploy the sample application to ECS, we can run the same command as in the local deployment:
$ docker compose up |
Docker Compose converts the Compose file to a CloudFormation template defining a set of AWS resources. Details on the resource mapping can be found in the documentation. To review the CloudFormation template generated, we can run the command:
$ docker compose convert |
To check the state of the services, we can run the command:
$ docker compose ps |
Similarly to the local run, we can query the frontend of the application:
$ curl mypro-LoadB-1ROWIHLNOG5RZ-1172432386.eu-west-3.elb.amazonaws.com:80 |
We can retrieve logs from the ECS containers by running the compose logs command:
$ docker compose logs |
To terminate the Compose application and release AWS resources, run:
$ docker compose down |
The Docker documentation provides several examples of Compose files, supported features, details on how to deploy and how to update a Compose application running in ECS, etc.
The following features are discussed in detail:
- use of private images
- service discovery
- volumes and secrets definition
- AWS-specific service properties for auto-scaling, IAM roles and load balancing
- use of existing AWS resources
Summary
We have covered the transition from local deployment of a Compose application to the deployment on Amazon ECS. We have used a minimal generic example for demonstrating how to use the Docker Compose cloud-capability. For a better understanding on how to update the Compose file and use specific AWS features, the documentation provides much more details.
Resources:
- Docker Compose embedded in the Docker CLI
- Compose to ECS support
- ECS-specific Compose examples:
- Deploying Docker containers to ECS:
- Sample used to demonstrate Compose commands:
Feedback
0 thoughts on "Docker Compose: From Local to Amazon ECS"