In May we announced the partnership between Docker and Microsoft to make it easier to deploy containerized applications from the Desktop to the cloud with Azure Container Instances (ACI). We are happy to let you know that all users of Docker Desktop now have the ACI experience available to them by default, allowing them to easily use existing Docker commands to deploy and manage containers running in ACI. 

As part of this I want to also call out a thank you to the MSFT team who have worked with us to make this all happen! That is a big thank you to Mike Morton, Karol Zadora-Przylecki, Brandon Waterloo, MacKenzie Olson, and Paul Yuknewicz.

Getting started with Docker and ACI 

As a new starter, to get going all you will need to do is upgrade your existing Docker Desktop to the latest stable version (2.5.0.0 or later), store your image on Docker Hub so you can deploy it (you can get started with Hub here) and then lastly you will need to create an ACI context to deploy it to. 

We have done a few blog posts now on the different types of things you can achieve with the ACI integration. 

• Getting started locally using Docker & ACI 
• Paul’s blog over at MSFT on getting started with Docker & ACI
• Deploying a Minecraft server & using volumes with ACI 
• Deploying to ACI as part of Github actions for CD 
• Docker & ACI integration into VSCode
• Dockers docs on the integration 

If you have other questions on the experience or would like some other guides then drop us a message in the Compose CLI repo so we can update our docs. 

What’s new in V1.0 

Since the last release of the CLI we have added a few new commands to make it easier to manage your working environment and also make it simpler for you to understand what you can clear up to save you money on resources you are not using.

To start we have add a volume inspect command alongside the volume create to allow you better management of your volumes: 

We are also very excited by our new top level prune command to allow you to better clear up your ACI working environment and manage your costs. 

docker prune --help

We have also added in a couple of interesting flags in here, we have the —dry-run flag to let you see what would be cleared up:

As you can see, this also lets you know the amount of compute resources you will be reclamining as well. At the end of a development session being able to do a force prune allows you to remove ‘all the things you have run’, giving you the confidence you won’t have left something running and get an unexpected bill. 

Lastly we have started to add a few more flags in based on your feedback, a couple of examples of these are the addition of the --format json and --quiet in commands ps, context ls, compose ls, compose ps, volume ls to output json or single IDs.

We are really excited about the new experience we have built with ACI, if you have any feedback on the experience or have ideas for other backends for the Compose CLI please let us know via our Public Roadmap

With Docker focusing on developers, we’ve been doubling down on the parts of Docker that developers love, like Desktop, Hub, and of course Compose. Millions of developers all over the world use Compose to develop their applications and love its simplicity but there was no simple way to get these applications running in the cloud.

Docker is working to make it easier to get code running in the cloud in two ways. First we moved the Compose specification into a community project. This will allow Compose to evolve with the community so that it may better solve more user needs and ensure that it is agnostic of runtime platform. Second, we’ve been working with Amazon and Microsoft on CLI integrations for Amazon ECS and Microsoft ACI that allow you to use docker compose up to deploy Compose applications directly to the cloud.

While implementing these integrations, we wanted to make sure that existing CLI commands were not impacted. We also wanted an architecture that would make it easy to add new backends and provide SDKs in popular languages. We achieved this with the following architecture:

The Node SDK and Compose CLI parts of this diagram are what we have open sourced today. This architecture is not final and we plan to merge the Compose CLI with the existing CLI at a later time.

Depending on the Docker Context that the user selects, the Compose CLI switches which backend is used for the command or API call. This allows us to pass commands which use existing contexts to the existing CLI transparently. The backend interface abstraction allows the implementation of a backend for any container runtime so that users can get the same Docker CLI UX they know and love for it along with the new APIs and SDK.

The Compose CLI can serve a gRPC API to provide similar functionality to that of the CLI commands. We chose to use gRPC as this allows us to generate high quality SDKs in popular languages like Node.js, Python, and Golang. While we currently only provide a Node SDK that supports single container management on ACI, there are plans to add Compose support, extend it to ECS and other backends, and add other language SDKs in the near future. The Node SDK is already used by VS Code to implement its Docker experience on ACI.

This work wouldn’t have been possible without help from our partners at Microsoft and AWS who helped us build the best possible experience for their respective platforms. Our team has enjoyed working with all of you! From Microsoft we’d specifically like to thank Mike Morton, Karol Zadora-Przylecki, Brandon Waterloo, MacKenzie Olson, and Paul Yuknewicz. From AWS we’d like to thank Carmen Puccio, David Killmon, Sravan Rengarajan, Uttara Sridhar, and David Duffey.

These tools are currently in beta so feedback and pull requests are welcome!

• Compose CLI source: https://github.com/docker/compose-cli
• Node SDK source: https://github.com/docker/node-sdk

To get started working with Compose in the Cloud you can download Docker Desktop here, get a free Hub account to deploy your images from here. Once you have you image saved to Docker Hub you will be able to deploy it to either ECS or ACI, to find out more about how to do this:

• Read about how to use the Amazon ECS integration here: https://docs.docker.com/engine/context/ecs-integration/
• Read about how to use the Microsoft ACI integration here: https://docs.docker.com/engine/context/aci-integration/

We are now pleased to announce that the Docker and ACI integration has moved into Docker Desktop stable 2.3.0.5 giving all Desktop users access to the simplest way to get containers running in the cloud. 

Getting started 

As a new starter, to get going all you will need to do is upgrade your existing Docker Desktop to the latest stable version (2.3.0.5), store your image on Docker Hub so you can deploy it (you can get started with Hub here) and then lastly you will need to create an ACI context to deploy it to. For a simple example of getting started with ACI you can see our initial blog post on the edge experience.

More CLI commands

We have added some new features since we first released the Edge experience, one of the biggest changes was the addition of the new docker volume command. We added this in as we wanted to make sure that there was an easy way to get started creating persistent state between spinning up your containers. It is always good to use a dedicated service for a database like Cosmo DB, but while you are getting started volumes are a great way to initial store state. 

This allows you to use existing and create new volumes, to get started with a new volume we can start by selecting our ACI context:

$ docker context use myaci 

Then we can create a volume in a similar way to the Docker CLI, though in this case we have a couple of cloud specific variables we do need to provide:

$ docker volume create --storage-account mystorageaccount --fileshare test-volume

Now I can use this volume either from my CLI:

docker run -v mystorageaccount/test-volume:/target/path myimage

Or from my Compose file:

myservice:
  image: nginx
  volumes:
    - mydata:/mount/testvolumes

volumes:
  mydata:
    driver: azure_file
    driver_opts:
      share_name: test-volume
      storage_account_name: mystorageaccount

Along with this the CLI now supports some of the most popular CLI commands that were previously missing including stop, start & kill.

Try it out

If you are after some ideas of what you can do with the experience you can check out Guilluame’s blog post on running Minecraft in ACI, the whole thing takes about ~15 minutes and is a great example of how simple it is to get containers up and running. 

Microsoft offers $200 of free credits to use in your first 30 days which is a great way to try out the experience, once you have an account you will just need Docker Desktop and a Hub repo with your images saved in and you can start deploying! 

To get started today with the new Azure ACI experience, download Docker Desktop 2.3.0.5 and try out the experience yourself. If you enjoy the experience, have feedback on it or other ideas on what Docker should be working on please reach out to us on our Roadmap.

Recently I wanted to set up a server but I wanted to persist one and as I have given away my last raspberry pi I needed to find a new way to do this. I decided that I would have a go at running this in Azure using the $200 free credits you get in your first month.

The first thing I decided to do was to check out the existing Docker Images for Minecraft servers to see if there were any that looked good to use, to do this I went to Docker Hub and searched for minecraft:

I liked the look of minecraft-server repo, so I clicked through to have a look at the image and link through to the Github repo.

To start I decide to just test out running this locally on my machine with the ‘simple get started’ Docker Run command:

$ docker run -d -p 25565:25565 --name mc -e EULA=TRUE
 itzg/minecraft-server

In the Docker Desktop Dashboard, I can see I have the container running and check the server logs to make sure everything has been initialized properly:

If I load up Minecraft I can connect to my server using local host and my open port: 

From there, I can try to deploy this in Azure to just get my basic server running in the cloud. 

With the Docker ACI integration, I can log into Azure using: 

$ docker login azure

Once logged in, I can create a context that will let me deploy containers to an Azure resource group (this proposes to create a new azure resource group or use an existing one): 

$ docker context create aci acicontext
Using only available subscription : My subscription (xxx)
? Select a resource group  [Use arrows to move, type to filter]
> create a new resource group
  gtardif (westeurope)

I can then use this new context : 

$ docker context use acicontext

I will now try to deploy my minecraft server using the exact same command I ran previously locally :

$ docker run -d -p 25565:25565 --name mc -e EULA=TRUE itzg/minecraft-server
[+] Running 2/2
 ⠿ Group mc  Created                     4.6s
 ⠿ mc        Done                        36.4s
mc

Listing my azure containers, I’ll see the public IP that has been provided for my Minecraft server:

$ docker ps 
CONTAINER ID        IMAGE                   COMMAND             STATUS         PORTS
mc                  itzg/minecraft-server                       Running        51.105.116.56:25565->25565/tcp

However, if I follow the logs of the ACI container, the server seems to be stuck in the initialization, and I cannot connect to it from Minecraft. 

$ docker logs --follow mc

In the logs we see the Minecraft server reserves 1G of memory, which happens to be the default memory allocated to the entire container by ACI ; let’s increase a bit the ACI limit with the –memory option : 

$ docker run -d --memory 1.5G -p 25565:25565 --name mc -e EULA=TRUE
itzg/minecraft-server

The server logs from ACI now show that the server initialized properly. I can run $ docker ps again to get the public IP of my container, and connect to it from Minecraft and start playing ! 

This is great, but now I want to find a way to make sure my data persists and reduce the length of the command I need to use to run the server.

To do this I will use a Compose file to document the command I am using, and next I will add a volume to this that I can mount my data to. 

version: '3.7'
services:
 minecraft:
   image: itzg/minecraft-server
   ports:
     - "25565:25565"
   environment:
     EULA: "TRUE"
   deploy:
     resources:
       limits:
         memory: 1.5G

Looking at our command from before we have moved our image name into the image section, our -p for ports into the ports and added our EULA acceptance into the environment variables. We also ensure the server container has enough memory to start.

The command to start this locally is now much simpler:

$ docker-compose --project-name mc up

And to deploy to ACI, still using the ACI context I created previously: 

$ docker compose --project-name mc2 up 
[+] Running 2/2
 ⠿ Group mc2  Created                                6.7s
 ⠿ minecraft  Done                                   51.7s

Of course with compose, this allows the compose application to include multiple containers (here we only have the “minecraft” one). The containers are visible in the progress display (here the “minecraft” line).
And listing the containers shows the application name and the container name mc2_minecraft : 

$ docker ps
CONTAINER ID                   IMAGE                  COMMAND      STATUS      PORTS
mc                             itzg/minecraft-server               Running     20.50.245.84:25565->25565/tcp
mc2_minecraft                  itzg/minecraft-server               Running     40.74.20.143:25565->25565/tcp

Next we will want to add a volume to include our Minecraft data and where we can load in other maps if we want. To do this I need to know what folder has the Minecraft data in the Docker image, if I go and inspect our running container in the Docker Dashboard I can see that it is the /Data directory:

If I wanted to add this back in my command line I would need to extend my command with something like:

docker run -d -p 25565:25565 --v /path/on/host:/data --name mc -e 
EULA=TRUE itzg/minecraft-server

I can add this under the volumes in my Compose file: 

version: '3.7'
services:
 minecraft:
   image: itzg/minecraft-server
   ports:
     - "25565:25565"
   environment:
     EULA: "TRUE"
   deploy:
     resources:
       limits:
         memory: 1.5G
   volumes:
     - "~/tmp/minecraft_data:/data"

Now when I do a docker compose up and come back to inspect I can see the /data folder in the container is now mounted to my local folder as expected. Navigating to this local folder I can see all Minecraft data.

Now let’s create an Azure File Share and deploy our application to mount /data to the Azure shared persistent folder so we can do the same thing in ACI. 

First I need to create an Azure storage account. We can do this using the Azure “az” command line, or through the Azure portal, I have decided to use the portal : 

I need to specify a name for the storage account, select the resource group to attach to it, then I let the other options default for this example. 

Once the ”minecraftdocker” storage account is created, I’ll create a file share that will hold all Minecraft files: 

I just need to specify a name for this file share and a size quota ; let’s call it “minecraft-volume”:

Then in my compose file, I’ll update the volume specification to point to this Azure File Share:

version: '3.7'
services:
 minecraft:
   image: itzg/minecraft-server
   ports:
     - "25565:25565"
   environment:
     EULA: "TRUE"
   deploy:
     resources:
       limits:
         memory: 1.5G
   volumes:
     - "minecraft:/data"
volumes:
 minecraft:
   driver: azure_file
   driver_opts:
     share_name: minecraftdocker     
     storage_account_name: minecraft-volume      

Note that the syntax for specifying ACI volumes in Compose files is likely to change in the future.

I can then redeploy my compose application to ACI, still with the same command line as before:

$ docker --context acitest compose  --project-name minecraft up
[+] Running 2/2
 ⠿ Group minecraft  Created                                5.3s
 ⠿ minecraft        Done                                  56.4s

And I can check it’s using the Azure File Share, just selecting the minecraft-volume Share in the Azure portal:

Security Note: in this Azure File Share web UI screenshot above, you can see the Minecraft server has created a server.properties with default values, and whitelist.json that contains an empty array.

Server.properties includes the whitelist disabled by default (“white-list = false”), and a default RCON password, that can be used for remote administration of the server (in our current setup, the RCON port has not been exposed so rcon cannot be used. You can expose port 25575 in addition to 25565 in the compose file if you want to use RCON ; don’t forget to change the password in server.properties first.

See https://minecraft.gamepedia.com/Server.properties for more details on Mineraft server properties.

You can download the server.properties file from Azure File Share, modify it, for example change RCON password or set “white-list” property to true and upload the file back to Azure File Share. Same for the whitelist.json file.
To restart the Minecraft server, you can simply re-issue the compose up command:

$ docker --context acitest compose  --project-name minecraft up

It will redeploy the container, but reuse any data that’s in the volume, so it will load the server.properties & whitelist.json files you have uploaded on the Azure File Share.

Trying to connect your Minecraft application if you’re not part of the white list, you will see:

To get started running your own Minecraft server you can download the latest Edge version of Docker Desktop. You can find the Minecraft image we used on Docker Hub, or start creating your own content from Dockers Official images.  Or if you want to create content like this to share, create a Docker account and start sharing your ideas in your public repos. 

At Docker we are making the Developer Experience (DX) more simple. As an extension of that we want to provide the same beloved Docker experience that developers use daily and integrate it with the cloud. Microsoft’s Azure ACI provided an awesome platform to do just that.

In this tutorial, we take a look at running single containers and multiple containers with Compose in Azure ACI. We’ll walk you through setting up your docker context and even simplifying logging into Azure. At the end of this tutorial, you will be able to use familiar Docker commands to deploy your applications into your own Azure ACI account.

Prerequisites

To complete this tutorial, you will need:

• Docker installed on your development machine. You can download and install Docker Desktop Edge version 2.3.3.0 or later from the links below:
  • Docker Desktop for Mac
  • Docker Desktop for Windows
• Docker Hub account. Get your free account here.
• An Azure account. Sign up for free.
• Git installed on your development machine.
• An IDE or text editor to use for editing files. I would recommend VSCode

Run Docker Container on ACI

The integration with Azure ACI is very similar to working with local containers. The development teams have thought very deeply about the developer experience and have tried to make the UX for working with ACI as close as possible to working with local containers.

Let’s run a simple Nginx web server on Azure ACI.

Log into Azure

You do not need to have the Azure CLI installed on your machine to run Docker images in ACI. Docker takes care of everything.

The first thing you need to do is to login to Azure.

$ docker login azure

This will open a browser window which will allow you to login to Azure.

Select your account and login. Once you are logged in, you can close the browser window.

Azure ACI Context

Docker has the concept of a context. You can think of a context as a place where you can run docker containers.It’s a little more complicated than this but this is a good enough description for now. In this tutorial, we use our local context and the new ACI context.

Let’s first take a look at what contexts we currently have on our local development machine. Run the following command to see a list of contexts.

$ docker context list
NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT               KUBERNETES ENDPOINT                                 ORCHESTRATOR

default *           moby                Current DOCKER_HOST based configuration   unix:///var/run/docker.sock   https://kubernetes.docker.internal:6443 (default)   swarm

Depending on if you have already created another context, you should only see one context. This is the default context that points to your local Docker engine labeled as “moby”. You can identify the current context that will be used for docker commands by the “*” beside the name of the active context.

Now let’s create an ACI context that we can run containers with. We’ll use the create aci command to create our context. 

Let’s take a look at the help for creating an aci context.

$ docker context create aci --help
Create a context for Azure Container Instances

Usage:
  docker context create aci CONTEXT [flags]

Flags:
      --description string       Description of the context
  -h, --help                     help for aci
      --location string          Location (default "eastus")
      --resource-group string    Resource group
      --subscription-id string   Location

Global Flags:
      --config DIRECTORY   Location of the client config files DIRECTORY (default "/Users/peter/.docker")
  -c, --context string     context
  -D, --debug              enable debug output in the logs
  -H, --host string        Daemon socket(s) to connect to

Underneath the Flags section of the help, you can see that we have the option to set the location, resource-group, and subscription-id.

You can pass these flags into the create command. If you do not, the docker cli will ask you these questions in interactive mode. Let’s do that now.

$ docker context create aci myaci

The first thing the cli will ask is what subscription you would like to use. If you only have one then docker will use that one.

Using only available subscription : Azure subscription 1 (b3c07e4a-774e-4d8a-b071-xxxxxxxxxxxx)

Now we need to select the resource group we want to use. You can either choose one that has been previously created or choose “create a new resource group”. I’ll choose to create a new one.

Resource group "c3eea3e7-69d3-4b54-83cb-xxxxxxxxxxxx" (eastus) created

Okay, our aci context is set up. Let’s list our contexts.

$ docker context list

You should see the ACI context you just created.

Run Containers on ACI

Now that we have our ACI context set up, we can now run containers in the cloud. There are two ways to tell Docker which context you want your commands to be applied to. 

The first is to pass the –context flag. The other is to tell Docker which context we want to use with all subsequent commands by switching contexts. For now, let’s use the –context flag.

$ docker --context myaci run -d --name web -p 80:80 nginx
[+] Running 2/2
 ⠿ web                         Created                                                                           
 ⠿ single--container--aci      Done                                                                                
web

Here you can see that Docker interacted with ACI and created a container instance named “web” and started a single instance.

Open your Azure portal and navigate to container instances.

We can also run Docker CLI commands that you are already familiar with such as ps and logs.

Switch Contexts

Let’s take a look at our running containers. But before we do that let’s switch our active context to the ACI context we setup above so we do not have to keep typing –context with every command.

$ docker context use myaci

Now let’s run the ps command without passing the –context flag.

$ docker ps
CONTAINER ID        IMAGE               COMMAND             STATUS              
PORTS
web                            nginx                                                              Running             52.224.73.190:80->80/tcp

Nice, since we told Docker to use the myaci context, we see a list of containers running in our Azure account and not on our local machine.

Let’s make sure our container is running. Copy the IP address of the container from the above ps output and paste it into your browser address bar. You can see our Nginx web server running!

Like I mentioned above, we can also take a look at the container’s logs. 

$ docker logs web

To stop and remove the container, run the following command.

$ docker rm web

BOOM!

That was pretty easy and the integration is smooth. With a few docker commands that you are already familiar with and a couple new ones, we were able to run a container in ACI from our development machine pretty quickly and simply.

But we’re not done!

Docker Compose

We can also run multiple containers using Docker Compose. With the ACI integration, we now have the ability to run compose commands from the docker cli against ACI. Let’s do that next.

Fork the Code Repository

I’m using a simple Python Flask application that logs timestamps to a Redis database. Let’s fork the repository and then clone the git repository to your local machine.

Open your favorite browser and navigate to: https://github.com/pmckeetx/timestamper

Click on the “fork” button in the top right corner of the window. This will make a “copy” of the demo repository into your GitHub account.

On your forked version of the repository, click the green “Code” button and copy the github url.

Open up a terminal on your local machine and run the following git command to clone the repository to your local development machine.

Make sure you replace the <<github username>> with your GitHub username.

git clone git@github.com:<<github username>>/timestamper.git

Build and Run Locally

Make sure you are in the root directory for the timestamper project and follow the following steps to build the images and start the application with Docker Compose.

First we need to add your Docker ID to the image in our docker-compose.yml file. Open the docker-compose.yml file in an editor and replace <<username>> with your Docker ID.

Next, we need to make sure we are using the local Docker context.

$ docker context use default

Now we can build and start our application using docker-compose.

$ docker-compose up --build
Building frontend
Step 1/7 : FROM python:3.7-alpine
 ---> 6ca3e0b1ab69
Step 2/7 : WORKDIR /app
...
frontend_1  |  * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
frontend_1  |  * Restarting with stat
frontend_1  |  * Debugger is active!
frontend_1  |  * Debugger PIN: 622-764-646

Docker will build our timestamper image and then run the Redis database and our timestamper containers.

Navigate to http://localhost:5000 and click the Timestamp! button a couple of times.

Compose on ACI

Now let’s run our application on ACI using the new docker compose integration.

We’ll first need to push our image to Docker Hub so ACI can pull the image and run it. Run the following command to push your image to your Docker Hub account.

$ docker-compose push
Pushing frontend (pmckee/timestamper:latest)...
The push refers to repository [docker.io/pmckee/timestamper]
6e899582609b: Pushed
...
50644c29ef5a: Layer already exists
latest: digest: sha256:3ce2607f101a381b36beeb0ca1597cce9925d17a0f826cac0f7e0365386a3042 size: 2201

Now that our image is on Hub, we can use compose to run the application on ACI.

First let’s switch to our ACI context.

$ docker context use myaci

Remember, to see a list of contexts and which is being used, you can run the list contexts command.

$ docker context list

Okay, now that we are using the ACI context, let’s start our application in the cloud.

$ docker compose up
[+] Running 3/3
 ⠿ timestamper     Created
 ⠿ frontend        Done
 ⠿ backend         Done

Let’s verify that our application is up and running. To get the IP address of our frontend, let’s list our running containers.

$ docker ps
CONTAINER ID           IMAGE                COMMAND             STATUS              PORTS

timestamper_frontend       pmckee/timestamper                       
Running             40.71.234.128:5000->5000/tcp

timestamper_backend         redis:alpine                             
Running

Copy the IP address and port listed above and paste into your favorite browser.

Let’s take a look at the logs for our Redis container.

$ docker logs timestamper_backend
1:C 13 Jul 2020 18:21:12.044 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
...
1:M 13 Jul 2020 18:21:12.046 # Server initialized
1:M 13 Jul 2020 18:21:12.047 * Ready to accept connections

Yes, sir! That is a Redis container running in ACI! Pretty cool.

After you play around a bit, you can take down the compose application by running compose down.

$ docker compose down

Conclusion

We saw how simple it is now to run a single container or run multiple containers using Compose on Azure with our ACI integration. If you want to help influence or suggest features, you can do that on our public Roadmap.

If you want to learn more about Compose and all the cool things that are happening around the OpenSource initiative, please checkout Awesome Compose and the OpenSource Compose specification.