One of the biggest challenges of developing and maintaining cloud-native applications at the enterprise level is having a consistent experience through the entire development lifecycle. This process is even harder for remote companies with distributed teams working on different platforms, with different setups, and asynchronous communication. 

At Kinsta, we have projects of all sizes for application hosting, database hosting, and managed WordPress hosting. We need to provide a consistent, reliable, and scalable solution that allows:

• Developers and quality assurance teams, regardless of their operating systems, to create a straightforward and minimal setup for developing and testing features.
• DevOps, SysOps, and Infrastructure teams to configure and maintain staging and production environments.

Overcoming the challenge of developing cloud-native applications on a distributed team

At Kinsta, we rely heavily on Docker for this consistent experience at every step, from development to production. In this article, we’ll walk you through:

• How to leverage Docker Desktop to increase developers’ productivity.
• How we build Docker images and push them to Google Container Registry via CI pipelines with CircleCI and GitHub Actions.
• How we use CD pipelines to promote incremental changes to production using Docker images, Google Kubernetes Engine, and Cloud Deploy.
• How the QA team seamlessly uses prebuilt Docker images in different environments.

Using Docker Desktop to improve the developer experience

Running an application locally requires developers to meticulously prepare the environment, install all the dependencies, set up servers and services, and make sure they are properly configured. When you run multiple applications, this approach can be cumbersome, especially when it comes to complex projects with multiple dependencies. And, when you introduce multiple contributors with multiple operating systems, chaos is installed. To prevent this, we use Docker.

With Docker, you can declare the environment configurations, install the dependencies, and build images with everything where it should be. Anyone, anywhere, with any OS can use the same images and have exactly the same experience as anyone else.

Declare your configuration with Docker Compose

To get started, you need to create a Docker Compose file, docker-compose.yml. This is a declarative configuration file written in YAML format that tells Docker your application’s desired state. Docker uses this information to set up the environment for your application.

Docker Compose files come in handy when you have more than one container running and there are dependencies between containers.

To create your docker-compose.yml file:

1. Start by choosing an image as the base for our application. Search on Docker Hub to find a Docker image that already contains your app’s dependencies. Make sure to use a specific image tag to avoid errors. Using the latest tag can cause unforeseen errors in your application. You can use multiple base images for multiple dependencies — for example, one for PostgreSQL and one for Redis.
2. Use volumes to persist data on your host if you need to. Persisting data on the host machine helps you avoid losing data if Docker containers are deleted or if you have to recreate them.
3. Use networks to isolate your setup to avoid network conflicts with the host and other containers. It also helps your containers to find and communicate with each other easily.

Bringing it all together, we have a docker-compose.yml that looks like this:

version: '3.8'

services:
  db:
    image: postgres:14.7-alpine3.17
    hostname: mk_db
    restart: on-failure
    ports:
      - ${DB_PORT:-5432}:5432
    volumes:
      - db_data:/var/lib/postgresql/data
    environment:
      POSTGRES_USER: ${DB_USER:-user}
      POSTGRES_PASSWORD: ${DB_PASSWORD:-password}
      POSTGRES_DB: ${DB_NAME:-main}
    networks:
      - mk_network
  redis:
    image: redis:6.2.11-alpine3.17
    hostname: mk_redis
    restart: on-failure
    ports:
      - ${REDIS_PORT:-6379}:6379
    networks:
      - mk_network
      
volumes:
  db_data:

networks:
  mk_network:
    name: mk_network

Containerize the application

Build a Docker image for your application

To begin, we need to build a Docker image using a Dockerfile, and then call that from docker-compose.yml.

Follow these five steps to create your Dockerfile file:

1. Start by choosing an image as a base. Use the smallest base image that works for the app. Usually, alpine images are minimal with nearly zero extra packages installed. You can start with an alpine image and build on top of that:

   docker
   FROM node:18.15.0-alpine3.17

2. Sometimes you need to use a specific CPU architecture to avoid conflicts. For example, suppose that you use an arm64-based processor but you need to build an amd64 image. You can do that by specifying the -- platform in Dockerfile:

   docker
   FROM --platform=amd64 node:18.15.0-alpine3.17

3. Define the application directory and install the dependencies and copy the output to your root directory:

    docker
    WORKDIR /opt/app 
    COPY package.json yarn.lock ./ 
    RUN yarn install 
    COPY . .

4. Call the Dockerfile from docker-compose.yml:

     services:
      ...redis
      ...db
      
      app:
        build:
          context: .
          dockerfile: Dockerfile
        platforms:
          - "linux/amd64"
        command: yarn dev
        restart: on-failure
        ports:
          - ${PORT:-4000}:${PORT:-4000}
        networks:
          - mk_network
        depends_on:
          - redis
          - db

5. Implement auto-reload so that when you change something in the source code, you can preview your changes immediately without having to rebuild the application manually. To do that, build the image first, then run it in a separate service:

     services:
      ... redis
      ... db
      
      build-docker:
        image: myapp
        build:
          context: .
          dockerfile: Dockerfile
      app:
        image: myapp
        platforms:
          - "linux/amd64"
        command: yarn dev
        restart: on-failure
        ports:
          - ${PORT:-4000}:${PORT:-4000}
        volumes:
          - .:/opt/app
          - node_modules:/opt/app/node_modules
        networks:
          - mk_network
        depends_on:
          - redis
          - db
          - build-docker

Pro tip: Note that node_modules is also mounted explicitly to avoid platform-specific issues with packages. This means that, instead of using the node_modules on the host, the Docker container uses its own but maps it on the host in a separate volume.

Incrementally build the production images with continuous integration 

The majority of our apps and services use CI/CD for deployment, and Docker plays an important role in the process. Every change in the main branch immediately triggers a build pipeline through either GitHub Actions or CircleCI. The general workflow is simple: It installs the dependencies, runs the tests, builds the Docker image, and pushes it to Google Container Registry (or Artifact Registry). In this article, we’ll describe the build step.

Building the Docker images

We use multi-stage builds for security and performance reasons.

Stage 1: Builder

In this stage, we copy the entire code base with all source and configuration, install all dependencies, including dev dependencies, and build the app. It creates a dist/ folder and copies the built version of the code there. This image is way too large, however, with a huge set of footprints to be used for production. Also, as we use private NPM registries, we use our private NPM_TOKEN in this stage as well. So, we definitely don’t want this stage to be exposed to the outside world. The only thing we need from this stage is the dist/ folder.

Stage 2: Production

Most people use this stage for runtime because it is close to what we need to run the app. However, we still need to install production dependencies, and that means we leave footprints and need the NPM_TOKEN. So, this stage is still not ready to be exposed. Here, you should also note the yarn cache clean on line 19. That tiny command cuts our image size by up to 60 percent.

Stage 3: Runtime

The last stage needs to be as slim as possible with minimal footprints. So, we just copy the fully baked app from production and move on. We put all those yarn and NPM_TOKEN stuff behind and only run the app.

This is the final Dockerfile.production:

docker
# Stage 1: build the source code 
FROM node:18.15.0-alpine3.17 as builder 
WORKDIR /opt/app 
COPY package.json yarn.lock ./ 
RUN yarn install 
COPY . . 
RUN yarn build 

# Stage 2: copy the built version and build the production dependencies FROM node:18.15.0-alpine3.17 as production 
WORKDIR /opt/app 
COPY package.json yarn.lock ./ 
RUN yarn install --production && yarn cache clean 
COPY --from=builder /opt/app/dist/ ./dist/ 

# Stage 3: copy the production ready app to runtime 
FROM node:18.15.0-alpine3.17 as runtime 
WORKDIR /opt/app 
COPY --from=production /opt/app/ . 
CMD ["yarn", "start"]

Note that, for all the stages, we start copying package.json and yarn.lock files first, installing the dependencies, and then copying the rest of the code base. The reason for this is that Docker builds each command as a layer on top of the previous one, and each build could use the previous layers if available and only build the new layers for performance purposes. 

Let’s say you have changed something in src/services/service1.ts without touching the packages. That means the first four layers of the builder stage are untouched and could be reused. This approach makes the build process incredibly faster.

Pushing the app to Google Container Registry through CircleCI pipelines

There are several ways to build a Docker image in CircleCI pipelines. In our case, we chose to use circleci/gcp-gcr orbs:

Minimum configuration is needed to build and push our app, thanks to Docker.

executors:
  docker-executor:
    docker:
      - image: cimg/base:2023.03
orbs:
  gcp-gcr: circleci/gcp-gcr@0.15.1
jobs:
  ...
  deploy:
    description: Build & push image to Google Artifact Registry
    executor: docker-executor
    steps:
      ...
      - gcp-gcr/build-image:
          image: my-app
          dockerfile: Dockerfile.production
          tag: ${CIRCLE_SHA1:0:7},latest
      - gcp-gcr/push-image:
          image: my-app
          tag: ${CIRCLE_SHA1:0:7},latest

Pushing the app to Google Container Registry through GitHub Actions

As an alternative to CircleCI, we can use GitHub Actions to deploy the application continuously.

We set up gcloud and build and push the Docker image to gcr.io:

jobs:
  setup-build:
    name: Setup, Build
    runs-on: ubuntu-latest

    steps:
    - name: Checkout
      uses: actions/checkout@v3

    - name: Get Image Tag
      run: |
        echo "TAG=$(git rev-parse --short HEAD)" >> $GITHUB_ENV

    - uses: google-github-actions/setup-gcloud@master
      with:
        service_account_key: ${{ secrets.GCP_SA_KEY }}
        project_id: ${{ secrets.GCP_PROJECT_ID }}

    - run: |-
        gcloud --quiet auth configure-docker

    - name: Build
      run: |-
        docker build \
          --tag "gcr.io/${{ secrets.GCP_PROJECT_ID }}/my-app:$TAG" \
          --tag "gcr.io/${{ secrets.GCP_PROJECT_ID }}/my-app:latest" \
          .

    - name: Push
      run: |-
        docker push "gcr.io/${{ secrets.GCP_PROJECT_ID }}/my-app:$TAG"
        docker push "gcr.io/${{ secrets.GCP_PROJECT_ID }}/my-app:latest"

With every small change pushed to the main branch, we build and push a new Docker image to the registry.

Deploying changes to Google Kubernetes Engine using Google Delivery Pipelines

Having ready-to-use Docker images for each and every change also makes it easier to deploy to production or roll back in case something goes wrong. We use Google Kubernetes Engine to manage and serve our apps, and we use Google Cloud Deploy and Delivery Pipelines for our continuous deployment process.

When the Docker image is built after each small change (with the CI pipeline shown previously), we take one step further and deploy the change to our dev cluster using gcloud. Let’s look at that step in CircleCI pipeline:

- run:
    name: Create new release
    command: gcloud deploy releases create release-${CIRCLE_SHA1:0:7} --delivery-pipeline my-del-pipeline --region $REGION --annotations commitId=$CIRCLE_SHA1 --images my-app=gcr.io/${PROJECT_ID}/my-app:${CIRCLE_SHA1:0:7}

This step triggers a release process to roll out the changes in our dev Kubernetes cluster. After testing and getting the approvals, we promote the change to staging and then production. This action is all possible because we have a slim isolated Docker image for each change that has almost everything it needs. We only need to tell the deployment which tag to use.

How the Quality Assurance team benefits from this process

The QA team needs mostly a pre-production cloud version of the apps to be tested. However, sometimes they need to run a prebuilt app locally (with all the dependencies) to test a certain feature. In these cases, they don’t want or need to go through all the pain of cloning the entire project, installing npm packages, building the app, facing developer errors, and going over the entire development process to get the app up and running.

Now that everything is already available as a Docker image on Google Container Registry, all the QA team needs is a service in Docker compose file:

services:
  ...redis
  ...db
  
  app:
    image: gcr.io/${PROJECT_ID}/my-app:latest
    restart: on-failure
    ports:
      - ${PORT:-4000}:${PORT:-4000}
    environment:
      - NODE_ENV=production
      - REDIS_URL=redis://redis:6379
      - DATABASE_URL=postgresql://${DB_USER:-user}:${DB_PASSWORD:-password}@db:5432/main
    networks:
      - mk_network
    depends_on:
      - redis
      - db

With this service, the team can spin up the application on their local machines using Docker containers by running:

docker compose up

This is a huge step toward simplifying testing processes. Even if QA decides to test a specific tag of the app, they can easily change the image tag on line 6 and re-run the Docker compose command. Even if they decide to compare different versions of the app simultaneously, they can easily achieve that with a few tweaks. The biggest benefit is to keep our QA team away from developer challenges.

Advantages of using Docker

• Almost zero footprints for dependencies: If you ever decide to upgrade the version of Redis or PostgreSQL, you can just change one line and re-run the app. There’s no need to change anything on your system. Additionally, if you have two apps that both need Redis (maybe even with different versions) you can have both running in their own isolated environment, without any conflicts with each other.
• Multiple instances of the app: There are many cases where we need to run the same app with a different command, such as initializing the DB, running tests, watching DB changes, or listening to messages. In each of these cases, because we already have the built image ready, we just add another service to the Docker compose file with a different command, and we’re done.
• Easier testing environment: More often than not, you just need to run the app. You don’t need the code, the packages, or any local database connections. You only want to make sure the app works properly, or need a running instance as a backend service while you’re working on your own project. That could also be the case for QA, Pull Request reviewers, or even UX folks who want to make sure their design has been implemented properly. Our Docker setup makes it easy for all of them to take things going without having to deal with too many technical issues.

Learn more

• Get the latest release of Docker Desktop.
• Have questions? The Docker community is here to help.
• New to Docker? Get started.
• Visit the Kinsta site to learn about the cloud platform.

We have just released a V2 of our GitHub Action to make using the Cache easier as well! We also want to call out a huge THANK YOU to @crazy-max (Kevin :D) for the of work he put into the V2 of the action, we could not have done this without him! 

Right now let’s have a look at what we can do! 

To start we will need to get a project setup, I am going to use one of my existing simple Docker projects to test this out:

The first thing I need to do is to ensure that I will be able to access Docker Hub from any workflow I create, to do this I will need to add my DockerID and a Personal Access Token (PAT) as secrets into GitHub. I can get a PAT by going to https://hub.docker.com/settings/security and clicking ‘new access token’, in this instance I will call my token ‘whaleCI’

I can then add this and my username as secrets into the GitHub secrets UI:

Great we can now start to set up our action workflow to build and store our images in Hub. In this CI flow I am using two Docker actions, the first allows me to log in to Docker Hub using my secrets store in my GitHub Repository. The second is the build and push action, in this I am setting the push flag to true (as I want to push!) and adding in my tag simply to always go to latest. Lastly in this I am also going to echo my image digest to see what was pushed. 

name: CI to Docker hub 

on:

  push:

    branches: [ master ]

  steps:

      -

        name: Login to DockerHub

        uses: docker/login-action@v1 

        with:

          username: ${{ secrets.DOCKER_HUB_USERNAME }}

          password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}

      -

        name: Build and push

        id: docker_build

        uses: docker/build-push-action@v2

        with:

          context: ./

          file: ./Dockerfile

          push: true

          tags: bengotch/simplewhale:latest

      -

        name: Image digest

        run: echo ${{ steps.docker_build.outputs.digest }}

Great, now I will just let that run for the first time and then tweak my Dockerfile to make sure the CI is running and pushing the new image changes:

Next we can look at how we can optimize this; the first thing I want to do is look at using my build cache. This has two advantages, first this will reduce my build time as it will not have to re-download all of my images and second it will reduce the number of pulls I complete against Docker Hub. To do this we are going to leverage the GitHub cache, to do this I need to set up my builder with a build cache.

The first thing I want to do is actually set up a Builder, this is using Buildkit under the hood, this is done very simply using the Buildx action.

  steps:
      -
        name: Set up Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@master

Next I need to set up my cache for my builder, here I am adding the path and keys to store this under using the github cache for this. 

     -
        name: Cache Docker layers
        uses: actions/cache@v2
        with:
          path: /tmp/.buildx-cache
          key: ${{ runner.os }}-buildx-${{ github.sha }}
          restore-keys: |
          ${{ runner.os }}-buildx-

And lastly having added these two bits to the top of my Action file I need to add in the extra attributes to my build and push step. Here I am setting the builder to use the output of the buildx step and then using the cache I set up for this to store to and retrieve from.

 -
        name: Login to Docker Hub
        uses: docker/login-action@v1 
        with:
          username: ${{ secrets.DOCKER_HUB_USERNAME }}
          password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
      -
        name: Build and push
        id: docker_build
        uses: docker/build-push-action@v2
        with:
          context: ./
          file: ./Dockerfile
          builder: ${{ steps.buildx.outputs.name }}
          push: true
          tags:  bengotch/simplewhale:latest
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache
      -
        name: Image digest
        run: echo ${{ steps.docker_build.outputs.digest }}

Great, now we can run it again and I can see that I am using the cache!

Now we can look at how we can improve this more functionally by adding in the ability to have our tagged versions we want to be released to Docker Hub behave differently to my commits to master (rather than everything updating latest on Docker Hub!). You might want to do something like this to have your commits go to a local registry to then use in nightly tests so you can always test what is latest while reserving your tagged versions for release to Hub. 

To start we will need to modify our previous GitHub workflow to only push to Hub if we get a particular tag:

on:
 push:
   tags:
     - "v*.*.*"

This now means our main CI will only fire if we tag our commit with V.n.n.n, let’s have a quick go and test this:

And when I check my GitHub action: 

Great!

Now we need to set up a second GitHub action file to store our latest commit as an image in the GitHub registry, you may want to do this to run your nightly tests or recurring tests against or to share work in progress images with colleagues. To start I am going to clone my previous GitHub action and add back in our previous logic for all pushes. 

Next I am going to change out our Docker Hub login to a GitHub container registry login

    	if: github.event_name != 'pull_request'
    	uses: docker/login-action@v1
    	with:
      	registry: ghcr.io
      	username: ${{ github.repository_owner }}
      	password: ${{ secrets.ghcr_TOKEN }}

And I will also need to remember to change how my image is tagged, I have opted to just keep latest as my only tag but you could always add in logic for this:

  tags: ghcr.io/nebuk89/simplewhale:latest

Now we will have two different flows, one for our changes to master and one for our pull requests. Next we will need to modify what we had before so we are pushing our PRs to the GitHub registry rather than to Hub. 

We could now look at how we set up either nightly tests against our latest tag, how we want to test each PR or if we want to do something more elegant with the tags we are using and make use of the Git tag for the same tag in our image. If you would like to look at how you can do one of these or get a full example of how to setup what we have gone through today please check out Chad’s repo which runs you through this and more details on our latest GitHub action: https://github.com/metcalfc/docker-action-examples 

And keep an eye on our blog for new posts coming in the next couple of weeks looking at how we can get this setup on other CIs, if there are some in particular you would like to see reach out to us on Twitter on @docker.
To get started setting up your GitHub CI with Docker Hub today sign up for a Docker account and have a go with Docker’s official GitHub actions.

We have also heard feedback that given the changes Docker introduced relating to network egress and the number of pulls for free users, that there are questions around the best way to use Docker Hub as part of CI/CD workflows without hitting these limits. This blog post covers best practices that improve your experience and uses a sensible consumption of Docker Hub which will mitigate the risk of hitting these limits and how to increase the limits depending on your use case. 

To get started, one of the most important things when working with Docker and really any CI/CD is to work out when you need to test with the CI or when you can do this locally. At Docker we think about how developers work in terms of their inner loop (code, build, run, test) and their outer loop (push change, CI build, CI test, deployment) 

Before you think about optimizing your CI/CD, it is always important to think about your inner loop and how it relates to the outer loop (the CI). We know that most people aren’t a fan of ‘debugging via the CI’, so it is always better if your inner loop and outer loop are as similar as possible. To this end it can be a good idea to run unit tests as part of your docker build command by adding a target for them in your Dockerfile. That way as you are making changes and re-building locally you can run the same unit tests you would run in the CI on your local machine with a simple command. Chris wrote a blog post earlier in the year about Go development with Docker, this is a great example of how you can use tests in your Docker project and re-use them in the CI. This creates a shorter feedback loop on issues and reduces the amount of pulls and builds your CI needs to do.

Once you get into your actual outer loop and Docker Hub, there are a few things we can do to get the most of your CI and deliver the fastest Docker experience. 

Firstly and foremost stay secure! When you are setting up your CI make sure you are using a Docker Hub access token rather than your password, you can create new access tokens from your security page on Docker Hub. 

Once you have this and have added it to whatever secrets store is available on your platform you will want to look at when you decide to push and pull in your CI/CD along with where from depending on the change you are making. The first thing you can do here to reduce the build time and reduce your number of calls is make use of the build cache to reuse layers you have already pulled. This can be done on many platforms by using buildX (buildkits) caching functionality and whatever cache your platform provides.

The other change you may want to make is only have your release images go to DockerHub, this would mean setting up functions to push your PR images to a more local image store to be quickly pulled and tested rather than promoting them all the way to production.

We know there are a lot more tips and tricks for using Docker in CI but really looking at how to do this around the recent Hub rate changes we think these are the top things you can do.

If you are still finding you have issues with Pull limits once you are authenticated you can consider upgrading to either a Pro or a Team account. This will give you 50,000 pulls in a 24 hour period along with unlimited image retention. In the near future this will also include Image Scanning (powered by Snyk) on push of new images to Docker Hub.

Look out for the next blog post in the series about how to put some of these practices into place with Github actions and feel free to give us ideas of what CI providers you would like to see us covering by dropping us a message on Twitter @Docker.

There was amazing attendance at Containers Today in Stockholm a couple of weeks ago. For those who were unable to make it, here is a quick overview of what I talked about at the event in my session around the future of the developer journey. 

Before we talk about what we think will change the journey, we need to think about why it changes. The fundamental goal of any change to the way of working for developers should be to reduce the number of boring, mundane and repetitive tasks that developers have to do or to allow them to reach new customers/solve new problems. Developers create amazing value for companies and provide solutions to customers’ real world problems. But if they are having to spend half of their time working out how to get things into the hands of their customers, then you are getting half the value.

Developer Evolution

The role of developers has changed a lot over the last ~40 years. Developers no longer deploy to mainframes or in house hardware, they don’t do waterfall deployments and not many of them write in machine code. Developers have to now think about web languages and ML, work in Extreme Agile DevOps teams (ok I made that up a bit) and deploy to the Cloud or to Edge devices. This change keeps happening as the entire industry tries to find new ways for developers to deliver value, deliver that value faster and reduce the number of mundane tasks for developers.

Today the process for getting value to customers is often described as the ‘outer loop’ of development, while the creative process of development, i.e. creating new features, is described as the ‘inner loop.’ 

In an ideal world, that outer loop is automated and uses modern CI/CD technologies to create a cycle of feedback that allows developers to create new features and fix bugs ever faster. This looks something like: 

Though this is an ideal world, this is not the case for a lot of developers. In the “The State of Developer Ecosystem 2019” report, only 45% of developers said that CI/CD was part of their regular tool set. This also means 55% of developers have yet to adopt it. This is one of the changes we think will start to accelerate in the near future. As CI/CD is democratized with new tools like Github Actions, we will see even small teams starting to adopt CI/CD over manual deployments.

Another big change in the outer loop is how people look at what they are passing through this outer loop. Developers are trying to get value out to customers but the concept of value is changing. As we moved away from monoliths, we have started to create individual services that are easier to work with and deploy. The idea of bundling these services with systems like Helm or CNAB tools is becoming more prominent as businesses consider that in isolation, a single container does not provide value to a customer. But a collection of these services together is the minimum set to deliver customer value. 

For the outer loop, some of the big changes we see are changes to how things are deployed. 57% of developers are still not using orchestration (Slashdata Developer Report 16th Edition). The growth in orchestration and in particular K8s is going to change how we think about the hand off between developers and operations as we move towards more microservices in production than ever before. The complexity of this change will be compounded as production is likely to be on multiple clouds or on premises, and as over 50% of companies have a hybrid strategy and a multi cloud strategy.

For developers, all of these things are going to be impactful at a significant scale as we aim to develop the next 500 million apps by 2023 (IDC Analyse the Future). With Edge/IoT growing 10X between 2017-2025 (Allied Market Research), we are also going to need to think of new ways for developers to work with these technologies and extend their inner loop outside of their immediate machine. 

So in summary, we are going to see more bundled apps being deployed via CI/CD to orchestrators in the cloud and on prem, and movement between them. We will be building more of these apps than ever and they have to run anywhere, considering how they interface with the Edge and work as part of our inner loop for this on the Cloud. And finally, we strongly believe that the use of containers is going to be a driving factor and the underlying technology to enable this.

What’s Next for Developers

At Docker, we have started to look ahead at how we can build technology to help developers adopt some of these tools. 

For local developers, we have looked at how to accelerate people creating the next 500 million new applications with Application templates, enabling developers to create new containerized applications in seconds from existing examples. We have been working out how we can improve our IDE integration to make developers lives better where they work today. We have added a layer of GUI abstraction into Docker Desktop to make it simpler to understand all of these components.

We are also thinking about how we can extend the local development environment and the inner loop for new objects. We have Docker Context to allow developers to work on a remote instance within their inner loop and ARM builds in Docker desktop for Working with the Edge.

And to deploy value in the future, we have Docker App for bundling the value of more than one container service to get it running in production.

We are also thinking about things like CI, moving between platforms and generally how to make it easier for the next 10 million developers to get started with Docker. We know that a lot of these technologies have been embraced by early adopters. But as all of these new technologies and changes start to reach the early majority, we need to think about how we are going to scale all of said technologies and keep mundane tasks away from developers! 

Containers Today was a fantastic event with great turnout. Hopefully this post gives some insight into my talk at the event and the trends we see. If you have other thoughts on what the future of the developer journey holds, please reach out to us! 

Getting Started: testcontainers-java

public class RedisBackedCacheIntTestStep0 {
    private RedisBackedCache underTest;

    @Before
    public void setUp() {
        // Assume that we have Redis running locally?
        underTest = new RedisBackedCache("localhost", 6379);
    }

    @Test
    public void testSimplePutAndGet() {
        underTest.put("test", "example");

        String retrieved = underTest.get("test");
        assertEquals("example", retrieved);
    }
}

public class ElasticsearchStorageRule extends ExternalResource {
 static final Logger LOGGER = LoggerFactory.getLogger(ElasticsearchStorageRule.class);
 static final int ELASTICSEARCH_PORT = 9200; final String image; final String index;
 GenericContainer container;
 Closer closer = Closer.create();

 public ElasticsearchStorageRule(String image, String index) {
   this.image = image;
   this.index = index;
 }
 @Override
 
  protected void before() {
   try {
     LOGGER.info("Starting docker image " + image);
     container =
         new GenericContainer(image)
             .withExposedPorts(ELASTICSEARCH_PORT)
             .waitingFor(new HttpWaitStrategy().forPath("/"));
     container.start();
     if (Boolean.valueOf(System.getenv("ES_DEBUG"))) {
       container.followOutput(new Slf4jLogConsumer(LoggerFactory.getLogger(image)));
     }
     System.out.println("Starting docker image " + image);
   } catch (RuntimeException e) {
     LOGGER.warn("Couldn't start docker image " + image + ": " + e.getMessage(), e);
   }

func TestNginxLatestReturn(t *testing.T) {
    ctx := context.Background()
    req := testcontainers.ContainerRequest{
        Image:        "nginx",
        ExposedPorts: []string{"80/tcp"},
    }
    nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
        ContainerRequest: req,
        Started:          true,
    })
    if err != nil {
        t.Error(err)
    }
    defer nginxC.Terminate(ctx)
    ip, err := nginxC.Host(ctx)
    if err != nil {
        t.Error(err)
    }
    port, err := nginxC.MappedPort(ctx, "80")
    if err != nil {
        t.Error(err)
    }
    resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port()))
    if resp.StatusCode != http.StatusOK {
        t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode)
    }
}

Creating the Test

ctx := context.Background()
req := testcontainers.ContainerRequest{
    Image:        "nginx",
    ExposedPorts: []string{"80/tcp"},
}
nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
    ContainerRequest: req,
    Started:          true,
})
if err != nil {
    t.Error(err)
}
defer nginxC.Terminate(ctx)

Modules

@Rule
public GenericContainer dslContainer = new GenericContainer(
    new ImageFromDockerfile()
            .withFileFromString("folder/someFile.txt", "hello")
            .withFileFromClasspath("test.txt", "mappable-resource/test-resource.txt")
            .withFileFromClasspath("Dockerfile", "mappable-dockerfile/Dockerfile"))

What I’m working on now

ctx := context.Background()
k := &KubeKindContainer{}
err := k.Start(ctx)
if err != nil {
  t.Fatal(err.Error())
}
defer k.Terminate(ctx)
clientset, err := k.GetClientset()
if err != nil {
  t.Fatal(err.Error())
}
ns, err := clientset.CoreV1().Namespaces().Get("default", metav1.GetOptions{})
if err != nil {
  t.Fatal(err.Error())
}
if ns.GetName() != "default" {
  t.Fatalf("Expected default namespace got %s", ns.GetName())

Calling All Coders

Try It Out

Docker Captain @GianArb gives the low down on how to write maintainable integration tests with #Docker
Click To Tweet