Dockerfiles now Support Multiple Build Contexts

The new releases of Dockerfile 1.4 and Buildx v0.8+ come with the ability to define multiple build contexts. This means you can use files from different local directories as part of your build. Let’s look at why it’s useful and how you can leverage it in your build pipelines.

When you invoke the docker build command, it takes one positional argument which is a path or URL to the build context. Most commonly, you’ll see docker build . making the current working directory the build context.

Inside a Dockerfile you can use COPY and ADD commands to copy files from your build context and make them available to your build steps. In BuildKit, we also added build mounts with RUN --mount that allow accessing build context files directly — without copying them — for extra performance.

Conquering Complex Builds

But, as builds got more complicated, the ability to only access files from one location became quite limiting. That’s why we added multi-stage builds where you can copy files from other parts of the Dockerfile by adding the --from flag and pointing it to the name of another Dockerfile stage or a remote image.

The new named build context feature is an extension of this pattern. You can now define additional build contexts when running the build command, give them a name, and then access them inside a Dockerfile the same way you previously did with build stages.

Additional build contexts can be defined with a new --build-context [name]=[value] flag. The key component defines the name for your build context and the value can be:

• Local directory – e.g. --build-context project2=../path/to/project2/src
• Git repository – e.g. --build-context qemu-src=https://github.com/qemu/qemu.git
• HTTP URL to a tarball – e.g. --build-context src=https://example.org/releases/src.tar
• Docker image – Define with a docker-image:// prefix, e.g. --build-context alpine=docker-image://alpine:3.15

 

On the Dockerfile side, you can reference the build context on all commands that accept the “from” parameter. Here’s how that might look:

# syntax=docker/dockerfile:1.4
FROM [name]
COPY --from=[name] ...
RUN --mount=from=[name] …

 

The value of [name] is matched with the following priority order:

• Named build context defined with --build-context [name]=..
• Stage defined with AS [name] inside Dockerfile
• Remote image [name] in a container registry

 

If no --from flag is set, files are loaded from the main build context.

Example #1: Pinning an Image

Let’s start with an example of how you can use build contexts to pin an image used by a Dockerfile to a specific version.

This is useful in many different cases. For example, you can use the new BuildInfo feature to capture all the build sources and run a build with the same dependencies as a previous build did, even if the image tags have been updated.

docker buildx imagetools inspect --format '{{json .BuildInfo}}' moby/buildkit

"sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/alpine:3.15",
        "pin": "sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300"
      },

 

docker buildx build --build-context alpine:3.15=docker-image://alpine:[email protected]:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300 .


When your Dockerfile uses alpine:3.15, even if it’s been updated with a newer version in the registry, your new build will still use the same exact image your previous build did.

As another example, you may just want to try a different image or different version for debugging or developing your image. A common pattern could be that you haven’t released your image yet, and it’s only in the test registry or staging environment. Let’s say you built your app and pushed it to a staging repository, but now want to use it in your other builds that would usually use the release image.

 

docker buildx build --build-context myorg/myapp=docker-image://staging.myorg.com/registry/myapp .

You can also think about the previous examples as a way to create an alias for an image.

Example #2: Multiple Projects

Probably the most requested use case for named contexts capability is the possibility to use multiple local source directories.

If your project contains multiple components that need to be built together, it’s sometimes tricky to load them with a single build context where everything needs to be contained in one directory. There’s a variety of issues: every component needs to be accessed by their full path, you can only have one .dockerignore file, or maybe you’d like each component to have its own Dockerfile.

If your project has the following layout:

project
├── app1
│   ├── .dockerignore
│   ├── src
├── app2
│   ├── .dockerignore
│   ├── src
├── Dockerfile

…with this Dockerfile:

#syntax=docker/dockerfile:1.4
FROM … AS build1
COPY –from=app1 . /src

FROM … AS build2
COPY –from=app2 . /src

FROM …
COPY –from=build1 /out/app1 /bin/
COPY –from=build2 /out/app2 /bin/

…you can invoke your build with docker buildx build –build-context app1=app1/src –build-context app2=app2/src .. Both of the source directories are exposed separately to the Dockerfile and can be accessed by their respective names.

This also allows you to access files that are outside of your main project’s source code. Normally when you’re inside the Dockerfile, you’re not allowed to access files outside of your build context by using the ../ parent selector for security reasons. But as all build contexts are passed directly from the client, you’re now able to use --build-context othersource=../../path/to/other/project to avoid this limitation.

Example #3: Override a Remote Dependency with a Local One

When exposing multiple source contexts to the builds there may be cases where your project always depends on multiple local directories, like in the previous example. Other times, however, you may want your dependencies to be loaded from a remote source by default, while still leaving you the option to replace it with a local source when you want to do some extra debugging.

As an example, let’s look at a common pattern where your app depends on another project that you build from source code using multi-stage builds.

Something like:

FROM golang AS helper
RUN apk add git
WORKDIR /src
ARG HELPERAPP_VERSION=1.0
RUN git clone https://github.com/someorg/helperapp.git && cd helperapp && git checkout $HELPERAPP_VERSION
WORKDIR /src/helperapp
RUN go build -o /out/helperapp .

FROM alpine
COPY –link –from=helper /out/helperapp /bin
COPY –link –from=build /out/myapp /bin

 

This works quite well. When you do a build, helperapp is built directly from its source repository and copied next to your app binary. Whenever you need to use a different version you can use the HELPERAPP_VERSION build argument to specify a different value.

But let’s say you’re developing your application and have found a bug. You’re not quite sure if the bug is in your application code or in the helper app. You’d want to make some local changes to the helperapp code to analyze what’s going on. The problem is that with your current code you’d need to push your changes to Github first so they can then be pulled down by the Dockerfile. Doing this for every code change would be very painful.

Instead, consider if we change the previous code to:

FROM alpine AS helper-clone
RUN apk add git
WORKDIR /src
ARG HELPERAPP_VERSION=1.0
RUN git clone https://github.com/someorg/helperapp.git && cd helperapp && git checkout $HELPERAPP_VERSION

FROM scratch AS helper-src
COPY –from=helper-clone /src/helperapp /

FROM golang:alpine AS helper
WORKDIR helperapp
RUN –mount=target=.,from=helper-src go build -o /out/helperapp .

FROM alpine
COPY –link –from=helper /out/helperapp /bin
COPY –link –from=build /out/myapp /bin

 

By default, this Dockerfile behaves exactly like the previous one, making a clone from GitHub to get the source code. But now, because we have added a separate stage helper-src that contains the source code for helperapp, we can use the new named contexts feature to override it with our local source directory when needed.

docker buildx build –build-context helper-src=../path/to/my/local/helper/checkout .

Now you can test all your local patches without a separate Dockerfile or without needing to move all your source code under the same directory.

Named Contexts in buildx bake

In addition to the `build` command, `docker buildx` also has a command called `bake`. Bake is a higher-level build command that allows you to define your build configurations in files instead of typing in a long list of flags for your build commands every time.

Additionally, it allows running many builds together, defining variables, and sharing definitions between your separate build configurations, etc. It accepts build configurations in JSON, HCL and Docker Compose YAML files. You can read more about it in the Buildx documentation.

We’ve also added named contexts support into bake. This is useful because if you write a Dockerfile that depends on multiple build contexts, you might forget that you need to pass these values with --build-context flag every time you invoke the build command.

With bake, you can define your target definition. For example:

hcl
target “binary” {
  contexts = {
    app1 = “app1/src”
    app2 = “app2/src”
  }
}

 

Now instead of remembering to use the --build-context flag with the correct paths every time, you can just call docker buildx bake binary and your build will run with the correct configuration. Of course, you can also use Bake variables, etc. in these fields for more complex cases.

You may also use this pattern to create special bake targets for the purpose of debugging or testing images in staging repositories.

hcl
target “myapp” {
 …
}

target “myapp-stage” {
  inherits = [“myapp”]
  contexts = {
    helperapp = “docker-image://staging.myorg.com/registry/myapp”
  }
}

With a Bake file like this, you can now call docker buildx bake myapp-stage to build your app with the exact configuration defined for your myapp target, except when your build is using helperapp image it will now be loaded from the staging repository instead of the release one that’s written into the Dockerfile.

Create Build Pipelines by Linking bake Targets

In addition to image, Git, URL, and local directories, Bake files also support another definition that you can use as a named context. You can set the source for the named context to point to another build target inside the Bake file. This way, you can chain together builds from multiple Dockerfiles that depend on each other and build them with a single command invocation.

Let’s say we have two Dockerfiles:

# base.Dockerfile
FROM alpine
…

# Dockerfile
FROM baseapp
...

 

Normally, you’d first build base.Dockerfile, then push it to a registry or leave it in the Docker image store. Then you’d build the second Dockerfile that loads the image by name.

An issue with this approach is that if you use the Docker image store, then it currently doesn’t support multi-platform local images. Using an external registry isn’t always very convenient either and, in both cases, some external change could update the base image in between two builds and make the second build use the wrong image. You also need to run the build commands twice and synchronize them manually.

Instead, you can define a Bake file with a build context defined with a target: prefix:

target “base” {
  dockerfile = “base.Dockerfile”
  platforms = [“linux/amd64”, “linux/arm64”]
}

target “myapp” {
  contexts = {
    baseapp = “target:base”
  }
  platforms = [“linux/amd64”, “linux/arm64”]
}

 

Now you can build your app by just running docker buildx bake myapp to build both Dockerfiles and link them as required. If you want to build both the base image and your app together, you can use docker buildx bake myapp base. Both of these targets are defined as multi-platform and Buildx will take care of linking the corresponding single-platform subimages with each other.

Note that you should always first consider just using multi-stage builds with a --target parameter in these conditions. Having self-contained Dockerfiles is a simpler solution as it doesn’t require passing extra parameters with your build. This pattern should be used when you can’t combine the Dockerfiles and need to keep them separate.

Please check out the new build context feature in Docker Buildx v0.8 release, included with the latest Docker Desktop.