How we test Docker Desktop with WSL 2

Recently we have released a new Edge version 2.2.3.0 of Docker Desktop for Windows. This can be considered as a release candidate for the next Stable version that will officially support WSL 2. With Windows 10 version 2004 in sight we are giving the next version of Docker Desktop the final touches to give you the best experience running Linux containers on Windows 10.

One of the great benefits is that with the next update of Windows 10 we will also support running Docker Desktop on Windows 10 Home. We worked closely with Microsoft during the last few months to make Docker Desktop and WSL 2 fit together.

In this blog post we look behind the scenes at how we set up new WSL 2 capable test machines to run automated tests in our CI pipeline.

It started with a laptop

Let’s keep in mind that all automation somehow starts with manual steps and you evolve from there to get better and more automated. At the beginning of this project we were given a laptop back at KubeCon 2019 with an early version of WSL 2.

With that single laptop our development team could start getting their hands on that new feature and integrating it into Docker Desktop. But of course, this doesn’t really scale for a whole team and we also needed automated tests.

The Docker Desktop test matrix

In the Docker Desktop team we run several test suites across several Windows and Mac machines with different operating system versions installed. Each code change is tested with a matrix of tests on selected machines.

One of our challenges was to add Windows machines to this matrix with WSL 2 enabled. At that time the Windows Insider program started to ship first releases and we could start automating the process to keep new test machines up to date.

On-demand test runners

The startup time of Docker Desktop is much faster with the WSL 2 backend. This gave us the option to run the end-to-end tests in virtual machines. We enhanced our CI infrastructure to spin up Windows 10 Insider machines in Azure on demand. This gave us more flexibility to keep the test machines at a working version of WSL 2 in our pool and also trying out the latest Insider builds.

Our internal CI dashboard shows all the test machines and the jobs running on them changed every few weeks. We constantly moved from one Insider release to the next. Currently we are concentrating on the final Slow Ring builds 19041.x, but we are also continuing with the next Fast Ring machines to have feedback from upcoming Windows builds.

Automated pipeline to build the test machines

The Azure VM images we use to spin up WSL 2 test machines are created with a separate CI pipeline. We use Packer to create the VM image from an ISO file and run provision scripts to prepare everything we need to run it as a CI runner. The pipeline of how we build and upload the VM image also contains more than just the build step. We first check the source code of the Packer template and the PowerShell and Unix shell scripts to fail fast if a code change broke something. The Packer build itself takes the longest time, it also runs a Windows Update in the VM to get the latest OS version. After the build we added a verification step using InSpec to check if the software we need is installed correctly.

The output of this Packer pipeline is an Azure VM image that can be used to spin up new on-demand runners in other CI pipelines. We normally run some tests in a canary environment to see if the VM image really boots up and attaches to our CI infrastructure. If everything is fine we update the configuration for the Docker Desktop CI for our end-to-end tests.

A new challenge: Windows 10 Home

With that automation for Windows 10 Pro machines at hand we were able to add Windows 10 Home very easily. Of course there were some challenges, for example Windows 10 Home does not provide Remote Desktop support. We added a VNC server to be able to attach to the cloud runners if we want to investigate problems.

Conclusion

In the last 12 months the Docker Desktop team worked hard to bring not only the WSL 2 support to Docker Desktop, but also enabled Windows 10 Home users to easily run Docker on their machines. We really look forward to the official release of Windows 10, version 2004 and love to hear your feedback.