Simon Ferquel – Docker https://www.docker.com Wed, 15 Feb 2023 02:10:59 +0000 en-US hourly 1 https://wordpress.org/?v=6.2.2 https://www.docker.com/wp-content/uploads/2023/04/cropped-Docker-favicon-32x32.png Simon Ferquel – Docker https://www.docker.com 32 32 Docker Desktop: WSL 2 Best practices https://www.docker.com/blog/docker-desktop-wsl-2-best-practices/ Mon, 04 May 2020 16:30:00 +0000 https://www.docker.com/blog/?p=26100 Docker Desktop WSL 2 backend has now been available for a few months for Windows 10 insider users and Microsoft just released WSL 2 on the Release Preview channel (which means GA is very close). We and our early users have accumulated some experience working with it and are excited to share a few best practices to implement in your Linux container projects!

Docker Desktop with the WSL 2 backend can be used as before from a Windows terminal. We focused on compatibility to keep you happy with your current development workflow.

But to get the most out of Windows 10 2004 we have some recommendations for you.

Fully embrace WSL 2

The first and most important best practice we want to share, is to fully embrace WSL 2. Your project files should be stored within your WSL 2 distro of choice, you should run the docker CLI from this distro, and you should avoid accessing files stored on the Windows host as much as possible.

For backward compatibility reasons, we kept the possibility to interact with Docker from the Windows CLI, but it is not the preferred option anymore.

Running docker CLI from WSL will bring you…

Awesome mounts performance

Both your own WSL 2 distro and docker-desktop run on the same utility VM. They share the same Kernel, VFS cache etc. They just run in separate namespaces so that they have the illusion of running totally independently. Docker Desktop leverages that to handle bind mounts from a WSL 2 distro without involving any remote file sharing system. This means that when you mount your project files in a container (with docker run -v ~/my-project:/sources <...>), docker will propagate inotify events and share the same cache as your own distro to avoid reading file content from disk repeatedly.

A little warning though: if you mount files that live in the Windows file system (such as with docker run -v /mnt/c/Users/Simon/windows-project:/sources <...>), you won’t get those performance benefits, as /mnt/c is actually a mountpoint exposing Windows files through a Plan9 file share. 

Compatibility with Linux toolchains and build scripts

Most reasonably sized projects involving Linux containers come with a bunch of automation scripts. Those scripts are often developed for Linux first (because most of the time, CI/CD pipelines for those projects run on Linux), and developers running on Windows are often considered second-class citizens. They are often using less polished versions of those scripts, and have to deal with subtle behavioral differences.

By fully embracing WSL 2, Windows developers can use the exact same build and automation scripts as on Linux. This means that Windows-specific scripts don’t need to be maintained anymore. Also, that means that you won’t experience issues with different line endings between Windows and Mac/Linux users!

What about my IDE?

If you want an IDE for editing your files, you can do that even if they are hosted within your WSL 2 distro. There are 3 different ways:

  • Use Visual Studio Code Remote to WSL extension

If your IDE is Visual Studio Code, using Remote to WSL is the best way to continue working on your project. Visual Studio Code architecture is based on a client/server approach where pretty much everything except rendering and input processing is done in a server process, while the UI itself runs in a client process. Remote to WSL leverages that to run the whole server process within WSL while the UI runs as a classic win32 process.

That means that you get the same experience as before, but all your language services, terminals etc. run within WSL.

For more information, see Microsoft’s VS Code Remote to WSL documentation.

  • Point your IDE to your distro network share

WSL provides a network share for each of your running distros. For example, if I have a project in my Ubuntu distro at `~/wall-e`, I can access it from Windows Explorer (and from any Windows Process) via the special network share `\\wsl$\Ubuntu\home\simon\wall-e`. 

  • Run an X11 server on Windows, and run a Linux native IDE

The setup is a bit more complicated, but you always have the possibility to run an X11 server on Windows (VcXsrv, X410,…) and configure your DISPLAY environment variable such that GUI apps on Linux get rendered properly.

Use BuildKit and multi-stage builds

Docker Desktop WSL 2 backend has access to all your CPU cores. To leverage this as much as possible (and also to get access to the latest build features), you should enable BuildKit by default.

The easiest way to do that is to add the following line to your ~/.profile file:

export DOCKER_BUILDKIT=1.

This way, anytime you run docker build, it will run the build with the awesome BuildKit which is capable of running different build stages concurrently.

Use resource limits

Docker Desktop WSL 2 backend can use pretty much all CPU and memory resources on your machine. This is awesome for most cases, but there is a category of workloads where this can cause issues. Indeed, some containers (mainly databases, or caching services) tend to allocate as much memory as they can, and leave other processes (Linux or Win32) starving. Docker provides a way to impose limits in allocatable memory (as well as quotas on CPU usage) by a container. You can find documentation about it here: https://docs.docker.com/config/containers/resource_constraints/.

Reclaim cached memory

WSL 2 automatically reclaims memory when it is freed, to make it available to Windows processes. However, if the kernel decides to keep content in cache (and with Docker, it tends to happen quite a lot), the amount of memory reclaimed might not be sufficient.

To reclaim more memory, after stopping your containers, you can run echo 1 > /proc/sys/vm/drop_caches as root to drop the kernel page cache and make WSL 2 reclaim memory used by its VM.

What next

We are excited for people to use Docker Desktop with WSL 2 and hope that the tips and tricks in this article will help you get the best performance for all of your workloads. 

If you have another tip or idea you want to share with us for using Docker send us a tweet @docker or if you have feedback on our implementation then raise a ticket against our Github Repo.

]]>
Introducing the Docker Desktop WSL 2 Backend https://www.docker.com/blog/new-docker-desktop-wsl2-backend/ Thu, 24 Oct 2019 15:00:47 +0000 https://engineering.docker.com/?p=453 What’s Changed Since the Tech Preview

Earlier this year, we released a technical preview of our vision for the future of Docker development on Windows using WSL 2. We received lots of feedback from Windows Insiders via different channels, and collated common failure cases. We also used it ourselves a lot, and took the time to evaluate its architecture.

Based on this analysis, we worked hard to redesign Docker Desktop’s WSL2 integration in a more robust and easier to maintain way, while ensuring we get feature parity with what we have today with our Hyper-V backend.

What is new?

Before digging into the details of the new backend architecture, let us see what new features we have:

  1. Kubernetes support: you can now enable Kubernetes when using the WSL 2 backend
  2. Updated daemon: our WSL 2 backend now runs our latest stable Docker Daemon
  3. VPN-friendly networking: our WSL 2 backend leverages our efforts in this area, using vpnkit to ensure a VPN-friendly networking stack
  4. And more: the WSL 2 backend is now at feature parity with our Hyper-V backend. HTTP proxy settings, trusted CA synchronization, version pack support, support for our new container UI…

This new backend can be enabled in Docker Desktop settings:

wsl2 docker settings

Once WSL 2 is generally available, we will remove this checkbox, and automatically switch on the WSL 2 backend on compatible machines.

Introducing our new architecture

Based on our users feedback, and on the requirements we identified internally, there are 3 major aspects that we wanted to change in the WSL integration architecture:

  • Run in an isolated environment: we want to run in a separate network/pid/mount namespace, to avoid as much as possible side effects from other applications running on WSL2
  • Leverage our current codebase to avoid re-implementing all the features we already implemented in our own Hyper-V VM
  • Have a complete integration with the existing UI our users are familiar with.

Architecture of the Hyper-V backend

architecture hyper v backend

To understand this new architecture, we need to step back a little and look at how the Hyper-V backend is designed, and how the Windows frontend communicates with it.

The most important thing is the Linux VM we run on Hyper-V. This Linux VM is entirely built using LinuxKit which makes it very easy for us to have precise control over everything that runs in it. We wrote a number of LinuxKit components, used both in our Hyper-V and Mac VMs: services controlling the lifecycle of Docker and Kubernetes, services to collect diagnostics in case of failure, services aggregating logs, etc. Those services are packaged in an iso file in the Docker Desktop installation directory (docker-desktop.iso).

On top of this base distro, at runtime we mount a second iso, that we call a version-pack iso. This file contains binaries and deployment / upgrade scripts specific to a version of the Docker Engine and Kubernetes. In Enterprise edition, this second iso is part of the version packs we publish, while in Community, a single version pack is supported (the docker.iso file, also present in the docker desktop installation folder).

Before starting the VM, we also attach a VHD to store container images and configs, as well as the Kubernetes data store.

To make those services reachable from the Windows side, we built a proxy that exposes Unix sockets as Windows named pipes, using Hyper-V Sockets under the hood.

How it translates to the new WSL 2 backend

The new WSL backend design is very close to that, with the difference that we don’t run the LinuxKit distro in a VM but… in a container.

archtecture wsl2

This will create 2 WSL distros for you: 

  • Docker-desktop, which I’ll call the bootstrapping distro
  • Docker-desktop-data, which I’ll call the data store distro

From a high level perspective, the bootstrapping distro essentially replaces Hyper-V, while the data store distro replaces the VHD that we previously attached to the VM.

The bootstrapping distro creates a Linux namespace with its own root filesystem based on the same 2 iso files we mentioned earlier (not entirely true, but close enough), and use the data-store distro as the backing store for container images etc. instead of a VHD (WSL 2 does not allow us to attach additional VHD at the moment, so we leverage cross-distro mounts for that). The first iso file is slightly modified from the original one: we have stripped out the Linux Kernel, and the system services provided out of the box by WSL 2. The second one (the version pack iso), is strictly identical to the one we use with Hyper-V (and on Mac as well). The bootstrapping distro also manages things like mounting the Windows 9p shares in a place that can be accessed by the Linuxkit container (to avoid using Samba for sharing Windows files with containers), and controls the lifecycle of the Linuxkit container (ensuring clean shutdown etc.).

This way, Docker runs in a contained environment that is very similar to Hyper-V and MacOS VMs. So close that we actually share the same code base for our Linuxkit components, we very quickly achieved feature parity with Hyper-V backend using the same version pack isos, and the same Windows side code base.

The big difference that this makes is that it starts 15 times quicker than our Hyper-V VM, and thanks to WSL 2 dynamic resource allocations, it can access all the resources of the machine, and consume as little as it really needs. It will make it able to run in environments with lower memory where it was previously difficult to allocate 2GB of the Hyper-V VM upfront.

In a longer term, it also opens the door to supporting Windows versions where Hyper-V is not available but WSL 2 is (Windows Home edition in particular).

Linux Workspaces support

With the tech preview architecture, Linux Workspaces were very easy to implement: the daemon ran in your own distro, so it had direct access to your filesystem. Exposing the daemon worked out of the box as well as bind mounts.

With the new architecture, things are a little bit trickier: we run in a separate distro, and inside an isolated namespace, so how can we achieve the same level of performance?

WSL 2 runs all distros in the same utility VM, sharing the same Kernel. Recently, Microsoft introduced cross-distros bindings, which enables access to the VHD of a given distro from another. Using this we are actually able to expose the daemon to the user distro, and we can access the user distro files from within our contained environment with native Linux performance.

To make bind-mounts a seamless experience, we introduced a docker api proxy similar to the one we use for enabling bind mounts of Windows files that translates paths relative to the user distro into a path to the same file accessible from the LinuxKit container.

Initial limitations

We are still working on polishing the Linux Workspace experience. Initially, you will have to deal with those following limitations:

  1. You can only mounts files backed by your distro VHD (that means you can’t bind mounts things within /tmp, /mnt, /var/run, /proc, /sys etc.). For most people, this should not be a problem, but mounting things like /var/run/docker.sock in a container won’t work initially. We are working with Microsoft on solving this issue, a future Windows Update will bring full bind mount support.
  2. We don’t provide client binaries yet. You need to install the docker cli and plugins using apt, yum or any other package manager on your distro. We will automate this in a later update.

Why did we do that?

When we released the tech preview, what we wanted was to put in the hands of real users, something that represents the vision we have about the future of Linux container development on Windows. We worked very quickly, aside from our main Docker Desktop project to build something we could experiment with, and collected a lot of user feedback (thank you Windows Insiders by the way, this helped a lot!).

We also challenged the tech preview architecture in term of long term concerns such as maintenance costs, problems diagnostics, stability, update handling, code sharing with other backends etc.

We then went back to the whiteboard, designed a few potential alternative architectures, and found that with minimal effort, we would be able to run our LinuxKit VM, with very small modifications, in a container within WSL 2. This approach makes it really easy to implement the same exact mechanisms as we have today for things like problems diagnostics, handling updates, ensuring version pack support, and feature parity with our other backends.

We went to a prototyping phase that proved successful and went on to integrate it directly in our Docker Desktop codebase.

Top issues driving this decision

Users want Kubernetes support

In a larger extent, we want feature parity with the Hyper-V backend. This includes Kubernetes support, of course but also many hidden features like:

  • Version pack support in Enterprise
  • Trusted CA synchronization
  • VPN-friendly networking
  • HTTP proxies support

Our new architecture solve all this.

Docker-compose cannot talk to the WSL 2 engine

The version of docker-compose we shipped with the tech preview was not aware of docker contexts. While we are working on fixing docker-compose, the new architecture makes it easier for client tools not supporting docker context yet to work with (including older versions of docker-compose).

Sometimes containers networking don’t work

We identified several reasons for that, and they are all related to a flaw in our original design: the tech preview runs within your own distro. In this distro you might be running other applications that might deal with network interfaces, iptables, etc. It can create conflicts that are completely outside of our control.

Even worse, all WSL2 distros run in the same network namespace… we have had reports from some users who had issues because they tried to run docker in 2 distros at once. Naturally the 2 Docker daemons killed each other by creating conflicting iptables rules.

Our new architecture is not subject to this issue any more.

Diagnostics are a nightmare

Running within a distro that we don’t control causes huge challenges in terms of problem diagnostics. Especially when adding support for Kubernetes.

There are too many ways the user can break everything, and it is very difficult to create a diagnostics system collecting only non-sensitive data on a system we don’t control. Running within an isolated container solves this issue for us, and we can leverage the same diagnostics services we already use in the Hyper-V and Mac backends.

Supporting more than one “hosting distro” is challenging

Not all our users want to run on Ubuntu, we recognize that and want to give them the choice. However automating installation of the integration package in a really “universal” way is very challenging. By running in our own WSL distro, within a container, we don’t have this issue anymore, and keep the same filesystem performance thanks to cross-distro bind mounts.

Maintenance cost of the integration package is very high

The architecture of the Hyper-V and Mac backends makes it easy for us to share a lot of code. Mac and Windows VMs are almost identical, and we want to keep this great productivity value on WSL 2 as well. The cost of implementing all the features in a new way with the integration package was too high.

Our new approach allows us to share the large majority of our Linux code base between Mac, Hyper-V and WSL 2 backends.

Handling upgrades is very hard

We have extensive experience at handling upgrades with Docker Desktop. It is hard, and we don’t want to duplicate this work.

The new architecture shares the same exact logic as what we have in place for Hyper-V and Mac backends.

The future of Docker Desktop

Once Microsoft makes WSL 2 generally available, we plan to enable the WSL 2 engine on all supported Windows versions by default. We will still support the Hyper-V backend until Microsoft stops supporting Windows versions without WSL 2 though, but only as a fallback mechanism.

By moving to this “WSL 2 first” approach, we also want to take advantage of its unique characteristics to unlock new features in the future. As an example, WSL 2 is supported on Windows 10 Home. We want to take advantage of that to reach new users in the future (nothing to announce yet, but it is definitely in our backlog).

This new backend paves the way for exciting new features to come, and we are eager to hear your feedback.

We will be releasing the new WSL 2 architecture as part of the next Docker Desktop Edge release. For Windows users already with WSL 2 Download Edge today to get access to the latest Docker architecture in the next couple of weeks.

]]>
Docker ❤️ WSL 2 – The Future of Docker Desktop for Windows https://www.docker.com/blog/docker-hearts-wsl-2/ Sun, 16 Jun 2019 11:59:43 +0000 https://engineering.docker.com/?p=321 One of Docker’s goals has always been to provide the best experience working with containers from a Desktop environment, with an experience as close to native as possible whether you are working on Windows, Mac or Linux. We spend a lot of time working with the software stacks provided by Microsoft and Apple to achieve this. As part of this work, we have been closely monitoring Windows Subsystem for Linux (WSL) since it was introduced in 2016, to see how we could leverage it for our products.

The original WSL was an impressive effort to emulate a Linux Kernel on top of Windows, but there are such foundational differences between Windows and Linux that some things were impossible to implement with the same behavior as on native Linux, and this meant that it was impossible to run the Docker Engine and Kubernetes directly inside WSL. Instead, Docker Desktop developed an alternative solution using Hyper-V VMs and LinuxKit to achieve the seamless integration our users expect and love today.

Microsoft has just announced WSL 2 with a major architecture change: instead of using emulation, they are actually providing a real Linux Kernel running inside a lightweight VM. This approach is architecturally very close to what we do with LinuxKit and Hyper-V today, with the additional benefit that it is more lightweight and more tightly integrated with Windows than Docker can provide alone. The Docker daemon runs well on it with great performance, and the time it takes from a cold boot to have dockerd running in WSL 2 is around 2 seconds on our developer machines. We are very excited about this technology, and we are happy to announce that we are working on a new version of Docker Desktop leveraging WSL 2, with a public preview in July. It will make the Docker experience for developing with containers even greater, unlock new capabilities, and because WSL 2 works on Windows 10 Home edition, so will Docker Desktop.

Collaborating with Microsoft

As part of our shared effort to make Docker Desktop the best way to use Docker on Windows, Microsoft gave us early builds of WSL 2 so that we could evaluate the technology, see how it fits with our product, and share feedback about what is missing or broken. We started prototyping different approaches and we are now ready to share a little bit about what is coming in the next few months.

Docker Desktop Future

We will replace the Hyper-V VM we currently use by a WSL 2 integration package. This package will provide the same features as the current Docker Desktop VM: Kubernetes 1-click setup, automatic updates, transparent HTTP proxy configuration, access to the daemon from Windows, transparent bind mounts of Windows files, and more.

wsl2 1

This integration package will contain both the server side components required to run Docker and Kubernetes, as well as the CLI tools used to interact with those components within WSL. We will then be able to introduce a new feature with Docker Desktop: Linux workspaces.

Linux Workspaces

When using Docker Desktop today, the VM running the daemon is completely opaque: you can interact with the Docker and Kubernetes API from Windows, but you can’t run anything within the VM except Docker containers or Kubernetes Pods.

With WSL 2 integration, you will still experience the same seamless integration with Windows, but Linux programs running inside WSL will also be able to do the same. This has a huge impact for developers working on projects targeting a Linux environment, or with a build process tailored for Linux. No need for maintaining both Linux and Windows build scripts anymore! As an example, a developer at Docker can now work on the Linux Docker daemon on Windows, using the same set of tools and scripts as a developer on a Linux machine:

wsl2 2 smaller

A developer working on the Docker Daemon using Docker Desktop technical preview, WSL 2 and VS Code remote

Also, bind mounts from WSL will support inotify events and have nearly identical I/O performance as on a native Linux machine, which will solve one of the major Docker Desktop pain points with I/O-heavy toolchains. NodeJS, PHP and other web development tools will benefit greatly from this feature.

Combined with Visual Studio Code “Remote to WSL”, Docker Desktop Linux workspaces will make it possible to run a full Linux toolchain for building containers on your local machine, from your IDE running on Windows.

Performance

With WSL 2, Microsoft put a huge amount of effort into performance and resource allocations: The VM is setup to use dynamic memory allocation, and can schedule work on all the Host CPUs, while consuming as little (or as much) memory it requires – within the limits of what the host can provide, and in a collaborative manner towards win32 processes running on the host.

Docker Desktop will leverage this to greatly improve its resource consumption. It will use as little or as much CPU and memory as it needs, and CPU/Memory intensive tasks such as building a container will run much faster than today.

In addition, the time to start a WSL 2 distribution and the Docker daemon after a cold start is blazingly fast – within 2s on our development laptops, compared to tens of seconds in the current version of Docker Desktop. This opens the door to battery-life optimizations by deferring the daemon startup to the first API call, and automatically stop the daemon when it is not running any container.

Zero-configuration bind mount support

One of the major issues users have today with Docker Desktop – especially in an enterprise environment – is the reliability of Windows file bind mounts. The current implementation relies on Samba Windows service, which may be deactivated, blocked by enterprise GPOs, blocked by 3rd party firewalls etc. Docker Desktop with WSL 2 will solve this whole category of issues by leveraging WSL features for implementing bind mounts of Windows files. It will provide an “it just works” experience, out of the box.

 

Technical Preview of Docker Desktop for WSL 2

Thanks to our collaboration with Microsoft, we are already hard at work on implementing our vision. We have written core functionalities to deploy an integration package, run the daemon and expose it to Windows processes, with support for bind mounts and port forwarding.

A technical preview of Docker Desktop for WSL 2 will be available for download in July. It will run side by side with the current version of Docker Desktop, so you can continue to work safely on your existing projects. If you are running the latest Windows Insider build, you will be able to experience this first hand. In the coming months, we will add more features until the WSL 2 architecture is used in Docker Desktop for everyone running a compatible version of Windows.

]]>