This post gives a general introduction with focus on the new version WSL 2. It provides some basic usage principles and outlines some advantages and disadvantages. In future posts to come, I will evaluate using WSL 2 as a software developer and especially to run Docker on Windows.

Windows Subsystem for Linux (WSL) adds Linux functionality, including command-line tools, utilities, and applications to Windows. It is the successor to the Microsoft Windows Services for UNIX. Technically it allows running unmodified L64 binaries on a Windows machine. You can also choose between the well-known user distributions directly from the Microsoft Store and then directly run bash and use tools like ssh and git.

The goal is to provide a seamless integration and interoperability of a Linux environment on a Windows machine especially for software development. This is needed due to the fact that many applications in web development run natively on Linux.

Adding Linux utilities to Windows was not invented by WSL. There are tools like Cygwin and Windows’ own ssh client. Using git on Windows is also possible with git bash. However, WSL tries to provide a more complete and integrated solution. WSL 2 adds a full Linux environment including an automated sharing of the file systems, environment variables and network resources. In addition, it allows interoperability from Powershell and Bash. You can actually choose from the most common Linux distributions yourself depending on your personal preferences. WSL feels more like a hybrid of a command-line tool and a full Linux VM, which was kind of the intention I suppose. The two different versions offer the same usability. However, their architecture generally differs.

From WSL 1 to WSL 2

WSL 1 and the current version WSL 2 have a major difference in their architecture. WSL 1 adds a Linux compatibility-interface, ’emulating’ a Linux system that you can work with. It supplies all the things the Linux user space needs to run. However, all system calls (Linux ABIs) will still be handled by the Windows NT kernel by converting them in a translation layer. This leads to longer response times for file I/O and a full Linux functionality cannot be offered. This is because Linux and Windows work fundamentally different in certain areas where a proper translation is not possible. In addition, new functionality on the Linux side needs some time to be also available within WSL 1 as it needs to be implemented first.

WSL2 changed this approach and switched to a virtualization approach instead. A full Linux kernel is virtualized in the so-called light-weight utility VM based on Hyper-V. Its design is based on the efforts made for Linux containers on Windows (LCOW) to minimize overhead and provide highest possible integration. The kernel itself is managed by Microsoft and especially optimized for the use within WSL 2. And all that is Open Source!

With a real Linux kernel, WSL 2 now offers full system call compatibility. Which means that you can use everything from a ‘real’ Linux installation. The WSL file system also changed to an ext4 format on a virtual hard disk (VHD). This allows way faster File IO within the WSL 2 file system as there is no need any more for a translation layer. Except for the VHD part, it is close to native performance as its an actual Linux Kernel and file system. This however does not apply if it involves the Windows file system, see the limits. Those two benefits were the main drivers behind this architectural change.

Just a quick visualization on how much quicker it is.

The following figure gives an outline of the architecture.

WSL 2 architecture: The operation of the light-weight utility VM happens in the background. It contains a full Linux Kernel and the installed Linux distributions. The interoperability between the filesystems is achieved with 9P protocol servers on both sides. WSL allows to invoke Linux application from Windows and vice versa due to shared environment variables.

Switching to a VM solution comes with the typical downsides. You now have two different machines running and a flawless integration highly depends on its configuration. Here comes the distinction of WSL 2 from a ‘normal’ VM into play. Installation and usage of WSL 2 doesn’t require you to touch any VM yourself. Its configuration is completely handled in the background. The focus is on the tightest possible interplay between both systems. You should basically not see that there are two different systems running. If this was achieved, I want to address in the following.  

As already mentioned, Hyper-V is used for the virtualization in WSL 2. This still comes with the disadvantage of not being able to use third-party virtualization. As WSL 1 does not need Hyper-V it still has its use cases.

Quickstart: How to get WSL 2 running

To start using WSL 2 you need Windows 10!

• For x64 systems: Version 1903 or higher, with Build 18362 or higher.
• For ARM64 systems: Version 2004 or higher, with Build 19041 or higher.

I followed this installation guide to get everything running so I won’t repeat it here. WSL 2 is quite new and still rapidly evolving leaving you with a more automated installation procedure at the time I wrote this blog post. Just run

wsl.exe –install

To actually use WSL you need to install your Linux distribution of choice from the Microsoft Store (this should also be possible from the command line in the future).  For a fresh installation WSL 2 will already be your default version. If this is not the case you can do:

wsl --set-default-version2

If you chose Ubuntu:

wsl -l -v

should give you

  NAME      STATE     VERSION
* UBUNTU    Stopped   2

To run a command within the WSL environment, you have two options. Run

wsl

to open up Bash in your terminal and start working as if you were on a Linux machine. Or you use WSL as a command-line tool adding it in front of each command.

wsl lsb_release -a

You may have not even noticed that you just started a VM, as booting it only took ~1 second. This is because the Linux Kernel is directly loaded into the VM’s address space without running any bootloader.

Basic commands

WSL 2 does not only allow you to run a single Linux distribution, but several of them in parallel. All of them will run within the same VM sharing the Linux Kernel. This comes in handy if you want to run some existing script which depends on the specific Linux distribution, like the installed package manager. You can simply decide on which Linux distribution you want to work

wsl -d <distribution_name>

or you change the default wsl distribution (which is invoked using wsl):

wsl --setdefault <distribution_name>
...
wsl sudo apt install cowsay

As the actual VM is already running in the background, switching between distributions is also fast.

If you don’t want to stop using WSL 1 you can easily convert between both versions

wsl --set-version Ubuntu 1  #2 for WSL 2

One thing to keep in mind when you exit your terminal, the current distro is still running to keep background processes alive. Running distros have to be stopped manually

wsl --terminate <distribution_name>

To kill all distros at the same time, do

wsl --shutdown

Configuration

The following considerations are related to the issues that WSL 2 had from the beginning. Again, WSL 2 is evolving and Microsoft lately fixed some of these issues, while others result from the way it’s build.

Restrict Memory usage

WSL 2 can by default consume up to 80% of your memory. To prevent this, you can restrict it globally. Create a .wslconfig file located in your users home directory (Windows) and add the following lines (or edit to your liking).

[wsl2]
memory=4GB # Limits VM memory in WSL 2 to 4 GB
processors=2# Makes the WSL 2 VM use two virtual processors
...

Memory consumption was quite an issue in the early days of WSL 2, where it sometimes did not free memory after running a process. This has been improved and reclaiming it manually is unnecessary in most cases. However, to allow a ‘faster’ user experience the Linux Kernel makes use of several caches. The page cache for example allows faster file system performance. It will only be freed when you closed all your terminal windows. To free it manually, run the following command:

sudo echo 1 > /proc/sys/vm/drop_caches

Disk space consumption

The light-weight utility VM and the Kernel is optimized for low overhead. However, especially when you decide to have several Linux distribution in parallel you should care about the disk space usage. By default, the Linux disk image will be installed on your C: drive. If you want to move it you can export and import it again.

wsl --export Ubuntu E:\wsl\ubuntu.tar

wsl --unregister Ubuntu

wsl --import Ubuntu E:\wsl\ E:\wsl\ubuntu.tar

Your ext4.vhdx virtual disk image is now located in E:\wsl\Ubuntu. This also simplifies backup and allows to clone your customized Linux distribution.

However, you have the common issue when using any VM using a virtual disk. If you delete something within its file system, the host can not reclaim the disk space automatically. To compact the image you can do the following (as admin from Powershell)

optimize-vhd -Path .\ext4.vhdx -Mode full

To get to the default path of your distributions

cd $env:LOCALAPPDATA\Packages\<distro_name_xxx>\LocalState

However, this is a Hyper-V feature. For Windows Home you can use diskpart.

Interoperability of WSL2 and the Windows host

The flawless usability in both directions is one of the key features of WSL. Running wsl from your Windows user folder will directly open up a shell at /mnt/c/Users/<your_username> within your Linux distro. Technically this is enabled via the 9P filesystem bridge which mounts your C: drive directly at /mnt/c/.

There are two options to access the WSL file system from Windows. You can either navigate to the network path \\wsl$\<your_distro_name> via Windows Explorer or you use the ‘WSL power’ to start e.g. Windows Explorer directly from within WSL 2.

explorer.exe .

This will run Windows Explorer opened in the current directory. This works as WSL shares environment variables including PATH with the Windows host, making path translations possible. So now you can just copy/paste and drag & drop with Windows Explorer within your Linux file system. This works for other Windows applications as well.

Tip: Instead of using \\wsl$\<your_distro_name> from the Windows side, you can map it to a network drive.

Integrations go even further like automated port forwarding, network sharing etc. The new WSL 2 architecture especially has its benefits for Docker on Windows. That’s why it’s now the default backend used by Docker Desktop. However, covering it here would just blow up this post. Hence, I will address it in my next post. Check here if new posts are already online.

The current limits of WSL 2

As introduced in the previous section, you can access both file systems from either side and it’s also possible to invoke Linux applications from Windows and vice versa. However, this has its limits mainly regarding performance. The file systems differ and appear as remotes to each other. For example, if you use grep on your Windows file system it will be way slower compared to its native speed. That’s one reason for keeping a WSL 1 Linux distribution available, as it does not suffer from the same effect.

This however is the major issue concerning the integration of WSL 2 into your Windows developer environment. If you want to compile, build and run a big project from within WSL 2, make sure that all your code is located inside the Linux file system as well! However, working with an IDE like Eclipse or IntelliJ from Windows on a remote code base is currently no valid option, especially when we talk about a big Java project! This is because they highly rely on file system events and those are not properly propagated via the 9P protocol. However, this may change in the future.

Currently you have the option to install the Linux version of your IDE within WSL. Since WSL does not include support for GUI apps so far, you need to add it yourself. This can be done via X-Server or a Remote Desktop connection. Microsoft promised to add GUI support until the end of 2020.

The only exception is Visual Studio Code. Its architecture is explicitly designed to work with remote servers. VS Code provides an extension to use WSL 2 as its remote server while the GUI basically represents the client. Each command is actually handled from within the WSL 2 environment and does therefore not suffer the same way.

More about using WSL 2 in your developer environment with VS Code and some workaround using IntelliJ or Eclipse will be treated in a blog post to come. So stay tuned and check here if it’s already online!

Summary

Windows Subsystem for Linux 2 offers full system call compatibility as it contains a ‘full’ Linux Kernel. Interoperability and high integration are available out of the box. You can run all your favorite Linux tools and they are running fast. Due to the two different file systems, a 100% seamless integration into any development environment could not be achieved. However, as WSL matures this will become better. If VS Code is your IDE of choice and you work a lot with Docker, it may be worth to try it out even now.