Checklist

  • You should be able to connect to linux.ms.mff.cuni.cz and 10.10.50.{7,9,10,11}:
    • Using key authentication
    • With a securely stored password-protected private key which you keep secret at all times
    • Without having to enter the password all the time
  • You should know most of the commands from the lab #0 and have an idea what each of them is doing, so that you know where to look.
  • From now on, the machines 10.10.50.{7,9,10,11} shall be known as c, d, e and f respectively. These machines will host our virtual machines (VMs).

Tasks for this week

This week, we will be playing with QEMU. QEMU is a key component in our virtualization stack: it uses KVM to control the execution of our VMs, and it emulates hardware for our VMs. You will learn more about the relationship between Linux, KVM and QEMU in the before class material for lecture #2. Here, we will focus on the their practical rather than theoretical aspects.

This week’s lab will prepare us for installation of Arch Linux on the lecture tomorrow.

Arch Linux, Archiso

Arch Linux is a lightweight Linux distribution focused on users who are willing to read the documentation and solve their own problems. Since we aim to create a lot of problems for ourselves in this course and then solve them ourselves, it’s a great fit! See Principles.

One of the well-known features of Arch Linux is that it doesn’t come with a graphical installer. Instead, you boot into a Live CD with Arch Linux (called Archiso) and install the system step-by-step from the command line.

  • The first step is to download Archiso, obviously. Please ssh into a random host and download Archiso. Hint: curl.
  • What are the advantages of this approach? What are the disadvantages?
  • What are the *.sig files? What is sha256sums.txt? What should you do with them?

Booting into Archiso

The simplest way to start a VM is just

qemu-system-x86_64 # That's it

Well, we can start a VM this way, but it’s not very useful. QEMU has a ton of options which control the many features of the VM, see qemu(1). For now, -cdrom is enough, since we want to insert Archiso CD into a virtual optical disk drive.

The next step is to use the -cdrom option to boot into the Archiso.

  • What is the meaning of x86_64?
  • What happens when you run just qemu-system-x86_64 with no options?
  • What happens when you boot into Archiso? Let’s compare this to what happens when you execute the same command on your laptop (or mine).

Detour: VNC + ssh port forwarding

  • What is VNC? (Google it!)
  • Why do we need VNC? Why does QEMU behave differently on the remote server?
  • What is ssh port forwarding? How does it apply to our situation? You may remember ssh port forwarding from the Introduction to Linux lab #10. Some of you didn’t attend this course, so let’s draw a picture.

Booting into Archiso (continued)

You should now see Arch Linux booting up. Or not? Hint: -m!

  • What is -m and why is it needed? What happens without it? Why?

OK, now you should see Arch Linux booting up. Notice anything? Hint: -accel!

  • What is -accel and why is it needed?
  • Quick’n’dirty benchmark of -accel tcg (default) and -accel kvm.

Attaching some storage

Let’s now boot into Archiso again, but this time, let’s also connect some storage devices too. We want our QEMU VM to see a virtual storage device attached. This virtual storage device needs to be backed by a real device (why?).

The easiest way to achieve that is to create a file on the host which will represent the storage space of the virtual storage device: when the guest read/writes data from/to the virtual storage device, those operations will be translated by QEMU into reads from/writes to the file.

We can create a backing file for the virtual storage device with dd:

dd if=/dev/zero of=img.raw bs=1G count=1 # Let's watch iotop
  • What does this do (dd(1)). What is /dev/zero?
  • What are /dev/null, /dev/random and /dev/urandom?
  • What happens when you dd if=/dev/null ... instead of dd if=/dev/zero ...? Why?

Start the VM again, this time with a virtual drive attached which is backed with this file (qemu(1), Google).

  • What are the correct QEMU flags to do this?

Let’s do this better: fallocate(1) the file instead of dd-ing it.

  • What is the difference between dd and fallocate?
  • What advantages does fallocate offer?

There’s one more issue related to Copy-on-Write (CoW) to deal with. /dev/sdb1 is Btrfs, which is a CoW file system. Our VM disk image file will receive lots of writes—which, from the perspective of the hypervisor, are random-access—resulting in significant fragmentation and deteriorated performance.

Thankfully, Btrfs allows us to disable CoW for directories and even for single files. This means we can have the data blocks of our VM disk images updated in-place. This comes with some drawbacks (for example, disabling CoW also disables checksums for the file), but since we will run Btrfs on top of the image file (in the VM) anyway, this is not an issue.

  • How do you disable CoW for a file/directory on Btrfs? Hint: chattr!
  • Create an 8 GiB fallocate-ed image file with CoW disabled.

Note on qcow2: QEMU also supports qcow2 (QEMU Copy-on-Write v2) images. The main selling point of qcow2 is that you don’t have to allocate the entire file upfront (dd-style) even if your host filesystem does not support sparse files (our Btrfs does). Besides that, qcow2 supports snapshots, image overlay, zlib compression and not too secure AES-based encryption (LUKS is superior). We don’t need these features, so let’s keep things simple with raw images.

Partitioning the virtual disk

We can now boot Archiso again and start partitioning our virtual disk—after all, that will be one important step of the installation process next week.

All of the following operations are destructive and they must happen in the VM only. Please make sure you’re working in the VM, not on your machine, or you will lose data!

  • Upon booting into the VM, you should see an 8 GiB drive /dev/sda attached. How do you verify this?
  • Use fdisk(8) to:
    • Create a new GUID partition table
    • Create four partitions, roughly 2 GiB each
    • Write the partition table to the virtual disk—make sure you work in the VM.

The partition table should look something like this:

Command (m for help): p
Disk /dev/loop0: 8 GiB, 8589934592 bytes, 16777216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 17277DE8-F761-524C-A768-54E330127ACA

Device          Start      End Sectors Size Type
/dev/sda1     2048  4196351 4194304   2G Linux filesystem
/dev/sda2  4196352  8390655 4194304   2G Linux filesystem
/dev/sda3  8390656 12584959 4194304   2G Linux filesystem
/dev/sda4 12584960 16775167 4190208   2G Linux filesystem

Creating file systems

Let’s create four different file systems on the four partitions. One of them should be Btrfs, choose any other three file systems from filesystems(5).

  • Create the four file systems on the four partitions.
  • Create four mount points and mount the four file systems. Create some files in each, then unmount.
  • Restart your VM and remount. You should see your files.

Homework

In the end, there was no homework from this lab.