Pacman cheat-sheet

Since we didn’t get to discuss Arch’s package manager during the lecture, here’s a mini-cheat-sheet:

  • Full system upgrade (FSU = install the latest versions of all installed packages): pacman -Syu
  • To install a package: pacman -Syu some-package
  • To show information about an installed package: pacman -Qi some-package
    • About a remote package: pacman -Si some-package
  • To remove a package: pacman -Rs some-package (add -n if you don’t want to keep backups of configs etc.)
  • To find which package owns a particular file in the system: pacman -Qo /path/to/some/file
  • To list all files which a package provides: pacman -Ql some-package
  • To figure out which package to install to obtain a file: pacman -Fy /path/to/some/file (this is really useful to find out which package provides a particular program, for example pacman -Fy nvim)
  • To clean package cache pacman -Sc (or pacman -Scc to clean it really hard)

By know you should know where to find the rest :-).

Warning about partial updates

Do not do the following, unless you know exactly what you are doing:

pacman -Sy some-package # Wrong!

and neither

pacman -Sy
# ... time goes by, you forget you did pacman -Sy without -u ...
pacman -S some-package # Wrong!

Rationale: Arch Linux is a rolling-release distro which explicitly does not support so-called partial updates: if you want to install a new package, you have two options:

  • Use pacman -Syu some-package, always (recommended). This fetches an up-to-date copy of the package database, updates all installed packages and then it proceeds to install some-package. It’s really hard to mess this up.

  • Use pacman -S some-package without preceding pacman -Sy. This option is discouraged, since you have to be sure you didn’t run pacman -Sy before. Furthermore, this will not work if your package database refers to an old version of some-package which isn’t available for download from your selected mirrors anymore (happens if you don’t update for a long time).

Partial updates may easily break your system. A real-world scenario:

  • You ssh into a machine to which you have no other means of access
  • You decide to install some-package which depends on libldap>=x.y
  • You run pacman -S some-package and get a bunch of errors since you didn’t update for a long time.
  • You don’t feel like doing a full system upgrade, so you just fetch the new package database and install a fresh version of some-package (i.e., not the version of some-package which the rest of your system assumes—the version of some-package which was compatible with your system was the one which was too old to be downloaded).
  • You run sudo pacman -Sy && sudo pacman -S some-package, what could go wrong?
  • Since some-package depends on libldap>=x.y, libldap is also updated to x.y as part of the update.
  • But sudo also depends on libldap, and when it starts, tries to load an old version of libldap which you no longer have installed!
  • So you run sudo pacman -U ... to downgrade libldap and… do you see the problem?
sudo: error in /etc/sudo.conf, line 0 while loading plugin “sudoers_policy”
sudo: unable to load /usr/lib/sudo/sudoers.so: libldap-2.4.so.2: cannot open shared object file: No such file or directory
sudo: fatal error, unable to load plugins
  • If you cannot su to root, you’re in trouble. You broke your only way of switching to root and need root to fix the machine.
  • I’m sure you wonder: why didn’t Pacman stop me? Well, the vast majority of packages in Arch Linux do not have these version constraints attached (>=x.y, <x.y, etc.). For example, sudo does not specify which version of libldap it needs, just that it needs libldap. As long as you have fresh sudo and fresh libldap installed, the packages will work. But if you decide to update one without the other, Pacman won’t stop you (since you don’t break any version constraints) and you will break something (sudo in this case).
  • This is not an accident. Pacman does provide version constraints, and a small number of packages do use it. It’s a design choice—you don’t care about old versions of programs (since that’s an incredible amount of work to do right) and instead you make sure that the latest versions of everything work well together.
  • In practice, this doesn’t happen very often, unless you leave your system out of date for months. But it’s a good idea to be careful anyway (or have a backup plan in case you break your system with an intentional partial update—don’t get angry with Pacman.)
  • A partial update also results when you interrupt a full system upgrade.

Tasks for this week

By now, you should have a single VM up and running. Our task for this week is to perform some basic system configuration—tasks which you are likely to perform on any Arch Linux server, whatever its responsibility might be. In fact, if you were setting up a laptop, many of the steps would still be needed.

Please read first: the ideas behind the assignments

This should hopefully address some of the feedback for the prior homework:

  • The assignments may seem a bit vague—that’s on purpose. Figuring out what exactly needs to be done, how and why is the most important part of each task. If you think however that some vital piece information is missing from an assignment, please let us know.
  • This way, you learn to navigate complex topics, read complicated technical literature, sharpen your google-fu and learn to fix various common problems.
  • With each task, you are just given some pointers. Follow the pointers. If we were to give you exact, precise instructions, the class would be pointless. Anybody can copy and paste commands to the terminal.
  • All of the tasks have a really simple solution. If your solution seems complex, read the linked materials again, google around, draw pictures and think hard.
  • If you are stuck, ask. We are still really happy to help. You can catch us after labs/lectures, or just write us an email.
  • The points awarded for your solutions will reflect the quality of your work. It’s not enough that “it works” if it only works by accident or when security is compromised. Also if your solution is super sloppy, we may get really sloppy awarding points :-).
  • This is going to be the default going forward.

Network configuration

  • The next lecture will be dedicated to networking, but we need networking now.
  • In the following, X = Y + n, where:
    • Y = 10 on c,
    • Y = 50 on d,
    • Y = 90 on e,
    • Y = 130 on f,
    • n is meta/n from your repository.
  • Your MAC address ($macaddr) is 52:54:00:36:H:01, where H = hex(X)
  • Attach a virtual Network Interface Controller (NIC) to your VM with the following command-line option (this will be explained next week):
-nic vde,sock=/tmp/vde-backbone.sock,mac=$macaddr
  • Configure systemd-networkd to set up the interface with the following parameters:
    • Your IP address is 10.0.0.X/24
    • Gateway is 10.0.0.1
    • DNS is 10.10.50.8
  • Hint: all information you need are on the linked systemd-networkd page. To get this done, you need to write < 10 lines of a config file and run ~5–10 commands, that’s it.
  • Hint: refer to notes about systemd from lecture #3.
  • Make sure that systemd-networkd starts after a reboot, so that your VM goes online automatically.

Installation clean-up

  • During the installation process, we skipped certain steps (locale configuration, timezone setup, hostname, etc.).
  • Use gw as your machine’s hostname. (This VM will be a gateway to your network.)
  • Refer to the installation guide and make sure to configure the missing bits.
  • Read General recommendations. Note that only a small subset is relevant for server VMs.
  • Perform full system upgrade (refer to Pacman cheat-sheet above)
  • During the next labs, we’ll clone this VM several times, so it pays off to take care and do this right.

Create new users

Create 2 new users:

  • $login, the user which you will use to ssh into the machine. Use a secure password, since other students may otherwise log in to your VMs through VNC.
  • dcepelik for me to be able to access your VMs. Please use the contents of meta/psk in your repo as the password for this account.

OpenSSH

  • Install and configure OpenSSH server
  • Make sure it’s started on boot
  • Verify that you can ssh into your VM from your workstation. Use the hypervisor as ProxyJump. (For example, I can ssh mff-adm-vm-gw to my gateway VM from my laptop, using key authentication, with an encrypted key, without having to enter the password—this should sound familiar.)
  • Make sure I can connect as dcepelik to your VM using ssh + key authentication. I will use the following public key (that is, I will not ssh-copy-id first.)
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKmfq/dgVzxo3gzI6Aegmqy6gdy1exGZ+T/gg4u7Xge5 d+nswi106-2022@dcepelik.cz
  • Do not allow password authentication for root over ssh.
  • Don’t use VNC to manage your VM anymore, use ssh instead. After all, it’s much more convenient. But please keep VNC enabled as we still need it to grade the Tmux basics task from homework #2.
  • Only connect through VNC when you actually need it, e.g. when your VM does not boot or sshd does not start.
  • When using ssh, log in as $login, not as root.

Configure sudo

  • Make sure users $login and dcepelik can use sudo to execute any command (must enter password).

Configure NTP client

  • Pick any reasonable NTP daemon, install it and configure it to keep your machine’s time in sync with internet time servers.

Homework

This homework has got a two-week deadline (strict):

  • Thursday 2022-11-03 9:00 Prague time for the Thursday group
  • Monday 2022-11-07 9:00 Prague time for the Monday group

Please try to get it done during the first week. As usual, if anything is unclear, don’t hesitate to ask.

Finish this week’s tasks

  • Finish all tasks from this week.
  • If you deviated from the requested setup in any way, or if you didn’t get something to work, or if you have any questions (and don’t want to ask directly), please write it to hw/03/00-setup in your repository. If you have nothing to say, please still create an empty file with this name. Push hw/03/00-setup to your repo once you want us to check your homework. Thanks!
  • Please make sure you VM is running (by the deadline at the latest) so that we can grade your homework. Run QEMU in tmux!
  • Bonus points to be awarded where appropriate, especially where it is obvious that you have taken extra care to configure something nicely (please mention this in 03/00-setup, otherwise we might not notice)
  • (100+30 points)

hw/03/01-ssh-fingerprints-gw

  • Please provide the fingerprints of keys used by sshd running on gw in 03/01-ssh-fingerprint-gw, so that I know for sure that I’m connecting to your VM and not someone else’s.
  • (10 points)

hw/03/02-feedback

  • If you have any valuable feedback, please do provide it here.
  • Points are only awarded for feedback which is actionable and can be used to improve the quality of the course.
  • Any constructive criticism is appreciated (and won’t be weaponized).

(Bonus) hw/03/05-vmbackup

Write a script which maintains a set of backups (copies) of your VM’s image. Features:

  • Create a new backup of an image
  • Delete all but n most recent backups of an image (keep at most n most recent backups, delete everything else)
  • List all backups of an image
  • Dry run mode which does not create nor delete backups, but prints what the script would have done
  • Other standard flags which all well-behaved commands have

To create a backup, just use cp:

# --reflink, --sparse: will be explained soon
cp -a --reflink=never --sparse=always -- image.raw backup

(The script could be used equally well to back up any other files, it doesn’t care about the fact that it’s backing up a QEMU VM image.)

Please note that it only makes sense to back up the image when no VM is using it, otherwise the backup may be inconsistent. This however is external to the script, and the script does not care.

You can assume that file names don’t contain newlines (\n), however your script must correctly handle all other characters allowed in a UNIX filename. We allow newlines since it’s not possible to write a truly portable script which handles newlines correctly without resorting to horrible hacks.

Other than that, you are free to design the script any way you like. Evaluation criteria in descending order of importance:

  • Correctness + simplicity
  • Design + implementation
  • Shellcheck with the usual flags is quiet
  • Portability (POSIX sh, portable flags of all commands used)

Please don’t test the script with actual VM images. Use an empty file (touch image.raw) for experiments. Once the script works, feel free to use it to maintain a reasonable set of backups on the hypervisors. We’ll switch to a better backup strategy (snapshots) soon.

My solution has < 80 SLOC (including usage). Can you beat that?

(0+30 points)

(Total = 110+60 points)

Don’t forget to git push all your changes! Also, make sure that VM still works by the deadline—otherwise we have no way of grading your work.