Labs: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14.

This is the second lab devoted to a mixture of maintenance tasks. It contains the bare minimum needed for management of your Linux machine. And it also contains few extras that might be useful for your shell scripts.

The things covered in these two labs require that you have your own Linux installation as they require superuser privileges. That will not work on the shared machines in IMPAKT.

Do not forget that the Before class reading is mandatory and there is a quiz that you are supposed to complete before coming to the labs.

Before class reading

Storage management

Files resides on file systems that are the structures on the actual block devices (typically, disks). Working with file systems and block devices is necessary when installing a new system, rescuing from a broken device, or simply checking available free space.

Mounts and mount-points

Recall that each file system (that we want to access) is accessible as a directory somewhere (compared to a drive letter in other systems, for example). When we can access /dev/sda3 under /home we say that /dev/sda3 is mounted under /home, /home is then called the mount point, /dev/sda3 is often called a volume.

Most devices are mounted automatically during boot. This includes / (root) where the system is as well as /home where your data resides. Note that /dev, /proc (among others) are special file systems that are mounted to these locations. Hence, the file /proc/uptime does not physically exist (i.e., there is no disk block with its content anywhere on your hard drive) at all.

The file systems that are mounted during boot are listed in /etc/fstab. You will rarely need to change this file on your laptop and this file was created for you during installation. Note that it contains volume identification (such as path to the partition), the mount point and some extra options.

When you plug-in a removable USB drive, your desktop environment will typically mount it automatically. Mounting it manually is also possible using the mount utility. However, mount has to be run under root to work (this thread explains several aspects why mounting a volume could be a security risk). Therefore, you need to play with this on your installations where you can become root. It will not work on any of the shared machines.

Technical note: the above text may seem contradictory, as mount requires root password yet your desktop environment (DE) may mount the drive automatically without asking for any password. Internally, your DE does not call mount but uses a different program (today, often based on Udisks and Polkit) that together verify that the mounted device is actually a removable one and that the user is a local one (i.e., it will not work over SSH). This program has set-uid bit set and thus runs with root privileges automatically (the concrete implementation may slightly differ).

To test the manual mounting, plug-in your USB device and unmount it in your GUI if it was mounted automatically (note that the usual path the device is mounted is somewhere under /media).

Your USB will probably be available as /dev/sdb1 or /dev/sda1 depending what kind of disk you have (consult the following section about lsblk to view list of drives).

Mounting disks is not limited to physical drives only. We will talk about disk images in the next section but there are other options, too. It is possible to mount a network drive (e.g., NFS or AFS used in MFF labs) or even create a network block device and then mount it.

Working with disk images

Linux has built-in support for working with disk images. That is, with files with content mirroring a real disk drive. As a matter of fact, you probably already worked with them when you set up Linux in a virtual machine or when you downloaded the USB disk image at the beginning of the semester.

Linux allows you to mount such image as if it was a real physical drive and modify the files on it. That is essential for the following areas:

  • Developing and debugging file systems (rare)
  • Extracting files from virtual machine hard drives
  • Recovering data from damaged drives (rare, but priceless)

Note that for recovering data from damaged drives, the typical approach is to try to copy the data from the file as-is on the lowest level possible (typically, copying the raw bytes without interpreting them as a file system or actual files). Only after you recover the disk (mirror) image, you run the actual recovery tools on the image. That prevents further damage to the hard drive and gives you plenty of time for the actual recovery.

In all cases, to mount the disk image we need to tell the system to access the file in the same way as it accesses other block devices (recall /dev/sda1 from the example above).

Services (and daemons too)

In the context of an operating system, the term service usually refers to any program that is running on the background (typically, no GUI, stdin from /dev/null) and provides some kind of service to other programs.

A typical example can be a printing service that takes care of printer discovery and provides end-user applications with list of printers (i.e. the end-user applications do not need to make the discovery themselves). Another example is a web server: it provides files over the HTTP protocol to web browsers.

In the world of Unix systems, such programs are often called daemons (this probably comes from ancient Greek mythology where daemon is a being working in the background), traditionally names of such programs end with the letter d. For example, the popular Apache web server is actually launched as a program httpd and the SSH server is running as sshd.

Daemons operate differently from normal programs. When started, they read their configuration (typically from a file under /etc/), start and listen for requests (imagine a web server listening on port 80). Changing their behavior is usually done by changing their configuration file and restarting them. Because they are started in background, they do not have access to an interactive stdin and the restart (or shutdown) is performed via signals.

Because the need to restart a running daemon is quite common (and sending signals is not very straightforward as you need to know the PID), there are special programs that are able to find the PID for you and send the right signal. We can call them control scripts and for some daemons you will find files daemond (with the actual daemon code) and daemonctl for controlling it.

Unified daemon control

Because the above principles are virtually the same for all daemons, there usually exists a set of scripts unifying this behavior. So, instead of calling a specific daemonctl, the distribution will typically offer a special command where you can control any daemon. Usually, one will use something along the following lines.

service [start|stop|restart] daemon

Currently, the most often used software for this task is called systemctl.

About logging

Most services provide so-called logs. There they record every significant action they performed.

For example, a web server typically logs which pages it served together with information about the client.

Usually, for each service you can specify how detailed the logging shall be. Debugging a configuration issue requires a more detailed level, on a production server you usually limit the amount of logged information to minimum for performance reasons.

Before class quiz

The quiz file is available in the 10 folder of this GitLab project.

Copy the right language mutation into your project as 10/before.md (i.e., you will need to rename the file).

The questions and answers are part of that file, fill in the answers in between the **[A1]** and **[/A1]** markers.

The before-10 pipeline on GitLab will test that your answers are in the correct format. It does not check for actual correctness (for obvious reasons).

Submit your before-class quiz before start of lab 10.

File management

Because you already know ranger and mc and ls/cd and plenty of file-modification utilities, this section will be extremely short. It will only strengthen your knowledge about file archiving as other utilities were already covered in previous labs.

Archiving and compression

Archiving on Linux systems typically refers to merging multiple files into one (for easier transfer) and compression of this file (to save space). Sometimes, only the first step (i.e., merging) is considered archiving.

While these two actions are usually performed together, Linux keeps the distinction as it allows combination of the right tools and formats for each part of the job. Note that on other systems where the ZIP file is the preferred format, these actions are blended into one.

The most widely used program for archiving is tar. Originally, its primary purpose was archiving on tapes, hence the name: tape archiver. It is always run with an option specifying the mode of operation:

  • -c to create a new archive from existing files,
  • -x to extract files from the archive,
  • -t to print the table of files inside the archive.

The name of the archive is given via the -f option; if no name is specified, the archive is read from standard input or written to standard output.

As usually, the -v option increases verbosity. For example, tar -cv prints names of files added to the archive, tar -cvv prints also file attributes (like ls -l). Plain tar -t prints only file names, tar -tv prints also file attributes. (Everything is printed to stderr, so that stdout can be still used for the archive.)

An uncompressed archive can be created this way:

tar -cf archive.tar.gz dir_to_archive/

A compressed archive can be created by piping the output of tar to gzip: `

tar -c dir_to_archive/ | gzip >archive.tar.gz

As this is very frequent, tar supports a -z switch, which automatically calls gzip, so that you can write:

tar -czf archive.tar.gz dir_to_archive/

tar has further switches for other (de)compression programs: bzip2, xz, etc.. Most importantly, the -a switch chooses the (de)compression program according to the name of the archive file.

If you want to compress a single file, plain gzip without tar is often used. Some tools or APIs can even process gzip-compressed files transparently.

To unpack an archive, you can again pipe gzip -d (decompress) to tar, or use -z as follows:

tar -xzf archive.tar.gz

Note that like many other file-system related programs, tar will overwrite existing files without any warning.

We recommend to install atool as a generic wrapper around tar, gzip, unzip and plenty of other utilities to simplify working with archives. For example:

apack archive.tar.gz dir_to_archive/
aunpack archive.tar.gz

Note that atool will not overwrite existing files by default (which is another very good reason why to use it).

Note that it is a good practice to always archive a single directory. That way, user that unpacks your archive will not have your files scattered in the current directory but neatly prepared in a single new directory.

To view the list of files inside an archive, you can execute als.

Mounting disks manually

sudo mkdir /mnt/flash
sudo mount /dev/sdb1 /mnt/flash

Your data shall be visible under /mnt/flash.

To unmount, run the following command.

sudo umount /mnt/flash

Note that running mount without any arguments prints list of currently active mounts. For this, root privileges are not required.

Specifying volumes

So far, we always used the name of the block device (e.g., /dev/sdb1) to specify the volume. While this is trivial on small systems, it can be incredibly confusing on larger ones – device names depend on the order in which the system discovered the disks. This order can vary between boots and it is even less stable with removable drives. You do not want to let a randomly connected USB flash disk render your machine non-bootable :-).

A more stable way is to refer to block devices using symlinks named after the physical location in the system. For example, /dev/disk/by-path/pci-0000:03:00.1-ata-6-part1 refers to partition 1 of a disk connected to port 6 of a SATA controller which resides as device 00.1 on PCI bus 0000:03.

In most cases, it is even better to describe the partition by its contents. Most filesystems have a UUID (universally unique identifier, a 128-bit number, usually randomly generated) and often also a disk label (a short textual name). You can run lsblk -f to view UUIDs and labels of all partitions and then call mount with UUID=number or LABEL=name instead of the block device name. Your /etc/fstab will likely refer to your volumes in one of these ways.

Mounting disk images

Disk images can be mounted in almost the same way as block devices, you only have to add the -o loop option to mount.

Recall that mount requires sudo privileges hence you need to execute the following example on your own machine, not on any of the shared ones.

To try that, you can download this FAT image and mount it.

sudo mkdir /mnt/photos-fat
sudo mount -o loop photos.fat.img /mnt/photos-fat
... (work with files in /mnt/photos-fat)
sudo umount /mnt/photos-fat

Alternatively, you can run udisksctl loop-setup to add the disk image as a removable drive that could be automatically mounted in your desktop:

# Using udisksctl and auto-mounting in GUI
udisksctl loop-setup -f fat.img
# This will probably print /dev/loop0 but it can have a different number
# Now mount it in GUI (might happen completely automatically)
... (work with files in /run/media/$(whoami)/07C5-2DF8/)
udisksctl loop-delete -b /dev/loop0

Repairing corrupted disks

The primary Linux tool for fixing broken volumes is called fsck (filesystem check). Actually, the fsck command is a simple wrapper, which selects the right implementation according to file system type. For the ext2/ext3/ext4 family of Linux file systems, the implementation is called e2fsck. It can be more useful to call e2fsck directly, since the more specialized options are not passed through the general fsck.

As we already mentioned, it is safer to work on a copy of the volume, especially if you suspect that the volume is seriously broken. This way, you do not risk breaking it even more. This can be quite demanding in terms of a disk space: in the end it all comes down to money – are the data worth more than buying an extra disk or even bringing it completely to a professional company focusing on this sort of work.

Alternatively, you can run e2fsck -n first, which only checks for errors, and judge their seriousness yourself.

Sometimes, the disk is too broken for fsck to repair it. (In fact, this happens rarely with ext filesystems – we have witnessed successful repairs of disks whose first 10 GB were completely rewritten. But on DOS/Windows filesystems like vfat and ntfs, automated repairs are less successful.)

Even if this happens, there is still a good chance of recovering many files. Fortunately, if the disk was not too full, most files were stored continuously. So we can use a simple program scanning the whole image for signatures of common file formats (recall, for example, how the GIF format looks like). Of course, this does not recover file names or the directory hierarchy.

The first program we will show is photorec (sudo dnf install testdisk). Before starting it, prepare an empty directory where to store the results.

It takes a single argument: the file image to scan. It then starts an interactive mode where you select where to store the recovered files and also guess on file system type (for most cases, it will be FAT or NTFS). Then it tries to recover the files. Nothing more, nothing less.

photorec is able to recover plenty of file formats including JPEG, MP3, ZIP files (this includes also ODT and DOCX) or even RTF files.

Another tool is recoverjpeg that focuses on photo recovery. Unlike photorec, recoverjpeg runs completely non-interactively and offers some extra parameters that allow you to fine-tune the recovery process.

recoverjpeg is not packaged for Fedora: you can try installing it manually or play with photorec only (and hope you will never need it).

Disk space usage utilities

The basic utility for checking available disk space is df (disk free).

Filesystem     1K-blocks    Used Available Use% Mounted on
devtmpfs         8174828       0   8174828   0% /dev
tmpfs            8193016       0   8193016   0% /dev/shm
tmpfs            3277208    1060   3276148   1% /run
/dev/sda3      494006272 7202800 484986880   2% /
tmpfs            8193020       4   8193016   1% /tmp
/dev/sda1        1038336  243188    795148  24% /boot

In the default execution (above), it uses one-kilobyte blocks. For a more readable output, run it with -BM or -BG (megas and gigas) or with -h to let it select the most suitable unit.

Do not confuse df with du which can be used to estimate file space usage. Typically, you would run du as du -sh DIR to print total space occupied by all files in DIR. You could use du -sh ~/* to print summaries for top-level directories in your $HOME. But be careful as it can take quite some time to scan everything.

Also, you can observe that the space usage reported by du is not equal to the sum of all file sizes. This happens because files are organized in blocks, so file sizes are typically rounded to a multiple of the block size. Besides that, directories also consume some space.

To see how volumes (partitions) are nested and which block devices are recognized by your kernel, you can use lsblk. On the shared machine, the following will appear:

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   480G  0 disk
├─sda1   8:1    0     1G  0 part /boot
├─sda2   8:2    0   7.9G  0 part [SWAP]
└─sda3   8:3    0 471.1G  0 part /

This shows that the machine has a 480G disk divided into three partitions: a tiny /boot for boostrapping the system, a 8G swap partition, and finally 470G left for system and user data. We are not using a separate volume for /home.

You can find many other output formats in the man page.

Inspecting and modifying volumes (partitions)

We will leave this topic to a more advanced course. If you wish to learn by yourself, you can start with the following utilities:

  • fdisk(8)
  • btrfs(8)
  • mdadm(8)
  • lvs(8), pvs(8) and related ones

SSH port forwarding

Generally, services provided by a machine should not be exposed over the network for random “security researchers” to play with. Therefore, a firewall is usually configured to control access to your machine from the network.

If a service should be provided only locally, it is even easier to let it listen on the loopback device only. This way, only local users (including users connected to the machine via SSH) can access it.

As an example, you will find that there is a web server listening on port 8080 of linux.ms.mff.cuni.cz. This web server is not available when you try to access it as linux.ms.mff.cuni.cz, but accessing it locally (when logged to linux.ms.mff.cuni.cz) works.

you@laptop$ curl http://linux.ms.mff.cuni.cz:8080                # Fails
you@laptop$ ssh linux.ms.mff.cuni.cz curl --silent http://localhost:8080  # Works

SSH can be used to create a secure tunnel, through which a subset of ports is forwarded to your machine. In essence, you will connect to a loopback device on your machine and SSH will forward that communication to the remote server and back, effectively making the remote port accessible.

The following command will make port 8888 behave as port 8080 on the remote machine. The 127.0.0.1 part refers to the loopback on the remote server (you can write localhost there, too.)

ssh -L 8888:127.0.0.1:8080 -N linux.ms.mff.cuni.cz

The -N makes this connection usable only for forwarding – use Ctrl-C to terminate it (without it, you will log-in to the remote machine, too).

Open http://localhost:8888 in your browser to check that you can see the same content as with the ssh linux.ms.mff.cuni.cz curl http://localhost:8080 command above.

You will often forward (remote) port N to (local) port N hence it is very easy to forgot about the proper order. However, the ordering of -L parameters is important and swithching the numbers (e.g. 8888:127.0.0.1:8080 instead of 8888:127.0.0.1:8080) will forward different ports (usually, you will learn about it pretty quickly, though). But do not worry if you are unable to remember it. That is why you have manual pages and even every-day users of Linux use them. It is not something to be ashamed or afraid of :-).

Reverse port forwarding

SSH allows to create also a so-called reverse forward.

It basically allows you to open a connection from the remote server to your local machine. Practically, you can setup a reverse port forward from your desktop you have at home to a machine in Rotunda, for example, and then use this reverse forward to connect from Rotunda back to your desktop.

This feature will work even if your machine is behind NAT, which makes direct connections from the outside impossible.

The following command sets the reverse port forward such that connecting to port 2222 on the remote machine (on loopback, i.e., on 127.0.0.1 or localhost) will be translated to connections to port 22 (ssh) on the local machine:

ssh -N -R 2222:127.0.0.1:22 u-plN.ms.mff.cuni.cz

When trying this, ensure that your sshd daemon is running (see the Systemd section below for the explanation why sudo systemctl start sshd might be needed) and use a different port than 2222 to prevent collisions.

In order to connect to your desktop via this port forward, you have to do so from rotunda lab via ssh -p 2222 your-desktop-login@localhost as the connection is only bound to the loopback interface, not to the actual network adapter available on lab computers. (Actually, ssh allows to bind the port forward on the public IP address, but this is often disabled by the administrator for security reasons.)

Network Manager

There are several ways how to configure networking in Linux. Server admins often prefer to use the bare ip command; on desktops most distributions today use the NetworkManager, so we will show it here too. Note that the ArchLinux Wiki page about NetworkManager contains a lot of information, too.

NetworkManager has a GUI (you probably used its applet without knowing about it), a TUI (which can be run with nmtui), and finally a CLI. We will (for obvious reasons) focus on the command-line interface here. Without parameters, nmcli will display information about current connections:

wlp58s0: connected to TP-Link_1CE4
        "Intel 8265 / 8275"
        wifi (iwlwifi), 44:03:2C:7F:0F:76, hw, mtu 1500
        ip4 default
        inet4 192.168.0.105/24
        route4 0.0.0.0/0
        route4 192.168.0.0/24
        inet6 fe80::9ba5:fc4b:96e1:f281/64
        route6 fe80::/64
        route6 ff00::/8

p2p-dev-wlp58s0: disconnected
        "p2p-dev-wlp58s0"
        wifi-p2p, hw

enp0s31f6: unavailable
        "Intel Ethernet"
        ethernet (e1000e), 54:E1:AD:9F:DB:36, hw, mtu 1500

vboxnet0: unmanaged
        "vboxnet0"
        ethernet (vboxnet), 0A:00:27:00:00:00, hw, mtu 1500

lo: unmanaged
        "lo"
        loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536

DNS configuration:
        servers: 192.168.0.1 8.8.8.8
        interface: wlp58s0

...

Compare the above with the output of ip addr. Notice that NetworkManager explicitly states the routes by default and also informs you that some interfaces are not controlled by it (here, lo or vboxnet0).

Changing IP configuration

While most networks offer DHCP (at least those you will connect to with your desktop), sometimes you need to setup IP addresses manually.

A typical case is when you need to connect two machines temporarily, e.g., to transfer a large file over a wired connection.

The only thing you need to decide on is which network you will create. Do not use the same one as your home router uses; our favourite selection is 192.168.177.0/24.

Assuming the name from above, the following command adds a connection named wired-static-temp on enp0s31f6:

sudo nmcli connection add \
    con-name wired-static-temp \
    ifname enp0s31f6 \
    type ethernet \
    ip4 192.168.177.201/24

It is often necessary to bring this connection up with the following command:

sudo nmcli connection up wired-static-temp

Follow the same procedure on the second host, but use a different address (e.g., .202). You should be able to ping the other machine now:

ping 192.168.177.201

To demonstrate how ping behaves when the connection goes down, you can try unplugging the wire, or doing the same in software:

sudo nmcli connection down wired-static-temp

nc (netcat)

Let us examine how to create network connections from the shell. This is essential for debugging of network services, but it is also useful for using the network in scripts.

The Swiss-army knife of network scripting is called netcat or nc. (Unfortunately, there exist multiple implementations of netcat, which differ in options and capabilities. We will show ncat, which is installed by default in Fedora. Your system might have a different one installed.)

Trivial things first: if you want to connect to a given TCP port on a remote machine, you can run nc machine port. This establishes the connection and wires the stdin and stdout to this connection. You can therefore interact with the remote server.

Netcat is often connected to other commands using pipes. Let us write a rudimentary HTTP client:

echo -en "GET / HTTP/1.1\r\nHost: www.kernel.org\r\n\r\n" | nc www.kernel.org 80

(We are using \r\n, since the HTTP protocol wants lines terminated by CR+LF. The Host: header is mandatory, because HTTP supports multiple web sites running on the same combination of IP address and port.)

We see that http://www.kernel.org/ redirects us to https://www.kernel.org/, so we try again using HTTPS. Fortunately, our version of netcat knows how to handle the TLS (transport-layer security) protocol used for encryption:

echo -en "GET / HTTP/1.1\r\nHost: www.kernel.org\r\n\r\n" | nc --ssl www.kernel.org 443

Now, let us build a simple server. It will listen on TCP port 8888 and when somebody connects to it, the server will send contents of a given file to the connection:

nc --listen 8888 <path-to-file

We can open a new shell and try receiving the file:

nc localhost 8888

We receive the file, but netcat does not terminate – it still waits for input from the stdin. Pressing Ctrl-D works, but it is easier to tell netcat to work in one direction only:

nc localhost 8888 --recv-only

OK, this works for transferring a single file over the network. (But please keep in mind that the transfer is not encrypted, so it is not wise to use it over a public network.)

When the file is transferred, the server terminates. What if we want to run a server, which can handle multiple connections? Here, redirection is not enough since we need to read the file multiple times. Instead, we can ask netcat to run a shell command for every connection and wire the connection to its stdin and stdout:

nc --listen 8888 --keep-open --sh-exec 'cat path-to-file'

Of course, this can be used for much more interesting things than sending a file. You can take any program which interacts over stdin and stdout and make it into a network service.

nmap (a.k.a. let me scan your network)

Warning: nmap is a very powerful tool. Unfortunately, even an innocent – but repeated – usage could be easily misinterpreted as a malicious scan of vulnerabilities that are susceptible to attack. Use this tool with care and experiment in your home network. Reckless scanning of the university network can actually ban your machine from connecting at all for quite some time.

nmap is the basic network scanning tool. If you want to know which network services are running on a machine you can try connecting to all of its ports to check which are opened. Nmap does that and much more.

Try first scanning your loopback device for internal services running on your machine:

nmap localhost

The result could look like this (the machine has a print server and a proxy HTTP server):

Starting Nmap 7.91 ( https://nmap.org ) at 2021-05-04 16:38 CEST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00011s latency).
Other addresses for localhost (not scanned): ::1
rDNS record for 127.0.0.1: localhost.localdomain
Not shown: 998 closed ports
PORT     STATE SERVICE
631/tcp  open  ipp
3128/tcp open  squid-http

Nmap done: 1 IP address (1 host up) scanned in 0.11 seconds

If you want to see more information, you can try adding -A switch.

nmap -A localhost

And if you run it under root (i.e. sudo nmap -A localhost) nmap can try to detect the remote operating system, too.

By default, nmap scans only ports frequently used by network services. You can specify a different range with the -p option:

nmap -p1-65535 localhost

This instructs nmap to scan all TCP ports (-p1-65535) on localhost.

Warning Again: do not use this on machines in the university network!

As an exercise, which web server is used on our GitLab? And which one is on our University website? Solution.

Systemd

Systemd one of the most widely used system service management tools in today’s Linux world.

We will not go into detail and just review the two most important commands: systemctl and journalctl.

Notice that systemd is a daemon, while systemctl is a command for controlling this daemon.

Starting and stopping a service

Starting a service with systemd is very simple. The following commands starts sshd: the SSH server:

sudo systemctl start sshd

If the service was already running, nothing will happen.

Check that you can now connect to your machine via the following command:

ssh your-login@localhost

To check the state of the service, the status subcommand is used (note that status can be run without sudo, but may display less information):

sudo systemctl status sshd
● sshd.service - OpenSSH Daemon
     Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: disabled)
     Active: active (running) since Mon 2021-03-01 14:31:40 CET; 2 months 3 days ago
   Main PID: 560 (sshd)
      Tasks: 1 (limit: 9230)
     Memory: 900.0K
        CPU: 16ms
     CGroup: /system.slice/sshd.service
             └─560 sshd: /usr/bin/sshd -D [listener] 0 of 10-100 startups

Warning: journal has been rotated since unit was started, output may be incomplete.

We see that the service is running, most items shall be self explanatory. The /usr/lib/systemd/system/sshd.service file contains service configuration itself (e.g., how to start/stop/restart the service), not the actual configuration of SSH daemon that is inside /etc/ssh.

It is safer to stop SSH daemon on your laptop if you are not going to use it:

sudo systemctl stop sshd

Enabling and disabling a service

If you wish to start the service with each boot, you can enable the service:

sudo systemctl enable sshd

Systemd will take care of proper ordering of the individual services (so that SSH server is started only after the network is initialized etc.).

If you no longer wish to have the SSH daemon started by default, call the command with disable instead.

Note that both enable and disable do not change the current state of the service: you still need to start/stop it if you do not want to wait for reboot. (For convenience, there is systemctl enable --now sshd, which also starts the service.)

Logs

Most system services keep logs of their work. The logs are usually stored under /var/log/. Some services produce logs on their own. Such logs are simple textual files, but their format is specific to the individual services and their configuration.

Many services use a central logging service, which keeps all its logs in a unified format and which can be configured for sorting logs, sending them over the network, removing old records, and so on.

On Fedora, the logging service is called journald. It keeps the log files in cryptographically signed binary files, which are not directly readable. But you can read the logs using the journalctl command.

For example, the following command shows logs for the SSH daemon:

journalctl -u sshd

More …

If you are interested in this topic, please, consult the relevant manual pages. Take these few paragraphs as a very brief introduction to the topic that allows you basic management of your system.

Printing with CUPS

Printing in Linux is handled by the CUPS subsystem that works out-of-the box with virtually every printer supporting IPP (internet printing protocol) and supports also many legacy printers.

Simple sudo dnf install cups installs the basic subsystem, extra drivers might be needed for specific models. OpenPrinting.org contains a searchable database to determine which (if any) drivers are needed. For example, for most HP printers, you would need to install the hplip package.

You typically want CUPS up and running on your system all the time, hence you need to enable it:

sudo systemctl enable --now cups

CUPS has a nice web interface that you can use to configure your printers. For many modern network-connected printers, even that is often unnecessary as they will be auto-discovered correctly.

If you have started CUPS already, try visiting http://localhost:631/. Under the Administration tab, you can add new printers. Selecting the right model helps CUPS decide which options to show in the printing dialog and enables proper functioning of grayscale printing and similar features.

Scanning with Sane

Scanner support on Linux is handled with SANE (Scanner Access Now Easy). As with printing, most scanners will be autodetected and if you already know GIMP, it has SANE support. Add it with sudo dnf install xsane-gimp.

Actual scanning of the image can be done from File -> Create -> XSane dialog where you select your device, scanning properties (e.g., resolution or colors) and then you can start the actual scan.

Periodically running tasks with Cron

There are many tasks in your system that needs to be executed periodically. Many of them are related to system maintenance, such as log rotation (removing of outdated logs), but even normal users may want to perform regular tasks.

A typical example might be backing up your $HOME or a day-to-day change of your desktop wallpaper.

From the administrator’s point of view, you need to install the cron daemon and start it. On Fedora, the actual package is called cronie, but the service is still named crond.

System-wide jobs (tasks) are defined /etc/cron.*/, where you can directly place your scripts. For example, periodic backup of your machine would typically go as a script backup.sh into /etc/cron.daily/.

If you want more fine-grained specification than the one offered by the cron.daily or cron.hourly directories, you can specify it in a custom file inside /etc/cron.d/.

There, each line specifies a single job: a request to run a specified command at specified time under the specified user (typically root). The time is given as a minute (0-59), hour (0-23), day of month (1-31), month (1-12), and day of week (0-6, 0 is Sunday). You can use * for “any” in every field. For more details, see crontab(5).

Therefore, the following will execute /usr/local/bin/backup.sh every day 85 minutes after midnight (i.e., at 1:25 am). The second line will call big-backup.sh on every Sunday morning.

25 1 * * * root /usr/local/bin/backup.sh
0  8 * * 0 root /usr/local/bin/big-backup.sh

Note that cron.d will typically contain a special call of the following form which ensures that the cron.hourly scripts are executed (i.e., the cronie deamon itself looks only inside /etc/cron.d/, the use of cron.daily or cron.monthly is handled by special jobs).

01 * * * * root run-parts /etc/cron.hourly

Running as a normal user

Normal (i.e., non-root) users cannot edit files under /etc/cron.d/. Instead, they have a command called crontab that can be used to edit their personal cron table (i.e., their list of cron jobs).

Calling crontab -l will list current content of your cron table. It will probably print nothing.

To edit the cron table, run crontab -e. It will launch your favourite editor where you can add lines in the above-mentioned format, this time without the user specification.

For example, adding the following entry will change your desktop background every day:

1 1 * * * /home/intro/bin/change_desktop_background.sh

Of course, assuming you have such script in the given location. If you really want to try it, the following script works for Xfce and uses Lorem Picsum.

#!/bin/bash

# Update to your hardware configuration
screen_width=1920
screen_height=1080

wallpaper_path="$HOME/.wallpaper.jpg"

curl -L --silent "https://picsum.photos/$screen_width/$screen_height" >"$wallpaper_path"

# Xfce
# Select the right path from xfconf-query -lvc xfce4-desktop
xfconf-query -c xfce4-desktop -p /backdrop/screen0/monitor0/workspace0/last-image -s "$wallpaper_path"

# LXDE
pcmanfm -w "$wallpaper_path"

# Sway
# For more details see `man 5 sway-output`
# You can also set a different wallpaper for each output (display)
# Run `swaymsg -t get_outputs` for getting specific output name
swaymsg output '*' bg "$wallpaper_path" fill

Graded tasks (deadline: May 1)

UPDATE: please, use user lab10 for 10/ip_addr.json task, sorry for the confusion.

10/port_forward.txt (25 points)

There is a web server running on linux.ms.mff.cuni.cz on port 8177.

This webserver is not accessible from the outside but only locally.

Use your SSH connection to make this server accessible on your local machine on port 8011. Then copy the designated text from http://127.0.0.1:8011/?student=LOGIN (replace LOGIN with your GitLab username, not UKCO) into your GitLab submission repository.

The copied text must include your username too.

10/ip_addr.json (35 points)

First, a bit of a context.

Because ping and ip addr are the starting points for virtually any debugging of network configuration, we want you to be able to read the output of ip addr. We have set-up a virtual machine connected to linux.ms.mff.cuni.cz and in this task you will read IP configuration. The task is not about creating a script but about reading the information manually.

For technical reasons, you will not have direct access to the machine but instead we have set-up a forced command wrapper on linux.ms.mff.cuni.cz that will print the configuration for you.

The actual task description starts here.

Use the private pair of the key you have uploaded as part of 07 and connect as user lab10 to linux.ms.mff.cuni.cz. This will print for you a result of ip addr of a certain machine.

Read this configuration and create a simple JSON with the following structure. The JSON would be a dictionary (mapping) where the key would be the network interface name and its value would be its IP address (IPv4 only), including network specification (e.g. 192.168.1.177/18). If the network is down (i.e., no IP address assigned), store the string "none".

As an example, the output from this lab would be stored in the following manner.

{
    "lo": "127.0.0.1/8",
    "enp0s31f6": "none",
    "wlp58s0": "192.168.0.105/24",
    "vboxnet0": "none"
}

The tests will check the basic format of your answer, for obvious reasons they do not compare your answers with the actual correct solution.

UPDATE: you need to have jq utility installed.

Regarding your public keys: at the moment of publication of this task we have added keys that were available in your repositories. If you have skipped that task, feel free to upload your key even after deadline.

We will try to update the list of authorized keys regularly.

Do not postpone this task to the last moment – we cannot guarantee that we will be able to update the configuration more often than every day.

10/fat.txt (40 points)

File linux.ms.mff.cuni.cz:~/lab10.fat.img is a disk image with a single file. Paste its (decompressed) content into 10/fat.txt (to GitLab).

Note that we can create the source file ~/lab10.fat.img only after you login to the remote machine for the first time.

If the file is not there, wait for the next work day for the file to appear.

Do not leave this task for the last minute and contact us if the file has not appeared as explained in the previous paragraph.

Learning outcomes

Conceptual knowledge

Conceptual knowledge is about understanding the meaning and context of given terms and putting them into context. Therefore, you should be able to …

  • explain difference between (normal) SSH port forward and a reverse port forward

  • explain what is a log and what are typical options for their management

  • explain what is a service (daemon)

  • explain life cycle and possible states of a service

  • explain why use of nmap is usually limited by network owners

  • explain why no special tools are required for working with disk images

  • explain difference between normal files, directory, symlink, device file and system-state file (/proc filesystem)

Practical skills

Practical skills is usually about usage of given programs to solve various tasks. Therefore, you should be able to …

  • use nmap

  • use SSH port forwarding

  • use systemctl to start/stop service

  • use systemctl to enable a service

  • use journalctl to view logs

  • mount and work with disks (both physical ones as well as disk images)

  • use df to get summary information (free space etc.) about mounted disks

  • use lsblk to view available storage devices

  • use nc (basic usage)

  • use dd for copying raw disk content (optional)

  • use photorec to restore files from a broken file system (optional)

  • use NetworkManager to setup static IP addresses (optional)

  • setup user cron table (optional)

  • configure printers in CUPS (optional)

  • use XSane for accessing scanners (optional)