Labs: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14.
This and the following lab devoted to a mixture of maintenance tasks. It contains the bare minimum needed for management of your Linux machine. And it also contains few extras that might be useful for your shells scripts.
The things covered in these two labs require that you have your own Linux installation as they require superuser privileges. That will not work on the shared machines in IMPAKT.
Do not forget that the Before class reading is mandatory and there is a quiz that you are supposed to complete before coming to the labs.
sudo
To try sudo
, you can try running fdisk -l
.
Generally, fdisk
is a tool for partitioning disks.
With -l
, it reads information about all disks on your system and displays
information about partitions on them.
Without sudo
, it will likely show only the following message:
fdisk: cannot open /dev/sda: Permission denied
Running it with sudo
displays the actual information.
sudo fdisk -l
Disk /dev/sda: 480 GiB, 515396075520 bytes, 1006632960 sectors
Disk model: QEMU HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xdc505942
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 2099199 2097152 1G 83 Linux
/dev/sda2 2099200 18620415 16521216 7.9G 82 Linux swap / Solaris
/dev/sda3 18620416 1006632959 988012544 471.1G 83 Linux
Note that sudo
typically asks for a password
(though that can be configured).
It is the password of the current user, not root’s.
(If you want to authenticate using the root’s password, you can use su
instead.)
dnf
(a.k.a. package manager in Fedora)
Note: Fedora used to have yum
as the package manager and it can be found
in many tutorials on the Internet (even in quite recent ones).
It is considered obsolete and you should better avoid it.
If you are used to yum
from older versions of Fedora or from other RPM-based
distributions, you will find dnf
very similar and in many situations faster
than yum
.
The package manager for Fedora is called DNF.
Note: if you decided to use a different distribution, you will need to update the commands to match your system. Generally, the operations would be rather similar but we cannot provide a tutorial for every package manager here.
You can use the search
command to get a list of packages which match the
given name.
Note that searching is not a privileged operation, hence it does not require sudo
.
dnf search arduino
dnf search atool
Note that searching for a very generic term can yield hundreds of results.
The output is in the following format:
atool.noarch : A perl script for managing file archives of various types
ratools.x86_64
The .noarch
and .x86_64
strings describe the nature of the package.
noarch
usually refers to a data package or package using interpreted languages,
while .x86_64
denotes a package with binaries for the x86-64 architecture
(e.g., written in C or Rust and then compiled to machine code).
To install a software package, run dnf
with the install
subcommand, giving it the
name of the package to install.
Here, sudo
is needed as we are modifying the system.
sudo dnf install atool
Some applications are not a part of any software repository, but you can still download them in a format understandable by your package manager. That is a better situation than installing the files manually, because your package manager knows about the files (although it cannot upgrade it automatically). One such example is the Zoom client which has to be installed like this:
sudo dnf install "https://zoom.us/client/latest/zoom_x86_64.rpm"
To upgrade the whole system, simply run sudo dnf upgrade
.
DNF will ask for confirmation and then upgrade all available packages.
Note that unlike on other systems, you can always choose when to upgrade. The system will never reboot the machine for you or display a message about needed restart, unless you explicitly ask for it.
If you want to install a whole group of packages, you can use dnf grouplist
to view their list and sudo dnf install @GROUP_NAME
to install it.
The commands above contain the basics for maintaining your Fedora installation with respect to package management. The following links provide more information. The official Wiki page is a good source of information if you already know the system a bit.
For beginners, this guide about DNF and this tutorial are probably a better starting point.
Working with processes
Execute ps -ef --forest
to view all process on your machine.
Because of your graphical interface, the list will be probably quite long.
Practically, a small server offering web pages, calendar and SSH access can have about 80 processes, for a desktop running Xfce with browser and few other applications, the number will rise to almost 300 (this really depends a lot on the configuration but it is a ballpark estimate). About 50–60 of these are actually internal kernel threads. In other words, a web/calendar server needs about 20 “real” processes, a desktop about 200 of them :-).
You can view the same information with htop
. You can also easily configure
it to display information about your system like amount of free memory
or CPU usage.
It can look like this (this one from a 32G/24CPU machine):
Date & Time: 2022-03-03 10:07:26 Uptime: 50 days, 20:55:31
1[|| 4.5%] 7[||| 5.0%] 13[|||| 8.9%] 19[| 3.2%]
2[| 1.3%] 8[| 1.9%] 14[|| 3.8%] 20[|| 4.5%]
3[|| 5.1%] 9[| 0.6%] 15[|||||| 17.8%] 21[||| 5.7%]
4[ 0.0%] 10[ 0.0%] 16[|| 4.4%] 22[|| 4.5%]
5[|| 4.5%] 11[|| 5.1%] 17[|| 4.5%] 23[||| 4.5%]
6[ 0.0%] 12[|| 5.1%] 18[|| 4.5%] 24[|| 4.5%]
Mem[||||||||||||||||||||||||||||||| 11.2G/31.3G] Tasks: 252, 1803 thr, 341 kthr; 1 running
Swp[|||||||||||||||||||||||||||||||||||||||||||||||| 3.06G/4.00G] Load average: 0.59 1.19 1.13
PID USER PRI NI VIRT RES SHR S CPU%+MEM% TIME+ Command
966 vojta 20 0 448M 9692 5824 S 0.0 0.0 0:01.01 | | | `-
3130 vojta 20 0 85856 3360 3356 S 0.0 0.0 0:00.03 | | `- /usr/lib/eclipse/eclipse -data /home/vojta/mff/eclips
3174 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 2h57:34 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.requi
3175 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 1h12:37 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3176 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 2:44.29 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3177 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 0:00.04 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3178 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 0:00.96 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3179 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 0:00.23 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3180 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 8:46.39 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3181 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 3:53.24 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3182 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 0:01.85 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3183 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 0:01.62 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3184 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 0:00.00 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3185 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 0:00.00 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3186 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 1:55.30 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3187 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 0:20.61 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3188 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 0:01.52 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3189 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 0:01.68 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
3193 vojta 20 0 20.8G 1274M 28068 S 0.0 4.0 17:29.22 | | | `- /usr/lib/jvm/java-11-openjdk/bin/java -Dosgi.re
F1 Help F2 Setup F3 Search F4 Filter F5 List F6 SortBy F7 Nice - F8 Nice + F9 Kill F10 Quit
Similar to MidnightCommander, function keys perform the most important actions and the help is visible in the bottom bar.
Signals
Open two terminals now.
Run the following command in the first one.
sleep 999
In the second terminal, find the PID of this process. Hint.
From the second terminal, we will now kill the program.
kill THE_PID
Looking back at the first terminal, you should see the following message.
Terminated (SIGTERM).
Start the sleep
command again, but send KILL
this time.
kill -9 THE_PID
The message probably changed to Killed
and the process was terminated
as well.
Reacting to signals in Python
Your program would typically react to TERM
(the default “soft” termination),
INT
(Ctrl-C from the keyboard) and perhaps to USR1
or USR2
(the only
user-defined signals).
System daemons (non-interactive programs handling system services) often react
to HUP
by reloading their configuration.
The following Python program reacts to Ctrl-C
by terminating (imports omitted):
# Actual signal callback
def on_signal(signal_number, frame_info):
print("")
print("Caught signal {} ({})".format(signal_number, frame_info))
sys.exit()
def main():
# Setting signal callback
signal.signal(signal.SIGINT, on_signal)
while True:
time.sleep(0.5)
print("Hit Ctrl-C...")
if __name__ == '__main__':
main()
Exercise: write a program that tries to print all prime numbers. When terminating, it stores the highest number found so far and on the next invocation, it continues from there. Solution.
Reacting to a signal in a shell script is also possible using the trap
command.
Note that a typical action for a signal handler in a shell script is clean-up of
temporary files.
#!/bin/bash
set -ueo pipefail
on_interrupt() {
echo "Interrupted, terminating ..." >&2
exit 17
}
on_exit() {
echo "Cleaning up..." >&2
rm -f "$MY_TEMP"
}
MY_TEMP="$( mktemp )"
trap on_interrupt INT TERM
trap on_exit EXIT
echo "Running as $$"
counter=1
while [ "$counter" -lt 10 ]; do
date "+%Y-%m-%d %H:%M:%S | Waiting for Ctrl-C (loop $counter) ..."
echo "$counter" >"$MY_TEMP"
sleep 1
counter=$(( counter + 1 ))
done
The command trap
receives as the first argument the command to execute on
the signal. Other arguments list the signals to react to.
Note that a special signal EXIT
means normal script termination.
Hence, we do not need to call on_exit
after the loop terminates.
Using -
instead of the handler causes the respective handler to be set to
default.
Without this reset in on_exit
, the handler would be called twice after the
user would hit Ctrl-C
(first for INT
caused by Ctrl-C
itself and then by the explicit call to exit
).
From now on, your shell scripts shall always include a signal handler for clean-up of temporary files.
Note the use of $$
which prints the current PID.
Alternatively, you can use pgrep <program_name>
to find a PIDs for running programs.
Similarly, you can use killall
to kill processes by name (but be careful as with great power comes great responsibility).
Consult the manual pages for more details.
Run the above script, note its PID and run the following in a new terminal.
kill THE_PID_PRINTED_BY_THE_ABOVE_SCRIPT
The script was terminated and the clean-up routine was called.
Compare with situation when you comment-out the trap
command.
Run the script again but pass -9
to kill
to specify that you want to
send signal nine (i.e., KILL
).
What happened? Answer.
While signals are a rudimentary mechanism, which passes binary events with no
additional data, they are the primary way of process control in Linux.
(If you need a richer communication channel, you can use D-Bus
instead.)
Reasonable reaction to basic signals is a must for server-style applications
(e.g., a web server should react to TERM
by completing outstanding requests
without accepting new connections, and terminating afterwards).
It shell scripts, it is considered good manners to always clean up temporary files.
User account management
So far, we used /etc/passwd
for getting information about accounts directly.
In practice, things can be more complicated as the information may come
from different sources. For example, in IMPAKT labs, the information is somehow
fetched from the CAS.
Generally, there can be multiple sources of user accounts. Thus, it is better
to use getent
to query them all instead of relying on /etc/passwd
only:
getent passwd YOUR_GITLAB_LOGIN
You can specify group
instead of passwd
to query groups, too.
Note that the output is actually in the same format as /etc/passwd
.
This is on purpose.
Without the login parameter, the command will (usually) list all accounts.
How many student accounts are on the machine u-pl17.ms.mff.cuni.cz
?
Hint.
Solution.
Creating new account
Creating a new account in Linux is rather straightforward.
The utility useradd
creates a new user by adding the appropriate entry
into /etc/passwd
and by creating a home directory.
Technically, nothing more needs to be done: the entry in /etc/passwd
means
that user id (numerical) is assigned to a human-readable name, creation of
the home directory ensures that the user has a writable directory to start with.
Practically, one can create a new user by editing /etc/passwd
manually, but
it is not recommended.
Graded tasks (deadline: Apr 24)
09/passwd.txt
(20 points)
Use the machine linux.ms.mff.cuni.cz
to retrieve information about
your account.
Paste the corresponding line of passwd
database about your account
into 09/passwd.txt
.
Automated tests verify only the format of your answer, not actual correctness.
Hint.09/signals.txt
(25 points)
Run program nswi177-signals
on linux.ms.mff.cuni.cz
.
You will need to send specific signals in given order to this program to complete this task.
The program will guide you: it will print which signals you are supposed to send.
Copy the last line of output (there will be two numbers) of this program
to 09/signals.txt
.
This task is only partially checked by automated tests.
09/countdown.sh
(35 points)
Write a script which gets one parameter: number of seconds (nonnegative integer) to count down, and each second it prints time left.
You can safely assume that the program will be invoked correctly under all circumstances.
If user hits Ctrl-C
during execution (or sends the TERM
signal), the
script aborts by printing the word Aborted
and exits with the status of 17.
Example:
$ ./countdown.sh 10
10
9
8
7
6
5
4
3
2
1
Each line will appear a second after the previous one. Use sleep 1
to wait between
printing the lines and before exiting.
Therefore, the first number is printed immediately, whereas after the final
line with 1
you still have to wait before your script terminates.
Note that the script will be a little bit slower than the requested time in seconds
(because of the overhead of calling the sleep
command itself) but that is fine.
Example execution when user hits Ctrl-C
is as follows (^C
denotes the place
where the user hit Ctrl-C
and is not an output from your program).
Note that the FAILED
message comes from the echo
and serves only to emphasize
the non-zero exit code.
$ ./countdown.sh 5 || echo "FAILED"
5
4
^C
Aborted
FAILED
09/dnf.txt
(20 points)
Which package version (not program version!) of msim-git
is installed on
linux.ms.mff.cuni.cz
?
This task is only partially checked by automated tests.
Use dnf help
to get a list of dnf
subcommands and find the right one to
finish this task (you will need one of the main commands).
It is fine to provide bare version or version with release.
Learning outcomes
Conceptual knowledge
Conceptual knowledge is about understanding the meaning and context of given terms and putting them into context. Therefore, you should be able to …
-
explain what account types exist on Linux and how they differ
-
explain why mechanisms such as sudo are needed and preferred to working as root all the time
-
explain what is a package and how it is used on Linux
-
explain what is a process signal
Practical skills
Practical skills is usually about usage of given programs to solve various tasks. Therefore, you should be able to …
-
use
getent
to retrieve information about existing accounts -
use
useradd
to create a new user account -
use a package manager for installation and removal of packages and for system updates
-
use
ps
andpgrep
-
use
htop
-
send a signal to running process