Labs: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11.

There will be an on-site test at the beginning of this lab, the topic is basic shell scripting (pipes).

The test will be held during the first half of the lab, please, make sure you arrive on time (other details are on this page).

The goal of this lab is to start using Linux in network environment and learn basic concepts needed for machines shared among multiple users. After the lab, you will be able to log in to a remote Linux machine, use Git over SSH and also have a look of what programs are currently running on a Linux machine.

We provide brief overview of several concepts that we believe you should already be familiar with. Feel free to skim these parts and focus on the new topics only.

Preflight checklist

  • You can perform basic Git operations using the command-line client.
  • You are ready for the first on-site test :-).

Networking introduction

This text assumes that you have basic knowledge of networking. Terms such as IP address, interface or port shall not be new for you. If you need a refresher, we have set-up a short page with a brief overview of networking.

Asymmetric cryptography

Before diving into details about how to access remote machines (over SSH), we need to make a brief refresh of some cryptography-related topics.

In this lab, we will be talking a lot about asymmetric cryptography. In general, it is a method of encryption/decryption where the user needs a different key for decryption of the message than the one that was used for message encryption.

This is different from symmetric ciphers. For example, the well-known Caesar cipher has a single key (the alphabet shift step) which is used for both encryption and decryption.

Asymmetric cryptography creates usually a pair of keys: a public key, that is usually used for encryption and a private one. For example, if you make your encryption key public and decryption key private, everybody can encrypt a message for you, but only you can decrypt it. This is secure if it is impossible (or hard enough) to derive the private key from the public one, which is usually the case.

This has an obvious advantage: you do not need to create a secret symmetric key for every pair of users who would want to communicate. Instead, everybody just distributes their public key and guards the single private key. (This is not as trivial as it looks: When Alice wants to send an encrypted message to Bob, she has to make sure that the public key does really belong to Bob. Otherwise, you can easily establish a secure connection, but to an attacker.)

Unfortunately, there is no good example of an asymmetric cipher as simple as the Caesar’s cipher. For an example, which is more complex, but still approachable, have a look at RSA.

Please note that selecting a good cipher is only a small step in communicating securely. If you want to learn more, please consult some real textbook on cryptography or attend one of our cryptographic courses. The above serves as a refresher to ensure we are on the same page for the rest of this lab.

Two uses for asymmetric cryptography

Asymmetric cryptography has two main uses. The first one is obvious: if we know the public key of the receiver of the message, we can use it to encrypt the message and send it over unprotected medium (and without fear that anyone else would be able to read it).

But it can be used also in reverse to authenticate the owner of the private key. We assume we are able to distribute the public keys safely here.

The mini-protocol is then able to authenticate (i.e., verify) that the other party is who they claim to be by proving the ownership of the private key (i.e., we assume that private keys were not stolen).

The method is very simple – the sender generates a random text and encrypts it with the public key of the receiver (the one we wish to verify). If the receiver is the real owner, they would be able to decrypt the random text and send it back to us. Inability to decrypt the text means that the receiver is not the owner of the private key.

Public & private key authentication

Typically a user authenticates to a service with a login and a password. While this method is quite natural and common for human users, it has several drawbacks.

The most obvious problem is password strength: people rarely have long passwords and not many people use any form of password manager. Using the same password at multiple places also allows administrator (or hacker) of one service to impersonate you in other services.

If you do not use a password manager, consider using one. The general idea is that you remember one (but strong!) password and this password encrypts rest of the passwords.

Therefore, all passwords can be generated to be long enough and unique for each service.

There are plenty of managers available, a simple one is pass that can use Git backend and has plenty of GUI clients available, including ones for Android and iOS.

Back to private/public key authentication. Some services allow the user to authenticate with their public key instead of using username/password.

The user uploads their public key to the server (using login and password for authenticating that operation) and when they want to log in, the server challenges them with a random secret encrypted with their public key. As the sole owner of the private key (and hence the only one able to decrypt), the user can decrypt the secret and confirm their identity. The operation then continues as with any other authenticated user.

Useful rules

For the public key authentication to work securely, the following is highly recommended (note that most of these rules apply to any other type of authentication, too).

The private key is like the password – it must not leak. Because the private key is usually a file, you must protect this file. Having an encrypted hard drive is a typical option for portable machines.

It is possible to protect the key itself with a passphrase (basically, another password). Then even a leaked private key file is not an immediate threat of identity theft. Note that there are helpers, such as ssh-agent(1), that can store the passphrase for some time so you do not have to enter it every time you use the key.

If you have multiple computers, it is preferred to use a different public/private key pair on each machine. If one machine is compromised, it it sufficient to remove its public key from all applications, while you can still use the other key pairs.

Cryptography overview: check you understand the basics

Select all true statements. You need to have enabled JavaScript for the quiz to work.

Using SSH

Enough of theory: let us connect to some remote machine. Let us explore SSH.

What is SSH?

SSH – that stands for Secure Shell – is a protocol for connecting to a different machine and running a shell there.

From a user perspective, after you SSH from a Linux machine into a different Linux machine, the shell may look the same and some commands would behave completely the same. Except they might be running on a different computer.

Note that this is intentional: remote shell is a natural way to control a Linux machine. No need to make it different from controlling it through a local shell.

SSH practically

Using SSH is very simple (unless you make it complex). To SSH to a remote machine, you need to know your credentials (i.e., login name and a password) and, of course, the name of the remote machine.

Then simply execute the following:

ssh YOUR_LOGIN@REMOTE_MACHINE_NAME

Note that the command ssh is often called a SSH client as it connects to the SSH server (similar to curl or wget being web clients connecting to a web server).

Your login will be usually case sensitive. For our (and university) servers it means that your login will be in lower case.

We have set up a remote machine for you on linux.ms.mff.cuni.cz. You will be using your GitLab (SIS/CAS) login and also the same password.

ssh your_gitlab_login@linux.ms.mff.cuni.cz

If your GitLab account was created manually, probably your SIS password will not work on this machine.

Please, contact us via this link and we will create an account for you manually.

The first login to the machine is a bit more complicated. SSH client wants from you a verification that you trust the remote server. It shows you a so-called server fingerprint:

The authenticity of host 'linux.ms.mff.cuni.cz (195.113.20.170)' can't be established.
ECDSA key fingerprint is SHA256:ltoc1TjoYhCZk6b8qSTAL6wFsRv7blw6u3h6NqCcYvI.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Based on your configuration, RSA or ED25519 key may be used instead:

RSA: SHA256:Z11Qbd6nN6mVmCSY57Y6WmxIJzqEFHFm47ZGiH4QQ9Y
ED25519: SHA256:/CVwDW388z6Z5VlhLJT0JX+o1UzakyQ+S01+34BI0BA

You should have received this fingerprint in a secure way before connecting to the server (for example, printed from your employer etc.). In this case, we hope that a potential attacker would not be able to break both into this web server and the SSH server at once. So we use the HTTPS protocol on the web as a secure way of verifying the SSH server.

The program then continues to ask your for a password and also informs you that the fingerprint was stored.

Usually, the password is not visible at all, i.e. nothing is printed (not even stars). That is fine: type your password and confirm with Enter.

On following logins, the SSH client checks that the fingerprint has not changed. A changed fingerprint belonging to the same machine (i.e., with the same DNS name) could indicate a man-in-the-middle attack. Do not connect when the fingerprint has changed. Always contact the administrator (e.g., the teachers in this particular case) and check with them what is happening.

If you were able to log-in, you will see an (almost) empty home directory and otherwise a normal Linux machine, this time without any graphical applications installed.

Try to run lscpu to see what machine you have logged in to.

Note that this machine is shared for all students of this course. Use it to solve graded tasks or to experiment with commands we show in the labs. You will be using it for the rest of the course for several tasks so keep it usable, please.

Do not use it for computationally intensive tasks or other tasks that are not related to this course.

We also strictly prohibit to use it for any kind of remote development with tools such as Visual Studio, IntelliJ IDEA or similar IDEs. These tools install huge blobs of code/data on the remote machine and because they rarely remove old versions, they are very quick in taking all free space available for themselves.

If we encounter any form of abusive use, we will block the offending account.

Whenever you are debugging issues with SSH, run the SSH client (ssh(1)) with the -v option that turns on verbose logging. It will print which keys were tried, full identification of the remote machine and other useful details.

When creating an issue about non-functional SSH, please, always paste the output of ssh -v there. Thank you!

Configuration of $PS1

We have already touched this a little bit in the previous lab as an extra for further reading.

$PS1 specified how your prompt looks like. From now on, you should ensure that your $PS1 shows also the machine name so that you always know where you are (i.e., which machine you are logged into).

Similarly to setting EDITOR in your ~/.bashrc you should ensure to specify the following (at least) if the default does not display the machine name.

PS1='\u@\h \w\$'

This ensures that you can see your username (\u), the hostname (\h) and also the working directory (\w).

Which ~/.bashrc you need to modify? Answer.

Return to previous lab if you want to see more options about setting your prompt.

As a personal tip: use colorful prompt without user/machine on your workstation but keep the non-colored version with \u@\h on all remote machines to keep short prompt on the personal machine but keep visual distinction when working on a remote one.

Using SSH to run one command

The command ssh is actually quite powerful and configurable. One important feature is that you can specify a command after the hostname and run this command directly. In this case, SSH never starts an interactive shell on the remote machine, but executes only the given command and returns.

Following commands prints uname -r on the remote machine.

ssh user@hostname uname -r

Investigate how (and why) the following command behaves differently.

The difference is visible only if you are SSHing to a different machine, that is when you execute ssh on your local machine.

ssh user@hostname uname -r >local_file.txt
ssh user@hostname "uname -r >remote_file.txt"

To verify that you are on a different machine, run on both uname -a and hostname -f (it provides the full DNS name of the current machine).

Can you see the differences?

You can also trying checking free -h or uptime.

SSH and passwordless authentication

To enable authentication with public key over SSH, we need to perform two steps. Generate the public-private key pair and copy the public key to the remote machine.

To generate the public key we need to run the following command (the -C is actually a comment, providing e-mail or your login is the usual approach but keeping it empty is possible too).

Key generation always happens on the local machine, i.e. the one that we will eventually connect from.

We may want to generate a new key pair on the remote machine too (e.g. on linux.ms.mff.cuni.cz). But then that pair would be used for authentication to another service, such as GitLab (i.e. linux.ms.mff.cuni.cz would be the local one in the communication).

ssh-keygen -t ed25519 -C "your_email@example.com"

The program would ask us where to store the generated private and public key. Keep the defaults. Choose if you want a passphrase or not (it is more secure to have one, but the use is a bit more cumbersome). After ssh-keygen finishes, check that the directory ~/.ssh contains id_ed25519 (the private key) and id_ed25519.pub (the public key).

Feel free to inspect their content, e.g. using cat. Surprised that they are text files?

Once we have the public key ready, we need to upload it to the remote machine. If you have multiple key pairs, read about the -i switch.

ssh-copy-id LOGIN@REMOTE_MACHINE

If you log in to the remote machine again, the SSH client should use the key pair and log you in without asking for a password. If not, run SSH with -vvv to debug the issue.

Note that the public key was stored into ~/.ssh/authorized_keys file on the remote machine. You can copy it there manually but using ssh-copy-id is easier.

If the copying fails with cryptic message about warning: here-document at line 251 delimited by end-of-file (wanted EOF), try upgrading the SSH client first.

If you use the image from us, simple sudo dnf upgrade openssh-clients should work.

Using keys only

Note that some services actually require that you authenticate using a key pair instead of a password as it is considered more secure.

The advantage is that any random attackers could keep guessing your password and still never get access to your machine.

Automated client ban

Typically, SSH server is also configured to ban any client that tries to login without success several times in a row. Our server does that, too.

Copying files

If you have deciphered what the following command does, you should by now have an idea how to copy files to and from a remote machine (we are not saying it would be the most effective way):

ssh user@hostname "uname -r >remote_file.txt"

Wondering how this helps you copy the files? Think about it a bit more before reading on, consider how pipes are used etc.

Have you really thought about it? :-)

To copy the files, we can leverage cat that simply dumps the file as-is to stdout. We will pipe its output to SSH and on the other machine run second cat to store the file.

cat README.md | ssh user@hostname "cat >remote-readme.md"

Now, please, ensure that you can explain what is happening. SSH is able to read stdin (after all, we were typing our commands locally and they appeared in the remote shell on our local screen) and hence we can tunnel the data through it.

There are also scp and rsync that can be used for copying multiple files over SSH easily but we will talk about these in later labs.

File managers

Many file managers allows you to use SSH transparently and copy between machines with the same ease as when working with local files.

For example, in mc, select Shell connection either in left or right panel and specify SSH connection. One of the panels will show the files on the remote machine. Try using F5 to copy files interactively.

Using machines in Rotunda

Apart from the machine linux.ms.mff.cuni.cz, there is also a full lab of machines available in the Rotunda computer lab on Malostranské náměstí.

All the Linux machines in the lab are also reachable via SSH. Again, use your SIS credentials to log in. Note that all machines in Rotunda (but not linux.ms.mff.cuni.cz!) share the same home directory, i.e., it does not matter which one you physically connect to. Your files will be available on all machines.

The same home directory is shared across all lab machines at Malostranské náměstí and in IMPAKT (i.e., SSHing into u-pl1.ms.mff.cuni.cz and creating a file there is the same as if you physically attend lab in N8/N11 and create the lab while logged in to the machine via GUI).

Unfortunately, the used file system and authentication mechanism does not allow to use public key authentication for Rotunda machines. You need to always specify your password. (linux.ms.mff.cuni.cz does not have this limitation and we expect you will use public key authentication there.)

Following machines are available.

  • Lab SU1
    • u1-1.ms.mff.cuni.cz
    • u1-2.ms.mff.cuni.cz
    • u1-14.ms.mff.cuni.cz
  • Lab SU2
    • u2-1.ms.mff.cuni.cz
    • u2-2.ms.mff.cuni.cz
    • u2-25.ms.mff.cuni.cz
  • Rotunda
    • u-pl1.ms.mff.cuni.cz
    • u-pl2.ms.mff.cuni.cz

The computers in Lab SU1 and SU2 can not be connected to via SSH during the day (or more specifically during the scheduled classes). That is to prevent remote users to interfere with students learning a subject. Rotunda lab machines are accessible 24/7.

SSH: review your knowledge

Select all true statements. You need to have enabled JavaScript for the quiz to work.

Unix-style access rights

So far we have silently ignored the fact that there are different user accounts on any Linux machine. And that users cannot access all files on the machine. In this section we will explain the basics of Unix-style access rights and how to interpret them.

After all, now you can log in to a shared machine and you should be able to understand what you can access and what you cannot.

Recall what we said about /etc/passwd earlier – it contains the list of user accounts on that particular machine (technically, it is not the only source of user records, but it is a good enough approximation for now).

Every running application, i.e., a process, is owned by one of the users from /etc/passwd (again, we simplify things a little bit). We also say that the process is running under a specific user.

And every file in the filesystem (including both real files such as ~/.bashrc and virtual ones such as /dev/sda or /proc/uptime) has some owner.

When a process tries to read or modify a file, the operating system decides whether the operation is permitted. This decision is based on the owner of the file, the owner of the process, and permissions defined for the file. If the operation is forbidden, the input/output function in your program raises an exception (e.g., in Python), or returns an error code (in C).

Since a model based solely on owners would be too inflexible, there are also groups of users (defined in /etc/group). Every user is a member of one or more groups, one of them is called the primary group. These are associated with every process of the user. Files have both an owning user and an owning group.

Files are assigned three sets of permissions: one for the owner of the file, one for users in the owning group, and one for all other users. The exact algorithm for deciding which set will be used is this:

  1. If the user running the process is the same as the owner of the file, owner access rights are used (sometimes also referred to as user access rights).
  2. If the user running the process is in a group that was set on the file group access rights are used.
  3. Otherwise, the system checks against other access rights.

Every set of permissions contains three rights: read (r), write (w), execute (x):

  • Read and write operations on a file are obvious.

  • The execute right is checked when the user tries to run the file as a program (recall that without chmod +x, the error message was in the sense of Permission denied: this is the reason).

    • Note that a script which is readable, but not executable, can be still run by launching the appropriate interpreter manually.
    • Note that when a program is run, the new process will inherit the owner and groups of its parent (e.g., of the shell that launched it). Ownership of the executable file itself is not relevant once the program was started. For example, /usr/bin/mc is owned by root, yet it can be launched by any user and the running process is then owned by the user who launched it.

The same permissions also apply to directories. Their meaning is a bit different, though:

  • Read right allows the user to list directory entries (files, symlinks, sub-directories, etc.).
  • Write right allows the user to create, remove, and rename entries inside that directory. Note that removing write permission from a file inside a writable directory is pointless as it does not prevent the user from overwriting the file completely with a new one.
  • Execute right on a directory allows the user to open the entries. (If a directory has x, but not r, you can use the files inside it if you know their names; however, you cannot list them. On the contrary, if a directory has r, but not x, you can only view the entries, but not use them.)

Permissions of a file or directory can be changed only by its owner, regardless of the current permissions. That is, the owner can deny access to themselves by removing all access rights, but can always restore them later.

root account

Apart from accounts for normal users, there is always an account for a so-called superuser – more often called simply just root – that has administrator privileges and is permitted to do anything with any file in the system. The permissions checks described above simply do not apply to root-owned processes.

Unlike other systems, Linux is designed in such way that end-user programs are always executed under normal users and never require root privileges. As a matter of fact, some programs (historically, this was a very common behaviour for IRC chat programs) would not even start under root.

Viewing and changing the permissions

Looking at the shortcuts of rwx for individual permissions, you may found them familiar:

drwxr-xr-x 1 intro intro 48 Feb 23 16:00 02/
drwxr-xr-x 1 intro intro 60 Feb 23 16:00 03/
-rw-r--r-- 1 intro intro 51 Feb 23 16:00 README.md

The very first column actually contains the type of the entry (d for directory, - for a plain file, etc.) and three triplets of permissions. The first triplet refers to owner of the file, the middle one to the group, and last to the rest of the world (other). The third and fourth columns contain the owner and the group of the file.

Typically, your personal files in your home directory will have you as the owner together with a group with the same name. That is a default configuration that prevents other users from seeing your files.

Do check that it is true for all directories under /home on the shared machine.

But also note that most of the files in your home directory are actually world-readable (i.e., anyone can read them).

That is actually quite fine because if you check permissions for your ~, you will see that it is typically drwx------. Only the owner can modify and cd to it. Since no one can actually change to your directory, no one will be able to read your files (technically, reading a file involves traversing the whole directory and checking access rights on the whole path).

To change the permissions, you can use chmod program. It has the general format of

chmod WHO[+=-]PERMISSION file1 file2 ...

WHO can be empty (for all three of user, group and others) or specification of u, g or o. And PERMISSION can be r, w or x.

It is possible to combine these, e.g. g-r,u=rw removes read permission for group and sets read-write for the owner (i.e. the file will not be executable by the owner regardless of a previous state of the executable bit).

Sticky and other bits

If you execute the following command, you will see a bit different output that you would probably expect.

ls -ld /usr/bin/passwd /tmp
drwxrwxrwt 23 root root   640 Mar  3 08:15 /tmp/
-rwsr-xr-x  1 root root 51464 Jan 27 14:47 /usr/bin/passwd*

You should have noticed that /tmp has t in place of an executable bit and passwd has s there.

Those are special variants of the executable bit. The t bit on a directory specifies that user can remove only their own files. The reason is obvious – you shall not be able to remove someone else’s files inside /tmp. Something that is otherwise impossible to specify with traditional (basic) permissions.

The s bit (set-uid) is a bit more tricky. It specifies that no matter who runs the shell, passwd will be running under the user owning the file (i.e., root for this file).

While it may look useless, it is a simple way to allow running certain programs with elevated (higher) permissions. passwd is a typical example. It allows the user to change their password. However, this password is stored in a file that is not readable by any user on the system except root (for obvious reasons). Giving the s bit to the executable means that the process would be running under root and would be able to modify the user database (i.e., /etc/passwd and /etc/shadow that contains the actual passwords).

Since changing the permissions can be done only by the owner of the file, there is no danger that a malicious user would add the s bit to other executables.

There are other nuances regarding Unix permissions and their setting, refer to chmod(1) for details.

Beyond traditional Unix permissions: POSIX ACL

The permission model described above is a rare example of a concept coming from Unix that is considered inflexible for use today. However, it is also considered as a typical example of a simple but well usable security model.

Many programs copied this model and you can encounter it in other places too. It is definitely something worth remembering and understanding.

The inflexibility of the system comes from the fact that allowing a set of users to access a particular file means creating a special group for these users. These groups are defined in /etc/group and changing them requires administrator privileges.

With an increasing number of users, the amount of possibly needed groups grows exponentially. On the other hand, for most situations, the basic Unix permissions are sufficient.

To tackle this problem, Linux offers also so-called POSIX access control lists where it is possible to assign an arbitrary list of users to any file to specify the permissions.

getfacl and setfacl are the utilities to control these rights but since these are practically needed rarely, we will leave their knowledge at the level of reading the corresponding manpages and acl(5).

Access rights checks

Let’s return a little bit to the access rights.

Change permission of some of your scripts to be --x. Try to execute them. What happens? Answer.

Remove writable bit for a file and write to it using stdout redirection. What happens?

Access rights quiz

Assuming the following output of ls -l (script.sh is really a shell script) and assuming user bob is in group wonderland while user lewis is not.

-rwxr-xr-- 1 alice wonderland 1234 Feb 20 11:11 script.sh

Select all true statements.

You need to have enabled JavaScript for the quiz to work.

Git over SSH

So far, we have used Git over HTTPS. Git can be used over SSH too. Then, the traffic is basically tunneled through an SSH connection and Git relies on the SSH wrapper for security as well as for (partial) authentication.

Virtually all Git servers (GitLab, GitHub, Bitbucket…) will require you to upload your public key if you want to use Git over SSH.

To actually use Git over SSH, we first need to tell GitLab about our SSH keys (recall the protocol that is used to authenticate the user).

GitLab and SSH keys

Copy your public key to GitLab. Navigate to right-top menu with your avatar, select Preferences and then SSH keys or visit this link.

Copy your public key there and name it. Typically, the name should mention your username and your machine. Note that GitLab will send you an e-mail informing you about a new key. Why? Hint. Answer.

Go to your project and clone it again. This time, use the Clone with SSH URL.

git clone git@gitlab.mff.cuni.cz:teaching/nswi177/2024/student-LOGIN.git

Have you noticed that this looks like SSH address? It actually is exactly that. The first part identifies the machine and the user (git) and after a colon is a local path.

This way, you can clone a Git directory from any SSH server by specifying its remote path there (here, GitLab does some mangling but the principle holds).

Note that the user we clone with is git – not you. This way, GitLab needs only one physical user account for handling Git requests and distinguishes the users via their public keys. How? Answer.

By the way, what happens if you try to run the following?

ssh git@gitlab.mff.cuni.cz

Note that you should prefer the SSH protocol for working with Git as it is much more comfortable for use.

Git on other platforms also offers generation of an SSH key but often the key is usable only by one application (different applications have incompatible key formats), while on Linux a single key is generally usable for Git, other machines, and other services.

Rest of the work with Git remains the same. git add, git commit, git push etc. will work the same but only the communication with GitLab goes through SSH tunnel. Note that you don’t have to reenter your credentials while doing git push. This is because git remembers how did you git clone the repository and will use the same URL (either HTTPS or SSH) for git push as well (unless you configure it otherwise). And since you used SSH for pulling, it will use SSH for pushing as well, which uses the public/private authentication you already set up.

Processes

Files in the system are the passive elements of it. The active parts are running programs that actually modify data. Let us have a look what is actually running on our machine.

When you start a program (i.e., an executable file), it becomes a process. The executable file and a running process share the code – it is the same in both. However, the process also contains the stack (e.g., for local variables), heap, current directory, list of opened files etc. etc. – all this is usually considered a context of the process. Often, the phrases running program and process are used interchangeably.

To view the list of running processes on our machine, we can use htop to view basic properties of processes. Similar to MidnightCommander, function keys perform the most important actions and the help is visible in the bottom bar. You can also configure htop to display information about your system like amount of free memory or CPU usage.

For non-interactive use we can execute ps -e (or ps -axufw for a more detailed list).

For illustration here, this is an example of ps output (with --forest option use to depict also the parent/child relation).

However, run ps -ef --forest on the shared machine to also view running processes of your colleagues.

Listing of processes is not protected in any way from other users. Every user on a particular machine can see what other users are running (including command-line arguments).

Keep in mind to never pass passwords as command line arguments and pass them always through files (with proper permissions) or interactively on stdin.

UID          PID    PPID  C STIME TTY          TIME CMD
root           2       0  0 Feb22 ?        00:00:00 [kthreadd]
root           3       2  0 Feb22 ?        00:00:00  \_ [rcu_gp]
root           4       2  0 Feb22 ?        00:00:00  \_ [rcu_par_gp]
root           6       2  0 Feb22 ?        00:00:00  \_ [kworker/0:0H-events_highpri]
root           8       2  0 Feb22 ?        00:00:00  \_ [mm_percpu_wq]
root          10       2  0 Feb22 ?        00:00:00  \_ [rcu_tasks_kthre]
root          11       2  0 Feb22 ?        00:00:00  \_ [rcu_tasks_rude_]
root           1       0  0 Feb22 ?        00:00:09 /sbin/init
root         275       1  0 Feb22 ?        00:00:16 /usr/lib/systemd/systemd-journald
root         289       1  0 Feb22 ?        00:00:02 /usr/lib/systemd/systemd-udevd
root         558       1  0 Feb22 ?        00:00:00 /usr/bin/xdm -nodaemon -config /etc/X11/...
root         561     558 10 Feb22 tty2     22:42:35  \_ /usr/lib/Xorg :0 -nolisten tcp -auth /var/lib/xdm/...
root         597     558  0 Feb22 ?        00:00:00  \_ -:0
intro        621     597  0 Feb22 ?        00:00:40      \_ xfce4-session
intro        830     621  0 Feb22 ?        00:05:54          \_ xfce4-panel --display :0.0 --sm-client-id ...
intro       1870     830  4 Feb22 ?        09:32:37              \_ /usr/lib/firefox/firefox
intro       1966    1870  0 Feb22 ?        00:00:01              |   \_ /usr/lib/firefox/firefox -contentproc ...
intro       4432     830  0 Feb22 ?        01:14:50              \_ xfce4-terminal
intro       4458    4432  0 Feb22 pts/0    00:00:11                  \_ bash
intro     648552    4458  0 09:54 pts/0    00:00:00                  |   \_ ps -ef --forest
intro      15655    4432  0 Feb22 pts/4    00:00:00                  \_ bash
intro     639421  549293  0 Mar02 pts/8    00:02:00                      \_ man ps
...

First of all, each process has a process ID, often just PID (but not this one). The PID is a number assigned by the kernel and used by many utilities for process management. PID 1 is used by the first process in the system, which is always running. (PID 0 is reserved as a special value – see fork(2) if you are interested in details.) Other processes are assigned their PIDs incrementally (more or less) and PIDs are eventually reused.

Note that all this information is actually available in /proc/PID/ and that is where ps reads its information from.

Execute ps -ef --forest again to view all process on your machine. Because of your graphical interface, the list will be probably quite long.

Practically, a small server offering web pages, calendar and SSH access can have about 80 processes, for a desktop running Xfce with browser and few other applications, the number will rise to almost 300 (this really depends a lot on the configuration but it is a ballpark estimate). About 50–60 of these are actually internal kernel threads. In other words, a web/calendar server needs about 20 “real” processes, a desktop about 200 of them :-).

Forcefully terminating processes

At this moment we will show probably the most important thing that you can do with processes. And that is their forceful termination.

Eventually we will learn about the concept of signals, at the moment we resort ourselves to the basic two commands.

The pgrep command can be used to find processes matching a given name.

Open two extra terminals and run sleep 600 in one and sleep 800 in the second one. The sleep program simply waits given amount of seconds before terminating.

In a third terminal, run the following commands to understand how the searching for the processes is done.

pgrep sleep
pgrep 600
pgrep -f 600

What have you learnt? Answer.

When we know the PID, we can use the kill utility to actually terminate the program. Try running kill PID with PID of one of the sleeps and look what happened in the terminal with sleep.

You should see something like this:

Terminated (SIGTERM).

This message informs us that the command was forcefully terminated.

Some programs ignore the kill command and do not terminate. We will explain why that is possible in some of the next labs but for now we want to mention that it is possible to add -9 to the kill command which instructs the operating system to be a bit more forceful and terminate the program without giving it any option to disagree ;-).

You can always kill your own processes but killing processes of someone else is not possible.

Quick check on processes

Select all true statements. You need to have enabled JavaScript for the quiz to work.

Going further

Few extra bits that will improve your user experience with SSH a lot but can be returned to any time later.

Using tmux for better SSH experience

tmux is a terminal multiplexer. That means that inside one terminal it opens several terminals for you that are running in parallel. It also allows you to send the session into background so that it remains there even if you log out or your remote connection is interrupted (this is useful for running long scripts). In other words, tmux gives you tabs (called windows) inside your existing session that can be minimized/iconified (if borrowing terms from GUI would explain the usefulness better).

The simplest way how to start tmux session is simply:

tmux

Alternatively, we can start session with some meaningful name:

tmux new -s <session_name>

To list all sessions run:

tmux ls

To connect/attach to the running session run:

tmux attach -t <session_name>

And finally, in order to kill the session we use:

tmux kill-session -t <session_name>

Inside the session we are able to create multiple windows, split the screen and much more. In order to invoke a tmux command, we need firstly to type tmux prefix. The default key binding is <Ctrl>-b.

In order to detach from session we can simply press (do not forget to type the prefix!):

d  detach from session

Operation with windows:

c  create window
w  list windows
n  next window
p  previous window
f  find window
,  name window
&  kill window

Sometimes it is useful to split the screen into several terminals. These splits are called panes.

Operation with panes (splits):

%  vertical split
"  horizontal split

o  swap panes
q  show pane numbers
x  kill pane
←  switch to left pane
→  switch to right pane
↑  switch to upward pane
↓  switch to downward pane

Other feature is that you can toggle writing simultaneously to all panes. Performing the same operation multiple times may seem not much useful, but you can for example open several different ssh connections in advance and then interactively control all these computers at the same time.

To toggle it, type the prefix and then write :set synchronize-panes. If you want to try this in Rotunda, please do not run computationally intensive tasks…

As usual with Linux tools, you can modify its behavior widely via rc configuration. For instance, in order to navigate among the panes with vim shortcuts, modify your .tmux.conf so it contains

bind-key C-h run "tmux select-pane -L"
bind-key C-j run "tmux select-pane -D"
bind-key C-k run "tmux select-pane -U"
bind-key C-l run "tmux select-pane -R"

Personal tip № 0: tmux is an excellent tools for collaboration, as multiple users can attach to the same session.

Personal tip № 1: when you give a lecture, you can attach to the tmux session from two terminals. Later on, you push the first one to the projector, while the second one is on you laptop screen. This eliminates the necessity of mirroring your screen. Together with pdfpc and a tiling window manager we get a Swiss-army knife for presentation.

There is much more to say. For more details see this tmux cheatsheet or manual pages.

SSH configuration

The SSH client is configured via the ~/.ssh/config file. Review the syntax of this file via man 5 ssh_config.

The file is divided into sections. Each section is related to one or more remote hosts. The section header is in the format Host pattern, where pattern might use wildcards.

The syntax is mostly self-explanatory, so we will only provide an example.

Host *
    IdentityFile ~/.ssh/id_ed25519

Host intro
    Hostname linux.ms.mff.cuni.cz
    User YOUR_SIS_LOGIN

Host mff0
    Hostname u-pl0.ms.mff.cuni.cz
    User YOUR_SIS_LOGIN

Host mff1
    Hostname u-pl1.ms.mff.cuni.cz
    User YOUR_SIS_LOGIN

With this ~/.ssh/config, we can type ssh intro and the ssh will start connection equivalent to

ssh YOUR_SIS-LOGIN@linux.ms.mff.cuni.cz

Tasks to check your understanding

We expect you will solve the following tasks before attending the labs so that we can discuss your solutions during the lab.

There are multiple options available, separate your answer with spaces or commas, e.g. **[A1]** 1,2 **[/A1]**.

Assume that we have a file `test.txt` for which `ls -l` prints the following:

    -rw-r----- 1 bjorn ursidae 13 Mar 21 14:54 test.txt

Which of the following users will be able to read the contents of the file?

 1. `bjorn` in group `ursidae`
 2. `bjorn` in groups `carnivora` and `mammalia`
 3. `iorek` in group `ursidae`
 4. `iorek` in groups `carnivora` and `mammalia`
 5. `root` (the superuser)
 6. everybody

**[A1]** ... **[/A1]**

Consider that the file from the previous example is stored within
the directory `/data` with the following permissions as printed by `ls -l`:


    drwxrwx-wx 3 bjorn ursidae 4096 Mar 21 14:53 data


Which of the following users will be able to delete the file?

 1. `bjorn` in group `ursidae`
 2. `bjorn` in groups `carnivora` and `mammalia`
 3. `iorek` in group `ursidae`
 4. `iorek` in groups `carnivora` and `mammalia`
 5. `root` (the superuser)
 6. everybody

You can assume that the root directory `/` is readable and executable
by everybody.

**[A2]** ... **[/A2]**

Continuing with the previous questions, which commands can be used to make
the file `test.txt` readable and writeable only to the owner and nobody else?

 1. `chmod u=rw test.txt`
 2. `chmod =rw test.txt`
 3. `chmod g= test.txt`
 4. `chmod o= test.txt`
 5. `chmod g=,o= test.txt`
 6. `chmod g-r test.txt`
 7. `chmod g-rwx test.txt`

**[A3]** ... **[/A3]**

This example can be checked via GitLab automated tests. Store your solution as 05/rights.md and commit it (push it) to GitLab.

Store your public keys into 05/key.[0-9].pub on GitLab (the wildcard denotes the ability to upload multiple keys if you are using multiple machines).

Do not lose the private part for it — we will use it for some other tasks later on.

This example can be checked via GitLab automated tests. Store your solution as 05/key.[0-9].pub and commit it (push it) to GitLab.

This will work only after you upload your keys to 05/key.[0-9].pub.

Then you should be able to clone the following repository (as usual, LOGIN is your GitLab login in lowercase) over SSH.

gitolite3@linux.ms.mff.cuni.cz:lab05-LOGIN

The repository (after cloning) will be empty (and Git will tell you so). That is fine. The point is to check that you can clone a different repository without having any web UI available.

Create file works.txt inside its root, commit it and push it back.

We are updating the keys daily, report issues only if you cannot access the repository 24 hours after changing your 05/key.[0-9].pub files.

Add the following public key to authorized keys on your account at linux.ms.mff.cuni.cz. We will check that you have completed this task by SSHing to linux.ms.mff.cuni.cz with your login and this key.

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPuoJJldIS6zOxunFxIFGk6tQlw0qpYSOxHYs57117o/ teachers-nswi177@d3s.mff.cuni.cz

We will create file FROM_TEACHERS once you have setup our key (again, give us some time to run our scripts).

This is to ensure that you can setup remote login for someone else without breaking anything :-).

Learning outcomes

Learning outcomes provide a condensed view of fundamental concepts and skills that you should be able to explain and/or use after each lesson. They also represent the bare minimum required for understanding subsequent labs (and other courses as well).

Conceptual knowledge

Conceptual knowledge is about understanding the meaning and context of given terms and putting them into context. Therefore, you should be able to …

  • explain basic principles of network communication (in OS agnostic manner)

  • explain basic principles of asymmetric cryptography

  • explain in detail how asymmetric cryptography (public and private key) can be used to authenticate a user

  • explain what is SSH and what functions it offers

  • explain what account types exist on Linux and how they differ (e.g. johndoe, root and nobody)

  • explain how differs execution of programs locally vs over SSH remotely

  • explain difference between using Git over HTTPS vs Git over SSH

  • explain basic access rights in Unix operating systems

  • explain what individual access rights r, w and x mean for normal files and what for directories

  • explain what is a set-uid bit

  • explain what is a process and how it differs from an executable file

  • explain difference of ownership of a file and a running process

  • optional: provide a high-level overview of POSIX ACLs

Practical skills

Practical skills are usually about usage of given programs to solve various tasks. Therefore, you should be able to …

  • set PS1 variable to distinguish different machines

  • use ssh command to login to a remote machine

  • execute commands on a remote machine using SSH

  • use hostname command

  • transfer files over SSH using cat

  • setup password-less authentication on a remote Linux machine using a private/public key pair

  • setup publickey-based authentication to GitLab

  • use git clone (and pull and push) over SSH

  • view and change basic access permissions of a file

  • use ps to view list of existing processes (including -e, -f and --forest switches)

  • use htop to interactively monitor existing processes

  • optional: configure SSH shortcuts

  • optional: use basic functions of tmux terminal multiplexer

This page changelog

  • 2024-03-11: Add notice about ssh -v.

  • 2024-03-14: Notice about case-sensitivity of logins.

  • 2024-03-18: Fix list of machines available in Rotunda.

  • 2024-04-08: Shared home in IMPAKT and Rotunda.