Labs: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14.
There will be an on-site test at the beginning of this lab, the topic is basic shell scripting (pipes).
The test will be held during the first half of the lab, please, make sure you arrive on time (other details are on this page).
The goal of this lab is to start using Linux in network environment and learn basic concepts needed for machines shared among multiple users. After the lab, you will be able to log in to a remote Linux machine, use Git over SSH and also have a look of what programs are currently running on a Linux machine.
We provide brief overview of several concepts that we believe you should already be familiar with. Feel free to skim these parts and focus on the new topics only.
Preflight checklist
- You can perform basic Git operations using the command-line client.
- You are ready for the first on-site test :-).
Networking introduction
This text assumes that you have basic knowledge of networking. Terms such as IP address, interface or port shall not be new for you. If you need a refresher, we have set-up a short page with a brief overview of networking.
Asymmetric cryptography
Before diving into details about how to access remote machines (over SSH), we need to make a brief refresh of some cryptography-related topics.
In this lab, we will be talking a lot about asymmetric cryptography. In general, it is a method of encryption/decryption where the user needs a different key for decryption of the message than the one that was used for message encryption.
This is different from symmetric ciphers. For example, the well-known Caesar cipher has a single key (the alphabet shift step) which is used for both encryption and decryption.
Asymmetric cryptography creates usually a pair of keys: a public key, that is usually used for encryption and a private one. For example, if you make your encryption key public and decryption key private, everybody can encrypt a message for you, but only you can decrypt it. This is secure if it is impossible (or hard enough) to derive the private key from the public one, which is usually the case.
This has an obvious advantage: you do not need to create a secret symmetric key for every pair of users who would want to communicate. Instead, everybody just distributes their public key and guards the single private key. (This is not as trivial as it looks: When Alice wants to send an encrypted message to Bob, she has to make sure that the public key does really belong to Bob. Otherwise, you can easily establish a secure connection, but to an attacker.)
Unfortunately, there is no good example of an asymmetric cipher as simple as the Caesar’s cipher. For an example, which is more complex, but still approachable, have a look at RSA.
Please note that selecting a good cipher is only a small step in communicating securely. If you want to learn more, please consult some real textbook on cryptography or attend one of our cryptographic courses. The above serves as a refresher to ensure we are on the same page for the rest of this lab.
Two uses for asymmetric cryptography
Asymmetric cryptography has two main uses. The first one is obvious: if we know the public key of the receiver of the message, we can use it to encrypt the message and send it over unprotected medium (and without fear that anyone else would be able to read it).
But it can be used also in reverse to authenticate the owner of the private key. We assume we are able to distribute the public keys safely here.
The mini-protocol is then able to authenticate (i.e., verify) that the other party is who they claim to be by proving the ownership of the private key (i.e., we assume that private keys were not stolen).
The method is very simple – the sender generates a random text and encrypts it with the public key of the receiver (the one we wish to verify). If the receiver is the real owner, they would be able to decrypt the random text and send it back to us. Inability to decrypt the text means that the receiver is not the owner of the private key.
Public & private key authentication
Typically a user authenticates to a service with a login and a password. While this method is quite natural and common for human users, it has several drawbacks.
The most obvious problem is password strength: people rarely have long passwords and not many people use any form of password manager. Using the same password at multiple places also allows administrator (or hacker) of one service to impersonate you in other services.
If you do not use a password manager, consider using one. The general idea is that you remember one (but strong!) password and this password encrypts rest of the passwords.
Therefore, all passwords can be generated to be long enough and unique for each service.
There are plenty of managers available, a simple one is pass that can use Git backend and has plenty of GUI clients available, including ones for Android and iOS.
Back to private/public key authentication. Some services allow the user to authenticate with their public key instead of using username/password.
The user uploads their public key to the server (using login and password for authenticating that operation) and when they want to log in, the server challenges them with a random secret encrypted with their public key. As the sole owner of the private key (and hence the only one able to decrypt), the user can decrypt the secret and confirm their identity. The operation then continues as with any other authenticated user.
Useful rules
For the public key authentication to work securely, the following is highly recommended (note that most of these rules apply to any other type of authentication, too).
It is possible to protect the key itself with a passphrase
(basically, another password).
Then even a leaked private key file is not an immediate threat of
identity theft.
Note that there are helpers, such as ssh-agent(1)
, that can store the
passphrase for some time so you do not have to enter it every time you
use the key.
If you have multiple computers, it is preferred to use a different public/private key pair on each machine. If one machine is compromised, it it sufficient to remove its public key from all applications, while you can still use the other key pairs.
Cryptography overview: check you understand the basics
Using SSH
Enough of theory: let us connect to some remote machine. Let us explore SSH.
What is SSH?
SSH – that stands for Secure Shell – is a protocol for connecting to a different machine and running a shell there.
From a user perspective, after you SSH from a Linux machine into a different Linux machine, the shell may look the same and some commands would behave completely the same. Except they might be running on a different computer.
Note that this is intentional: remote shell is a natural way to control a Linux machine. No need to make it different from controlling it through a local shell.
SSH practically
Using SSH is very simple (unless you make it complex). To SSH to a remote machine, you need to know your credentials (i.e., login name and a password) and, of course, the name of the remote machine.
Then simply execute the following:
ssh YOUR_LOGIN@REMOTE_MACHINE_NAME
Note that the command ssh
is often called a SSH client as it connects to the
SSH server (similar to curl
or wget
being web clients connecting
to a web server).
We have set up a remote machine for you on linux.ms.mff.cuni.cz
.
You will be using your GitLab (SIS/CAS) login and also the same password.
ssh your_gitlab_login@linux.ms.mff.cuni.cz
If your GitLab account was created manually, probably your SIS password will not work on this machine.
Please, contact us via this link and we will create an account for you manually.
The first login to the machine is a bit more complicated. SSH client wants from you a verification that you trust the remote server. It shows you a so-called server fingerprint:
The authenticity of host 'linux.ms.mff.cuni.cz (195.113.20.170)' can't be established.
ECDSA key fingerprint is SHA256:ltoc1TjoYhCZk6b8qSTAL6wFsRv7blw6u3h6NqCcYvI.
Are you sure you want to continue connecting (yes/no/[fingerprint])?
Based on your configuration, RSA or ED25519 key may be used instead:
RSA: SHA256:Z11Qbd6nN6mVmCSY57Y6WmxIJzqEFHFm47ZGiH4QQ9Y
ED25519: SHA256:/CVwDW388z6Z5VlhLJT0JX+o1UzakyQ+S01+34BI0BA
You should have received this fingerprint in a secure way before connecting to the server (for example, printed from your employer etc.). In this case, we hope that a potential attacker would not be able to break both into this web server and the SSH server at once. So we use the HTTPS protocol on the web as a secure way of verifying the SSH server.
The program then continues to ask your for a password and also informs you that the fingerprint was stored.
Enter
.
On following logins, the SSH client checks that the fingerprint has not changed. A changed fingerprint belonging to the same machine (i.e., with the same DNS name) could indicate a man-in-the-middle attack. Do not connect when the fingerprint has changed. Always contact the administrator (e.g., the teachers in this particular case) and check with them what is happening.
If you were able to log-in, you will see an (almost) empty home directory and otherwise a normal Linux machine, this time without any graphical applications installed.
Try to run lscpu
to see what machine you have logged in to.
Note that this machine is shared for all students of this course. Use it to solve graded tasks or to experiment with commands we show in the labs. You will be using it for the rest of the course for several tasks so keep it usable, please.
Do not use it for computationally intensive tasks or other tasks that are not related to this course.
We also strictly prohibit to use it for any kind of remote development with tools such as Visual Studio, IntelliJ IDEA or similar IDEs. These tools install huge blobs of code/data on the remote machine and because they rarely remove old versions, they are very quick in taking all free space available for themselves.
If we encounter any form of abusive use, we will block the offending account.
Whenever you are debugging issues with SSH, run the SSH client (ssh(1)
)
with the -v
option that turns on verbose logging.
It will print which keys were tried, full identification of the remote machine
and other useful details.
When creating an issue about non-functional SSH, please, always paste the
output of ssh -v
there. Thank you!
Configuration of $PS1
We have already touched this a little bit in the previous lab as an extra for further reading.
$PS1
specified how your prompt looks like. From now on, you should ensure
that your $PS1
shows also the machine name so that you always know where
you are (i.e., which machine you are logged into).
Similarly to setting EDITOR
in your ~/.bashrc
you should ensure to specify
the following (at least) if the default does not display the machine name.
PS1='\u@\h \w\$'
This ensures that you can see your username (\u
), the hostname (\h
) and
also the working directory (\w
).
Which ~/.bashrc
you need to modify?
Answer.
Return to previous lab if you want to see more options about setting your prompt.
\u@\h
on all remote machines to
keep short prompt on the personal machine but keep visual distinction
when working on a remote one.
Using SSH to run one command
The command ssh
is actually quite powerful and configurable.
One important feature is that you can specify a command after the hostname and
run this command directly.
In this case, SSH never starts an interactive shell on the remote machine,
but executes only the given command and returns.
Following commands prints uname -r
on the remote machine.
ssh user@hostname uname -r
Investigate how (and why) the following command behaves differently.
The difference is visible only if you are SSHing to a different machine,
that is when you execute ssh
on your local machine.
ssh user@hostname uname -r >local_file.txt
ssh user@hostname "uname -r >remote_file.txt"
To verify that you are on a different machine, run on both uname -a
and hostname -f
(it provides the full DNS name of the current machine).
Can you see the differences?
You can also trying checking free -h
or uptime
.
SSH and passwordless authentication
To enable authentication with public key over SSH, we need to perform two steps. Generate the public-private key pair and copy the public key to the remote machine.
To generate the public key we need to run the following command (the -C
is actually a comment, providing e-mail or your login is the usual approach
but keeping it empty is possible too).
Key generation always happens on the local machine, i.e. the one that we will eventually connect from.
We may want to generate a new key pair on the remote machine too
(e.g. on linux.ms.mff.cuni.cz
).
But then that pair would be used for authentication to another service, such
as GitLab (i.e. linux.ms.mff.cuni.cz
would be the local one in the communication).
ssh-keygen -t ed25519 -C "your_email@example.com"
The program would ask us where to store the generated private and public
key.
Keep the defaults.
Choose if you want a passphrase or not (it is more secure to have one,
but the use is a bit more cumbersome).
After ssh-keygen
finishes, check that the directory ~/.ssh
contains id_ed25519
(the private key) and id_ed25519.pub
(the public key).
Feel free to inspect their content, e.g. using cat
.
Surprised that they are text files?
Once we have the public key ready, we need to upload it to the remote machine.
If you have multiple key pairs, read about the -i
switch.
ssh-copy-id LOGIN@REMOTE_MACHINE
If you log in to the remote machine again, the SSH client should use
the key pair and log you in without asking for a password.
If not, run SSH with -vvv
to debug the issue.
Note that the public key was stored into ~/.ssh/authorized_keys
file
on the remote machine.
You can copy it there manually but using ssh-copy-id
is easier.
If the copying fails with cryptic message about warning: here-document at line 251 delimited by end-of-file (wanted EOF), try upgrading the SSH client first.
If you use the image from us, simple sudo dnf upgrade openssh-clients
should work.
Using keys only
Note that some services actually require that you authenticate using a key pair instead of a password as it is considered more secure.
The advantage is that any random attackers could keep guessing your password and still never get access to your machine.
Automated client ban
Typically, SSH server is also configured to ban any client that tries to login without success several times in a row. Our server does that, too.
Copying files
If you have deciphered what the following command does, you should by now have an idea how to copy files to and from a remote machine (we are not saying it would be the most effective way):
ssh user@hostname "uname -r >remote_file.txt"
Wondering how this helps you copy the files? Think about it a bit more before reading on, consider how pipes are used etc.
Have you really thought about it? :-)
To copy the files, we can leverage cat
that simply dumps the file as-is
to stdout.
We will pipe its output to SSH and on the other machine run second cat
to store the file.
cat README.md | ssh user@hostname "cat >remote-readme.md"
Now, please, ensure that you can explain what is happening. SSH is able
to read stdin
(after all, we were typing our commands locally and they
appeared in the remote shell on our local screen) and hence we can tunnel
the data through it.
There are also scp
and rsync
that can be used for copying multiple files
over SSH easily but we will talk about these in later labs.
File managers
Many file managers allows you to use SSH transparently and copy between machines with the same ease as when working with local files.
For example, in mc
, select Shell connection either in left or right
panel and specify SSH connection.
One of the panels will show the files on the remote machine.
Try using F5
to copy files interactively.
Using machines in Rotunda
Apart from the machine linux.ms.mff.cuni.cz
, there is also a full lab of
machines available in the Rotunda computer lab on Malostranské náměstí.
All the Linux machines in the lab are also reachable via SSH.
Again, use your SIS credentials to log in.
Note that all machines in Rotunda (but not linux.ms.mff.cuni.cz
!) share the
same home directory, i.e., it does not matter which one you physically connect to.
Your files will be available on all machines.
u-pl1.ms.mff.cuni.cz
and creating
a file there is the same as if you physically attend lab in N8/N11 and create
the lab while logged in to the machine via GUI).
Unfortunately, the used file system and authentication mechanism does not allow
to use public key authentication for Rotunda machines.
You need to always specify your password.
(linux.ms.mff.cuni.cz
does not have this limitation and we expect you will
use public key authentication there.)
Following machines are available.
- Lab SU1
u1-1.ms.mff.cuni.cz
u1-2.ms.mff.cuni.cz
- …
u1-14.ms.mff.cuni.cz
- Lab SU2
u2-1.ms.mff.cuni.cz
u2-2.ms.mff.cuni.cz
- …
u2-25.ms.mff.cuni.cz
- Rotunda
u-pl1.ms.mff.cuni.cz
u-pl2.ms.mff.cuni.cz
The computers in Lab SU1 and SU2 can not be connected to via SSH during the day (or more specifically during the scheduled classes). That is to prevent remote users to interfere with students learning a subject. Rotunda lab machines are accessible 24/7.
SSH: review your knowledge
Unix-style access rights
So far we have silently ignored the fact that there are different user accounts on any Linux machine. And that users cannot access all files on the machine. In this section we will explain the basics of Unix-style access rights and how to interpret them.
After all, now you can log in to a shared machine and you should be able to understand what you can access and what you cannot.
Recall what we said about /etc/passwd
earlier – it contains the list of user accounts
on that particular machine
(technically, it is not the only source of user records, but it is a good
enough approximation for now).
Every running application, i.e., a process, is owned by one of the users
from /etc/passwd
(again, we simplify things a little bit).
We also say that the process is running under a specific user.
And every file in the filesystem (including both real files such as ~/.bashrc
and virtual ones such as /dev/sda
or /proc/uptime
) has some owner.
When a process tries to read or modify a file, the operating system decides whether the operation is permitted. This decision is based on the owner of the file, the owner of the process, and permissions defined for the file. If the operation is forbidden, the input/output function in your program raises an exception (e.g., in Python), or returns an error code (in C).
Since a model based solely on owners would be too inflexible, there are also
groups of users (defined in /etc/group
). Every user is a member of one or
more groups, one of them is called the primary group. These are associated
with every process of the user. Files have both an owning user and an owning group.
Files are assigned three sets of permissions: one for the owner of the file, one for users in the owning group, and one for all other users. The exact algorithm for deciding which set will be used is this:
- If the user running the process is the same as the owner of the file, owner access rights are used (sometimes also referred to as user access rights).
- If the user running the process is in a group that was set on the file group access rights are used.
- Otherwise, the system checks against other access rights.
Every set of permissions contains three rights: read (r
), write (w
), execute (x
):
-
Read and write operations on a file are obvious.
-
The execute right is checked when the user tries to run the file as a program (recall that without
chmod +x
, the error message was in the sense of Permission denied: this is the reason).- Note that a script which is readable, but not executable, can be still run by launching the appropriate interpreter manually.
- Note that when a program is run, the new process will inherit the owner
and groups of its parent (e.g., of the shell that launched it). Ownership
of the executable file itself is not relevant once the program was started.
For example,
/usr/bin/mc
is owned byroot
, yet it can be launched by any user and the running process is then owned by the user who launched it.
The same permissions also apply to directories. Their meaning is a bit different, though:
- Read right allows the user to list directory entries (files, symlinks, sub-directories, etc.).
- Write right allows the user to create, remove, and rename entries inside that directory. Note that removing write permission from a file inside a writable directory is pointless as it does not prevent the user from overwriting the file completely with a new one.
- Execute right on a directory allows the user to open the entries.
(If a directory has
x
, but notr
, you can use the files inside it if you know their names; however, you cannot list them. On the contrary, if a directory hasr
, but notx
, you can only view the entries, but not use them.)
Permissions of a file or directory can be changed only by its owner, regardless of the current permissions. That is, the owner can deny access to themselves by removing all access rights, but can always restore them later.
root
account
Apart from accounts for normal users, there is always an account for a so-called superuser – more often called simply just root – that has administrator privileges and is permitted to do anything with any file in the system. The permissions checks described above simply do not apply to root-owned processes.
Unlike other systems, Linux is designed in such way that end-user
programs are always executed under normal users and never require root privileges.
As a matter of fact, some programs (historically, this was a very common behaviour
for IRC chat programs) would not even start under root
.
Viewing and changing the permissions
Looking at the shortcuts of rwx
for individual permissions, you may found
them familiar:
drwxr-xr-x 1 intro intro 48 Feb 23 16:00 02/
drwxr-xr-x 1 intro intro 60 Feb 23 16:00 03/
-rw-r--r-- 1 intro intro 51 Feb 23 16:00 README.md
The very first column actually contains the type of the entry (d
for directory, -
for a plain file, etc.)
and three triplets of permissions.
The first triplet refers to owner of the file, the middle one to the group,
and last to the rest of the world (other).
The third and fourth columns contain the owner and the group of the file.
Typically, your personal files in your home directory will have you as the owner together with a group with the same name. That is a default configuration that prevents other users from seeing your files.
Do check that it is true for all directories under /home
on the shared
machine.
But also note that most of the files in your home directory are actually world-readable (i.e., anyone can read them).
That is actually quite fine because if you check permissions for your ~
,
you will see that it is typically drwx------
.
Only the owner can modify and cd
to it.
Since no one can actually change to your directory, no one will be able to read
your files
(technically, reading a file involves traversing the whole directory and checking
access rights on the whole path).
To change the permissions, you can use chmod
program.
It has the general format of
chmod WHO[+=-]PERMISSION file1 file2 ...
WHO
can be empty (for all three of user, group and others) or specification
of u
, g
or o
.
And PERMISSION
can be r
, w
or x
.
It is possible to combine these, e.g. g-r,u=rw
removes read permission
for group and sets read-write for the owner (i.e. the file will not
be executable by the owner regardless of a previous state of the executable bit).
Sticky and other bits
If you execute the following command, you will see a bit different output that you would probably expect.
ls -ld /usr/bin/passwd /tmp
drwxrwxrwt 23 root root 640 Mar 3 08:15 /tmp/
-rwsr-xr-x 1 root root 51464 Jan 27 14:47 /usr/bin/passwd*
You should have noticed that /tmp
has t
in place of an executable
bit and passwd
has s
there.
Those are special variants of the executable bit.
The t
bit on a directory specifies that user can remove only their
own files.
The reason is obvious – you shall not be able to remove someone else’s
files inside /tmp
.
Something that is otherwise impossible to specify with traditional
(basic) permissions.
The s
bit (set-uid) is a bit more tricky.
It specifies that no matter who runs the shell, passwd
will be
running under the user owning the file (i.e., root
for this file).
While it may look useless, it is a simple way to allow running
certain programs with elevated (higher) permissions.
passwd
is a typical example.
It allows the user to change their password.
However, this password is stored in a file that is not readable by
any user on the system except root
(for obvious reasons).
Giving the s
bit to the executable means that the process would be
running under root
and would be able to modify the user database
(i.e., /etc/passwd
and /etc/shadow
that contains the actual
passwords).
Since changing the permissions can be done only by the owner of the file,
there is no danger that a malicious user would add the s
bit
to other executables.
There are other nuances regarding Unix permissions and their setting, refer to
chmod(1)
for details.
Beyond traditional Unix permissions: POSIX ACL
The permission model described above is a rare example of a concept coming from Unix that is considered inflexible for use today. However, it is also considered as a typical example of a simple but well usable security model.
Many programs copied this model and you can encounter it in other places too. It is definitely something worth remembering and understanding.
The inflexibility of the system comes from the fact that allowing
a set of users to access a particular file means creating a special
group for these users.
These groups are defined in /etc/group
and changing them requires administrator
privileges.
With an increasing number of users, the amount of possibly needed groups grows exponentially. On the other hand, for most situations, the basic Unix permissions are sufficient.
To tackle this problem, Linux offers also so-called POSIX access control lists where it is possible to assign an arbitrary list of users to any file to specify the permissions.
getfacl
and setfacl
are the utilities
to control these rights but since these are practically needed rarely,
we will leave their knowledge at the level of reading the corresponding
manpages and acl(5)
.
Access rights checks
Let’s return a little bit to the access rights.
Change permission of some of your scripts to be --x
.
Try to execute them.
What happens?
Answer.
Remove writable bit for a file and write to it using stdout redirection. What happens?
Access rights quiz
Assuming the following output of ls -l
(script.sh
is really a shell script)
and assuming user bob
is in group wonderland
while user lewis
is not.
-rwxr-xr-- 1 alice wonderland 1234 Feb 20 11:11 script.sh
Select all true statements.
You need to have enabled JavaScript for the quiz to work.Git over SSH
So far, we have used Git over HTTPS. Git can be used over SSH too. Then, the traffic is basically tunneled through an SSH connection and Git relies on the SSH wrapper for security as well as for (partial) authentication.
Virtually all Git servers (GitLab, GitHub, Bitbucket…) will require you to upload your public key if you want to use Git over SSH.
To actually use Git over SSH, we first need to tell GitLab about our SSH keys (recall the protocol that is used to authenticate the user).
GitLab and SSH keys
Copy your public key to GitLab. Navigate to right-top menu with your avatar, select Preferences and then SSH keys or visit this link.
Copy your public key there and name it. Typically, the name should mention your username and your machine. Note that GitLab will send you an e-mail informing you about a new key. Why? Hint. Answer.
Go to your project and clone it again. This time, use the Clone with SSH URL.
git clone git@gitlab.mff.cuni.cz:teaching/nswi177/2024/student-LOGIN.git
Have you noticed that this looks like SSH address?
It actually is exactly that.
The first part identifies the machine and the user (git
)
and after a colon is a local path.
This way, you can clone a Git directory from any SSH server by specifying its remote path there (here, GitLab does some mangling but the principle holds).
Note that the user we clone with is git
– not you.
This way, GitLab needs only one physical user account for handling Git requests
and distinguishes the users via their public keys.
How?
Answer.
By the way, what happens if you try to run the following?
ssh git@gitlab.mff.cuni.cz
Note that you should prefer the SSH protocol for working with Git as it is much more comfortable for use.
Rest of the work with Git remains the same. git add
, git commit
, git push
etc.
will work the same but only the communication with GitLab goes through SSH
tunnel. Note that you don’t have to reenter your credentials while doing git push
.
This is because git remembers how did you git clone
the repository and will use
the same URL (either HTTPS or SSH) for git push
as well (unless you configure it
otherwise). And since you used SSH for pulling, it will use SSH for pushing as well,
which uses the public/private authentication you already set up.
Processes
Files in the system are the passive elements of it. The active parts are running programs that actually modify data. Let us have a look what is actually running on our machine.
When you start a program (i.e., an executable file), it becomes a process. The executable file and a running process share the code – it is the same in both. However, the process also contains the stack (e.g., for local variables), heap, current directory, list of opened files etc. etc. – all this is usually considered a context of the process. Often, the phrases running program and process are used interchangeably.
To view the list of running processes on our machine,
we can use htop to view basic properties of processes.
Similar to MidnightCommander, function keys perform the most important actions
and the help is visible in the bottom bar.
You can also configure htop
to display information about your system like amount of free memory
or CPU usage.
For non-interactive use we can execute ps -e
(or ps -axufw
for a more detailed list).
For illustration here, this is an example of ps
output (with --forest
option use to depict also the parent/child relation).
However, run ps -ef --forest
on the shared machine to also view running
processes of your colleagues.
Listing of processes is not protected in any way from other users. Every user on a particular machine can see what other users are running (including command-line arguments).
Keep in mind to never pass passwords as command line arguments and pass them always through files (with proper permissions) or interactively on stdin.
UID PID PPID C STIME TTY TIME CMD
root 2 0 0 Feb22 ? 00:00:00 [kthreadd]
root 3 2 0 Feb22 ? 00:00:00 \_ [rcu_gp]
root 4 2 0 Feb22 ? 00:00:00 \_ [rcu_par_gp]
root 6 2 0 Feb22 ? 00:00:00 \_ [kworker/0:0H-events_highpri]
root 8 2 0 Feb22 ? 00:00:00 \_ [mm_percpu_wq]
root 10 2 0 Feb22 ? 00:00:00 \_ [rcu_tasks_kthre]
root 11 2 0 Feb22 ? 00:00:00 \_ [rcu_tasks_rude_]
root 1 0 0 Feb22 ? 00:00:09 /sbin/init
root 275 1 0 Feb22 ? 00:00:16 /usr/lib/systemd/systemd-journald
root 289 1 0 Feb22 ? 00:00:02 /usr/lib/systemd/systemd-udevd
root 558 1 0 Feb22 ? 00:00:00 /usr/bin/xdm -nodaemon -config /etc/X11/...
root 561 558 10 Feb22 tty2 22:42:35 \_ /usr/lib/Xorg :0 -nolisten tcp -auth /var/lib/xdm/...
root 597 558 0 Feb22 ? 00:00:00 \_ -:0
intro 621 597 0 Feb22 ? 00:00:40 \_ xfce4-session
intro 830 621 0 Feb22 ? 00:05:54 \_ xfce4-panel --display :0.0 --sm-client-id ...
intro 1870 830 4 Feb22 ? 09:32:37 \_ /usr/lib/firefox/firefox
intro 1966 1870 0 Feb22 ? 00:00:01 | \_ /usr/lib/firefox/firefox -contentproc ...
intro 4432 830 0 Feb22 ? 01:14:50 \_ xfce4-terminal
intro 4458 4432 0 Feb22 pts/0 00:00:11 \_ bash
intro 648552 4458 0 09:54 pts/0 00:00:00 | \_ ps -ef --forest
intro 15655 4432 0 Feb22 pts/4 00:00:00 \_ bash
intro 639421 549293 0 Mar02 pts/8 00:02:00 \_ man ps
...
First of all, each process has a process ID, often just PID
(but not this one).
The PID is a number assigned by the kernel and used by many utilities for process management.
PID 1 is used by the first process in the system, which is always running.
(PID 0 is reserved as a special value – see fork(2)
if you are interested in details.)
Other processes are assigned their PIDs incrementally (more or less) and PIDs are eventually
reused.
Note that all this information is actually available in /proc/
PID/
and that is where ps
reads its information from.
Execute ps -ef --forest
again to view all process on your machine.
Because of your graphical interface, the list will be probably quite long.
Practically, a small server offering web pages, calendar and SSH access can have about 80 processes, for a desktop running Xfce with browser and few other applications, the number will rise to almost 300 (this really depends a lot on the configuration but it is a ballpark estimate). About 50–60 of these are actually internal kernel threads. In other words, a web/calendar server needs about 20 “real” processes, a desktop about 200 of them :-).
Forcefully terminating processes
At this moment we will show probably the most important thing that you can do with processes. And that is their forceful termination.
Eventually we will learn about the concept of signals, at the moment we resort ourselves to the basic two commands.
The pgrep
command can be used to find processes matching a given name.
Open two extra terminals and run sleep 600
in one and sleep 800
in the
second one.
The sleep
program simply waits given amount of seconds before terminating.
In a third terminal, run the following commands to understand how the searching for the processes is done.
pgrep sleep
pgrep 600
pgrep -f 600
What have you learnt? Answer.
When we know the PID, we can use the kill
utility to actually terminate the
program.
Try running kill PID
with PID of one of the sleeps and look what happened
in the terminal with sleep
.
You should see something like this:
Terminated (SIGTERM).
This message informs us that the command was forcefully terminated.
Some programs ignore the kill
command and do not terminate.
We will explain why that is possible in some of the next labs but for now
we want to mention that it is possible to add -9
to the kill
command
which instructs the operating system to be a bit more forceful and
terminate the program without giving it any option to disagree ;-).
You can always kill your own processes but killing processes of someone else is not possible.
Quick check on processes
Going further
Few extra bits that will improve your user experience with SSH a lot but can be returned to any time later.
Using tmux
for better SSH experience
tmux
is a terminal multiplexer.
That means that inside one terminal it opens several terminals for you that are
running in parallel.
It also allows you to send the session into background so that it remains
there even if you log out or your remote connection is interrupted (this is useful for running long scripts).
In other words, tmux
gives you tabs (called windows) inside your existing session
that can be minimized/iconified (if borrowing terms from GUI would explain
the usefulness better).
The simplest way how to start tmux
session is simply:
tmux
Alternatively, we can start session with some meaningful name:
tmux new -s <session_name>
To list all sessions run:
tmux ls
To connect/attach to the running session run:
tmux attach -t <session_name>
And finally, in order to kill the session we use:
tmux kill-session -t <session_name>
Inside the session we are able to create multiple windows, split the screen and much more.
In order to invoke a tmux
command, we need firstly to type tmux
prefix.
The default key binding is <Ctrl>-b
.
In order to detach from session we can simply press (do not forget to type the prefix!):
d detach from session
Operation with windows:
c create window
w list windows
n next window
p previous window
f find window
, name window
& kill window
Sometimes it is useful to split the screen into several terminals. These splits are called panes.
Operation with panes (splits):
% vertical split
" horizontal split
o swap panes
q show pane numbers
x kill pane
← switch to left pane
→ switch to right pane
↑ switch to upward pane
↓ switch to downward pane
Other feature is that you can toggle writing simultaneously to all panes. Performing the same operation multiple times may seem not much useful, but you can for example open several different ssh connections in advance and then interactively control all these computers at the same time.
To toggle it, type the prefix and then write :set synchronize-panes
.
If you want to try this in Rotunda,
please do not run computationally intensive tasks…
As usual with Linux tools, you can modify its behavior widely via rc
configuration.
For instance, in order to navigate among the panes with vim
shortcuts,
modify your .tmux.conf
so it contains
bind-key C-h run "tmux select-pane -L"
bind-key C-j run "tmux select-pane -D"
bind-key C-k run "tmux select-pane -U"
bind-key C-l run "tmux select-pane -R"
Personal tip № 0: tmux
is an excellent tools for collaboration, as multiple users can attach to the same session.
Personal tip № 1: when you give a lecture, you can attach to the tmux
session from two terminals.
Later on, you push the first one to the projector, while the second one is on you laptop screen.
This eliminates the necessity of mirroring your screen.
Together with pdfpc
and a tiling window manager we get a Swiss-army knife for presentation.
There is much more to say. For more details see this tmux cheatsheet or manual pages.
SSH configuration
The SSH client is configured via the ~/.ssh/config
file.
Review the syntax of this file via man 5 ssh_config
.
The file is divided into sections.
Each section is related to one or more remote hosts.
The section header is in the format Host pattern
, where pattern might use wildcards.
The syntax is mostly self-explanatory, so we will only provide an example.
Host *
IdentityFile ~/.ssh/id_ed25519
Host intro
Hostname linux.ms.mff.cuni.cz
User YOUR_SIS_LOGIN
Host mff0
Hostname u-pl0.ms.mff.cuni.cz
User YOUR_SIS_LOGIN
Host mff1
Hostname u-pl1.ms.mff.cuni.cz
User YOUR_SIS_LOGIN
With this ~/.ssh/config
, we can type ssh intro
and the ssh
will start connection equivalent to
ssh YOUR_SIS-LOGIN@linux.ms.mff.cuni.cz
Tasks to check your understanding
We expect you will solve the following tasks before attending the labs so that we can discuss your solutions during the lab.
Learning outcomes
Learning outcomes provide a condensed view of fundamental concepts and skills that you should be able to explain and/or use after each lesson. They also represent the bare minimum required for understanding subsequent labs (and other courses as well).
Conceptual knowledge
Conceptual knowledge is about understanding the meaning and context of given terms and putting them into context. Therefore, you should be able to …
-
explain basic principles of network communication (in OS agnostic manner)
-
explain basic principles of asymmetric cryptography
-
explain in detail how asymmetric cryptography (public and private key) can be used to authenticate a user
-
explain what is SSH and what functions it offers
-
explain what account types exist on Linux and how they differ (e.g.
johndoe
,root
andnobody
) -
explain how differs execution of programs locally vs over SSH remotely
-
explain difference between using Git over HTTPS vs Git over SSH
-
explain basic access rights in Unix operating systems
-
explain what individual access rights
r
,w
andx
mean for normal files and what for directories -
explain what is a set-uid bit
-
explain what is a process and how it differs from an executable file
-
explain difference of ownership of a file and a running process
-
optional: provide a high-level overview of POSIX ACLs
Practical skills
Practical skills are usually about usage of given programs to solve various tasks. Therefore, you should be able to …
-
set
PS1
variable to distinguish different machines -
use
ssh
command to login to a remote machine -
execute commands on a remote machine using SSH
-
use
hostname
command -
transfer files over SSH using
cat
-
setup password-less authentication on a remote Linux machine using a private/public key pair
-
setup publickey-based authentication to GitLab
-
use
git clone
(andpull
andpush
) over SSH -
view and change basic access permissions of a file
-
use
ps
to view list of existing processes (including-e
,-f
and--forest
switches) -
use
htop
to interactively monitor existing processes -
optional: configure SSH shortcuts
-
optional: use basic functions of
tmux
terminal multiplexer
This page changelog
-
2024-03-11: Add notice about
ssh -v
. -
2024-03-14: Notice about case-sensitivity of logins.
-
2024-03-18: Fix list of machines available in Rotunda.
-
2024-04-08: Shared home in IMPAKT and Rotunda.