Organizational stuff

  • Grading rules have been finalized.
  • Note for 0th homework: never, ever push your private key—not even to demonstrate that it’s password protected as requested. The key is yours and yours only, keep it secure at all times!

Introduction

Being able to use the computer efficiently is one of the most important take-aways of this course. Of the many things you will learn, this one will benefit you for the years to come, whether your career is in research or in the IT industry.

The point of the lecture is to show you the extent to which you can customize your Linux work environment to suit your needs.

A non-goal is to configure all that I present at once—that would be very difficult. Instead, you should focus on how your work environment affects the way you use your computer, identify inefficiencies in the process and then hopefully get rid of them, perhaps using some of the tricks I mentioned.

Note on English

This class will be taught in English. Rationale: it makes sense to use English whenever we can, everybody else does. Plus, this year there are many students who don’t speak Czech.

Systray

I’m using a minimal systray based on Dzen which only shows the information I care about all the time. Conky is a heavier alternative. You probably want to configure something similar (unless provided by your desktop environment, if you are using any).

Tiling window manager

Currently the incumbent windowing system for Linux is Xorg (also known as just X, or X11), slowly being replaced by Wayland. By itself, the X window system doesn’t get you very far though, since you have no useful way of managing the windows. You need a window manager (WM) for that. One class of WMs which is relevant for our discussion about efficiency is tiling windows managers.

Tiling window managers are great efficiency boosters. A tiling WM divides the screen into rectangular areas hosting clients (X11 terminology for graphical programs using the X server to render stuff). Unlike a typical window-based GUI you may know from other operating systems, tiling WMs are usually keyboard-centric. Mouse is frowned upon for several reasons discussed below.

Tiling WMs roughly divide into two broad classes:

  • Manual tiling WMs which leave you in complete charge of window placement.
  • Dynamic tiling WMs which attempt to optimize the layout for you.

Some of these are installed on the lab machines, so give them a try! :-) If you want to start using them on your machine, see xinit on ArchWiki.

You should learn the following:

  • Horizontally and vertically splitting windows frames (sticking with Herbstluftwm terminology)
  • Launching programs (dmenu is a nice and simple menu program which can be used as an app launcher)
    • Both dmenu and dwm are Suckless.org projects which aims to develop software that sucks less! You may want to check their other projects too.
  • Resizing frames
  • Moving windows between frames
  • Moving windows between tags (“desktops”)
    • I showed tags 1..9 which each use a different layout and hold a different set of clients
    • It is useful to devote each tag to a different task (one for browsing, other for mail, then one tag for each machine you’re ssh-ed into when doing the homework, etc.)
  • Check out the Herbstluftwm tutorial.

The Mouse

…is a great input device for gaming and graphics. Since we do neither in this course, we won’t be using it.

While you can use the keyboard both for text input and to interact with your environment, the mouse can only be used for the latter. This means you need to frequently switch between the mouse and the keyboard, moving one of your hands off the keyboard.

  • This repetitive movement can cause fatigue and muscle and joint pain if practiced for many hours every day for years.
  • It requires you to frequently reposition your hand onto the keyboard. That slows you down.

The trackpoint was invented specifically to allow you to use the mouse while not moving your hands off the keyboard and it works pretty well.

The Keyboard

Is where your hands should be at all times. There are markers on F and J keys which you can find with your index fingers without having to look at the keyboard. Speaking of which:

Don’t look at the keyboard! Look at the screen at the text you’re typing. Reasoning is largely the same: avoid eye and neck strain due to frequent switching between looking at the screen and looking at the keyboard. Learn this now.

Don’t use standard Czech keyboard layout (QWERTZ, “Czech QWERTY”, “Czech programming” and other abominations) for programming, they suck. We recommend that you use the US layout (QWERTY) + UCW variant of the CZ layout:

setxkbmap -layout us,cz -variant ,ucw -option grp:switch

Press Right Alt + z, you’ll figure out the rest from here. This allows you to use a reasonable keyboard layout while also being able to use relevant diacritics.

Note on alternative (non-QWERTY) layouts: those can work very well. One thing to keep in mind is that it makes you incompatible with virtually every keyboard out there. It is a lot of fun bringing your own keyboard to a job interview. At the same time, the gains from touch typing vastly eclipse any gains from using a more optimized layout, at least in our opinion.

Shell

The $SHELL variable holds the name of the shell path to the shell binary you’re running (unless you overwrite it).

Prompt

Prompt is really useful, customize it. Whatever you set as the PS1 variable will be interpreted by the shell and the output used as your prompt:

  • PS1='> ' (as minimal as it gets)
  • PS1='$(date)> ' (not useful but shows how PS1 works)
  • PS1='$?> ' (prior command’s exit code, very useful)
  • PS1='$(prompt_exit)%B%0~%b%# $(prompt_git)' with the following functions defined in ~/.zhsrc:
prompt_git () {
        [ -n "$cwd_git_root" ] || return
        branch="$(git rev-parse --abbrev-ref HEAD 2>/dev/null)"
        printf "(%s) " "$branch"
}

prompt_exit () {
        local ec=$?
        [ "$ec" -ne 0 ] && printf "%d " "$ec"
}

Command input

Be careful with copy/paste:

  • Ctrl+C/Ctrl+V copy/paste in graphical applications
  • Ctrl+C/Ctrl+V have a different meaning in terminal based applications; they send the interrupted signal (SIGINT) and enter the verbatim insert mode (mostly useless these days) respectively
  • Ctrl+Shift+C/Ctrl+Shift+V copy/paste in terminal based applications (mostly)

Note that there are two three “clipboards”:

  • The “clipboard” clipboard, the one you use with Ctrl+C/Ctrl+V and Ctrl+Shift+C/Ctrl+Shift+V
  • The “selection” (also called “primary”) clipboard. It is enough to select something with the mouse and it will become the contents of the selection clipboard; middle mouse button or Shift+Insert pastes pastes the contents
  • There is also “secondary selection” clipboard, but it’s seldom used.

Make sure you know how to use the clipboard with your terminal: take advantage of code examples in the text and run them (if you think it’s safe to do so).

You may be used to hitting Ctrl+S to save a file. If you do that in a terminal, most likely you will freeze the screen: Ctrl+S sends a “STOP” terminal sequence which prevents further output to the terminal. Ctrl+Q resumes output.

Use Tab completion. Virtually every interactive shell (Bash, Zsh, Ksh, Fish, …) supports Tab completion for file names:

$ touch an-incredibly-long-useless-file-name-which-is-a-pain-to-write
$ rm an-# hit Tab at this place

Tab completion will greatly improve your speed of input, and will prevent typos. You can hit Tab blindly to get a list of files in the current directory.

Tab completion goes beyond just filenames, you can also autocomplete the names of commands. grml-zsh-config is an Arch Linux package providing Tab completion for Zsh which is able to autocomplete even things like certain options of certain commands, names of systemd units, etc. You might want to give that a try to make life a bit easier on you.

Use shell history. You can navigate the history with the arrow keys, but that isn’t very powerful. It’s much better to use Ctrl+R to trigger reverse search (with glob support). For Zsh, $HISTFILE and $HISTSIZE control the location and maximum number of entries in a history file:

$ cat /etc/zsh/zshenv
# [...]
# Consult zshparam(1)
export HISTFILE="$ZDOTDIR/history"
export SAVEHIST=10000000
export HISTSIZE="$SAVEHIST"
# [...]

$ cat /etc/zsh/zshrc
# [...]
# Consult zshoptions(1)
setopt inc_append_history
setopt share_history
setopt hist_verify
setopt hist_ignore_space
# [...]

For Bash, the following do a good job:

shopt -s histappend

export HISTCONTROL=ignorespace
export HISTFILESIZE=-1
export HISTSIZE=-1

This way, your shell will keep history virtually forever. You will be building a knowledge base automatically from the commands you used in the past several years. This is incredibly useful in combination with reverse search.

If Ctrl+R isn’t good enough for you, give RESH a try.

History expansion is useful:

  • ^foo^bar to run the previous command where the first foo is replaced with bar.
  • sudo !! to run the previous command with sudo—that’s just an example, !! expands to the previous command.
  • !:1, !:2 etc. expand to the previous command’s first, second, … argument
  • Depending on your shell, Esc+. may insert last argument of the previous command (!$ almost always works)
  • Many things are possible
  • setopt prompt_subst to prompt before executing a command line with history substitution

You can change line editing mode with set -o vi (Vim key bindings) and set -o emacs (Emacs key bindings). I use the following hybrid bindings:

bindkey -v

bindkey "^A" beginning-of-line
bindkey "^B" backward-word
bindkey "^E" end-of-line
bindkey "^F" forward-word
bindkey "^H" history-incremental-pattern-search-backward
bindkey "^K" kill-line
bindkey "^L" clear-screen
bindkey "^U" kill-whole-line
bindkey "^W" backward-kill-word
bindkey "^Y" yank
bindkey '^N' down-history
bindkey '^O' accept-line-and-down-history
bindkey '^P' up-history

bindkey -M viins '\e.' insert-last-word # Esc+.

autoload -U edit-command-line
zle -N edit-command-line
bindkey -M vicmd v edit-command-line

Then:

  • You can use Ctrl+L or the clear command to clear the screen.
  • Ctrl+W to delete a word
  • Ctrl+A to move cursor to beginning of command line
  • Ctrl+E to move cursor to end of command line
  • Esc to switch to Vim normal mode while editing the command line
    • v in normal mode to edit command line in Vim
    • Many other Vim key bindings work in this mode

Aliases create useful shortcuts for frequently used or unwieldy commands. Run alias to list your aliases. For example, I use the following Git aliases:

alias g=git
alias ga='g add'
alias gaa='g add .'
alias gau='g add -u'
alias gcane='g commit --amend --no-edit'
alias gd='g diff -w'
alias gdc='gd --cached -w'
alias gpr='g pull --rebase'
alias gs='g status'
alias gsh='git show -w'

Add aliases to ~./zshrc to make them persistent.

For Git specifically, you can also just add a single alias—alias g=git—and then put the following into your ~/.gitconfig:

alias.a=add
alias.c=commit --verbose
alias.ca=commit --amend --verbose
alias.cane=commit --amend --no-edit
alias.co=checkout
alias.cob=checkout -b
alias.d=diff
alias.dc=diff --cached
alias.ds=diff --stat
alias.f=fetch
alias.l=log
alias.lfp=log --first-parent
alias.lg=log --graph --oneline --decorate --all
alias.p=pull
alias.s=status

This has two advantages:

  1. Git will correct spelling errors and suggest what you meant.
  2. If you have autocompletion enabled, g ca<Tab> will suggest the ca and cane aliases. Also, it will suggest options, e.g. g c --<Tab> autocompletes.

Final note: sometimes, it may be better to just write a tiny shell script instead of an alias. When you modify an alias, you need to reload all your shells for the change to take effect—whereas the changes made to a shell script take immediate effect.

Some commands such as cd add useful behavior: you can use cd - to jump to previous directory (in fact, cd -$n works where $n is a number)

Terminal

The $TERM variable holds the name of your terminal emulator (unless you overwrite it).

As mentioned before, your terminal likely supports scrollback. If not, you can always use tmux to add scrollback support to simple terminals such as st.

Note that scrollback is not the same thing as shell history. Scrollback is the complete transcript of the terminal session: the commands you typed and their output.

You should be able to navigate (scroll) the transcript using keyboard only, and to search the transcript up and down.

Also, make sure you know how to adjust font size and colors. It’s helpful when you need to consult something with people around who don’t have a great view of your screen.

Sometimes you can mess up your terminal by accidentally sending a control sequence to it. All sorts of things can then happen—you can type from right to left, backspace may move cursor to the right instead to the left (and still work), letters may appear twice. If that happens, just reset(1) the terminal.

Git

Git will be extremely useful for you, it is the de-facto standard for software development.

I gave an extremely terse demo of Git. Basically, I showed how you can commit and push your changes while I can pull them, and how I can commit and push feedback and points back into your repository so that you can pull them.

I mentioned that conflicts may arise where you’re trying to push your changes but the upstream (Github or sourcehut) has some commits which you don’t have locally, and that git pull has several conflict resolution strategies (e.g., merge and rebase) which can usually resolve the issue automatically.

See gittutorial for more.

Vim

I also showed a tiny bit of Vim. See e.g. this basic tutorial or this cheatsheet.

Vim has an incredibly useful but little known feature called persistent undo which is disabled by default. Make sure to enable it, it will be incredibly helpful while fixing broken config files.

In a nutshell,

set undofile
set undodir=~/.cache/nvim/undodir
set undolevels=10000
set undoreload=50000

Please note that this leaves all your edits on disk in the undodir, and it would be very unfortuntate if you accidentally made them public. For example, if you remove a hard-coded credential from a script, it will still be present in the undodir.

Also useful

  • Zathura, a great little PDF viewer with Vim keybindings
  • Using snapshots on a laptop to rescue deleted files (happens often to me) and using snapshots as incremental backups sent to another device
    • I showcased my own super-simple snapshot manager (snap), but there are many others, such as Snapper and Btrbk
  • It is a very good idea to use a screen lock (xscreensaver, slock, i3lock, etc.) and make sure your screen gets locked automatically after a brief period of inactivity (xautolock)
  • osdd or On-Screen Display Daemon is a nifty tool to overlay various notifications (mail, irc, volume/brightness control, etc.)
  • Ranger is a powerful curses-based file manager

Ad-hoc scripts

Once you identify a repetitive task, automate it away. Over the years, I accumulated various ad-hoc scripts. I pasted some of them here for inspiration.

Keyboard setup

I run this from ~/.xinitrc:

#!/bin/sh
set -eux

setxkbmap \
        -layout us,cz \
        -variant ,ucw \
        -option grp:switch,caps:swapescape,altwin:swap_alt_win

Brightness and color temperature adjustment

I didn’t like adjusting color temperature and brightness separately. In the evening, I usually turn the brightness of the screen of my laptop down; at the same time, I like to reduce the amount of blue light emitted by the screen (with redshift). This shell script does both, and is hooked to the brightness up/down keys of my keyboard:

#!/bin/sh
set -eux

usage() {
        cat <<-EOF
Usage: brite [-v] ++
       brite [-v] --
       brite -h
Increase or decrease screen brightness.
        EOF
}

base=/sys/class/backlight/intel_backlight

while getopts hv opt; do
        case $opt in
        v) set -x;;
        h) usage; exit 0;;
        *)  usage >&2; exit 1;;
        esac
done
shift $((OPTIND - 1))

[ $# -eq 1 ] || { usage >&2; exit 1; }

case $1 in
inc) sortopts=-n ;;
dec) sortopts=-nr;;
*)   usage >&2; exit 1;
esac

cur=$(cat "$base/brightness")

levels() {
        cat <<-EOF | sort -s "$sortopts" -k2,2 | uniq -f1
                2200    1
                2500    2
                2800    100
                3000    250
                3500    480
                4800    800
                6500    1400
                6500    2500
                6500    4200
                6500    7500
                6500    12000
                6500    19393
                unused  $cur
        EOF
}

{
        levels | sed -n "/\s$cur\$/{n;p}"
        levels | tail -n1 # if there's no match
} | head -n1 | {
        read -r temperature brightness
        echo $temperature $brightness
        printf "%d\n" "$brightness" > "$base/brightness"
        redshift -oP -O "$temperature"
}

Send Btrfs snapshots to an external NVMe SSD drive

Snapshots are great, I don’t know how anybody can work without them—I recover accidentally deleted files at least once a month. But they aren’t backups on their own: since you keep them on the same drive where your data resides, if that drive dies, all your backups go with it.

It is therefore a very good idea to backup onto an external device. I am using an NVMe USB 3.2 gen2×2 (20 Gb/s) enclosure with a fast 2 TB SSD, and I leverage the fact that snapshots are differential in nature—so it’s enough to sync the difference between the current state of my laptop and the last snapshot on the external drive.

This script deals with unlocking the LUKS container (the external drive is encrypted, of course), syncing the snapshots (with snap) and verifying integrity of Btrfs on the external drive.

#!/bin/sh
set -eu

luks_uuid=b8c231b6-0ea0-4978-b4af-af0aa54e4bde
luks_key="$HOME/etc/cryptbackup.key"
name=cryptbackup
dmdev="/dev/mapper/$name"
btrfs_uuid=8c751844-bc7a-455e-b7fd-2d4b2ccae640
mount=/mnt/backup

# Unlock a volume with the right UUID.
[ -e "/dev/mapper/$name" ] ||
        sudo cryptsetup \
                luksOpen \
                --key-file="$luks_key" \
                UUID="$luks_uuid" \
                "$name"
printf "Device %s is unlocked\n" "$name"

# Mount the LUKS volume.
[ -n "$(lsblk -no MOUNTPOINT "$dmdev")" ] ||
        sudo mount UUID="$btrfs_uuid" "$mount"
printf "Device %s is mounted at %s\n" "$name" "$mount"

sudo snap -blvX root-backup
sync
printf "\nSnapshots synchronized to %s\n\n" "$name"

df -h / "$mount"

sudo umount "$mount"
sync
printf "\nDevice %s is unmounted\n\n" "$name"

sudo btrfs check \
        --readonly \
        --check-data-csum \
        --progress \
        "$dmdev"
printf "Btrfs on %s passed btrfs check\n\n" "$name"

sudo cryptsetup luksClose "$name"
sync
printf "Device %s is locked\n" "$name"

printf "You backed up. Good for you!\n"
[ ! -e "$dmdev" ] &&
        printf "Device %s is now safe to remove!\n" "$name"

Remap Ctrl+W in Firefox

I’m used to deleting words with Ctrl+W, but Firefox insists on closing tabs instead when I press Ctrl+W in the address bar, which is incredibly frustrating. I run this script after Firefox updates, it unmaps Ctrl+W for me:

#!/bin/sh
set -eu

usage() {
        cat <<-EOF
        Usage: firefox-unmap-ctrlw [-hv]

        Options:

          -h  Print this usage message and exit.
          -v  set -x
        EOF
}

while getopts hv opt; do
        case $opt in
        h) usage; exit 0;;
        v) set -x;
        esac
done
shift $((OPTIND-1))

[ $# -eq 0 ] || { usage >&2; exit 1; }

: "${OMNI_JA:=/usr/lib/firefox/browser/omni.ja}"
: "${TMP_OMNI_JA:=/tmp/omni.ja}"
: "${TMP_EXTRACT:=/tmp/omni}"
: "${BROWSER_XHTML:=$TMP_EXTRACT/chrome/browser/content/browser/browser.xhtml}"

cleanup() {
        rm -f  -- "$TMP_OMNI_JA"
        rm -fr -- "$TMP_EXTRACT"
}

trap cleanup EXIT INT TERM

sudo cp "$OMNI_JA" "$OMNI_JA.orig"
mkdir -p "$TMP_EXTRACT"
(
        cd "$TMP_EXTRACT"
        unzip -q "$OMNI_JA"
)
patch "$BROWSER_XHTML" <<-EOF
        288,289c288
        <     <key id="key_close" data-l10n-id="close-shortcut" command="cmd_close" modifiers="accel" reserved="true"/>
        <     <key id="key_closeWindow" data-l10n-id="close-shortcut" command="cmd_closeWindow" modifiers="accel,shift" reserved="true"/>
        ---
        >     <key id="key_close" data-l10n-id="close-shortcut" command="cmd_close" modifiers="control,shift" reserved="true"/>
EOF
(
        cd "$TMP_EXTRACT"
        zip -0DXqr "$TMP_OMNI_JA" *
)
sudo mv "$TMP_OMNI_JA" "$OMNI_JA"

find "$HOME/.cache/mozilla/firefox" -type d -name 'startupCache' | xargs rm -fr --

Then, you can bind Ctrl+W for GTK apps (including Firefox) in ~/.config/gtk-3.0/gtk.css:

@binding-set text-entry
{
  bind "<Control>w"
  {
    "delete-from-cursor" (word-ends, -1)
  };
}

entry    { -gtk-key-bindings: text-entry; }
textview { -gtk-key-bindings: text-entry; }

gitdo: Running a command for every file tracked by Git

#!/bin/sh
set -eu

usage() {
	cat <<EOF
Usage: $prog [-ghv] -- COMMAND...

Execute COMMAND on all (non-excluded) files within a Git repository.
The file names will be provided as command line arguments to COMMAND.

Options:
  -g  Operate on the entire repository, not just the current subtree.
  -h  Print this usage message and exit.
  -v  Be verbose
EOF
}

prog=$(basename "$0")

if ! git rev-parse --show-toplevel >/dev/null 2>&1; then
	echo >&2 "$prog: not within a Git repo: $(pwd)"
	exit 1
fi

dir=$(pwd)

while getopts "ghv" opt; do
	case "$opt" in
	g) dir=$(git rev-parse --show-toplevel);;
	d) echo="echo";;
	h) usage; exit 0;;
	v) set -x;;
	*) usage 2>&1; exit 1;;
	esac
done

shift $((OPTIND-1))

git -C "$dir" ls-files -z --exclude-standard |
	gawk -v "dir=$dir" 'BEGIN { ORS = RS = "\0"; } $0 = dir "/" $0' |
	xargs -0r realpath -z --relative-to="$(pwd)" -- |
	xargs -0r "$@" --

Suspend, then hibernate

This script suspends the laptop using Zzz. Unless the laptop is woken up (by opening the lid) in a specified time-frame, it will be hibernated.

#!/bin/sh
set -eu

usage() {
	cat <<-EOF
	Usage: suspend-then-hibernate [-hv] DELAY

	Suspend the system to memory (S3).
	After DELAY expires, hibernate to disk (S4), providing lid is closed.

	Options:

	  -h  Print this usage message and exit.
	  -v  set -x
	EOF
}

while getopts hv opt; do
	case $opt in
	h) usage; exit 0;;
	v) set -x;;
	*) usage >&2; exit 1;;
	esac
done

shift $((OPTIND - 1))
[ $# -eq 1 ] || { usage >&2; exit 1; }

delay=$1; shift

start=$(date +%s)
rtcwake -m mem -s "$delay"
lid_status=$(cat /proc/acpi/button/lid/LID/state | sed -E 's/^state: +//')
bat_status=$(cat /sys/class/power_supply/BAT0/status)
now=$(date +%s)
elapsed=$(( now - start ))

[ "$lid_status" =   "closed"      ] || exit 0
[ "$elapsed"    -ge "$delay"      ] || exit 0

ZZZ

Key take-aways

  • Your computer’s interface can be heavily customized to suit your needs.
  • Customizing it will undoubtedly pay off in the long run.
  • It is much better to invest the time and effort and acquire some good habits now, at the start of your career, than later on.
  • The biggest mistake is not typing at the keyboard properly.
  • At the very least, you should know your shell, terminal and text editor really well.
  • Lock your screen.
  • Boring work should be scripted away.
  • Simplicity and elegance are more important than features.