Here are final tasks for this course.

Deadline for the final tasks is 10th July 2022. There will be no extensions.

Use your standard submission repository and keep the usual naming as described by the task title.

Note for many of the tasks we provide a partial solution for you to start with. Starting with our solution is not mandatory but it should help you jump-start your implementation.

We expect that you will be using Git branches and tools such as Shellcheck or Pylint to simplify your work. While we will not explicitly check that you have used branching, take it as a good habit to get into.

For some of the tasks we will assign also points for the quality of your implementation (identifier naming, layers of abstractions used but also Git commit messages, etc.).

Updates

Here are some updates regarding tests.

Tests are available as part of GitLab CI (as was usual for quizzes and lab tasks).

As a reminder: it is possible to run the tests locally. Assuming you cloned your submission repository student-LOGIN and tests repository to the same directory (e.g. you have ~/nswi177/student-LOGIN and ~/nswi177/tests), you can execute the .bats files from tests manually (see examples for each task below).

Note that BATS supports -f for filtering tests that are executed which might be also useful.

Important note: we tried to write the tests to be as robust as possible and as orthogonal as possible (so that thumbnail-related tests do not require theming support and vice versa). But bugs are still possible: if you think your solution is okay and the tests think otherwise, please, let us know: we will have a look.

If you create your own tests and want to share them, let us know as well, please. We would be happy to add them to our suite.

Implement a simple HTML gallery generator in Bash.

We expect the user to organize his folder like in the listing below. After running your script, the generated gallery will reside in the public_html subdirectory.

.
├── albums
│   ├── 2021-09-airport
│   │   ├── dsc00045.jpg
│   │   ├── dsc00097.jpg
│   │   └── dsc00122.jpg
│   ├── 2022-04-bicycles
│   │   ├── img0001.jpg
│   │   └── img0017.jpg
│   └── 2022-05-mountain-trip
│       ├── HEADER.md
│       └── 00001.jpg
├── gallery.rc
└── HEADER.md

As we can deduce from the listing above, the gallery is organized into folders that are represented by separate directories under albums. The resulting HTML generates a separate directory for each album too, so that we have nice-looking URLs.

The generator also reads the HEADER.md file to get extra content for the index page that is displayed above the list of albums.

The user may optionally specify site name inside a configuration file gallery.rc.

Several example galleries are available inside the examples repository. We have saved the photos with rather low resolution and with rather low quality in general to make the files as small as possible and speed up your development.

You can start with our implementation which is able to detect the list of albums and generate a simple gallery for each album with pseudo-thumbnails.

Your task is split to several subtasks where each should demonstrate your various scripting abilities. Most subtasks are completely independent (have you heard the echoes of Git branches?) and it is not necessary to implement all of them to pass the course.

Changing output directory

Allow the user to change where the HTML files are generated instead of the default public_html.

The user will be able to change that either via -d or --publish-dir parameter on the command-line specifying a directory name (if the directory exists, its contents will be overwritten without confirmation) or through a variable publish_dir in the gallery.rc.

Specification on the command-line takes precedence over gallery.rc.

We expect you will use getopt.

Theming support

Our solution uses a template and a stylesheet that is located next to the script. Add support for a -t (or --theme-dir) command-line option or theme_dir configuration option to load index.tpl.html and album.tpl.html from an alternate directory.

Our implementation uses $data_files_dir to refer to files that are in the same directory as our script.

Proper thumbnail generation

Our version does not actually generate any thumbnails. Instead, it creates an <img> tag pointing to the original file with width and height overridden.

Your task is to extend this version to call ImageMagick’s convert (recall lab 05) to actually resize the image and let the link point to the original file.

In the first version, feel free to resize the images to the sizes we have used, for an extended version allow the user to specify the size via a thumbnail_size configuration option in gallery.rc.

ImageMagick is able to resize the image either forcefully (i.e. ignoring its ratio of width to height) or to fit into the given rectangle (no distortion). Using file or identify (from ImageMagick) you can read the actual thumbnail size to provide a correct (undistorted) thumbnail.

The CSS we provide should wrap the images quite nicely once they have the same height.

Update: for tests, we had to codify the naming of the thumbnail images. Full-sized images are still expected to be named 00000001.jpg etc., for thumbnails we expect naming of thumb.00000001.jpg.

Update: see this test template for details how thumbnail size is expected to be propagated to the template.

Update: we expect you will use ImageMagick format specification of the thumbnail size, so 300x200 to define maximum size (keeping aspect ratio) or even x200 to specify height only (and keep aspect ratio). Most probably, you have used $thumbnail_size directly and things will simply continue to work.

Album configuration

Allow the user to configure each album separately. If the script finds album.rc inside the album directory, it will read album-specific configuration from there.

If title is specified, it would be used instead of the directory name for album name.

If front_image is specified, it should be a (relative) path to an image that is supposed to be used on the index page instead of the first photo.

EXIF information

If the image contains EXIF information, it should be added to the gallery. The exact format will be defined by the tests (and templates), we will be interested in Date/Time Original field only.

Update: for tests, we expect this information to be stored inside the meta JSON as date_time. See this template for details.

Updates

Our implementation requires Pandoc with support for --metadata-file (Pandoc 2.3, released in 2018). See this Forum issue for instructions on updating Pandoc on Ubuntu.

You will also need Tidy to be installed.

Following command can be used to test your solution (inside student-LOGIN, assuming you clone the tests repository as mentioned above).

../tests/final/21/publish_dir.bats

There are separate tests for each feature. It is also possible to execute all of them via the following command.

bats ../tests/final/21/*.bats

22/ (templater) (500 points)

In this task, you will prepare a full-fledged Python package which should be installable with pip install git+ssh://....?subdir=22 (i.e. we expect to see 22/setup.cfg file).

After installation, the package will add an nswi177-jinja-templater executable into your $PATH that is able to render Jinja templates.

Again, a partial solution is available from us, your task will be to convert it to a proper package (this is a mandatory part) and extend it a little bit.

The Jinja package is a templating engine. We have seen templates in Pandoc and Jinja is a more powerful take on the same topic. You provide the Jinja engine with a template and a set of variables and Jinja renders the template, expanding variables, for loops etc. as needed.

The core of the task – Jinja configuration – is already done for you. Again, your task consists of several subtasks, which are mostly independent.

Proper Python package (mandatory)

Along the lines of assignment for lab 11, convert the source code from us into a proper Python package.

Notice above how pip install can be given a subdirectory of a Git repository to install from there.

All the tests that we will provide will start by installing your package like that.

YAML header (mandatory)

If you open the example input files, you will see that they contain a special header enclosed in ---. Inside it, the variables are assigned in key: value form and lists are created via -.

This is a so-called YAML header and it defines variables that will be used inside the template. YAML is a quite complex format, so we expect you to use a proper library for parsing it.

However, extracting the --- delimited block is something that you can implement yourself as it is basically a simple for loop only.

Add arabic2roman macro

An extremely nice feature of Jinja is that you can easily add your own macros (pipes) to provide new functions to the templates.

Our example contains l2gal for conversion between SI litres and imperial gallons.

Your task is to add a macro arabic2roman that is able to convert a number to its Roman representation (e.g., convert 12 to XII).

It is required that you find a library for this task.

Your implementation should work with positive integers only. For invalid input, print NaN as the rendered text and emit a warning to standard error output.

Add --use-us-gallons option

Extend the program with an option --use-us-gallons (note that we are using argparse) that switches l2gal computation from imperial to US gallons.

Add -V option for extra variables

Add option -V for adding extra (string) variables. The option can be specified multiple times. Each time, its argument has a key=value syntax. For example:

nswi177-jinja-templater -V course=NSWI177 -V "name=Introduction to Linux" ...

Advanced users may look up the action class to create a smart implementation for storing these values in a dictionary directly.

Updates

Tests (assuming the setup mentioned at the beginning of the page) can be executed via:

../tests/final/22/templater.bats

The above invocation creates a new virtual environment for each test. That is perfect for CI, for local development it is often better to use your current virtual environment (e.g. because you have not yet added all dependencies to setup.cfg etc.).

The tests support NO_INSTALL that ignores any installation and merely executes the nswi177-jinja-templater executable.

env NO_INSTALL=true ../tests/final/22/templater.bats

23/ (disk) (500 points)

Your task is to recover three picture files from a broken disk image. Some of the pictures are easy to get, some will require a bit more work.

You might wish to refresh your knowledge about disk management from lab 10.

You will have your own disk that you can get through a HTTP server running on http://localhost:8080/23/ on machine linux.ms.mff.cuni.cz.

Hint: use a graphical browser to access the page.

Once you recover the pictures, you will notice that each of them contains one word (we have used random words from a Czech dictionary).

Store the word into a file named x.txt where x is the first letter of the word. Therefore, if the recovered image would look like this, you might use the following command to create your solution file.

echo fedora >23/f.txt

Updates

Actually, one of the images (it should be the one you recover the first using the simplest method) contains also the words prvni heslo which means first password. Please, use only the last word from this image.