You will receive access to a new repository with the API designed by another team. Your task is to write tests for the public API and setup CI for the project. This task is done individually, which means that you should receive two sets of tests for your API.

API tests

The purpose of this task is to practise writing tests against a public API without access to the the implementation (i.e. blackbox style testing). Your tests should cover all the functionality provided by the library—review the API requirements from task 1, check whether the library provides some extra functionality and write the tests.

The character of your tests will depend on the API provided by the library. Even though your tests may turn out to be almost like unit tests (because the API will provide you with a set of classes full of methods), you should test the API from the perspective of the client code.

This means that your test classes should be in a different package (to make package private elements inaccessible), nor should they rely on “friend” access.

Use the requirements to define the expected behaviour in the tests. If the requirements do not clearly define some behavior, check if the API defines it (in the sample client code or in the documentation) and write tests according to that. If in doubt, define the expected behavior yourself using the tests. Do not forget to test for expected failures (i.e., error codes returned or exceptions thrown) too.

Because there is no implementation, your tests will (obviously) fail. While that can be a bit discouraging, a failing test is also the primary driving force in Test Driven Development, because it makes the expected behavior explicit and thus makes it clear what needs to be implemented.

We intentionally do not provide any metrics on how many tests you should provide. Use common sense to provide a reasonable set of tests that would help your colleagues when writing the implementation.

Finally, use a well-known unit testing framework that nicely integrates with the language, avoid inventing your own.

CI in GitLab

The second part of this task is to setup a simple CI for the project. Simply put, configure the project so that tests are executed automatically on each commit (push) to GitLab.

If you have never configured CI in GitLab, we suggest to consult the reference manual and examples for unit testing.

Suggestions

The following is a short summary of things to watch out for in your tests, because they keep appearing over and over in the submitted tests.

Naming

  • Test method names should reflect the situation being tested and the expected outcome.

    • StackTest.popFailsOnEmptyStack() is a better name than StackTest.testPop().

    • Like any other names, they definitely should not be numbered.

  • Avoid test prefix in the name if you language supports annotations, e.g., @Test (Java) or [Test] (C#), or similar.

Coverage

  • Cover different situations in a systematic fashion, based on the requirements from task-1.

  • Don’t forget to test robustness with respect to invalid inputs. The API should be explicit in how it reacts to such inputs.

Implementation

  • Make the test code obvious for the reader.

    • The reader must be able to see that you are arranging the right situation, executing the right action, and performing the right asserts to determine the test outcome.

    • In other words, strive for the Arrange-Act-Assert (AAA) structure in your test code.

  • Keep tests small and test one situation at a time.

    • This also means that any testable conditions from the arrange phase should be tested using separate tests (not asserts). The assert phase should only concern the particular situation being tested.
  • Make test inputs “visible”.

    • Unlike in non-test code, constant literals are tolerated and often preferable, because they are part of the scenario being tested (e.g., invalid inputs, boundary conditions).

Exceptions

  • Declare expected exceptions using test annotations, don’t catch them.

    • Unexpected exception should crash (fail) the test.

    • If multiple statements may throw the expected exception, use the assert throws facility of your testing framework, which typically provides an assertThrows(() method which takes code to execute as a lambda and an exception type to expect. This may also indicate that the API does not provide exception types corresponding to the provided level of abstractions.

Parametrization

  • Consider using parametrized (theory) tests for cases where everything but the input data changes and provide the data declaratively.

    • Strive to keep the tests obvious even when using parametrized (theory) tests. This typically requires capturing the expected outcomes in the provided data. You can always comment on the data, or include an extra (dummy) string in the data containing a textual description.
  • Avoid code duplication (while keeping the code obvious).

    • Extract common parts into test setup phase or create parametrized helper methods (i.e., build a vocabulary) to setup various scenarios, just make sure the code remains readable.

Sources

  • Treat test code with the same care as your other (non-test) code.

    • Put the code in the right package, split the code into multiple files (if it makes sense), use white space (empty lines) to show boundaries between syntactic elements (methods), etc.

Submission

When you are satisfied with your solution to this task, make sure it is in the master branch of your task-4 repository and tag the commit using the task-4-submission tag.