Task 2 - Apache Thrift

This task is about cross-language communication using remote procedure calls (RPC), using Apache Thrift. Implement a simple client-server application which will use given a communication protocol. The protocol is based on a given language-independent interface description in Thrift IDL. Then, extend the protocol while maintaining full run-time compatibility.

Preparation

Download the input source code https://d3s.mff.cuni.cz/legacy/files/teaching/nswi080/labs/Files/sources-2.zip . Install Apache Thrift and try the example https://d3s.mff.cuni.cz/legacy/files/teaching/nswi080/labs/Files/lab-2thrift-en.html . We will use C++ to implement the server. Choose a different programming language supported by Thrift for implementing the client.

The task requires understanding of the following:

• Mapping of Thrift IDL features to constructs of the target programming language.
  • This mapping may not be fully documented in Thrift's documentation. You may run the Thrift compiler on the provided IDL file, and explore the generated code in the target language. If still unclear, ask on our mailing list.
• Implementation of a server for a thrift service using a simple IO stack, with a binary protocol, multiplex, and using a processor factory to create a separate handler for each client.
• Connecting to the running server and calling methods of a service.

Application

The application that will be implemented does not perform any meaningful task, it is designed to merely show working with various data types using cross-language RPC.

The main idea is that the client logs in to the server, then lets the server perform a search for some items. The result of the search is a list of items, which are returned to the client one by one. The client generates a summary based on the items and saves it on the server.

Because the focus of this task is just the communication not an actual functionality of the application, we will work with just random data without any meaning.

Communication

The client and server should communicate like this:

1. The client logs in
   • Client calls method logIn of a Login service, with a user name an integer key.
   • The server checks the validity of the key, and if the key does not match, throws an InvalidKeyEception, which will contain the correct key.
   • The client calls logIn again, now with the correct key.
2. Searching for items
   • In the basic version, the server only supports searching for items of type ItemA and fetching them one by one.
   • The client repeatedly calls fetch to fetch the items.
     • The returned FetchResult either contains an item from the result, or indicates why no item was returned.
       • If the state field of FetchResult is ENDED, then no item was returned, because there are no more items (all were already fetched).
       • If the state field of FetchResult is PENDING, then no item was returned, because the server is still searching for items and does not currently have any items to return.
         • In this case, the client should wait for a bit and try again.
       • If the state field of FetchResult is ITEMS, then the item field contains an item.
3. Creating a summary
   • A "summary" in this case will be a multimap-like collection, represented as a map form a string to a set of strings.
     • The keys in the map will be the names of fields in the item types, such as "fieldX", "fieldY", etc.
     • The sets will contain all values of that field across all the items.
   • The client collects all field values from all the fetched items into the multimap.
     • The values of all the fields should be converted to strings. If the field is a collection, then the values should be comma-separated.
   • The client saves the summary on the server using saveSummary in Reports.
   • The server verifies that the summary is accurate and indicates that by a boolean return value.
4. The client logs out
   • The client calls logOut.

Your tasks are

1. Implement the Client

Implement a client which will communicate with the server using the protocol defined by the IDL file and the description above.

• Use a programming language of your choice.
• The name used to log in and the query string may be command-line options of the program.
• The rest of what the client does will depend on what the server returns.
• In each step, print a message to standard output to show the progress of the application.

Connect to Server

Test that the client works with our server at lab.d3s.mff.cuni.cz:5001.

• There are 3 services in multiplex, named Login, Search and Reports.
• The transport and protocol stack is the same as in the lab example (multiplex over a binary protocol).
• Choose a unique name for login.

2. Implement the Server

Implement the server side of the application in C++.

• For the expected key in login, you may use a random value per connection, or some hash of the user name.
• For generating random data, you can use code in the file database.hpp. (Note: The database generates multiple types of items, to be used in the next part of the assignment. In the basic version of the server, we are only interested in items of type "ItemA".)
• Sometimes, return status PENDING from fetch to simulate a long-running search.
• In saveSummary, check that the summary is correct - you can use code in the file summary.hpp to generate the expected summary and compare it with the client-provided summary.
• In all methods, if the client passes invalid arguments or calls methods in a wrong order (for example, fetching items while not logged in), throw a ProtocolException.
• The Thrift IDL compiler, in addition to the files required for compilation, generates an empty skeleton implementation for each service. You can use parts of this code as the basis for your implementation.

3. Protocol update

An important feature of Thrift and its IDL is the ability to evolve the interface while keeping compatibility. For example, new fields can be added to user-defined types, fields which are not required may be removed, function parameters may be added, etc.

The client and server may use different definition of a type or a function.

• Fields are matched by their id (field names are not important). As long as matching fields have the same types on both sides, communication is possible.
• A field which is not declared as required, may be missing on one side.
• A function parameter with a default value may be missing on the caller side.
• If a function is called with more parameters than it receives, the additional parameters are ignored.

New features

1. Extend the interface with new types of an items (ItemB, ItemC) in addition to ItemA. The C++ variants of these types are in items.hpp.
   • Now we can search for multiple types of items, where the search result can be a mix of different item types.
2. Modify the Search service in some way to allow the client to initiate a search, specifying:
   • which types of items to search for,
   • limit on maximum number of items to search for.
3. Modify the Search service in some way to allow fetching multiple items at once.
   • The client decides the number of items to be fetched.
4. Update the implementation of the server to support these new features.
5. Update the implementation of the client to support these new features.
   • Let the client specify on the command line: the set of types of items to search for, the search limit and the number of items fetched at a time.

• The update client should be compatible with the old server and the updated server should be compatible with the old client. (All four possible combinations should work, but the new functionality can be used only when both sides are updated.)

Instructions

• Submit by e-mail, a zip file containing:

  • Source code of the basic client, including build scripts if necessary.
  • Source code of the basic server, including build scripts.
  • The updated Thrift IDL file.
  • Source code of the exteded client, including build scripts if necessary.
  • Source code of the exteded server, including build scripts.

• You must use the provided interface file and modify it only as described.

• You may use the provided C++ code an modify it as you wish.

• The server must be able to serve multiple clients at the same time without interference.

• Do not crash or leak memory, perform undefined behavior, race conditions (if multiple threads access shared mutable data, the accesses must be synchronized, for example using a mutex lock).

• Properly handle exceptions (do not crash or terminate just because the other side does not follow the protocol, etc.).

• Keep the implementation simple.

• It should be easy to build and run (preferrably should work on the computers in the school labs).

• You may use parts of the example codes and scripts.

• If you are using git to keep track of different versions, do not submit your git history - all parts of the solution should accessible directly as files.

• If something is unclear, ask on the mailing list.