Locality Aware Task Scheduling in Parallel Data Stream Processing

Abstract:

Parallel data processing and parallel streaming systems become quite popular. They are employed in various domains such as real-time signal processing, OLAP database systems, or high performance data extraction.One of the key components of these systems is the task scheduler which plans and executes tasks spawned by the system on available CPU cores. The multiprocessor systems and CPU architecture of the day become quite complex, which makes the task scheduling a challenging problem. In this paper, we propose a novel task scheduling strategy for parallel data stream systems, that reflects many technical issues of the current hardware. We were able to achieve up to 3× speed up on a NUMA system and up to 10% speed up on an older SMP system with respect to the unoptimized version of the scheduler. The basic ideas implemented in our scheduler may be adopted for task schedulers that focus on other priorities or employ different constraints.

BibTeX:

@inproceedings{falt_locality_2015,
    title = {{Locality Aware Task Scheduling in Parallel Data Stream Processing}},
    author = {Falt, Zbyněk and Kruliš , Martin and Bednárek, David and Yaghob, Jakub and Zavoral, Filip},
    year = {2015},
    booktitle = {{Intelligent Distributed Computing VIII}},
    editor = {Camacho, David and Braubach, Lars and Venticinque, Salvatore and Badica, Costin},
    publisher = {Springer International Publishing},
    series = {{Studies in Computational Intelligence}},
    location = {Cham},
    doi = {10.1007/978-3-319-10422-5_35},
    isbn = {978-3-319-10422-5},
    pages = {331--342},
}

Proceedings paper