Capturing Performance Assumptions Using Stochastic Performance Logic

Abstract:

Compared to functional unit testing, automated performance testing is difficult, partially because correctness criteria are more difficult to express for performance than for functionality. Where existing approaches rely on absolute bounds on the execution time, we aim to express assertions on code performance in relative, hardware-independent terms. To this end, we introduce Stochastic Performance Logic (SPL), which allows making statements about relative method performance. Since SPL interpretation is based on statistical tests applied to performance measurements, it allows (for a special class of formulas) calculating the minimum probability at which a particular SPL formula holds. We prove basic properties of the logic and present an algorithm for SAT-solver-guided evaluation of SPL formulas, which allows optimizing the number of performance measurements that need to be made. Finally, we propose integration of SPL formulas with Java code using higher-level performance annotations, for performance testing and documentation purposes.

BibTeX:

@inproceedings{bulej_capturing_2012,
    title = {{Capturing Performance Assumptions Using Stochastic Performance Logic}},
    author = {Bulej, Lubomí r and Bureš, Tomáš and Keznikl, Jaroslav and Koubková, Alena and Podzimek, Andrej and Tůma, Petr},
    year = {2012},
    booktitle = {{Proc. 3rd ACM/SPEC International Conference on Performance Engineering (ICPE)}},
    publisher = {ACM},
    series = {{ICPE '12}},
    location = {New York, NY, USA},
    doi = {10.1145/2188286.2188345},
    isbn = {978-1-4503-1202-8},
    pages = {311--322},
    url = {http://doi.acm.org/10.1145/2188286.2188345},
}

Proceedings paper