Do We Teach Useful Statistics for Performance Evaluation?

Abstract:

Basic topics from probability and statistics -- such as probability distributions, parameter estimation, confidence intervals and statistical hypothesis testing -- are often included in computing curricula and used as tools for experimental performance evaluation. Unfortunately, data collected through experiments may not meet the requirements of many statistical analysis methods, such as independent sampling or normal distribution. As a result, the analysis methods may be more tricky to apply and the analysis results may be more tricky to interpret than one might expect. Here, we look at some of the issues on methods and experiments that would be considered basic in performance evaluation education.

BibTeX:

@inproceedings{bulej_do_2017,
    title = {{Do We Teach Useful Statistics for Performance Evaluation?}},
    author = {Bulej, Lubomír and Horký, Vojtech and Tůma, Petr},
    year = {2017},
    booktitle = {{Companion Proc. 8th ACM/SPEC on Intl. Conf. on Performance Engineering}},
    publisher = {ACM},
    series = {{ICPE '17 Companion}},
    doi = {10.1145/3053600.3053638},
    isbn = {978-1-4503-4899-7},
    pages = {185--189},
    url = {http://doi.acm.org/10.1145/3053600.3053638},
}

Proceedings paper