Detailed Analysis and Optimization of CUDA K-means Algorithm

Abstract:

K-means is one of the most frequently used algorithms for unsupervised clustering data analysis. Individual steps of the k-means algorithm include nearest neighbor finding, efficient distance computation, and cluster-wise reduction, which may be generalized to many other purposes in data analysis, visualization, and machine learning. Efficiency of the available implementations of k-means computation steps therefore directly affect many other applications. In this work, we examine the performance limits in the context of modern massively parallel GPU accelerators. Despite the existence of many published papers on this topic, we have found that crucial performance aspects of the GPU implementations remain unaddressed, including the optimizations for memory bandwidth, cache limits, and workload dispatching on problem instances of varying cluster count, dataset size, and dimensionality. We present a detailed analysis of individual computation steps and propose several optimizations that improve the overall performance on contemporary GPU architectures. Our open-source prototype exhibits significant speedup over the current state-of-the-art implementations in virtually all practical scenarios.

BibTeX:

@inproceedings{krulis_detailed_2020,
    title = {{Detailed Analysis and Optimization of CUDA K-means Algorithm}},
    author = {Kruliš, Martin and Kratochvíl, Miroslav},
    year = {2020},
    booktitle = {{49th International Conference on Parallel Processing - ICPP}},
    publisher = {Association for Computing Machinery},
    series = {{ICPP '20}},
    location = {New York, NY, USA},
    doi = {10.1145/3404397.3404426},
    isbn = {978-1-4503-8816-0},
    pages = {1--11},
    url = {https://doi.org/10.1145/3404397.3404426},
}

Proceedings paper