Sampling Within k-Means Algorithm to Cluster Large Datasets

- 01 Aug 2011

TL;DR: This work uses a sample from a dataset to decrease runtime by reducing the amount of data analyzed in a sampling based k-means algorithm, which is significantly more ecient than the existing algorithm with comparable accuracy.

Abstract: Due to current data collection technology, our ability to gather data has surpassed our ability to analyze it. In particular, k-means, one of the simplest and fastest clustering algorithms, is ill-equipped to handle extremely large datasets on even the most powerful machines. Our new algorithm uses a sample from a dataset to decrease runtime by reducing the amount of data analyzed. We perform a simulation study to compare our sampling based k-means to the standard k-means algorithm by analyzing both the speed and accuracy of the two methods. Results show that our algorithm is significantly more ecient than the existing algorithm with comparable accuracy.

AI Agents for this Paper

Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps

Most frequently asked questions

1. What are the contributions in "Sampling within k-means algorithm to cluster large datasets" ?

The authors perform a simulation study to compare their sampling based k-means to the standard k-means algorithm by analyzing both the speed and accuracy of the two methods.

2. What future works have the authors mentioned in the paper "Sampling within k-means algorithm to cluster large datasets" ?

Further work on this project might include a more comprehensive study both on more varied test datasets as well as on real weather datasets.. The authors would like to analyze this further to see how accurate the algorithm is for even lower sample sizes.. Therefore, the authors conclude that analysts can use their algorithm, expecting accurate results in considerably less time.

Table 4.5: Example Differences in Runtime

Table 4.2: Summary of Results Obtained from Generated Data: Comparison with True Clusters.

Table 4.4: Summary of Results Obtained from Second Version of Generated Data: Comparison with True Clusters.

Table 4.3: Best Results Obtained from Second Version of Generated Data: Comparison with True Clusters.

Table 4.1: Best Results Obtained from Generated Data: Comparison with True Clusters.

Figure 1.1: Ratio of Total Times of Twenty trials (t1/t2)

Citations

Journal Article•10.1177/0047287513496475

Required Sample Sizes for Data-Driven Market Segmentation Analyses in Tourism:

Sara Dolnicar, +3 more

- 01 Apr 2014

- Journal of Travel Research

TL;DR: Under all simulated data circumstances, a sample size of 70 times the number of variables proves to be adequate and will provide guidance to data analysts in academia and industry who wish to conduct reliable and valid segmentation studies.

...read moreread less

213

Journal Article•10.1016/J.ESWA.2014.11.050

Segmentation of Terahertz imaging using k-means clustering based on ranked set sampling

Mohamed Walid Ayech, +1 more

- 15 Apr 2015

- Expert Systems With Applications

TL;DR: Experimental results show that k-means clustering based on ranked set sampling is more efficient than other clustering techniques such as the k- Means based on the fundamental sampling design simple random sampling technique, the standard k-Means and the k -meansbased on the Bradley refinement of initial centers.

...read moreread less

•10.1002/NOP2.1103

Mentors' competence in mentoring nursing students in clinical practice: Detecting profiles to enhance mentoring practices.

Kristina Mikkonen, +10 more

- 02 Nov 2021

TL;DR: The European Commission, Erasmus+ and KA2: Strategic partnership for providing funding for the project Quality mentorship for developing competent nursing students (QualMent) as discussed by the authors.

...read moreread less

•Journal Article•10.3390/SYM11080963

Fast Color Quantization by K-Means Clustering Combined with Image Sampling

Mariusz Frackiewicz, +2 more

- 01 Aug 2019

- Symmetry

TL;DR: This method significantly accelerated the color quantization without noticeable loss of image quality and compared with other methods based on a limited sample of pixels (coreset-based algorithm) showed an advantage of the proposed method.

...read moreread less

Proceedings Article•10.1145/2980258.2980308

Ensemble based Parallel k means using Map Reduce for Aspect Based Summarization

V. Priya, +1 more

- 25 Aug 2016

TL;DR: A new parallel k means clustering approach based on Map Reduce framework for aspect based summary generation, which particularly incorporates bagging and ensembling techniques is proposed, which was found to improve considerably than other state of art approaches.

...read moreread less

...

Expand

References

•Journal Article

R: A language and environment for statistical computing.

R Core Team

- 01 Jan 2014

- MSOR connections

TL;DR: Copyright (©) 1999–2012 R Foundation for Statistical Computing; permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and permission notice are preserved on all copies.

...read moreread less

410.8K

Some methods for classification and analysis of multivariate observations

James B. MacQueen

- 01 Jan 1967

TL;DR: The k-means algorithm as mentioned in this paper partitions an N-dimensional population into k sets on the basis of a sample, which is a generalization of the ordinary sample mean, and it is shown to give partitions which are reasonably efficient in the sense of within-class variance.

...read moreread less

28.1K

•Journal Article•10.1109/TIT.1982.1056489

Least squares quantization in PCM

S. P. Lloyd

- 01 Mar 1982

- IEEE Transactions on Information Theory

TL;DR: In this article, the authors derived necessary conditions for any finite number of quanta and associated quantization intervals of an optimum finite quantization scheme to achieve minimum average quantization noise power.

...read moreread less

16K

Least Squares Quantization in PCM

S. P. Lloyd

- 01 Jan 1982

TL;DR: The corresponding result for any finite number of quanta is derived; that is, necessary conditions are found that the quanta and associated quantization intervals of an optimum finite quantization scheme must satisfy.

...read moreread less

9.6K

Some methods for classi cation and analysis of multivariate observations

J. Mcqueen

- 01 Jan 1967

2.3K

Sampling Within k-Means Algorithm to Cluster Large Datasets

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What are the contributions in "Sampling within k-means algorithm to cluster large datasets" ?

2. What future works have the authors mentioned in the paper "Sampling within k-means algorithm to cluster large datasets" ?

Figures

Citations

Required Sample Sizes for Data-Driven Market Segmentation Analyses in Tourism:

Segmentation of Terahertz imaging using k-means clustering based on ranked set sampling

Mentors' competence in mentoring nursing students in clinical practice: Detecting profiles to enhance mentoring practices.

Fast Color Quantization by K-Means Clustering Combined with Image Sampling

Ensemble based Parallel k means using Map Reduce for Aspect Based Summarization

References

R: A language and environment for statistical computing.

Some methods for classification and analysis of multivariate observations

Least squares quantization in PCM

Least Squares Quantization in PCM

Some methods for classi cation and analysis of multivariate observations

Related Papers (5)

Competitive K-Means, a New Accurate and Distributed K-Means Algorithm for Large Datasets

A modified hyperplane clustering algorithm allows for efficient and accurate clustering of extremely large datasets

A novel K-means based clustering algorithm for big data

High performance parallel $$k$$k-means clustering for disk-resident datasets on multi-core CPUs

Data clustering: 50 years beyond K-means