Ward's method

Abstract: The Ward error sum of squares hierarchical clustering method has been very widely used since its first description by Ward in a 1963 publication. It has also been generalized in various ways. Two algorithms are found in the literature and software, both announcing that they implement the Ward clustering method. When applied to the same distance matrix, they produce different results. One algorithm preserves Ward's criterion, the other does not. Our survey work and case studies will be useful for all those involved in developing software for data analysis using Ward's hierarchical clustering method.

Abstract: We propose a hierarchical clustering method that minimizes a joint between-within measure of distance between clusters. This method extends Ward's minimum variance method, by defining a cluster distance and objective function in terms of Euclidean distance, or any power of Euclidean distance in the interval (0,2]. Ward's method is obtained as the special case when the power is 2. The ability of the proposed extension to identify clusters with nearly equal centers is an important advantage over geometric or cluster center methods. The between-within distance statistic determines a clustering method that is ultrametric and space-dilating; and for powers strictly less than 2, determines a consistent test of homogeneity and a consistent clustering procedure. The clustering procedure is applied to three problems: classification of tumors by microarray gene expression data, classification of dermatology diseases by clinical and histopathological attributes, and classification of simulated multivariate normal data.

Abstract: The claim that Ward's linkage algorithm in hierarchical clustering is limited to use with Euclidean distances is investigated. In this paper, Ward's clustering algorithm is generalised to use with l1 norm or Manhattan distances. We argue that the generalisation of Ward's linkage method to incorporate Manhattan distances is theoretically sound and provide an example of where this method outperforms the method using Euclidean distances. As an application, we perform statistical analyses on languages using methods normally applied to biology and genetic classification. We aim to quantify differences in character traits between languages and use a statistical language signature based on relative bi-gram (sequence of two letters) frequencies to calculate a distance matrix between 32 Indo-European languages. We then use Ward's method of hierarchical clustering to classify the languages, using the Euclidean distance and the Manhattan distance. Results obtained from using the different distance metrics are compared to show that the Ward's algorithm characteristic of minimising intra-cluster variation and maximising inter-cluster variation is not violated when using the Manhattan metric.