Pairwise comparison

Abstract: : The Analytic Hierarchy Process serves as a framework for people to structure their own problems and provide their own judgements based on knowledge, reason or feelings, to derive a set of priorities for activities to which they, for example, wish to allocate effort or resources. In this process transitivity of preference is studied through a new approach to consistency - which need not always strictly hold for the results to be acceptable. Also since hierarchic structures may not be complete, not all alternatives need to be directly comparable. It is necessary to construct a pairwise comparison matrix of the relative contribution or impact of each element on each governing objective or criterion in the adjacent upper level. In such a matrix of the elements by the elements, the elements are compared in a pairwise manner with respect to a criterion in the next level. In comparing the i,j elements, people prefer to give a judgement which indicates the dominance as an integer. Thus, if the dominance does not occur in the i,j position while comparing the ith element with the jth element then it is given in the j,i position as a ji and its reciprocal is automatically assigned to aij.

Abstract: In this paper, a new method, called best-worst method (BWM) is proposed to solve multi-criteria decision-making (MCDM) problems. In an MCDM problem, a number of alternatives are evaluated with respect to a number of criteria in order to select the best alternative(s). According to BWM, the best (e.g. most desirable, most important) and the worst (e.g. least desirable, least important) criteria are identified first by the decision-maker. Pairwise comparisons are then conducted between each of these two criteria (best and worst) and the other criteria. A maximin problem is then formulated and solved to determine the weights of different criteria. The weights of the alternatives with respect to different criteria are obtained using the same process. The final scores of the alternatives are derived by aggregating the weights from different sets of criteria and alternatives, based on which the best alternative is selected. A consistency ratio is proposed for the BWM to check the reliability of the comparisons. To illustrate the proposed method and evaluate its performance, we used some numerical examples and a real-word decision-making problem (mobile phone selection). For the purpose of comparison, we chose AHP (analytic hierarchy process), which is also a pairwise comparison-based method. Statistical results show that BWM performs significantly better than AHP with respect to the consistency ratio, and the other evaluation criteria: minimum violation, total deviation, and conformity. The salient features of the proposed method, compared to the existing MCDM methods, are: (1) it requires less comparison data; (2) it leads to more consistent comparisons, which means that it produces more reliable results.

Abstract: The area under the ROC curve, or the equivalent Gini index, is a widely used measure of performance of supervised classification rules. It has the attractive property that it side-steps the need to specify the costs of the different kinds of misclassification. However, the simple form is only applicable to the case of two classes. We extend the definition to the case of more than two classes by averaging pairwise comparisons. This measure reduces to the standard form in the two class case. We compare its properties with the standard measure of proportion correct and an alternative definition of proportion correct based on pairwise comparison of classes for a simple artificial case and illustrate its application on eight data sets. On the data sets we examined, the measures produced similar, but not identical results, reflecting the different aspects of performance that they were measuring. Like the area under the ROC curve, the measure we propose is useful in those many situations where it is impossible to give costs for the different kinds of misclassification.