Abstract: (1997). Multiple Comparisons: Theory and Methods. Journal of Quality Technology: Vol. 29, No. 3, pp. 359-359.

Abstract: In this paper we offer a multiplicity of approaches and procedures for multiple testing problems with weights Some rationale for incorporating weights in multiple hypotheses testing are discussed Various type-I error-rates and different possible formulations are considered, for both the intersection hypothesis testing and the multiple hypotheses testing problems An optimal per family weighted error-rate controlling procedure a la Spjotvoll (1972) is obtained This model serves as a vehicle for demonstrating the different implications of the approaches to weighting Alternative approach es to that of Holm (1979) for family-wise error-rate control with weights are discussed, one involving an alternative procedure for family-wise error-rate control, and the other involving the control of a weighted family-wise error-rate Extensions and modifications of the procedures based on Simes (1986) are given These include a test of the overall intersec tion hypothesis with general weights, and weighted sequentially rejective procedures for testing the individual hypotheses The false discovery rate controlling approach and procedure of Benjamini & Hochberg (1995) are extended to allow for different weights

Abstract: The Simes method for testing intersection of more than two hypotheses is known to control the probability of type I error only when the underlying test statistics are independent. Although this method is more powerful than the classical Bonferroni method, it is not known whether it is conservative when the test statistics are dependent. This article proves that for multivariate distributions exhibiting a type of positive dependence that arise in many multiple-hypothesis testing situations, the Simes method indeed controls the probability of type I error. This extends some results established very recently in the special case of two hypotheses.

Abstract: r Abstract: We present a simple method that allows statistical inferences to be made about the significance of regional effects in statistical parametric maps (SPMs) when the approximate location of the effect is specified in advance. The test can be thought of as analogous to assessing activations with uncorrected P values based on the height of SPMs but, in this instance, using the spatial extent or volume of the nearest activated region. The advantage of the current test is that it eschews a correction for multiple comparisons even though the exact location of the expected activation may not be known. Hum. Brain Mapping 5:133-136, 1997. r 1997 Wiley-Liss, Inc.

Abstract: Use of logical constraints among hypotheses and correlations among test statistics can greatly improve the power of step-down tests. An algorithm for uncovering these logically constrained subsets in a given dataset is described. The multiple testing results are summarized using adjusted p values, which incorporate the relevant dependence structures and logical constraints. These adjusted p values are computed consistently and efficiently using a generalized least squares hybrid of simple and control-variate Monte Carlo methods, and the results are compared to alternative stepwise testing procedures.

Abstract: The difference in statistical power between the original Bonferroni and five modified Bonferroni procedures that control the overall Type I error rate is examined in the context of a correlation matrix where multiple null hypotheses, H0: ρij = 0 for all i ≠ j, are tested. Using 50 real correlation matrices reported in educational and psychological journals, a difference in the number of hypotheses rejected of less than 4% was observed among the procedures. When simulated data were used, very small differences were found among the six procedures in detecting at least one true relationship, but in detecting all true relationships the power of the modified Bonferroni procedures exceeded that of the original Bonferroni procedure by at least .18 and by as much as .55 when all null hypotheses were false. The power difference decreased as the number of true relationships decreased. Power differences obtained for the average power were of a much smaller magnitude but still favored the modified Bonferroni procedur...

Abstract: We examine the operating characteristics of 17 methods for correcting p-values for multiple testing on synthetic data with known statistical properties. These methods are derived p-values only and not the raw data. With the test cases, we systematically varied the number of p-values, the proportion of false null hypotheses, the probability that a false null hypothesis would result in a p-value less than 5 per cent and the degree of correlation between p-values. We examined the effect of each of these factors on family-wise and false negative error rates and compared the false negative error rates of methods with an acceptable family-wise error. Only four methods were not bettered in this comparison. Unfortunately, however, a uniformly best method of those examined does not exist. A suggested strategy for examining corrections uses a succession of methods that are increasingly lax in family-wise error. A computer program for these corrections is available.

Abstract: Factorial Designs. Further Aspects of Design and Modelling. Analysis of Variance. Random or Fixed Assessors in Analysis of Variance. More Complex ANOVA Situations. Replicates in Sensory Analysis. Multiple Comparisons. Two Detailed Examples. References and Relevant Literature. Appendices. Index.

Abstract: Two contrasting views toward the evaluation of multiple tests of constraints and control of Type 1 errors in structural equation modeling are presented. (a) Exploring; data helps researchers make decisions about inclusion of relevant model parameters and control of Type 1 errors hinders this process. (b) Exploring data is not likely to yield meaningful models unless we can limit the process on the basis of methods and theory, and controlling Type I errors is a useful device: to force us to limit our searches. Also, in evaluating multiple tests of constraints for applications other than exploratory analyses, we should control for Type I errors as we do in testing multiple comparisons in analysis of variance. We argue for the second perspective and present examples to illustrate methods for controlling Type 1 errors when making model comparisons.

Abstract: Suppose that there are $k \geq 2$ different systems (i.e., stochastic processes), where each system has an unknown steady-state mean performance and unknown asymptotic variance. We allow for the asymptotic variances to be unequal and for the distributions of the k systems to be different. We consider the problem of running independent, single-stage simulations to make multiple comparisons of the steady-state means of the different systems. We derive asymptotically valid (as the run lengths of the simulations of the systems tend to infinity) simultaneous confidence intervals for each of the following problems: all pairwise comparisons of means, all contrasts, multiple comparisons with a control and multiple comparisons with the best. Our confidence intervals are based on standardized time series methods, and we establish the asymptotic validity of each under the sole assumption that the stochastic processes representing the simulation output of the different systems satisfy a functional central limit theorem. Although simulation is the context of this paper, the results naturally apply to (asymptotically) stationary time series.

Abstract: A powerful test for population association of a disease with alleles at a bi-allelic marker locus is the transmission/disequilibrium test (TDT). A generalization of the test to multi-allelic marker loci is proposed which utilizes the maximal association of individual alleles with the disease, given by the maximum TDT statistic, TDT(max). To overcome the multiple testing problem encountered when using the maximal association to test the null hypothesis of no disease-marker association, a randomization procedure is developed. An investigation of the power of the test suggests that the randomization procedure performs almost as well as a recently proposed likelihood based test of linkage disequilibrium. The advantage of the new test is that it can be applied sequentially, based on a one-sided version of the TDT statistic, for investigating patterns of association of several individual alleles with the disease.

Abstract: Pruning is a common technique to avoid overfitting in decision trees. Most pruning techniques do not account for one important factor — multiple comparisons. Multiple comparisons occur when an induction algorithm examines several candidate models and selects the one that best accords with the data. Making multiple comparisons produces incorrect inferences about model accuracy. We examine a method that adjusts for multiple comparisons when pruning decision trees - Bonferroni pruning. In experiments with artificial and realistic datasets, Bonferroni pruning produces smaller trees that are at least as accurate as trees pruned using other common approaches.

Abstract: Methods of multiple comparisons were applied to linkage analysis in the case of genome scanning. Data for Problem 2A were used. p-Values were calculated for all 440,400 possible tests of linkage. Plots of distribution functions and false discovery rate are shown.

Abstract: The analysis of data from clinical trials often includes subgroup analyses, which are performed to examine the treatment effect within various sets of patients based on baseline and/or demographic variables. The goals of these analyses are to establish the consistency of the results across the subgroups and to identify important prognostic factors. The p-values for such analyses are usually presented without any adjustment for the multiple analyses: This approach has been criticized because of the possibility of misleading false positives. Conservative approaches have been proposed to resolve this problem; however, these approaches are usually so conservative that significant results are rarely observed after adjustment. Here an approximate technique for use when the variable of interest has a normal distribution is presented.

Abstract: Preface. Introduction to Simultaneous Statistical Inference. Classification of Multiple Comparison Methods. Multiple Comparisons With a Control. Multiple Comparisons With the Best. All-Pairwise Comparisons. Abuses and Misconceptions in Multiple Comparisons. Multiple Comparison in the General Linear Model. Some Useful Probabilistic Inequalities. Some Useful Geometric Lemmas. Sample Size Computations. Accessing Computer Codes. Tables of Critical Values. Bibliography. Subject Index.

Abstract: When testing a family of comparisons or contrasts across k treatment groups, researchers are often encouraged to maintain control over the familywise Type I error rate. For common families such as comparisons against a reference group, sets of orthogonal and/or nonorthogonal contrasts, and all possible pairwise comparisons, numerous simultaneous (and more recently sequential) testing methods have been proposed. Many of the simultaneous methods can be shown to be a form of Krishnaiah’s (e.g., 1979) finite intersection test (FIT) for simultaneous multiple comparisons, which controls the familywise error rate to precisely a under conditions assumed in standard ANOVA scenarios. Other methods, however, merely represent conservative approximations to a FIT procedure, yielding suboptimal power for conducting simultaneous testing. The purpose of the current article is threefold. First, we discuss how FIT methodology represents a paradigm that unifies many existing methods for simultaneous inference, as well as ho...

Abstract: Problems of multiple comparison were discussed without assuming technical knowledge of statistics. For the first question concerning why to use multiple comparison procedures, theoretical bases of statistical inference and multiple comparison (including type I error rate and familywise error rate) were briefly outlined. For the second question concerning how to properly use multiple comparison procedures, multiple comparison procedures were introduced, and their characteristics were compared. Families of comparisons are different among Dunnett's, Tukey's and Scheffe's tests. Assumptions of dose-response relationship are different among Dunnett's, Williams' tests and linear regression analysis. Duncan's test does not control familywise error rate at a fixed level. For the last question concerning what is remarked for multiple comparison, approaches to several problems such as abnormality and heteroscedasticity were provided. Philosophy and strategy to multiple comparison problems were discussed.

Abstract: A sequentially rejective (SR) testing procedure introduced by Holm (1979) and modified (MSR) by Shaffer (1986) is considered for testing all pairwise mean comparisons.For such comparisons, both the SR and MSR methods require that the observed test statistics be ordered and compared, each in turn, to appropriate percentiles on Student's t distribution.For the MSR method these percentiles are based on the maximum number of true null hypotheses remaining at each stage of the sequential procedure, given prior significance at previous stages, A function is developed for determining this number from the number of means being tested and the stage of the test.For a test of all pairwise comparisons, the logical implications which follow the rejection of a null hypothesis renders the MSR procedure uniformly more powerful than the SR procedure.Tables of percentiles for comparing K means, 3 < K < 6, using the MSR method are presented.These tables use Sidak's (1967) multiplicative inequality and simplify the use of t ...

Abstract: Investigations of sample size for planning case-control studies have usually been limited to detecting a single factor. In this paper, we investigate sample size for multiple risk factors in strata-matched case-control studies. We construct an omnibus statistic for testing M different risk factors based on the jointly sufficient statistics of parameters associated with the risk factors. The statistic is non-iterative, and it reduces to the Cochran statistic when M = 1. The asymptotic power function of the test is a non-central chi-square with M degrees of freedom and the sample size required for a specific power can be obtained by the inverse relationship. We find that the equal sample allocation is optimum. A Monte Carlo experiment demonstrates that an approximate formula for calculating sample size is satisfactory in typical epidemiologic studies. An approximate sample size obtained using Bonferroni's method for multiple comparisons is much larger than that obtained using the omnibus test. Approximate sample size formulas investigated in this paper using the omnibus test, as well as the individual tests, can be useful in designing case-control studies for detecting multiple risk factors.

Abstract: Multiple comparisons are commonly seen in clinical trials and many other fields. An example, which is the focus of this paper, is the comparison of several test treatments (possibly different doses of a compound) with placebo (control). It is well known that steps must be taken to control the type I error rate when multiple testing is performed. We introduce the concept of the strong stagewise family error rate and propose a multiple comparison procedure to control this error rate. The proposed procedure is compared to Dunnett's step-down procedure when there are two test treatment groups and a placebo group.

Abstract: Research was conducted to provide educational researchers with a choice of pairwise multiple comparison procedures (P-MCPs) to use with single group repeated measures designs. The following were studied through two Monte Carlo (MC) simulations: (1) The T procedure of J. W. Tukey (1953); (2) a modification of Tukey's T (G. Keppel, 1973); (3) the Dunn-Bonferroni procedure (DB); (4) the sequentially rejective Bonferroni procedure (J. P. Shaffer, 1986) (SB); (5) Hayter's modification of Fisher's Least Significant Difference Test (A. J. Hayter, 1986) (FH); (6) a modified range procedure combining others (SRW); (7) a multiple range procedure based on Ryan-Welsch critical values (MRW) (T. A. Ryan and R. E. Welsch); (8) E. Peritz's (1.970) procedure (P); and (9) Welsch's step-up procedure (W). The first MC study was exploratory and was based on variance-covariance matrices that were created so as to conform to different sphericity values. Power in this study was examined for a fixed set of mean differences. The second MC study, based on the results of the first, used variance-covariance matrices found in 100 real repeated measures data sets. Based on study results, the stepwise tests SB, FH, SRW, MRW, and P and the T and W are not recommended, but the DB procedure is recommended for use with single group repeated measures data. (Contains 5 tables, 10 figures, and 34 references.) (SLD) ******************************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ******************************************************************************** co Co cv ti cv 2 Pairwise Multiple Comparisons In Single Group Repeated Measures Analysis Robert S. Barcikowski Ohio University and National Institute For Educational Development, Okahandja, Namibia PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) Ronald S. Elliott Ohio University U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) trThis document has been reproduced as received from the person or organization originating it. Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy.

Abstract: The SAS System was used to program an algorithm for constructing both simple and complex multiple comparisons among product-moment correlation coefficients. The statistical procedure is attributable to Marascuilo (1966) and is based on Scheff6's method (1959). This SAS program is applicable to designs with either equal or unequal sample sizes in the cells.

Abstract: This article presents a multiple hypothesis test procedure that combines two well known tests for structural change in the linear regression model, the CUSUM test and the recursive t test. The CUSUM test is run through the sequence of recursive residuals as usual; if the CUSUM plot does not violate the critical lines, one more step is taken to perform the t test for hypothesis of zero mean based on all recursive residuals. The asymptotic size of this multiple hypothesis test is derived; power simulation results suggest that it outperforms the traditional CUSUM test and complements other tests that are currently stressed in econometrics.

Abstract: SUMMARY Bayes/frequentist correspondences between the p-value and the posterior probability of the null hypothesis have been studied in univariate hypothesis testing situations. This paper extends these comparisons to multiple testing and in particular to the Bonferroni multiple testing method, in which p-values are adjusted by multiplying by k, the number of tests considered. In the Bayesian setting, prior assessments may need to be adjusted to account for multiple hypotheses, resulting in corresponding adjustments to the posterior probabilities. Conditions are given for which the adjusted posterior probabilities roughly correspond to Bonferroni adjusted p-values.

Abstract: In this paper a likelihood-based method for analyzing mixed discrete and continuous regression models is proposed. We focus on marginal regression models, that is, models in which the marginal expectation of the response vector is related to covariates by known link functions. The proposed model is based on an extension of the general location model of Olkin and Tate (1961, Annals of Mathematical Statistics 32, 448-465), and can accommodate missing responses. When there are no missing data, our particular choice of parameterization yields maximum likelihood estimates of the marginal mean parameters that are robust to misspecification of the association between the responses. This robustness property does not, in general, hold for the case of incomplete data. There are a number of potential benefits of a multivariate approach over separate analyses of the distinct responses. First, a multivariate analysis can exploit the correlation structure of the response vector to address intrinsically multivariate questions. Second, multivariate test statistics allow for control over the inflation of the type I error that results when separate analyses of the distinct responses are performed without accounting for multiple comparisons. Third, it is generally possible to obtain more precise parameter estimates by accounting for the association between the responses. Finally, separate analyses of the distinct responses may be difficult to interpret when there is nonresponse because different sets of individuals contribute to each analysis. Furthermore, separate analyses can introduce bias when the missing responses are missing at random (MAR). A multivariate analysis can circumvent both of these problems. The proposed methods are applied to two biomedical datasets.

Abstract: Basic Probability and Statistical Distributions Introductory Concepts in Probability Families of Discrete Distributions Families of Continuous Distributions The Exponential Class Families of Multivariate Distributions Summary Exercises Fundamentals of Statistical Inference Introductory Concepts in Statistical Estimation Nature and Properties of Estimators Techniques for Constructing Statistical Estimators Statistical Inference - Testing Hypotheses Statistical Inference - Confidence Intervals Confidence Intervals for Some Special Distributions Semi-Parametric Inference Summary Exercises Fundamental Issues in Experiment Design Basic Terminology in Experiment Design The Experimental Unit Random Sampling and Randomization Sample Sizes and Optimal Animal Allocation Dose Selection Summary Exercises Data Analysis of Treatment versus Control Differences Two-Sample Comparisons - Testing Hypotheses Two-Sample Comparisons - Confidence Intervals Summary Exercises Treatment-versus-Control Multiple Comparisons Comparing More than Two Populations Multiple Comparisons via Bonferroni's Inequality Multiple Comparisons among a Control - Normal Sampling Multiple Comparisons among Binomial Populations Multiple Comparisons with a Control - Poisson Samling All-Pairwise Multiple Comparisons Summary Exercises Trend Testing Simple Linear Regression for Normal Data William's Test for Normal Data Trend Tests for Proportions Cochran-Armitage Trend Test for Counts Overdispersed Discrete Data Distribution-Free Trend Testing Nonparametric Tests for Nonmonotone ("Umbrella") Trends Summary Exercises Dose-Response Modeling and Analysis Dose-Response Models on a Continuous Scale Dose-Response Models on a Discrete Scale Potency Estimation for Dose-Response Data Comparing Dose-Response Curves Summary Exercises Introduction to Generalized Linear Models (GLiMs) Review of Classical Linear Models Generalizing the Classical Linear Model Generalized Linear Models Examples and Illustrations Summary Exercises Analysis of Cross-Classified Tabular/Categorical Data RxC Contingency Tables Statistical Distributions for Categorical Data Statistical Tests of Independence in RxC Tables Log-Linear Models and Relationships to GLiMs Tables of Proportions Summary Exercises Incorporating Historical Control Information Guidelines for Using Historical Control Data Two-Sample Hypothesis Testing - Normal Distribution Sampling Two-Sample Hypothesis Testing - Binomial Sampling Trend Testing with Historical Controls Summary Exercises Survival Data Analysis Survival Data Lifetime Distributions Estimating the Survivor Function Nonparametric Methods for Comparing Survival Curves Regression Models for Survival Data Summary Exercises Appendices References