Comparing Performances of Multiple Comparison Methods in Commonly Used 2 x C Contingency Tables

Abstract

This study aims at mentioning briefly multiple comparison methods such as Bonferroni, Holm-Bonferroni, Hochberg, Hommel, Marascuilo, Tukey, Benjamini-Hochberg and Gavrilov-Benjamini-Sarkar for contingency tables, through the data obtained from a medical research and examining their performances by simulation study which was constructed as the total 36 scenarios to 2 x 4 contingency table. As results of simulation, it was observed that when the sample size is more than 100, the methods which can preserve the nominal alpha level are Gavrilov-Benjamini-Sarkar, Holm-Bonferroni and Bonferroni. Marascuilo method was found to be a more conservative than Bonferroni. It was found that Type I error rate for Hommel method is around 2 % in all scenarios. Moreover, when the proportions of the three populations are equal and the proportion value of the fourth population is far at a level of +/- 3 standard deviation from the other populations, the power value for Unadjusted All-Pairwise Comparison approach is at least a bit higher than the ones obtained by Gavrilov-Benjamini-Sarkar, Holm-Bonferroni and Bonferroni. Consequently, Gavrilov-Benjamini-Sarkar and Holm-Bonferroni methods have the best performance according to simulation. Hommel and Marascuilo methods are not recommended to be used because they have medium or lower performance. In addition, we have written a Minitab macro about multiple comparisons for use in scientific research.