An Application of Permutation Methods to Combining Tests and Box-Cox Transformation for Small Samples
MetadataShow full item record
Permutation testing is a nonparametric methodology that determines statistical significance based on the empirical distribution of a test statistic generated by permuting the data. The only assumption of this method is the exchangeability under the null hypothesis. In the absence of treatment effect, exchangeability is consistent with randomization designs. In biomedical research, in particular clinical trials, randomized experimental designs are more common than random sampling designs and these sample sizes are usually small. First, we propose a double permutation method for combining tests, which is a linear combination of order statistics of p-values of possible candidate statistical tests. We illustrate this approach in the context of two-sample survival comparisons based on use of the log-rank test, Gehan's Wilcoxon test and the likelihood ratio test based on assumed exponentiality. In a situation where all of the candidate tests are parametric-based, we propose a double permutation method for combining parametric tests with likelihood-based weights, which is a linear combination of p-values of possible candidate parametric statistical tests. We demonstrate this approach in the context of two-sample comparisons by likelihood ratio tests based on assumed Weibull, log-normal and normal distributions. We also propose a permutation approach for Box-Cox transformation and illustrate this approach in the context of simple linear regression. Monte Carlo simulations are conducted in order to study the small sample properties of these three proposed testing procedures. These results show that all three proposed methods achieve adequate power compared with traditional methods in small samples.