I have a dataset of 256 rows with 61 columns/variables. Each row should be considered a vector of dimension 61. If I randomly split it, by rows, in 2 groups, how could I prove that the 2 groups are similar? The origin of the data is biomedical and nonlinear approaches should be preferable.
1
-
$\begingroup$ If you split the data randomly the two groups will always be similar because you know that they come from the same distribution. $\endgroup$– stmaxCommented Apr 21, 2016 at 12:46
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
7
You can't actually prove that the two groups are similar but you can establish a confidence level/threshold. Furthermore, it is possible that the two groups won't be similar (depending on your threshold for similarity) if, for example, only one of the two groups contains strong outliers.
That said, you can make comparisons based on assumptions regarding the underlying data distribution. For example, if the data can be assumed to be distributed as a multivariate normal distribution, you can use Hotelling's two-sample T-squared statistic (a multivariate generalization of the Student's t-test) to test your confidence interval.
-
$\begingroup$ Thx. Not fully understanding the concept yet, I have performed the test you suggested. Splitting into matrix A & B, the "result" was higher than 70 (in general >100). If I try A against A (or B x B) it is 0. How should I interpret the result A x B (> 100)? Another point, is this approach considering the whole vector as the element of comparison? $\endgroup$ Commented Apr 21, 2016 at 16:51
-
$\begingroup$ The linked page states that the T-squared statistic is distributed as a noncentral F-distribution. So for a chosen confidence level, you would find the corresponding point on the noncentral F distribution CDF, which translates to the corresponding T-squared value. The value of T2=100 corresponds to a particular significance level. You probably want the opposite: pick your confidence level and compute the corresponding T2 value to use as your threshold. $\endgroup$– bogatronCommented Apr 21, 2016 at 18:18
-
$\begingroup$ Regarding the "element of comparison", the matrices
AandBare each considered "as a whole" in the sense that their mean, covariance, & sample sizes are used in the computation. But the original data matrix (before splitting intoAandB) is not considered as a whole (implicitly or explicitly). In other words, if you perform two random splits of the data, you should not expect identical results for the T-squared statistic. $\endgroup$– bogatronCommented Apr 21, 2016 at 18:26 -
$\begingroup$ Thx. I am working with R, Hotelling package (hotel.test(a,b)), and after your comments I have decided to run 1000 simulations. The results are quite variable for T2 (mean 80, almost bell-shape, range 40-160) and pValue (mean 0.5, almost uniformly distributed between 0-1). Curiously, for AxA or BxB, pValue=1, T2=0. Any insights from those results? $\endgroup$ Commented Apr 21, 2016 at 19:35
-
$\begingroup$ Don't know what you mean by
pValue. AssumingA&Bare same size, 128 observations is not a lot for 61 variables. So you may be suffering the Curse of Dimensionality. For comparison, you might try re-running your analysis using a subset of attributes (say ~10) to see how T2 varies. Maybe do the same using the top few principal components. And for extra credit, plot the histogram of Mahalanobis distances of all points in groupBfrom the mean/covariance of groupA. $\endgroup$– bogatronCommented Apr 21, 2016 at 20:41
$\begingroup$
$\endgroup$
1
There are other recent methods like Principal Difference Analyses specifically designed to address these sort of problems. Am not sure if the methd is available as an R package, you can get the concept/algorithm from the manuscript. See http://arxiv.org/abs/1510.08956
