1
$\begingroup$

I have a dataset of 256 rows with 61 columns/variables. Each row should be considered a vector of dimension 61. If I randomly split it, by rows, in 2 groups, how could I prove that the 2 groups are similar? The origin of the data is biomedical and nonlinear approaches should be preferable.

$\endgroup$
  • $\begingroup$ If you split the data randomly the two groups will always be similar because you know that they come from the same distribution. $\endgroup$ – stmax Apr 21 '16 at 12:46
1
$\begingroup$

You can't actually prove that the two groups are similar but you can establish a confidence level/threshold. Furthermore, it is possible that the two groups won't be similar (depending on your threshold for similarity) if, for example, only one of the two groups contains strong outliers.

That said, you can make comparisons based on assumptions regarding the underlying data distribution. For example, if the data can be assumed to be distributed as a multivariate normal distribution, you can use Hotelling's two-sample T-squared statistic (a multivariate generalization of the Student's t-test) to test your confidence interval.

$\endgroup$
  • $\begingroup$ Thx. Not fully understanding the concept yet, I have performed the test you suggested. Splitting into matrix A & B, the "result" was higher than 70 (in general >100). If I try A against A (or B x B) it is 0. How should I interpret the result A x B (> 100)? Another point, is this approach considering the whole vector as the element of comparison? $\endgroup$ – RgrNormand Apr 21 '16 at 16:51
  • $\begingroup$ The linked page states that the T-squared statistic is distributed as a noncentral F-distribution. So for a chosen confidence level, you would find the corresponding point on the noncentral F distribution CDF, which translates to the corresponding T-squared value. The value of T2=100 corresponds to a particular significance level. You probably want the opposite: pick your confidence level and compute the corresponding T2 value to use as your threshold. $\endgroup$ – bogatron Apr 21 '16 at 18:18
  • $\begingroup$ Regarding the "element of comparison", the matrices A and B are each considered "as a whole" in the sense that their mean, covariance, & sample sizes are used in the computation. But the original data matrix (before splitting into A and B) is not considered as a whole (implicitly or explicitly). In other words, if you perform two random splits of the data, you should not expect identical results for the T-squared statistic. $\endgroup$ – bogatron Apr 21 '16 at 18:26
  • $\begingroup$ Thx. I am working with R, Hotelling package (hotel.test(a,b)), and after your comments I have decided to run 1000 simulations. The results are quite variable for T2 (mean 80, almost bell-shape, range 40-160) and pValue (mean 0.5, almost uniformly distributed between 0-1). Curiously, for AxA or BxB, pValue=1, T2=0. Any insights from those results? $\endgroup$ – RgrNormand Apr 21 '16 at 19:35
  • $\begingroup$ Don't know what you mean by pValue. Assuming A & B are same size, 128 observations is not a lot for 61 variables. So you may be suffering the Curse of Dimensionality. For comparison, you might try re-running your analysis using a subset of attributes (say ~10) to see how T2 varies. Maybe do the same using the top few principal components. And for extra credit, plot the histogram of Mahalanobis distances of all points in group B from the mean/covariance of group A. $\endgroup$ – bogatron Apr 21 '16 at 20:41
0
$\begingroup$

There are other recent methods like Principal Difference Analyses specifically designed to address these sort of problems. Am not sure if the methd is available as an R package, you can get the concept/algorithm from the manuscript. See http://arxiv.org/abs/1510.08956

$\endgroup$
  • $\begingroup$ Thx. I will check it. $\endgroup$ – RgrNormand Jul 22 '16 at 4:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.