0
$\begingroup$

I have a dataset where I have features from winning tennis players and the other half are from a losing tennis players:

winner_age, winner_rank / loser_age, loser_rank

In order to conduct a proper EDA, do I have to concatenate both losers and winner to further analysis or split the data frame in two?

| improve this question | |
$\endgroup$
0
$\begingroup$

So, the question asks that during Exploratory Data Analysis whether it is better to analyse the data with both winning and losing tennis players in one data frame or split them into two. In this question, I am going to assume that you will be using this data in a classification task (do correct me if I am wrong!): to discriminate between winning and losing tennis players.

I believe the best approach would be to analyse with both winning and losing players together. The main reasons for it are:

  • When you are running this data through a model, you won't split the data by class (winning vs losing players)
  • More importantly, in the context of feature engineering, the usual rule of thumb normally is the you choose features which can greatly discriminate between the classes. So you can only do that by reviewing the feature distributions by classes to see if adding those features to the model will improve generalisation performance of the model.
| improve this answer | |
$\endgroup$
  • $\begingroup$ Ok. Great makes sense. Thanks for the reply. The problem is that I have more than 40 features and Im afraid to ending plotting twice the same graph for winners and losers. Do I concatenate the features just for some cases ? $\endgroup$ – TOMAS Jul 7 at 19:53
  • $\begingroup$ So you could still look at the distributions for each feature if you have the time. But with a high number of dimensions, you can condense them into a smaller number of latent features using an algorithm like Principal Component Analysis (builtin.com/data-science/…). $\endgroup$ – shepan6 Jul 7 at 21:24
  • $\begingroup$ I just tried PCA but I lose the feature names. It just picks the best features but then I dont have the names. If I want plot the features I cant without the names. $\endgroup$ – TOMAS Jul 8 at 16:31
  • $\begingroup$ That is why I used the terminology "latent features" because these features do not correspond to same features you started with. They will choose a lower dimension which prevents minimum information loss. Hence PCA is a way to compress the dimensionality of the data. $\endgroup$ – shepan6 Jul 8 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.