How to approach this data set for linear regression?

Question

I am new to data science and am currently working on a data science project and have to answer a few questions about the following data set with 18k data points: https://www.kaggle.com/karangadiya/fifa19

The question I have to answer is as follows: To which extent can you determine the value of a player using the players most important features?

Features in this case being the columns from 'Crossing' to 'GKReflexes'. I grouped up all the players as follows: Att = ['RF', 'LF', 'RS', 'LS', 'RS', 'RW', 'LW', 'CF', 'ST'] # attack positions Mid = ['LAM', 'CAM', 'RAM', 'LM', 'LCM', 'CM', 'RCM', 'RM', 'LDM', 'CDM', 'RDM'] # midfield positions Def = ['LCB', 'CB', 'LB', 'RCB', 'RB', 'LWB', 'RWB'] # defense positions GK

I already looked at the data and was able to determine that certain player features are much higher for certain positions.

I plan on using multiple linear regression to answer the research question, but I'm not sure on how to split the data into a train and test set.

I'm thinking of one of the following approaches:

1. First split the large data set into smaller data sets for each position (GK, DEF, ATT and MID) and then make a train and test set for each position.

or

2. Leave the large dataset as it is with all of the positions combined and split that into a training and test set.

I am currently leaning more towards the first approach because I think I'll be able to better determine the price of the player using the features that are important to the players position, but I am not sure if that's the correct way to go at it.

Answer 1

So in the first approach you want to manually choose some features that seem more important?

If so, I think you might not be able to accurately diagnose the most important features, I think it is better to first choose the most impactful features using an algorithm like backward-elimination, etc to figure out what are the most important features, then ignore the less important columns and split the dataset to training and test error.

Answer 2

I can't know without reading the assignment or seeing the available data, but I suspect that you are not intended to split the data by offense/midfield/defense position and then train models for each set.

Doing so is you making an assessment that a player's position is an important predictor of player value, but not allowing your modelling process to capture its significance. It's much more likely that position is intended to serve as a categorical predictor.

Including position as a predictor (rather than a meta-category) allows your model to determine the marginal value of all other predictors while considering player position. This is the similar to the approach you describe in (1), but gives you more information about how, exactly, position affects player value.

---

Splitting data into training and test sets has nothing to do with choosing variables or assigning a priori categories. Training and test sets are random assignments of observations into subsets of data so that you can train the model (on the training set) and then evaluate its performance on data that wasn't used in training (on the test set, to estimate how well the model will generalize).

So even if you do divide your data by player position, each of those subsets of data would need to be divided into training and test sets as part of model development.

---

There definitely are cases where you will want to categorize data in various ways before working on a model, but those will nearly always be based on existing domain knowledge. Never make that sort of decision based on something you "saw in the data" that gave you an impression such a division might exist.

Approaching things that way bakes your assumptions (which may or may not be valid) into every subsequent part of your research, with the result that your model starts to reflect your guesses about what the data is probably like rather than using the modelling process to explain the data actually is like.

Answer 3

What exactly do you mean by linear regression? If you speak about OLS, you could build a model like:

salary = POSITIONS + feature_1 + ... + feature_x

"POSITIONS" is a dummy (or one hot) encoded representation of positions (a matrix). So you have a separate intercept for each position (aka "fixed effects" in econometrics). This helps to "isolate" effects of position on salary (other things equal). It also works in other models, because you help the model to learn that the position is important to explain salary.

This model internally "splits" the positions. So you also do a test set in which all features are represented (your track 2).

In a linear model, you can easily imagine the effect. Please, see my ugly painting below.

This model will not necessarily give a good fit (because of the parametric assumptions behind). But you could further augment the model by introducing interaction terms between positions and features.

Feature selection would be a different problem. In this case you would need to loop over different models and compare them or you would need to "shrink" coefficients via Ridge or Lasso. Some more background here: https://datascience.stackexchange.com/a/52220/71442