I'm new to machine learning, and I'm working with some international head-to-head sports competition data. I used relational data creation techniques in tidyverse to join several data sources to create an event based dataset where each row is the outcome of a unique match up between 2 teams and their measurable traits, with the obvious goal of finding the importance of those traits on the outcome.
Note = In general, I'm trying a few different ways to organize the data to get hands on experience with creating and analyzing data sets effectively, so if how I set it up isn't how you'd do it, don't go too hard on me. Specifically with have a repeat in variables based on home versus away, so that I could try to get all the event data into one and only one observation (example = "average_speed_home" and "average_speed_away"). I know and will try other data configurations, but any suggestions related to that would be nice, but it's not the main reason I'm asking for help.
My main question is related to what I should do with ID variables that I used to create the data set in relation to data splitting for machine learning. I've read on a few posts that I should keep those variables for the data splitting because it could create bias if I don't. But, the ID variables I have, aren't really factors that I want to included when creating my models.
Specifically, I used Home and Away variables for certain teams in the matchup, so I could pivot wider and include all the data for each event in one observation, but those team differences are already shown in the other variables that are dedicated to to either home or away for a certain trait, such as "average_speed_home" and "average_speed_away". The Home and Away variables now just say a Nationality. Since several of the yearly competitions are included in the data set, I wouldn't want to analyze the nationality impact on the results because the composition of the teams change quite frequently and the results might be biased due to recency of success (and also all the actual skills and performance metrics are included as their own variables).
Also, I'm unsure about whether to leave the match # ID variable as well because each line is a unique value that is essentially just an observation count.
Does the distribution of the data matter for data splitting when the data is event-based and standardized such that the only differences would be what actual National Teams faced each other in the matchup? And if I'm only looking for the numerical values of certain skills and their impact on the outcome, should I worry about including the ID variables when data splitting?
TL;DR: I don't want to include certain ID variables for the model creation, but I'm unsure if getting rid of those variables before the data split would create bias, especially in standarized event-based data.
What's the general rule of thumb for when to get rid of seemingly unimportant ID variables that were only used to create the relational data set? When is it better to get rid of them before the data split or after the data split?
Thank you.