New answers tagged

0

Another related problem, which I think worth considering, before generating more features, is to determine which columns are important for classification task, i.e. improve prediction of target variable. One common way is to rely on feature importance scores, but a disadvantage of this is that the scores are only available after training model. To guess ...


1

You may encode each level and concatenate - If we ignore the path till file1 as its same across all the names. Then we need 1 digit for subfile and 2 digit for targetfile c:/users/file1/subfile1/targetfile_0 - [0 00] c:/users/file1/subfile1/targetfile_1 - [0 01] c:/users/file1/subfile1/targetfile_2 - [0 10] c:/users/file1/subfile2/targetfile_0 - [1 00] ...


0

If you want to use neural networks this post on Kaggle might help: https://www.kaggle.com/c/m5-forecasting-accuracy/discussion/159052 It has a short list of resources for categorical embeddings and LSTM (I think). If you think your dataset has periodic patterns, and you only need to answer your questions (not deploy a model). I would take a look a FB Prophet:...


1

I think you can keep as many as you want, and it'll be alright. Sometimes it's even worth to delete the very rare classes to have more stable features. In addition, for linear regression, you shouldn't include all of them, as you might have a collinearity issue. To sum up, no problem with not keeping them all.


Top 50 recent answers are included