Is this a case for too much dimensionality? 1881 samples, 2562 features

Question

I have been working on creating a multi-class classification model for medical data. I have 1881 samples, 2562 features each, and 6 classes total. My distribution of classes is as follows:

{1: 83, 2: 1021, 4: 169, 5: 229, 6: 288, 3: 91}

Can someone tell me how this dataset will impact my model's performance? I have done some research that this could lead to problems with dimensionality however I would appreciate some clarification on if that is the subject here and how I would fix it.

I am still relatively new to working with AI models so absolutely any help is greatly appreciated, let me know if you require any more information.

Answer 1

In most data science scenarios, thousands of features are not relevant. Just a few are enough, but it depends on the data.

In general, some data preprocessing is necessary to take the most relevant features. This could be done thanks to a correlation map.

https://medium.com/@szabo.bibor/how-to-create-a-seaborn-correlation-heatmap-in-python-834c0686b88e

The features with correlated values around 0 with other features could be removed, and the strongly correlated features could be merged into one.

If you want to have also a clearer view of your data, you can apply a dimensionality reduction algorithm to squeeze your data into 2 or 3 dimensions, and get clusters of similar features. It also works for grouping similar samples.

https://umap-learn.readthedocs.io/en/latest/clustering.html

One last tip: start with smaller samples with fewer features to reduce the processing time and build an efficient model quickly. Then increase them to cover all features and samples.