What else can I do to help my model my classification task?

Question

I have a classification task that I'm currently getting really low accuracy metrics on (my highest accuracy score is about 20%). So far I've run 5 models: quadratic disc analysis, logistic regression, knn, random forest, and naive bayes (Gaussian but will try categorical soon). I've used GridSearchCV (10-folds) for all. My dataset has ~1500 data points with no more than 9 features.

My only dummy variable covered gender, and I've already left one option out to avoid the dummy trap. My other explanatory variable is age group, which I've encoded to preserve order. Finally, my dependent variable is actually a vector (using multioutput from sklearn) of binary target variables.

For more color on the dependent variable: the original feature was a question that allowed for 6 response choices but respondents could elect for multiple of them. I've essentially turned them into dummy variables (did not drop one of them) and turned them into a vector to predict using sklearn's multioutput tools.

Any idea where I can improve the model?

Answer 1

There are several steps you can take to improve the performance of your classification model:

1. Data preprocessing: Ensure that the data is cleaned and preprocessed properly. Check for missing values, outliers, and ensure that the data is normalized or standardized if necessary (look at preprocessing module).
2. Feature engineering: Try to create new features that can better represent the underlying patterns in the data (look at decomposition module).
3. Model selection: Consider using more advanced machine learning algorithms such as gradient boosting (xgboost), Random Forest, or NN. Complex algorithms can often capture more complex relationships in the data.
4. Hyperparameter tuning: Optimize the hyperparameters of your models using cross-validation techniques such as GridSearchCV and RandomizedCV (look at model selection)
5. Address class imbalance: If your data is imbalanced, you can try techniques such as oversampling or undersampling to balance the classes (understand also how to stratify your data)
6. Evaluate performance metrics: Look beyond accuracy and evaluate other performance metrics such as precision, recall and F1-score to get a better understanding of how well your model is performing.

Finally, it's important to keep in mind that there may be limitations to what can be achieved with the available data. If the underlying patterns are inherently complex or noisy, it may be difficult to achieve high levels of accuracy. In such cases, it's important to carefully evaluate the performance metrics and determine what level of performance is acceptable for your use case (compare your results with a dumb (random) classifier).