How to know whether machine learning is possible for a given data set. I have been given a data set, I should check whether machine learning is possible or not for that data set. How can I do that. How do you come to conclusion that machine learning can be performed for the given data set or not.
For this feasibility study, following will be high level steps :
- For each feature perform PCA with rest of the features as train_x and feature as train_y. If you find a feature that can be predicted by other features; ML can be applied on Dataset
- Can a Human solve it ? As a person; can you find patterns for a given feature, based on other features ?
- Exploratory data analysis with Weka, Dataframe + Matplotlib or similar tools . https://datascienceguide.github.io/exploratory-data-analysis
I have asked the same question myself many times. To add a bit of a context: I work with relatively small data (~100 observations per experiment) of environmental nature, which is often sparse and/or imbalanced.
My empirical answer is very trivial: just try it! Also, do not forget that ML is a loosely defined term - some "classical" statistical tools may very well fall under it.
To begin, use the domain knowledge to set up appropriate research questions and think about what your data can tell you. Then start exploring the basics: draw a correlation plot, check whether the data is normally distributed, etc. Then you can apply unsupervised learning to look for more complicated relationships. Perhaps afterwards you may do some supervised one to make predictions.
One important remark. On one hand, do not get discouraged by poor PCA performance - it may pretty much happen that your data relationships are not linear. On the other hand, do not expect to build a Neural Network for every single problem ever - oftentimes they are not needed. Just go from a low level of complexity to a higher one, till it makes sense to continue.
Hope it helps!