How should classification be done for a very small data set?

Question

I am looking at data from the London Data Store based on social characteristics between London boroughs.

Since there are only about 30 London boroughs, the data sets I am looking at are naturally very small. For example, I might be fitting regression/correlations to a plot of about 30 points.

1. What are appropriate ways to conduct classification on such small data sets, and why? 'Why' is important.

I was thinking of something like SVM, or Naive Bayes. Or regression if the data is continuous.

2. What are very inappropriate ways to conduct classification here?

Answer 1

I don't think you need some classification algorithm, you can use your basic understanding on data/ Business Knowledge to do the classification. As the number of data points are too low, the model cannot give you good/generalised results.

Even if you try applying some complex algorithm like SVM/NN, it is of no use as the data is too low.

If you still want to apply some machine learning algorithm and then you can apply Naive Bayes, Decision Tree as these are the basic algorithms, can do the job.