# If my data looks like this, is linear regression just never gonna be useful?

Clearly you can see that there is no linear relationship. If there was one it would be a vertical line which is not that useful. I need some type of regression rather than classification as the output is continuous. The only options I can think of are neural networks and SVR (support vector regressors). Is linear regression a waste of time here? What would you use?

The use case is - I am trying to predict the rgb color of a polygon based on its area, length, index, complexity, #of lines, # of curves, and intensity (black and white).

In this first step I am merely trying to predict the amount of red from 0 to 1. (scaled down from 255).

Edit:

here are the labels - rgb values: http://www.heypasteit.com/clip/2HZX

here is the data with the 7 features: http://www.heypasteit.com/clip/2HZY

• its really hard to see this data... can you jitter your plots? Or zoom in towards x=0... what's the scale on the x-axis? Or better yet, do you have a sample of the actual data? Mar 3 '16 at 23:49
• added samples of the data in those urls Mar 4 '16 at 0:03
• It seems some of your inputs have exaggerated values. Try ignoring them, or excluding them from the plots.
– Emre
Mar 4 '16 at 0:10
• That might possibly help. In general, are there times where linear regression is just a waste and clearly not gonna work - like if you see data like it is here? Or is there always something that you can do to get the data into a form where linear regression might work - like removing outliners, etc. Mar 4 '16 at 0:17
• It works when there is a linear relationship between the features and the response. If you picked the features yourself, you should have some idea about whether such a relationship holds, otherwise you find out when you try. It helps to know something about the process that relates the input to the output.
– Emre
Mar 4 '16 at 0:22

I tried running a RANSAC model on your data, but got worse results than a straight linear regressor. The ten-fold cross-validated mean absolute error over all three response variables (r,g,b) for the linear model was about 0.37. I also ran a random forest model for comparison, and got about the same score. This suggests that the linear model is not shabby, but it is up to you to decide if either is good enough.

import sklearn.linear_model, sklearn.cross_validation, sklearn.ensemble, pandas

sklearn.cross_validation.cross_val_score(sklearn.linear_model.LinearRegression(), features, labels[:len(features)], 'mean_absolute_error', 10, -1).mean()


I also tried visualizing the data with the outliers clipped, but did not find it very enlightening, so I did not include it here.

• hmm. Can you paste your exact code? I tried the code you pasted and got a regression score of -0.277155605577. Is your 0.37 perhaps after clipping some outliers? Mar 8 '16 at 23:35
• This is the actual code, and I still get 0.37. Are you using the data set you posted? Note that sklearn outputs the negative of the score for its own convenience, so your score was actually 0.277. Obviously, the MAE/MSE can not be negative.
– Emre
Mar 8 '16 at 23:44
• ah well that makes me feel better. Still 0.27 or 0.37 are both pretty horrible. I think I need to look for a different algorithm to up the accuracy, perhaps a neural network? Mar 9 '16 at 0:00
• Sure. To get a better sense of what a good score is, I would run the algorithm on a test set, and have it show you the predicted color beside the real one. There are well-known limits to human sensitivity to color difference.
– Emre
Mar 9 '16 at 0:07
• in your code sample, it seems to predict labels using all 3 features ('r','g','b') through the linear regression. I figured that this was impossible and that linear regression can only predict 1 feature. Is this indeed what your code does, or does it specify a particular color somewhere that I'm not seeing? Mar 9 '16 at 1:20