Based on three datasets, I have produced the scatterplot below in Python: enter image description here

I am trying to fit a line on each dataset, but when I check the metrics this is what I get:

  • Set 1 (red): R squared=0.002, p-value=0.651
  • Set 2 (purple): R squared=0.008, p-value=0.378
  • Set 3 (blue): R squared=0.001, p-value=0.714

My question: are such data sets impossible to fit? Is there any kind of data transformation I could apply, based on the scatterplot shape?

The goal is to predict from regression.

  • $\begingroup$ There's an infinite number of transformations, eventually you can find one that can give you any answer you want. Just ignore the fact you chose a transformation from the infinity of transformations when you write it up with your significant p-value. $\endgroup$ – Spacedman Nov 30 '15 at 13:45
  • $\begingroup$ Less sarcastic response: What is the question you are asking of the data? Chasing transformations to get a "good fit" or a large R-squared is not the way we do statistics. $\endgroup$ – Spacedman Nov 30 '15 at 13:45
  • $\begingroup$ I didn't mean to force a fit - I just wanted to make sure there is any. $\endgroup$ – FaCoffee Nov 30 '15 at 13:54
  • $\begingroup$ You can fit a line to any data. Are you trying to determine trend, predict from regression, cluster, classify, ...? $\endgroup$ – user13684 Nov 30 '15 at 14:56
  • 1
    $\begingroup$ The data certainly appears non-linear. Have you considered regression trees (CART)? $\endgroup$ – user13684 Nov 30 '15 at 15:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.