Multivariate linear regression accounting for threshold / data cleaning

Question

I am trying to make a linear regression model for the sale price of a house based on many variables (based on the data from this Kaggle challenge https://www.kaggle.com/c/house-prices-advanced-regression-techniques)

The distribution above is the 2nd-floor size in square feet and the y-axis is the sale price. It shows a clear linearity, except for the fact that homes without a 2nd floor clearly are not for sale for 0 dollars.

I have many variables like this that have some threshold either upper or lower that has a large distribution of response for a single value. If I simply exclude these values then the intercept for this curve will be through the origin. Should I let that be the case and assume that the $0 price tag will be corrected by the other variables in my regression?

What is the best way to treat/fit data such as this?

Thanks!

Answer 1

You could try including "Does not have a 2nd floor" as a separate categorical variable, which you could encode as 1 or 0 in your linear model. Something like:

price ~ no_second_floor + area_of_second_floor + [Other Variables]

Event without the other variables, it's already going to be able to fit the data better than price ~ area_of_second_floor because, instead of forcing the price of the single-floor houses to $0, it would be able to fit it to the average price of all the single-floor houses in your data set - which is the best you can do for those houses until you add other variables besides area_of_second_floor.

(Actually, it wouldn't be forced to $0 if you were to include a constant term in your model, but the constant term that best lets you fit the linear portion of the data still probably isn't what you want for the single-floor houses, some of which - as you see - are quite expensive).

Answer 2

Every regression analysis should include a residual analysis as a further check on the adequacy of the chosen regression model. A plot of all residuals on the y-axis vs. the predicted values on the x-axis, called a residual vs. fit plot, is a good way to check the linearity and equal variance assumptions.

To answer your question, I believe if you did this EDA (Exploratory Data Analysis) you would find that including your single floor data does indeed violate the linearity that you have to assume if you are making a linear regression model.

To get around your intercept at $0 dollars issue, try subtracting the mean of your two story houses from each data point (centering the data ). This should give you meaningful results at the intercept. Your new intercept should be the mean of your two story houses.

After you do your EDA with the residuals, if you find your data is not linear try doing some transformation of the variables to get you to a linear data set.

I have used the following as a reference to answer your question. I encourage you to check it out: especially the section on transforming variables.

Chapter 9 Simple Linear Regression - CMU Statistics