Choosing input variables for Linear Regression for higher accuracy

Question

I have a data set describing water levels of rivers. It has following attributes:

Value                   float64
Variable Name            object
Variable Unit Code       object
Date Pulled              object
Site Code                 int64
Site Latitude           float64
Site Longitude          float64
Site Name                object

I am using Linear Regression in trying to predict the Value using Site Code, Site Longitude and Site Latitude but getting results which are no way near the actual values. I am wondering if it's my choice of independent variables that is creating this problem or should I try other types of Regression? If it's the first case, which approaches are recommended to get higher accuracy rate or more precisely, what should one do if there are not many relevant independent variables?

Answer 1

I am using Linear Regression in trying to predict the Value using Site Code, Site Longitude and Site Latitude

Unfortunately none of these three variables can go directly into linear regression.

Site Code looks like a numerical variable, but it is actually categorical. For example Site Code = 10 is probably orthogonal to Site Code = 5, and should not be interpreted as twice as significant. The correct way to handle this is to create a boolean dummy indicator variable for each possible site code. You can also use this method for some of your other object variables which also appear to be categorical.

Latitude and Longitude will not work well in a linear regression because their relationship is highly non-linear. For example two points can have the same latitude/longitude but be very far apart. One typical method is to convert (lat,lng) pairs into predefined zones, and treat the zone as a categorical variable.

The final type of variable you have is a date. You could convert this directly to a categorical variable, but it might be better to take only the month to reduce the number of categories and generalize seasonal effects better.

Answer 2

While it is always preferred to have more informative variables for building a model, in reality though, perhaps like your case, we have to live with what we have. Although, if I were you I would think of digging other external data relevant to water level like geographical data, climate (there should be many things out there that you can add to your current feature space). Anyhow your current situation, my educated guess is that a simple regression is not a good choice of model anyway. Simply it won't capture any nonlinear correlation between your independent and dependent variables suitable for prediction i.e. water levels of rivers.

I would strongly suggest, if you want to build a better predictive model hopefully performing better than a simple regression and still fast and easy, to go with Gradient Boosting Decision Trees (GBDT) either using LightGBM, XGBoost, or recently my favorite Catboost implementations (otherwise you could think of Neural Network as well, but depends how much data you have etc). Each has its own pros and con, please check out this nice video by Mateusz Susikin in PyData Conference 2017 going through some of their differences.

Please note when building a GBDT model, you need to be careful how to encode your categorical variables, and not least, how to include your independent continuous variables if they exist. Please go through these stackexchange post1, post2 where I discuss ways to handle them, or at least they may give you some hints.