3
$\begingroup$

I am building a regression model. Each sample/object in my dataset has some numerical and categorical features. Let's call those $f_1, f_2, f_3$ for now. The output that I'm trying to learn is a $2D$ grid of numerical values. Kind of like an $3\times 3$ image with values in it. The output variables are correlated.

For my first attempt at the problem, I thought of using $f_1, f_2, f_3$ as input features and then flatten my image into 9 output features $out_1,...,out_9$. There are multiple techniques that can optimize for those variables independantly/jointly so this is fine.

The other option that I thought of was to use the output grid coordinates as input features. So I would have $5$ input features: $f_1, f_2, f_3$ (as before) and $f_4, f_5$ are my $x, y$ coordinates for the target variable. In this case I would have more samples and my output would consist of only $1$ output target variable.

My problem is that I'm not sure whether any of the options is fundamentally wrong or just bad practice. Is there anything I should watch out for in training and testing if I proceed with either approach?

Improve this question
$\endgroup$
2
  • $\begingroup$ In option 2, if you're using the output coordinates as input features, will this affect the testing set? And furthermore, will these be part of the data you'll use the model on afterwards? Because it's part of the output, it's part of what you want to predict. $\endgroup$
    – Sterls
    Apr 27, 2019 at 3:21
  • $\begingroup$ @Sterls after building the option 2 model, I would just run the prediction 9 times ( once for each cell in my output ) and then use those 9 values as my output grid. $\endgroup$
    – Houssam
    Apr 27, 2019 at 9:35

2 Answers 2

Reset to default
4
$\begingroup$

There is a couple of separate questions I can identify here: one is whether it is appropriate to include coordinate features and the other is whether it is appropriate to convert a multivariate regression (more than one outcome variable - 9 in your example) into a regression with single dependent variable.

Coordinate features

If you think that the dependent variable value in a particular cell of your grid is somehow affected by the location of the cell within the grid (in other words, that values in the same cell are correlated), then you should probably include coordinate features (which can be simple cell identifiers or something more complex, like a more specific identification of where the cell is located within the grid relative to a particular point). It is possible to even use target-based encoding (i.e., use the mean of dependent variable values in the same cell as a new feature), though that gets into issues like data leakage and overfitting (which can be addressed separately). You may also possibly want to create interaction terms, allowing for the effects of non-coordinate features to vary by level of a coordinate feature.

This is different than modeling correlation between values across different cells. If you expect that there is some spatial autocorrelation between cells in this 2-d grid, e.g., a value in cell (1,1) is correlated with values in cells near it: (1,2), (2,1), (2,2), you should probably model that relationship by including some set of spatial features (not just coordinate identifiers), like spatially clustered covariates, or by using a spatially lagged dependent variable model.

If you are not sure if spatial correlation is present, you can estimate a model without any spatial features and then test for spatial autocorrelation (spatial dependence between residuals of regression) using Moran's I (a positive Moran’s I hints at data is clustered, a negative Moran’s I implies data is dispersed.)

Multiple vs single dependent variable

If you go with option 2 as you described it, with one output variable, you will be modeling a completely different variable than if you were modeling 9 (in your example) separate variables. Each of the original 9 variables likely has its own statistical properties, but if you combine them into one, you will have a new variable that has its own properties (some combination of the 9). So if you consider the grid to be 9 separate variables, you should stick with modeling 9 separate variables.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you so much for the detailed answer ! :) $\endgroup$
    – Houssam
    Apr 27, 2019 at 9:45
0
$\begingroup$

You will likely you will have an extremely multi-modal distribution if you are performing a regression. I.e. if there are multiple locations which are clustered and you include it as a numeric variable, there will be a highly non linear relationship.

Perhaps try including the nearest zip code as a categorical variable. That will pick up the clusters more effectively.

Coordinates can be useful if you are determining clusters. Clusters are essentially the categorical variables like zip code obtained via unstructured data algo's. The only difference is a zip code is determined via manual classification and a cluster might take into account other features and is obtained via k-means or another algo.

Once you determine the clusters via certain features, you can include them as a categorical variable in your regression.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.