1
$\begingroup$

I'm currently working on a project involving the prediction of tenancy lengths. I've so far managed to get to a point where I've processed the data and pruned my Random Forest model (via sklearn in Python) to the following accuracy levels (in days):

Train MAE: 131
Train R^2: 0.906
Test MAE: 259 (using cross-validation)
Test R^2: 0.651

enter image description here

While the model is decent for the industry, there's more performance to squeeze out of it. It currently overestimates results and has poor accuracy on the test data imo.

I'd like to further develop a Neural Network approach, as my initial implementation of an MLP Regressor seems promising:

Train MAE: 301
Train R^2: 0.582
Test MAE: 338 (using cross-validation)
Test R^2: 0.522

enter image description here

My question is how can I improve my results for the prediction (using Python) other than using GridSearch to play around with the MLPRegression function in sklearn? Are there any other models that could be useful in this situation? (I have tried also decision trees, gradient boosting)

In case it is relevant, my dataset contains ~5000 entries since 2008 onwards of individual tenancies, containing: tenancy dates, rent, repair costs, property information and replacements, client information, etc., currently at 41 variables.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

You may have done this already, but if your target values are positive integers, it could be worth transforming your output layer in such a way that constrains it to take an appropriate range of values.

Tenancy length values presumably have some recognisable distribution that might fruitfully be modelled by a known statistical process (Poisson process, survival analysis, etc.), and so instead of using your NN to predict the tenancy length directly, you could use the NN to parametrise a relevant probability distribution, and make a point estimate of that distribution in your output layer.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Hm, that's a pretty good idea, thanks! I have indeed conducted survival analysis on tenancy length previously, so I have some idea on how the distributions look like. The problem is I'd have no idea where to even start to transform my output layer in sklearn's NN. Would I have to create a custom NN or is there something I'm overlooking? $\endgroup$
    – Silviu Tofan
    Oct 12, 2016 at 8:10
  • $\begingroup$ Having a look at the scikit-learn docs for MLPRegressor, it looks like it takes a parameter out_activation_, which is the name of a function to apply at the output layer. $\endgroup$
    – R Hill
    Oct 12, 2016 at 8:56
  • $\begingroup$ Not sure if I understand how that works unfortunately.. Sorry, I'm quite inexperienced with this. $\endgroup$
    – Silviu Tofan
    Oct 12, 2016 at 9:30
  • $\begingroup$ Looking at it a bit more, out_activation_ is an attribute of the model, not a parameter, so you can't set it yourself. It looks like MLPRegressor won't support this, so you probably will have to create a custom NN to achieve this. $\endgroup$
    – R Hill
    Oct 12, 2016 at 9:58
  • $\begingroup$ Just what I was afraid of. Thank you very much. I guess my current predictions have to be enough for the time being until I get the chance to look into how this would be achieved. $\endgroup$
    – Silviu Tofan
    Oct 12, 2016 at 10:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.