I'm having issues with fitting a Random Forest model to a completely new dataset. I'm trying to predict tenancy lengths for current tenants. I have a dataset with tenancy information since 2008, with both tenants that have finished their contract and others which are still in their properties.
Steps undertaken so far:
1) Remove the tenancies that are still on going. This is to actually get target values for "Tenancy Length".
2) Find a suitable model to train and test on the finished tenancies. I'm happy with a Random Forest, with MAE of ~35 on train and ~70 on test.
3) Create a new dataframe in python that only has the previously excluded information (tenancies that have not yet finished) in order to apply the .predict method on these. I've also combined the previous 'train' and 'test' into a single train, as there is no further need to split that data up.
df = finished tenancies; currentDf = ongoing tenancies
#Select the features and target variable from finished tenancies data X, y = df.iloc[:, 5:].values, df.iloc[:, 4].values clf = RandomForestRegressor() #with relevant params clf.fit(X, y) yPred= clf.predict(X)
This results in a 'new' training set consisting of all the finished tenancies.
4) I move on and predict my current tenancies:
X2 = currentDF.iloc[:, 5:].values y2Pred = clf.predict(X2)
While I cannot create a residual plot for this (as I have no previously known information regarding targets), I've noticed that 'Tenancy Length' strongly correlates with 'Tenancy Start', which was not the case in my trained model.
Does anyone have any idea what's going on? Why would something like this be happening? I can't for the life of me figure out if I'm doing something wrong, yet I doubt it's just coincidence for some reason.