Timeline for Predicting contract churn/cancellation: Great model results does not work in the real world

3 events

when toggle format	what		by	license	comment
Jun 15, 2017 at 9:46	comment	added	Ernst Dinkelmann		Perhaps to solve that issue of it only learning to identify contracts about to lapse, is perhaps to, for each lapsed contract, create a number of samples (eg representing how the features looked right before cancellations, 5 days before, 10 days before, 15 days before, etc). This is a form of oversampling (and it's OK as it's a class-imbalance problem anyways), which is better than for example SMOTE based oversampling and certainly better than random over-sampling. I'll see if my data sources allow me to do it this way and I will report back.
Jun 15, 2017 at 9:39	comment	added	Ernst Dinkelmann		Most features that are important (based on feature importance measures) are the ones that change over time. E.g. the client's broker displays certain behaviour before contracts are cancelled. So using features as at the initialization would miss out on the behavioural aspects completely as well as any other things that happen close to cancellation. I really doubt using features at at initialization would solve this problem. However, I do value the idea that it then only learns to identify contracts about to cancel. This is certainly a weakness.
Jun 14, 2017 at 15:55	history	answered	epattaro	CC BY-SA 3.0