0
$\begingroup$

We have a few variables that are highly predictive in our modeling task. Is it sound to train models with a superset of features even though some are known NOT to be available at predict time? & if so, what are some suitable ML techniques to effectively design "optional predict-time" features? If not what are some of the fallacies of this?

That is, both learning & evaluation data come from the same distribution, except training has a few more variables (without any missing values) but these variables are impossible to exist at predict time.

For context, let's say we are trying to explain how the specific specs of a product from a manufacturer affect its pricing in the public market. We have customer star rating information for released products, but want our model to generalize to new not yet released products with similar types of specs. How can we leverage the star-rating feature for learning?

Perhaps using one of the many imputation strategies to fill-in lacking prediction inputs with values learned at training time, but perhaps there are other techniques.

$\endgroup$

1 Answer 1

2
$\begingroup$

You are describing several timestamps in the lifecycle of a product:

  1. $t_1$ launch, with features A, B, C
  2. $t_2$ initial inference time for sales figures
  3. $t_3$ customers award stars, assigned to feature D
  4. $t_4$ later inference time for sales figures

At all times we are traversing a product manifold which definitely has some structure to it. There are good and bad products, which will be reflected by sales and by customer stars. Retail price might be among the A, B, C features.

There are several models you can learn from this setup.

  • Given A, B, C, predict number of stars D.
  • Given A, B, C, predict monthly sales.
  • Given A, B, C, D, predict monthly sales.

We assume A, B, C do not change over the product's lifetime (so, for example, no timeseries of ad spend), and we similarly assume that D is a scalar quantity which becomes available after 90 days and thereafter does not change (not a timeseries). In particular, the timeseries of sales figures is not an input feature.

Apply chain rule.
Train a model to predict (A, B, C) --> D.
Train a model to predict (A, B, C, D) --> sales, and also
train a model to predict (A, B, C) --> sales.
Compare performance.

$\endgroup$
2
  • $\begingroup$ Interesting chain rule approach. In our real problem we know optional features help predict the target & we use the target to explain influence of the mandatory features. So the suggestion is to learn to approximate the optional features from the mandatory ones (rather than just impute the optional when predicting) & inject these values into our main model at predict time? $\endgroup$
    – eliangius
    Apr 5 at 11:00
  • $\begingroup$ Yes, that’s right. $\endgroup$
    – J_H
    Apr 5 at 19:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.