2
$\begingroup$

In the tutorials, I have noticed only similar data has been used with models training and prediction.

I was wondering how cases where you can't find training data that is similar to your final use case(test data)?

What happens if I am building a recommendation system for house rentals but the closest training data I can find is for movies. And it has fewer features than I would have in the house scenario.

How common are scenarios like this and how are they generally handled?

$\endgroup$
2
$\begingroup$

One important assumption of data for a lot of Machine Learning algorithms is that the data from the training set and test set are i.i.d (independent and identically distributed) and come from the same probability distribution. More explanation here. That said, you cannot train a model for movies recommendation and use for house rentals. Or not that I'm aware of.

However, it's possible to train a model for house rentals recommendation if you have a lot of data for movies recommendation and a small amount of data for house rentals recommendation, using a technique called Transfer Learning. The idea is simple: If two tasks are similar, we should be able to use what we learn from one task to learn the other task faster and better.

Following is the general process (for neural networks):

  • Using movies data, train a model for movies recommendation
  • Fine-tune this recommendation system using house rentals data: Freeze the most part of the movies recommender, retrain a few last layers with house rentals data.

This is just a direction about how it can be done, there's quite a lot of related literature for you to learn more. I would say this is a not very uncommon scenario.

$\endgroup$
1
$\begingroup$

You should have the same number of features that are used for training. one can't use trained model in use cases with less number of features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.