Suppose we have two data sets of movie reviews; one from IMDB and one from Rotten Tomatoes (RT). Each entry has a written-review and a score attached to it. The concatenated datasets might look like
The Lion King|9.8|"This is one of the best movies ever!"|IMDB
The Lion King|4.5|"This move absolutely made my childhood"|RT
Our task is to predict scores from reviews. This is similar to the Kaggle IMDB sentiment analysis challenge, except we're trying to predict a continuous variable from two data sources. For a single data set, we might train a model in Keras with an embedding and lstm layer. In R:
model <- keras_model_sequential() %>% layer_embedding(input_dim = features, output_dim = 32) %>% layer_lstm(units = 32) %>% layer_dense(units = 1, activation = "linear")
where features is the number of words in our corpus.
How do we combine the IMDB and Rotten Tomatoes data sets to maximize our predictions?
My first thought is to include the site column as a categorical variable and then let the model learn how to best combine them. Would this cause a skew in the predictions?
We could also try multi-task learning, but most of these methods seem to be designed to predict repeated observations. Such as 5 different people reviewing the same 100 movies.
Some things to consider:
- IMDB and RT don't have all the same movies (i.e. not a repeated observation problem)
- There are words local to each dataset. i.e. the word "rotten" might be a frequent word on RT, but never appears in the IMDB dataset.
- The outcome measures are different. RT uses a 5 star scale, whereas IMDB uses a 10 star one.
How do we deal with this heterogeneity?