By using the functional API you can easily share weights between different parts of your network. In your case we have an Input $x$ which is our input, then we will have a Dense layer called shared. Then we will have three different Dense layers called sub1, sub2 and sub3 and then three output layers called out1, out2 and out3.
x = Input(shape=(n, ))
Since we are talking about multiple different types of targets (classes versus numerical for example) we already need a composite loss function. I will consider how to balance the different composite parts of the loss function outside of the scope of this answer but if you look into multi-task learning there are solutions to this. What you could do (both in ...
You have one classification task and one regression task, but sklearn's multioutput meta-estimators only support two tasks of the same type.
The best solution here is to train two models:
A binary classifier to predict $target1$
A regressor to predict $target2$
from sklearn.cross_validation import train_test_split
from sklearn.ensemble ...
You should design a multi-task model (MTM).
MTM has the ability to share learned representations from input between several tasks. More precisely, we try to simultaneously optimize a model with m types of loss function, one for each task. Consequently, MTM will learn more generic features, which should be used for several tasks, at its earlier layers. Then, ...
You should break this down into two models. I would solve this in the following manner:
The first model would predict if its either Target 1 or Target 2
by looking at 100 columns
The second model then would look at the 100 columns and additionally the output of model 1 and then predict 0 or 1 in case of target 1 or 0-100 in case of target 2.
I do not ...
After playing around with tf.data.map operations I found the answer was easier than expected, I simply had to preprocess the data and put all the labels for each output of the model as a different key of a dictionary.
First I create a dataset from the tfrecords file
dataset = tf.data.TFRecordDataset(tfrecords_file)
Next, I parse data from the file
Following your considerations:
IMDB and RT don't have all the same movies
I would build a larger dataset of movies, aggregate all observations available, and run a model on that.
There are words local to each dataset. i.e. the word "rotten" might be
a frequent word on RT, but never appears in the IMDB dataset.
I think this doesn't constitute a ...
You need to pass an array object. You don't need to map the label array with the output node name ( like TensorFlow low-level APIs ). Pass the list containing the label arrays directly.
model.fit(X_train, [ label1, label2 ] ,batch_size=10,verbose=1 ,callbacks=[checkpoint],validation_data=(X_test,y_test),epochs=40)
Adjust the optimizer hyper-parameters
1) use amsgrad = True
2) try lower beta_1
3) adjust lr from 1e-6 to .1 or use a learning rate finder
4) use either the weight_decay(learning schedule) in the optimizer or add a Callback that will adjust the learning rate while training
adjust the parameters here.....
optimizer=Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, ...
So, it turns out that my problem had a lot more to do with network structure than anything else. I was trying to share too many lower levels, and then still not leaving enough room for the later branches to learn individual features/locations. In the end, I re-structured my network so that there was only 1-2 shared convolutional layers, then each primary ...
MTL is supposed to enable your shared layer to generalise better.For eg. in a text classification problem your encoder would be trained on separate tasks:
one could be multi class classification(original task) itself and let's take identifying the sentiment of the sentence(again a classification task; but different than the original task) as the other task. ...
You have already found the answer by yourself: if there are missing values, you should not backpropagate on them. I do not know of any particular name for this action.
I do not think that there are already implemented functions to do so, but it seems quite straightforward to embed your loss function (e.g. mean squared error) into a personalized one.
Every Blob in caffe can be assigned a nonzero loss weight.
And you can have an arbitrary number of outputs.
This means you can just learn n different networks on the same data with different targets and assign every loss function it's own weight. Caffe takes care of adding up all the loss.