1
$\begingroup$

I am using Keras to build my architecture.

The regression problem I am trying to solve has outputs different for different training samples.

Suppose that I have very first two rows of y_train = [[16, 3], [6], ... ] corresponding to my very first two rows of X_train (input data). I would like to assign the output of the dense layer's unit to length of these y_train rows.

For example, for 1st training sample, I would like model.add(dense(2)) as y_train[0] has length 2, for 2nd training sample, I would like model.add(dense(1)) as y_train[0] has length 1, and so on.

I also think that I can call Lambda Layer instead of output dense layer and in that Lambda Layer I can wrap length of every row of y_train and assign it as output units by using output_shape argument of Lambda Layer. But, I don't know how can I access my y_train at training time of my model so that I can use y_train inside lambda layer to make this scenario happen either, nonetheless.

Can anybody help me on this variable length output problem?

$\endgroup$
1
  • $\begingroup$ It is not clear what you want your final input to look like. Please post what your dataset should look like if you could make it happen. $\endgroup$ – grldsndrs Jul 31 '19 at 17:36
1
$\begingroup$

If you want to use arbitrary length input and outputs, there are two common options: train a single model for each length or use a recurrent neural network (RNN) architecture. A non-RNN neural network cannot handle arbitrary length because the shape of the weights depend on the length of the input. From your example, it seems like you want to keep different copies of weights based on the length. You could do that, but in theory, it is the same as training separate models. The weights from one length will only be updated by inputs of the same length, so why not save yourself the hassle and have separate models?

Although RNNs are typically used for natural language processing and understanding, it does not mean they cannot be used to solve other deep learning problems. Specifically for your problem, you should use a many-to-many architecture.

Source: http://karpathy.github.io/2015/05/21/rnn-effectiveness/

Source: http://karpathy.github.io/2015/05/21/rnn-effectiveness/

In Keras, you can use the LSTM layer and set the parameter return_sequences to True, which will return an output the same length as your input sequence. Documentation here: https://keras.io/layers/recurrent/#lstm.

$\endgroup$
4
  • $\begingroup$ I don't want output having the same length as my input sequence. Instead I want to assign the length of row of y_train as output unit to my output dense layer, i.e. I want output layer to have different number of neurons for every other training sample in my X_train. Can you please help me on getting a solve for this? $\endgroup$ – ANIKET SAXENA Aug 1 '19 at 16:45
  • $\begingroup$ I don't quite understand. Can you provide some more examples of input X_train and the corresponding output y_train shape you want? $\endgroup$ – Richard Aug 1 '19 at 17:42
  • $\begingroup$ Can you please check out this question: datascience.stackexchange.com/questions/56770/… where I have explained my problem in detail along with ten samples of my y_train? Kindly check this question and provide some valuable response for this. $\endgroup$ – ANIKET SAXENA Aug 2 '19 at 13:05
  • $\begingroup$ I am not able to comment on your other post since I do not have enough reputation. It is not clear to me how you are deciding the number of output neurons based on the input. "As you can see in the attached image, first row has value [1,0]. So in this case I want output dense layer to have only 1 unit, i.e. model.add(Dense(1)). For third row value is [1,7], so here I want output dense layer to have 2 units, i.e. model.add(Dense(2))." Why does [1,0] get one output unit and why does [1,7] get 2? Also, you do know that the number of units in your output layer means the length of the output right? $\endgroup$ – Richard Aug 3 '19 at 0:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.