0
$\begingroup$

I am new to here and new to Deep Learning too, so apologies in advance for any ill formatted code or wordings. I have a data set where I track 4 variables with 2 categorical and 3 numerical fields, over 4 time steps.

I was able build a dataframe like this:

enter image description here

  1. cat1 - SckitLearn Label Encoded variable names
  2. cat2 - SckitLearn Label Encoded time steps
  3. num1 : num3 - SckitLearn Normalized variables

I got it turn into a list of numpy arrays like this:

enter image description here

I want to embed and concatenate all these fields before feeding it into a LSTM in Keras, using this function:

def build_concat(df):
  global inputs
  inputs = []
  global embeddings
  embeddings = []
  cat_cols = df.filter(like='cat')
  num_cols = df.filter(like='num')
  for cat_col in cat_cols:
    cat_input = Input(shape=(4,4), name=cat_col)
    unique_cat = cat_cols[cat_col].nunique()
    embedding_size = min(np.ceil((unique_cat)/2), 20)
    embedding_size = int(embedding_size)
    cat_dim = unique_cat + 1
    inputs.append(cat_input)
    embeddings.append(Embedding(cat_dim, embedding_size, input_length = (4,4),
        name=str(cat_col) + "_emb")(cat_input))
  for num_col in num_cols:
    num_input = Input(shape=(4,4), name=num_col)
    inputs.append(num_input)
    embeddings.append(num_input)

But I get this error :

ValueError: Shape must be rank 3 but is rank 2 for '{{node so2_model/concat/concat}} = ConcatV2[N=5, T=DT_FLOAT, Tidx=DT_INT32](so2_model/cat1_emb/embedding_lookup/Identity_1, so2_model/cat2_emb/embedding_lookup/Identity_1, IteratorGetNext:2, IteratorGetNext:3, IteratorGetNext:4, so2_model/concat/concat/axis)' with input shapes: [2,4,2], [2,4,2], [2,4], [2,4], [2,4], [].

This is my embedding list :

[<KerasTensor: shape=(None, 4, 4, 2) dtype=float32 (created by layer 'cat1_emb')>, <KerasTensor: shape=(None, 4, 4, 2) dtype=float32 (created by layer 'cat2_emb')>, <KerasTensor: shape=(None, 4, 4) dtype=float32 (created by layer 'num1')>, <KerasTensor: shape=(None, 4, 4) dtype=float32 (created by layer 'num2')>, <KerasTensor: shape=(None, 4, 4) dtype=float32 (created by layer 'num3')>]

This is my input list, before embedding :

[<KerasTensor: shape=(None, 4, 4) dtype=float32 (created by layer 'cat1')>, <KerasTensor: shape=(None, 4, 4) dtype=float32 (created by layer 'cat2')>, <KerasTensor: shape=(None, 4, 4) dtype=float32 (created by layer 'num1')>, <KerasTensor: shape=(None, 4, 4) dtype=float32 (created by layer 'num2')>, <KerasTensor: shape=(None, 4, 4) dtype=float32 (created by layer 'num3')>]

Thank you for your help!

$\endgroup$
0
$\begingroup$

You should reshape your numerical layers so that they have a shape (None,4,4,1). To concatenate, you need the all but one axis to be equal.

If you use:

tf.keras.layers.Reshape((4,4,1))(num_input)

this should work.

$\endgroup$
2
  • $\begingroup$ Thanks! :) However I found that I have a different issue now. I made a smaller data set to experiment on Embeddings. I only took 1 cat variable with 4 levels and 1 numerical variable. I embedded the cat layer with cat dimension = 5 (I have 4 levels) and embedding size = 2. I found that the embedded output = (None,4,1,2) and the numerical variable output = (None,4,1). I am trying to concatenate these two, but I don't know how to. I need to input all these into an LSTM, so I need to preserve the input format in 3D. $\endgroup$
    – LHS
    Apr 6 at 20:13
  • $\begingroup$ add an extra 1: python tf.keras.layers.Reshape((4,4,1,1))(num_input) $\endgroup$ Apr 7 at 8:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.