1
$\begingroup$

I'm relatively new to the world of recurrent neural nets and I'm trying to build a classifier using an LSTM model to predict HIV activity from a given molecule (the original dataset can be found here ).

I have sequences of different lengths (from few dozen to almost 400 characters) but I'm not sure how to proceed. Let's say that I have a dataset structured like so:

import pandas as pd
import random
import string

random.seed(42)
seqs = [[random.choice(string.ascii_letters) for i in range(random.randint(1,10))] for i in range(5)]
classes = [random.randint(0,1) for i in range(5)]
df = pd.DataFrame({
    "seq": seqs,
    "class": classes
})
print(df)

    seq                             class
0   [b, V]                          0
1   [p, o, i, V, g]                 0
2   [f, L, B, c, b, f, n, o, G]     1
3   [b, J, m, T, P, S, I, A, o, C]  0
4   [r, Z, a, W, Z, k, S, B, v, r]  0

I know I should:

  • one hot encode the elements,
  • perform masking and padding

But I don't know how to perform it in Keras/TF2 and I can't find any resources online that explain how to code something similar.

$\endgroup$
0
$\begingroup$

masking and padding are indeed popular preprocessing choices when working with LSTM. The easiest way is to pad all the short sequences to the same length as the longest sequence.

Here is an example of masking and padding in Keras

https://www.tensorflow.org/guide/keras/masking_and_padding

Is that what you're looking for?

raw_inputs = [
    [711, 632, 71],
    [73, 8, 3215, 55, 927],
    [83, 91, 1, 645, 1253, 927],
]

# By default, this will pad using 0s; it is configurable via the
# "value" parameter.
# Note that you could "pre" padding (at the beginning) or
# "post" padding (at the end).
# We recommend using "post" padding when working with RNN layers
# (in order to be able to use the
# CuDNN implementation of the layers).
padded_inputs = tf.keras.preprocessing.sequence.pad_sequences(
    raw_inputs, padding="post"
)
print(padded_inputs)

Output:

[[ 711  632   71    0    0    0]
 [  73    8 3215   55  927    0]
 [  83   91    1  645 1253  927]]

Besides, you can also add start and end tokens to your sequences, like this example (https://www.tensorflow.org/tutorials/text/nmt_with_attention)

def preprocess_sentence(w):
  w = unicode_to_ascii(w.lower().strip())

  # creating a space between a word and the punctuation following it
  # eg: "he is a boy." => "he is a boy ."
  # Reference:- https://stackoverflow.com/questions/3645931/python-padding-punctuation-with-white-spaces-keeping-punctuation
  w = re.sub(r"([?.!,¿])", r" \1 ", w)
  w = re.sub(r'[" "]+', " ", w)

  # replacing everything with space except (a-z, A-Z, ".", "?", "!", ",")
  w = re.sub(r"[^a-zA-Z?.!,¿]+", " ", w)

  w = w.strip()

  # adding a start and an end token to the sentence
  # so that the model know when to start and stop predicting.
  w = '<start> ' + w + ' <end>'
  return w

```
$\endgroup$
2
  • $\begingroup$ I'm there, but I don't know how to one-hot encode my data and how to use masking with my data one hot encoded $\endgroup$ May 18 at 20:16
  • $\begingroup$ I don't think you need to one-hot encode your data. A vocabulary lookup to map a character to an integer will be enough, like in the part "Padding sequence data" in the first link. $\endgroup$
    – TQA
    May 19 at 12:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.