Pass 2 different kinds of X training data to ML model simultaneously

Question

I'm trying to classify if a book is fiction/nonfiction based on title and summary.

This is 2 distinct types of information - is there a way to segment title and summary before feeding it to a model, rather than concatenating the information?

For example:

Title: "such a long journey"

Summary: "it is bombay in 1971, the year india went to..."

Label: "fiction" (where fiction =1)

Current procedure:

What I've been doing until now is concatenating the information, so the above becomes,

example = "such a long journey it is bombay in 1971, the year india went to..."
label = 1

Then the usual setup, something like,

X.append(example)
y.append(label)
...
X = lemmatize(X)
...
X_train, X_test, y_train, y_test = split_data(X,y)

vectorizer = TfidfVectorizer(...)
X_train = vectorizer.fit_transform(X_train)
X_test = vectorizer.transform(X_test)

classifier.fit(X_train, y_train)
y_predict = classifier.predict(X_test)

But feeding the data concatenated feels intuitively wrong. Is there a better way to do this?

If for some reason its possible with a library other than sklearn (keras, tensorflow) I'd be also open to hearing about that.

---

UPDATE

Going from,

X = ['two'],['two'],['four'],['two'],['four'],['four']]
y = ['human','human','dog','human','dog','dog']

to,

X = [['two','hello'],['two','hello'],['four','bark'],['two','hi'],['four','bark'],['four','woof']]
y = ['human','human','dog','human','dog','dog']

causes errors to be thrown.

'list' object has no attribute 'lower' is X is a list, and 'numpy.ndarray' object has no attribute 'lower' if X is an array.

The error is thrown when I call,

X_train = vectorizer.fit_transform(X_train)

Is it possible to pass in a vector of features?

Answer 1

You could just apply two independent vectorization steps on your input X (one vectorizer for description and another for summary) and then concatenate the obtained feature matrices into a single feature matrix.

Doing it this way you will have features such as description_"such", description_"long", ..., summary_"bombay", summary_"1971", ..., so any model you apply will be able to:

1. Use all the features from the description and summary altogether
2. Give a different weight to desctiption tokens and summary tokens

Answer 2

In neural network, it is possible to create a model with multiple inputs. Rather than concatenating, you can create separate input for the title and another for the summary where the input size of the title is smaller than that of the summary. In keras, you can do:

MAX_TITLE_LENGTH = 50
MAX_SUMMARY_LENGTH = 500
EMBEDDING_DIM = 50
CLASS_SIZE = 20

input_title = Input(shape=(MAX_TITLE_LENGTH,), name='input_title')
input_summary = Input(shape=(MAX_SUMMARY_LENGTH,), name='input_summary')

emb_title = Embedding(VOCAB_SIZE, EMBEDDING_DIM, input_length=MAX_TITLE_LENGTH)(input_title)
emb_summary = Embedding(VOCAB_SIZE, EMBEDDING_DIM, input_length=MAX_SUMMARY_LENGTH)(input_summary)

a = LSTM(128)(emb_title)
a = Dense(128, activation='relu')(a)

b = LSTM(512)(emb_summary)
b = Dense(128, activation='relu')(b)

z = Concatenate()([a, b])
z = Dense(128, activation='relu')(z)
output = Dense(CLASS_SIZE, activation='softmax')(z)

model = Model(inputs=[input_title , input_summary], outputs=output)

---

Also, regarding your error:

'list' object has no attribute 'lower'

Are you perhaps trying to convert the string in a list to lowercase? If so, check your code. It's trying to convert the list rather than the string, thus, giving this error.

Answer 3

The error you see is because the input you pass to vectorizer is a list of lists (or array of arrays), while, as stated in the TfidfVectorizer syntax you should pass a sequence of strings