# Pass 2 different kinds of X training data to ML model simultaneously

I'm trying to classify if a book is fiction/nonfiction based on title and summary.

This is 2 distinct types of information - is there a way to segment title and summary before feeding it to a model, rather than concatenating the information?

For example:

Title: "such a long journey"

Summary: "it is bombay in 1971, the year india went to..."

Label: "fiction" (where fiction =1)

Current procedure:

What I've been doing until now is concatenating the information, so the above becomes,

example = "such a long journey it is bombay in 1971, the year india went to..."
label = 1


Then the usual setup, something like,

X.append(example)
y.append(label)
...
X = lemmatize(X)
...
X_train, X_test, y_train, y_test = split_data(X,y)

vectorizer = TfidfVectorizer(...)
X_train = vectorizer.fit_transform(X_train)
X_test = vectorizer.transform(X_test)

classifier.fit(X_train, y_train)
y_predict = classifier.predict(X_test)


But feeding the data concatenated feels intuitively wrong. Is there a better way to do this?

If for some reason its possible with a library other than sklearn (keras, tensorflow) I'd be also open to hearing about that.

## UPDATE

Going from,

X = ['two'],['two'],['four'],['two'],['four'],['four']]
y = ['human','human','dog','human','dog','dog']


to,

X = [['two','hello'],['two','hello'],['four','bark'],['two','hi'],['four','bark'],['four','woof']]
y = ['human','human','dog','human','dog','dog']


causes errors to be thrown.

'list' object has no attribute 'lower' is X is a list, and 'numpy.ndarray' object has no attribute 'lower' if X is an array.

The error is thrown when I call,

X_train = vectorizer.fit_transform(X_train)


Is it possible to pass in a vector of features?

• There's no reason to concatenate; pretty much any scikit-learn algorithm handles vector features. – Adrian Keister Sep 13 '18 at 16:05
• @AdrianKeister I've updated my question. – tim_xyz Sep 21 '18 at 22:34
• Try creating a model.with multiple inputs and then merge them after few layers – Aditya Sep 22 '18 at 12:34

You could just apply two independent vectorization steps on your input X (one vectorizer for description and another for summary) and then concatenate the obtained feature matrices into a single feature matrix.

Doing it this way you will have features such as description_"such", description_"long", ..., summary_"bombay", summary_"1971", ..., so any model you apply will be able to:

1. Use all the features from the description and summary altogether
2. Give a different weight to desctiption tokens and summary tokens

In neural network, it is possible to create a model with multiple inputs. Rather than concatenating, you can create separate input for the title and another for the summary where the input size of the title is smaller than that of the summary. In keras, you can do:

MAX_TITLE_LENGTH = 50
MAX_SUMMARY_LENGTH = 500
EMBEDDING_DIM = 50
CLASS_SIZE = 20

input_title = Input(shape=(MAX_TITLE_LENGTH,), name='input_title')
input_summary = Input(shape=(MAX_SUMMARY_LENGTH,), name='input_summary')

emb_title = Embedding(VOCAB_SIZE, EMBEDDING_DIM, input_length=MAX_TITLE_LENGTH)(input_title)
emb_summary = Embedding(VOCAB_SIZE, EMBEDDING_DIM, input_length=MAX_SUMMARY_LENGTH)(input_summary)

a = LSTM(128)(emb_title)
a = Dense(128, activation='relu')(a)

b = LSTM(512)(emb_summary)
b = Dense(128, activation='relu')(b)

z = Concatenate()([a, b])
z = Dense(128, activation='relu')(z)
output = Dense(CLASS_SIZE, activation='softmax')(z)

model = Model(inputs=[input_title , input_summary], outputs=output)