Clustering of news combining headline and main article

Question

I want to classify German police news articles and do an automated classification/clustering with regards to the kind of crime committed. Thus far I am not getting great results. Often times the headline is pretty telling but sometimes it is not that revealing, so I need to include the main article to do a proper clustering on all articles.

However the headline is a very good indicator and I don't want to throw it away. I could simply combine the words from headline and main article but I think there should be more weight given to the headline. Has somebody an idea how to do this?

Answer 1

Put more weight on the title.

For example if you are using tf-idf, you first compute the tf-idf vectors of headline and body separately. Then you combine them into a single vector by simply adding both vectors and rescaling to unit length. That would put 50% of the weight onto the headline even though it is much shorter.

Answer 2

One idea here is to train two distinct recurrent networks and then merge their outputs, and then have some dense layers after. This way the model can learn different information for the headline and story, something like this:

from keras.layers.core import Dense, Activation
from keras.layers import concatenate, LSTM

modela = Sequential()
modela.add(LSTM(100,input_shape=(headline_len,1)))
modela.add(Activation('relu'))
modela.add(Dense(50))

modelb = Sequential()
modelb.add(LSTM(100,input_shape=(story_len,1)))
modelb.add(Activation('relu'))
modelb.add(Dense(50))

merged_output = concatenate([modela.output, modelb.output])   

model_combined = Sequential()
model_combined.add(Activation('relu'))
model_combined.add(Dense(50))
model_combined.add(Activation('relu'))
model_combined.add(Dense(1))
model_combined.add(Activation('linear'))

final_model = Model([modela.input, modelb.input], model_combined(merged_output))

final_model.compile(loss='mean_squared_error', optimizer='rmsprop', metrics=['mae'])

Something like this, but your architecture may vary.