3
$\begingroup$

I want to classify German police news articles and do an automated classification/clustering with regards to the kind of crime committed. Thus far I am not getting great results. Often times the headline is pretty telling but sometimes it is not that revealing, so I need to include the main article to do a proper clustering on all articles.

However the headline is a very good indicator and I don't want to throw it away. I could simply combine the words from headline and main article but I think there should be more weight given to the headline. Has somebody an idea how to do this?

$\endgroup$
1
$\begingroup$

Put more weight on the title.

For example if you are using tf-idf, you first compute the tf-idf vectors of headline and body separately. Then you combine them into a single vector by simply adding both vectors and rescaling to unit length. That would put 50% of the weight onto the headline even though it is much shorter.

$\endgroup$
  • $\begingroup$ that sounds like a pretty smart idea, thank you very much $\endgroup$ – Hans Geber Apr 26 '19 at 19:51
  • $\begingroup$ so I found out that gensim has the option for the LSI-Model to do an update with a decay parameter. Of course the coherence score goes down a little bit, after I update with the corpus from the news, but thus far it looks pretty promising $\endgroup$ – Hans Geber Apr 29 '19 at 12:55
1
$\begingroup$

One idea here is to train two distinct recurrent networks and then merge their outputs, and then have some dense layers after. This way the model can learn different information for the headline and story, something like this:

from keras.layers.core import Dense, Activation
from keras.layers import concatenate, LSTM

modela = Sequential()
modela.add(LSTM(100,input_shape=(headline_len,1)))
modela.add(Activation('relu'))
modela.add(Dense(50))

modelb = Sequential()
modelb.add(LSTM(100,input_shape=(story_len,1)))
modelb.add(Activation('relu'))
modelb.add(Dense(50))

merged_output = concatenate([modela.output, modelb.output])   

model_combined = Sequential()
model_combined.add(Activation('relu'))
model_combined.add(Dense(50))
model_combined.add(Activation('relu'))
model_combined.add(Dense(1))
model_combined.add(Activation('linear'))

final_model = Model([modela.input, modelb.input], model_combined(merged_output))

final_model.compile(loss='mean_squared_error', optimizer='rmsprop', metrics=['mae'])

Something like this, but your architecture may vary.

$\endgroup$
  • $\begingroup$ thank you for your help. I have to admit, I am not too familiar with this technique. Do you have a link, where they describe this a little bit more? $\endgroup$ – Hans Geber Apr 26 '19 at 19:53
  • $\begingroup$ There are many resources for this sort of thing, so your milage may vary, but here is a tutorial on Recurrent Neural Nets (an LSTM is a type of RNN): wildml.com/2015/09/… $\endgroup$ – Andy M Apr 26 '19 at 20:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.