1
$\begingroup$

I did a lot of Googling but could not find a paper that presents an algorithm which will produce dense feature vectors for short text input. I would be happy to find feature extraction algorithm which would perform at least as good as sparse word unigram and bigram feature vectors.

Currently I am exploring the idea of using LDA (Latent Dirichlet Allocation) but there are problems with processing short text (2-7 words per document).

The task at hand is short text classification. The number of classes for my data ranges from 10 to 20 classes. The classes are fairly well represented and the word unigram and bigram features work well. I would like to compute dense feature vectors for other experiments.

Any pointers to papers, preferrably simple to implement, would be appreciated.

| improve this question | |
$\endgroup$
  • $\begingroup$ So the inputs are queries? Over what are you trying to match them? $\endgroup$ – Adam Bittlingmayer Dec 21 '15 at 12:14
  • $\begingroup$ @A.M.Bittlingmayer The input is a dataset of very sparse vectors. The dataset consists of examples that can be split into 10-20 groups depending on the annotation (which is available). The task is to transform this dataset into much lower dimensionality while keeping the information. $\endgroup$ – Vladislavs Dovgalecs Dec 21 '15 at 16:45
1
$\begingroup$

At my work we've had some success just using averages of pre-trained embeddings (e.g. the GloVe vectors) for classifying short texts. Have you tried that?

| improve this answer | |
$\endgroup$
  • $\begingroup$ Yes, I did. Unless I did something very wrong when making averages (in Python numpy), but the classification results were just horrible. Perhaps the GloVe vectors were trained on insufficiently large & clean collection of short texts (I have +100K) $\endgroup$ – Vladislavs Dovgalecs Feb 14 '17 at 17:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.