9
$\begingroup$

Which freely available datasets can I use to train a text classifier?

We are trying to enhance our users engagement by recommending the most related content for him, so we thought If we classified our content based on a predefined bag of words we can recommend to him engaging content by getting his feedback on random number of posts already classified before.

We can use this info to recommend for him pulses labeled with those classes. But we found If we used a predefined bag of words not related to our content the feature vector will be full of zeros, also categories may be not relevant to our content. so for those reasons we tried another solution that will be clustering our content not classifying it.

Thanks :)

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ I think more details about your problem are necessary before anyone could recommend a dataset. $\endgroup$
    – Neil Slater
    Commented Jun 18, 2014 at 16:40
  • 3
    $\begingroup$ For what purpose? Spam filtering? Sentiment analysis? Without a clear purpose it is very difficult to suggest a dataset. $\endgroup$
    – lsdr
    Commented Jun 18, 2014 at 18:37
  • $\begingroup$ @lsdr Looking at the answers, it seems that the question does not necessarily need more details. $\endgroup$
    – Amir Ali Akbari
    Commented Jun 19, 2014 at 17:25
  • $\begingroup$ @AmirAliAkbari I think they came after an edit. I retracted my close-vote, anyway. $\endgroup$
    – Rubens
    Commented Jun 19, 2014 at 17:30
  • $\begingroup$ A more appropriate place for this question is opendata.stackexchange.com $\endgroup$
    – sheldonkreger
    Commented May 29, 2015 at 17:38

4 Answers 4

Reset to default
14
$\begingroup$

Some standard datasets for text classification are the 20-News group, Reuters (with 8 and 52 classes) and WebKb. You can find all of them here.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks :), I already visited it before but I found it's classifications are weak not abstract enough or it may be not related to my content $\endgroup$
    – Abdelmawla
    Commented Jun 21, 2014 at 10:25
7
$\begingroup$

One of the most widely used test collection for text categorization research (link below). I've used many times. Enjoy your exploration :)

http://www.daviddlewis.com/resources/testcollections/reuters21578/ or http://archive.ics.uci.edu/ml/datasets/Reuters-21578+Text+Categorization+Collection

Improve this answer
$\endgroup$
5
$\begingroup$

There is a bunch of datasets made free by UC Irvine to play with here. Among those datasets, there are a few dozen textual datasets that might help you guys with your task.

Those are kind of generic datasets, so depending on your purpose they should not be used as the only data to train your models, or else your model -- while it might work -- will not produce quality results.

Improve this answer
$\endgroup$
1
$\begingroup$

Apart from the suggestions above, there is an extremely useful pdf - Benchmarking Text Collections for Classification and Clustering Tasks which contains various datasets along with the benchmarks in order to test our models. This includes 20ng Collection, Reuters and many of the above suggested datasets. I hope it helps!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.