2
$\begingroup$

Matlab is a great tool for some mathematical experiments, Neural Networks, Image Processing ...

I would like to know if there is such a comprehensive and strong tool for data manipulation and NLP tasks? such as tokenization, POS tagging, parsing, training, testing ....

However I am new to NLP and I need a tool which let me experiment, get familiar and progress

$\endgroup$
4
  • $\begingroup$ Matlab is not the only good tool for maths, there are also Octave, R, Python/NumPy, SciLab, etc. Same applies to NLP - there are NLTK, Standford NLP, GATE and numerous others. Often it's a question of personal preference, so no single good answer can be given to your question. In general, try to avoid questions starting with "what is the best ..." - they are primary opinion-based and are forbidden on most SE sites. $\endgroup$
    – ffriend
    Sep 26 '14 at 7:34
  • 2
    $\begingroup$ Then I would suggest trying out NLTK. It's pretty simple, and Python's REPL allows to do quick experiments and immediately see the result. $\endgroup$
    – ffriend
    Sep 26 '14 at 8:02
  • $\begingroup$ @ffriend thank you very much, I am going to try it, just a question are these toolkit general purpose? In fact my target language is Persian $\endgroup$
    – Ahmad
    Sep 26 '14 at 19:30
  • $\begingroup$ Short answer: no, but it is solvable. Natural languages differ in both - words and rules. You can't expect English lemmatizer to correctly find root of a French word. You also can't expect standard whitespace-based tokenizer to split German adjetcive/noun pairs that are written without whitespaces. But most NLP libraries are flexible enough to allow you extend them for your specific purposes. Sometimes it is done via custom modules, sometimes - via training data, and sometimes there's already ready-to-use library for you (e.g. see Hazm for Persian support). $\endgroup$
    – ffriend
    Sep 26 '14 at 20:27
2
$\begingroup$

I have just finished my Ph.D. and have used some NLP in it. My university didn't offer any NLP courses. So I ended up teaching myself NLP. I used this book.

Which serves as a great introduction to NLP using NLTK (Natural Language Tool Kit). It gives a good introduction into programming with Python. So handy if you've never programmed in Python before too.

I would highly suggest using nltk from nltk.org (sorry can't post more than two links)

The book I used is now out of date as NLTK is now on version 3.0, the book mentioned previously is for NLTK 2.x. But the Authors are working on a new version of the book for NLTK 3.X, you can view the unfinished book here.

I would highly suggest using NLTK and if you're new to natural language processing. I would highly suggest you try and get yourself a copy of the following book:

Foundations of Statistical Natural Language Processing by Manning and Shutze

Even though it doesn't contain any code, it servers a great introduction to natural language processing.

$\endgroup$
1
  • $\begingroup$ Thank you very much for your full response. I check them. $\endgroup$
    – Ahmad
    Nov 26 '14 at 6:49
0
$\begingroup$

Please take a look at this paper:

Survey on Various Natural Language Processing Toolkits

In this work several NLP tools are introduced. Examples are:

  • Carabao Language Toolkit
  • GATE
  • Stanford NLP toolkit
  • NLTK

Just google the title of the paper and you will find the .pdf file.

$\endgroup$
1
  • $\begingroup$ Thank you, I think I will use NLTK. The paper was good but not an excellent comparison $\endgroup$
    – Ahmad
    Sep 26 '14 at 19:43
0
$\begingroup$

There's a NLP toolbox for MATLAB called MatlabNLP. It includes modules for tokenization, preprocessing (stop word removal, text cleaning, stemming), and learning algorithms (linear regression, decision trees, support vector machines and a Naïve Bayes). A module for POS tagging is coming soon.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.