What is the best technology to be used to create my custom bag of words with N-grams to apply to. I want to know a functionality that can be achieved over GUI. I cannot use spot fire as it is not available in the organization. Though i can get SAP Hana or R-hadoop. But R-hadoop is bit challenging, any suggessions.
Add a comment
|
4 Answers
$\begingroup$
$\endgroup$
3
create my custom bag of words with N-grams to apply to
My initial recommendation would be to use the NLTK library for Python. NLTK offers methods for easily extracting bigrams from text or ngrams of arbitrary length, as well as methods for analyzing the frequency distribution of those items. However, all of this requires a bit of programming.
a functionality that can be achieved over GUI
That's tricky. Have you looked into GATE? I'm not exactly sure if/how GATE does what you want, but it does offer a GUI.
answered Sep 9, 2014 at 15:05
-
$\begingroup$ I did look at GATE, i was not successful. I am not familiar with python. Any other tool.. I know spotfire but its way too expensive.. $\endgroup$– Anup AshSep 10, 2014 at 16:33
-
$\begingroup$ Sorry, I don't know of any easy-to-use free tools that do exactly what you want to do. $\endgroup$ Sep 11, 2014 at 13:56
-
$\begingroup$ This is as easy as it is going to get, if you want to DIY without spending money. $\endgroup$– EmreOct 10, 2014 at 16:22
$\begingroup$
$\endgroup$
1
You can use SKlearn, It is a python library. It is simplest method which i like with minimal code. You can follow this link http://scikit-learn.org/stable/modules/feature_extraction.html
-
$\begingroup$ Thanks Harjeet, SKlearn is simple but it does not solve my problem. SKlearn is more of analytical algorithm approach which R also provides but I am looking for something like we do in Google, put in n-words and it gives all suitable hits.. $\endgroup$– Anup AshSep 10, 2014 at 16:43
$\begingroup$
$\endgroup$
Check out LightSide for a GUI introduction to text mining in general, and more specifically for generating features quickly. It was developed (and I believe is still being developed) by researchers at CMU and it's how I got hooked on text mining. There is quite a bit of functionality right out of the box, and you can quickly import CSV data into the application, extract features, and start running experiments. It also makes use of suites of algorithms from several other prominent, well-regarded open source toolkits like Weka and LibLinear so you know you're using something credible under the hood. That being said, both of these last mentioned toolkits are definitely worth checking out for added functionality, even if they have a bit of a steeper learning curve. Hope that helps.
$\begingroup$
$\endgroup$
Another library that I use that has not been mentioned yet is gensim. Its Dictionary module allows you to convert a list of words into a list of (id,count) pairs. It also has an allow_update variable that updates the dictionary in the case that new words are encountered at runtime.
It also has built-in support for TF-IDF, LSI, and LDA, among other models.
