What is the best technology to be used to create my custom bag of words with N-grams to apply to. I want to know a functionality that can be achieved over GUI. I cannot use spot fire as it is not available in the organization. Though i can get SAP Hana or R-hadoop. But R-hadoop is bit challenging, any suggessions.
create my custom bag of words with N-grams to apply to
My initial recommendation would be to use the NLTK library for Python. NLTK offers methods for easily extracting bigrams from text or ngrams of arbitrary length, as well as methods for analyzing the frequency distribution of those items. However, all of this requires a bit of programming.
a functionality that can be achieved over GUI
That's tricky. Have you looked into GATE? I'm not exactly sure if/how GATE does what you want, but it does offer a GUI.
You can use SKlearn, It is a python library. It is simplest method which i like with minimal code. You can follow this link http://scikit-learn.org/stable/modules/feature_extraction.html
Check out LightSide for a GUI introduction to text mining in general, and more specifically for generating features quickly. It was developed (and I believe is still being developed) by researchers at CMU and it's how I got hooked on text mining. There is quite a bit of functionality right out of the box, and you can quickly import CSV data into the application, extract features, and start running experiments. It also makes use of suites of algorithms from several other prominent, well-regarded open source toolkits like Weka and LibLinear so you know you're using something credible under the hood. That being said, both of these last mentioned toolkits are definitely worth checking out for added functionality, even if they have a bit of a steeper learning curve. Hope that helps.
Another library that I use that has not been mentioned yet is gensim. Its Dictionary module allows you to convert a list of words into a list of (id,count) pairs. It also has an
allow_update variable that updates the dictionary in the case that new words are encountered at runtime.
It also has built-in support for TF-IDF, LSI, and LDA, among other models.