Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [text]

The tag has no usage guidance.

0
votes
0answers
13 views

Description matching between two columns

I have a DataFrame with two columns, let's call them column A and column B. Each column is identified with a unique ID and a description. Usually, each idA is in a 1-1 relation with idB. ...
1
vote
1answer
17 views

Can I treat text review analysis as a regression problem?

I am playing with a dataset that contains tripadvisor restaurant reviews and their labels (either 1, 2, 3, 4 or 5 stars). Initially I was thinking of using it as a classification problem, applying ...
1
vote
0answers
12 views

Ordering quotes in a list based on user input and text analysis

A little bit of context: I have a website that has many quotes. These quotes are organized automatically by Solr into lists of quotes, so e.g. there is a list called 'Smart Quotes' that includes ...
0
votes
0answers
32 views

Where and when can I find the faster implementation of Google Wavenet?

https://deepmind.com/blog/wavenet-launches-google-assistant This link talks about how Google researchers have found a way to make the original Wavenet 1000 times faster. Where and when can I expect ...
0
votes
2answers
73 views

Street address clustering?

I have a huge dataset of addresses. I have another data stream that contains addresses that I need to match against those in the original dataset. As all the addresses are user-provided, matching them ...
0
votes
0answers
20 views

Automatic classification of products

I've been playing with some simple algorithms on a couple of projects recently and it got me thinking whether it's possible to automatically categorise products based on an existing set of product/...
2
votes
1answer
36 views

Changing multiple models into 1 model

I am working for a recruitment company on developing machine learning algorithms to automatically classify job applicants as either to be interviewed or not be interveiwed. The data is highly ...
1
vote
2answers
74 views

Accuracy reduces drastically when using TruncatedSVD with hashingvector

I have around 0.8 million product description with categories. There are around 280 categories. I want to train a model with given dataset so that in future I can predict Category for the given ...
1
vote
0answers
38 views

Where can I find a dataset for long sequence text chunking? [closed]

Context: I have documents with reviews of articles that have the following structure: Introduction: a description of the review, dates and metadata that will be discarded. (avg~180 words, std~30 ...
1
vote
2answers
30 views

Grouping company information

I have 3 different datasets with company information, in all of them I have company name, but is not perfect: For example: Dataset A: Company name: Facebook Dataset B: Company name: Facebook, Inc ...
3
votes
1answer
192 views

Is there any clear tutorial for how to use AutoEncoders with text as input

I have a pandas dataframe that describes some fields of the register. I have used one hot encoding to encode the feature vectors that are not numbers. Finally my dataset now has 4000 rows * 4 columns. ...
2
votes
0answers
176 views

What is the minimum number of times a word needs to appear in word2vec training corpus for quality results?

When training a word2vec model with, eg, gensim, you can specify the minimum times a word needs to be seen (with the parameter min_count). The default value for this seems to be 5. Are there any ...
2
votes
1answer
62 views

Why is spam detection a classification problem and not a class modelling problem

Trying to get my feet wet with machine learning on text. The most common dataset I've seen in this space is the sms dataset with classes ham and spam. And the most common and successful approach ...
0
votes
1answer
1k views

One hot encoding at character level with Keras

I am reading Chollet's book on deep learning at the moment and in the NLP chapter he says: ...
0
votes
0answers
30 views

How to view incorrectly recognized text

Need ur help guys. There is a neural network that classifies the sentiment of the reviews. The accuracy is not 100%, hence there are texts that are recognized by the network incorrectly. How can I see ...
2
votes
1answer
776 views

How does ,the Mutlinomial Bayes's alpha parameter, affects the text classification task?

I would like to know how the alpha parameter, in Multinomial Bayes, affects the text classification task. I know that this parameter is correlated to the algorithm'...
0
votes
0answers
21 views

How to extract and visualize data from DBPedia?

I am trying to understand how can I download DBPedia and visualize the hierarchy to analyze the data. Then I want to extract certain sub-trees from it. Any help would be appreciated.
2
votes
1answer
1k views

Which type auto encoder gives best results for text

I did I couple of examples for auto encoders for images and they worked fine. Now I want to do an auto encoder for text that takes as input a sentence and returns the same sentence. But when I try to ...
0
votes
0answers
20 views

Analyzing chat data set

I have a problem analyzing a dataset; I'm not so sure what to look for, But these are the variables: My data set is a collection of chat lines with the corresponding timestamps of when they typed. I ...
0
votes
0answers
15 views

Disambiguating different Dates / section headers within text

In medical records (converted from PDF to Plain Text), what are the approaches to disambiguate between Valid Dates - Date of Service (such as Admission date/Discharge date/Date of consultation) and ...
0
votes
0answers
43 views

Identify if a text is a quote (philosophy)

Any pointers as to how to approach the problem of classifying a text into philosophical quote or not. I can harvest quotes and plain texts for a supervised training set. My problem statement is to ...
0
votes
0answers
111 views

How to create a DTM from a CSV with quanteda?

I am having trouble creating a DTM from a CSV with quanteda. I keep getting the error: Error in tokens.default(corpus, what = "word", remove_punct = TRUE, ...
0
votes
0answers
36 views

How to identify how survey takers answer a double barreled question?

I am analyzing the results of a survey where a survey taker is asked a, "double-barreled question." I am applying text analytics in order to answer the question, "which part of the question are survey ...
1
vote
0answers
43 views

Categorize text as Body, Heading in a loosely formatted document

Given a semi-structured document with only texts and images, and some style properties present on the text, What is the best possible way to classify the text present in the document to any formal ...
4
votes
1answer
2k views

How do you apply SMOTE on text classification?

Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique used in an imbalanced dataset problem. So far I have an idea how to apply it on generic, structured data. But is it ...
0
votes
0answers
16 views

How would you go about segmenting a document based on topic?

Something similar to TextTiling algorithm, but that expects the document to be broken into paragraphs, whereas in my use case the document has natural language and has not been formatted. LDA seems ...
1
vote
0answers
345 views

Multiclass classification with many classes and wide range of sample sizes

I'm working on a free text classification problem with over 100 classes in the training data. There is huge variation in the sample sizes of the classes: ranging from 1 to around 6000. I am using a ...
0
votes
0answers
61 views

How can I store a large feature set for fast prediction?

I'm using sklearn's CountVectorizer to vectorize text for a classification problem. This makes my linear model have a feature for every unigram and bigram in the training data. In addition, my model ...
0
votes
0answers
12 views

how to use text data to extract variables to supplement a dataset using multiple files?

I am trying to do a political science large-N research where the cases are the countries , However , some variables were never coded in structured data sets . So in order to test them and due to the ...
1
vote
1answer
250 views

Unsupervised clustering of unstructured text by document type

I have 100,000+ PDF healthcare documents from which I have extracted text. I would like to cluster these documents by type (e.g. pathology report, doctor visit notes, prescription orders, etc.) The ...
0
votes
2answers
45 views

Classify text labels in to a similar category [closed]

I'm trying to classify same kind of text labels in to one category. For example, if I have labels like qty, quantity, qty_no all of them should direct to Quantity. ...
1
vote
2answers
284 views

Text classification problem using Python or R

I am a novice in machine learning and new to NLP. I am looking for ideas on how to solve the below two problems. I have a dataset with two columns, "Titles" and "Description". Titles column has names ...
0
votes
3answers
1k views

Grouping of similar looking text

I have a data frame which has two columns, "Title" and "Description". The title column has a bunch of titles related to clinical lab tests. Unfortunately, most of the titles are a repeat of the same ...
1
vote
1answer
45 views

How to add incorporate meta data into text classification? [closed]

I have a collection of statements which I need to classify into 5 classes. Each statement have meta data in different columns: Author|Editor| date of release| statement | Class How can one use the ...
0
votes
0answers
218 views

Orange: Text mining - export corpus to table

I've been playing around with Orange Text Mining with the intent to: Extract tags and topics that can be linked to documents for better search Extract Names and Locations I've done nr 1, but need to ...
1
vote
0answers
965 views

Doc2vec to calculate cosine similarity - absolutely inaccurate

I'm trying to modify the Doc2vec tutorial to calculate cosine similarity and take Pandas dataframes instead of .txt documents. I ...
-3
votes
1answer
31 views

Algorithms/services to know an “iPhone case” is not an “iPhone”, in the context of complex item descriptions? [closed]

We are trying to implement a highly accurate search, based on user-entered search terms, into a large product database. For example, if the user searches for "iPhone", then one of these is ...
3
votes
4answers
14k views

Sentence similarity prediction

I'm looking to solve the following problem: I have a set of sentences as my dataset, and I want to be able to type a new sentence, and find the sentence that the new one is the most similar to in the ...
1
vote
2answers
121 views

Discover string “motifs” in python

If I have many strings from different sources that tend to exhibit some common patterns. Is there a way to extract these common motifs? For example, in a list (of millions) that includes strings <...
0
votes
1answer
65 views

Where can I find datasets with labeled duplicate text documents?

I'm working on detecting duplicate text documents using a classifier. I am looking for training data - a corpus of text documents and corresponding metadata which lists out pairs of duplicate ...
-1
votes
1answer
60 views

Text standardisation for manually entered data

I am working on a project that involves dealing with manually entered text data. I have a dataset of customs records where the customs officers manually enter the name and address of companies ...
0
votes
0answers
144 views

Binary Text Classifier

I'm looking for binary classification methods for text. I've found SVM, Naive Bayes, word2vec, doc2vec, and GloVe, but before I try implementing something I want to know which may be a better approach ...
0
votes
1answer
73 views

What methods to create singular content classification from inconsistent inbound info?

I am attempting to aggregate professional profile info from multiple sources, imposing a consistent taxonomy. Specifically, the current problem is how to impose a preferred taxonomy on profiles with ...
-1
votes
1answer
230 views

ML project ideas for dataset [closed]

Not sure if this is the right forum, but currently i have a dataset which contains a list of TV shows. Each record contains pricing between competitors (price in provider 1. Example: Itunes) TV show ...
1
vote
0answers
334 views

Methods for string classifications

I have a list of some 100 millions of strings, each of different length. Examples: nsdgnlnesef ngmrlxkvgrmksefsfnlj <...
0
votes
3answers
255 views

Using training data generated with pure regular expressions - Can machine learning surpass the accuracy of your regular expression?

For text classification with machine learning - If your training data was generated purely with regular expressions, is it possible to train a machine learning model with this training data which will ...
1
vote
1answer
1k views

What are some function/package in R to find similarity of individual words not in the context of sentences?

what are some function/package in R/python to find similarity of individual words not in the context of sentences? As a novice, I searched and found methods like adist, cosine similarity, word2vec, ...
5
votes
1answer
2k views

How to use TFIDF vectors with multinomial naive bayes?

Say we have used the TFIDF transform to encode documents into continuous-valued features. How would we now use this as input to a Naive Bayes classifier? Bernoulli naive-bayes is out, because our ...
0
votes
3answers
1k views

How to use binary text classifier(built using SVM with TF-IDF) to classify new text document?

I have built binary text classifier using SVM on TF-IDF for news articles(Sports: Non-Sports). But I not sure how to classify new document using this model. Since TF-IDF is calculated based on the ...
2
votes
0answers
173 views

Section/Topic segmentation in HTML and plaintext documents

GROBID (https://github.com/kermitt2/grobid) is terrific for fine-grained (section, chapter) segmentation of PDF files, but I need to do the equivalent for HTML and plaintext files. I've tried ...