Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [text]

The tag has no usage guidance.

0
votes
0answers
7 views

Text segmentation based on most probable classes

I am working on a text classification problem. As training data, I have human annotated text, which was manually segmented into sections and then these sections are labeled with some class. Training ...
0
votes
1answer
9 views

Size of Output vector from AvgW2V Vectorizer is less than Size of Input data

Hi, I have been seeing this problem for quite some time. Whenever I tried vectorizing input text data though avgw2v vectorization technique. The size of vectorized data is less than the size of the ...
0
votes
1answer
16 views

How to use correlation matrix when the dataset contains multiple columns with text data? [closed]

How to use it with Amazon fine food reviews dataset?
2
votes
1answer
49 views

CNN to many outputs

I have a dataset with 100 columns (categorial one-hot encoded) and 1 column with text data (simple sentences) and i want to build a neural network to arround 380.000 outputs labels. I have no idea ...
0
votes
1answer
31 views

Multi-Class Text Classification: Doc2Vec performing very bad compared to Hashing Vector

I have a multi-class text classification problem in hand this is similar to product category mapping where we map products to its correct Category based on the text content provided. I first created ...
0
votes
2answers
45 views

Use text similarity (cosine) instead of machine learning to classify companies into industries

I am building an industry classifier. I.e. classifying companies into industries based on a company's description. Each company can only have one industry. I took 2000 companies and assigned them ...
0
votes
0answers
16 views

Description matching between two columns

I have a DataFrame with two columns, let's call them column A and column B. Each column is identified with a unique ID and a description. Usually, each idA is in a 1-1 relation with idB. ...
1
vote
1answer
18 views

Can I treat text review analysis as a regression problem?

I am playing with a dataset that contains tripadvisor restaurant reviews and their labels (either 1, 2, 3, 4 or 5 stars). Initially I was thinking of using it as a classification problem, applying ...
1
vote
0answers
12 views

Ordering quotes in a list based on user input and text analysis

A little bit of context: I have a website that has many quotes. These quotes are organized automatically by Solr into lists of quotes, so e.g. there is a list called 'Smart Quotes' that includes ...
0
votes
0answers
38 views

Where and when can I find the faster implementation of Google Wavenet?

https://deepmind.com/blog/wavenet-launches-google-assistant This link talks about how Google researchers have found a way to make the original Wavenet 1000 times faster. Where and when can I expect ...
0
votes
2answers
109 views

Street address clustering?

I have a huge dataset of addresses. I have another data stream that contains addresses that I need to match against those in the original dataset. As all the addresses are user-provided, matching them ...
0
votes
0answers
23 views

Automatic classification of products

I've been playing with some simple algorithms on a couple of projects recently and it got me thinking whether it's possible to automatically categorise products based on an existing set of product/...
2
votes
1answer
37 views

Changing multiple models into 1 model

I am working for a recruitment company on developing machine learning algorithms to automatically classify job applicants as either to be interviewed or not be interveiwed. The data is highly ...
1
vote
2answers
128 views

Accuracy reduces drastically when using TruncatedSVD with hashingvector

I have around 0.8 million product description with categories. There are around 280 categories. I want to train a model with given dataset so that in future I can predict Category for the given ...
1
vote
0answers
42 views

Where can I find a dataset for long sequence text chunking? [closed]

Context: I have documents with reviews of articles that have the following structure: Introduction: a description of the review, dates and metadata that will be discarded. (avg~180 words, std~30 ...
1
vote
2answers
31 views

Grouping company information

I have 3 different datasets with company information, in all of them I have company name, but is not perfect: For example: Dataset A: Company name: Facebook Dataset B: Company name: Facebook, Inc ...
3
votes
1answer
525 views

Is there any clear tutorial for how to use AutoEncoders with text as input

I have a pandas dataframe that describes some fields of the register. I have used one hot encoding to encode the feature vectors that are not numbers. Finally my dataset now has 4000 rows * 4 columns. ...
2
votes
0answers
247 views

What is the minimum number of times a word needs to appear in word2vec training corpus for quality results?

When training a word2vec model with, eg, gensim, you can specify the minimum times a word needs to be seen (with the parameter min_count). The default value for this seems to be 5. Are there any ...
2
votes
1answer
87 views

Why is spam detection a classification problem and not a class modelling problem

Trying to get my feet wet with machine learning on text. The most common dataset I've seen in this space is the sms dataset with classes ham and spam. And the most common and successful approach ...
1
vote
1answer
2k views

One hot encoding at character level with Keras

I am reading Chollet's book on deep learning at the moment and in the NLP chapter he says: ...
0
votes
0answers
32 views

How to view incorrectly recognized text

Need ur help guys. There is a neural network that classifies the sentiment of the reviews. The accuracy is not 100%, hence there are texts that are recognized by the network incorrectly. How can I see ...
2
votes
1answer
1k views

How does ,the Mutlinomial Bayes's alpha parameter, affects the text classification task?

I would like to know how the alpha parameter, in Multinomial Bayes, affects the text classification task. I know that this parameter is correlated to the algorithm'...
0
votes
0answers
22 views

How to extract and visualize data from DBPedia?

I am trying to understand how can I download DBPedia and visualize the hierarchy to analyze the data. Then I want to extract certain sub-trees from it. Any help would be appreciated.
3
votes
1answer
2k views

Which type auto encoder gives best results for text

I did I couple of examples for auto encoders for images and they worked fine. Now I want to do an auto encoder for text that takes as input a sentence and returns the same sentence. But when I try to ...
0
votes
0answers
22 views

Analyzing chat data set

I have a problem analyzing a dataset; I'm not so sure what to look for, But these are the variables: My data set is a collection of chat lines with the corresponding timestamps of when they typed. I ...
0
votes
0answers
15 views

Disambiguating different Dates / section headers within text

In medical records (converted from PDF to Plain Text), what are the approaches to disambiguate between Valid Dates - Date of Service (such as Admission date/Discharge date/Date of consultation) and ...
0
votes
0answers
46 views

Identify if a text is a quote (philosophy)

Any pointers as to how to approach the problem of classifying a text into philosophical quote or not. I can harvest quotes and plain texts for a supervised training set. My problem statement is to ...
0
votes
0answers
133 views

How to create a DTM from a CSV with quanteda?

I am having trouble creating a DTM from a CSV with quanteda. I keep getting the error: Error in tokens.default(corpus, what = "word", remove_punct = TRUE, ...
0
votes
0answers
53 views

How to identify how survey takers answer a double barreled question?

I am analyzing the results of a survey where a survey taker is asked a, "double-barreled question." I am applying text analytics in order to answer the question, "which part of the question are survey ...
1
vote
0answers
47 views

Categorize text as Body, Heading in a loosely formatted document

Given a semi-structured document with only texts and images, and some style properties present on the text, What is the best possible way to classify the text present in the document to any formal ...
6
votes
1answer
3k views

How do you apply SMOTE on text classification?

Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique used in an imbalanced dataset problem. So far I have an idea how to apply it on generic, structured data. But is it ...
0
votes
0answers
25 views

How would you go about segmenting a document based on topic?

Something similar to TextTiling algorithm, but that expects the document to be broken into paragraphs, whereas in my use case the document has natural language and has not been formatted. LDA seems ...
1
vote
0answers
398 views

Multiclass classification with many classes and wide range of sample sizes

I'm working on a free text classification problem with over 100 classes in the training data. There is huge variation in the sample sizes of the classes: ranging from 1 to around 6000. I am using a ...
0
votes
0answers
63 views

How can I store a large feature set for fast prediction?

I'm using sklearn's CountVectorizer to vectorize text for a classification problem. This makes my linear model have a feature for every unigram and bigram in the training data. In addition, my model ...
0
votes
0answers
13 views

how to use text data to extract variables to supplement a dataset using multiple files?

I am trying to do a political science large-N research where the cases are the countries , However , some variables were never coded in structured data sets . So in order to test them and due to the ...
1
vote
1answer
332 views

Unsupervised clustering of unstructured text by document type

I have 100,000+ PDF healthcare documents from which I have extracted text. I would like to cluster these documents by type (e.g. pathology report, doctor visit notes, prescription orders, etc.) The ...
0
votes
2answers
46 views

Classify text labels in to a similar category [closed]

I'm trying to classify same kind of text labels in to one category. For example, if I have labels like qty, quantity, qty_no all of them should direct to Quantity. ...
1
vote
2answers
302 views

Text classification problem using Python or R

I am a novice in machine learning and new to NLP. I am looking for ideas on how to solve the below two problems. I have a dataset with two columns, "Titles" and "Description". Titles column has names ...
1
vote
3answers
2k views

Grouping of similar looking text

I have a data frame which has two columns, "Title" and "Description". The title column has a bunch of titles related to clinical lab tests. Unfortunately, most of the titles are a repeat of the same ...
1
vote
1answer
59 views

How to add incorporate meta data into text classification? [closed]

I have a collection of statements which I need to classify into 5 classes. Each statement have meta data in different columns: Author|Editor| date of release| statement | Class How can one use the ...
0
votes
0answers
239 views

Orange: Text mining - export corpus to table

I've been playing around with Orange Text Mining with the intent to: Extract tags and topics that can be linked to documents for better search Extract Names and Locations I've done nr 1, but need to ...
2
votes
0answers
1k views

Doc2vec to calculate cosine similarity - absolutely inaccurate

I'm trying to modify the Doc2vec tutorial to calculate cosine similarity and take Pandas dataframes instead of .txt documents. I ...
-3
votes
1answer
31 views

Algorithms/services to know an “iPhone case” is not an “iPhone”, in the context of complex item descriptions? [closed]

We are trying to implement a highly accurate search, based on user-entered search terms, into a large product database. For example, if the user searches for "iPhone", then one of these is ...
8
votes
4answers
20k views

Sentence similarity prediction

I'm looking to solve the following problem: I have a set of sentences as my dataset, and I want to be able to type a new sentence, and find the sentence that the new one is the most similar to in the ...
1
vote
2answers
165 views

Discover string “motifs” in python

If I have many strings from different sources that tend to exhibit some common patterns. Is there a way to extract these common motifs? For example, in a list (of millions) that includes strings <...
0
votes
1answer
74 views

Where can I find datasets with labeled duplicate text documents?

I'm working on detecting duplicate text documents using a classifier. I am looking for training data - a corpus of text documents and corresponding metadata which lists out pairs of duplicate ...
-1
votes
1answer
60 views

Text standardisation for manually entered data

I am working on a project that involves dealing with manually entered text data. I have a dataset of customs records where the customs officers manually enter the name and address of companies ...
0
votes
0answers
187 views

Binary Text Classifier

I'm looking for binary classification methods for text. I've found SVM, Naive Bayes, word2vec, doc2vec, and GloVe, but before I try implementing something I want to know which may be a better approach ...
0
votes
1answer
74 views

What methods to create singular content classification from inconsistent inbound info?

I am attempting to aggregate professional profile info from multiple sources, imposing a consistent taxonomy. Specifically, the current problem is how to impose a preferred taxonomy on profiles with ...
-1
votes
1answer
243 views

ML project ideas for dataset [closed]

Not sure if this is the right forum, but currently i have a dataset which contains a list of TV shows. Each record contains pricing between competitors (price in provider 1. Example: Itunes) TV show ...