Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [text]

The tag has no usage guidance.

1
vote
2answers
29 views

Input data of variable length - two scenarios

I'm trying to figure out how I could train a neural network with inputs that have variable length. This issue comes up in the following 2 scenarios I'm trying to solve. Scenario 1: I have a long list ...
1
vote
0answers
22 views

python - Identify variable in similar sentences

I'm looking to solve the following problem: I have a list of similar sentences as my dataset, and I want to be able to type a new sentence, which is also similar to the sentences in my dataset and ...
0
votes
0answers
20 views

How to predict entity (token) classes with a Keras neural network?

I am trying to build a neural network that predicts one of several classes for every token in a document. I have a dataset that provides a class for every token. The majority of the tokens have the ...
0
votes
0answers
13 views

Match a two items from two different receipts

I have two different invoices or receipts. One is a Purchase order one is something like a receipt(acknowledgement). Suppose I have ordered(PO) Wine: White Wine Red Wine Rose Wine And I receive ...
2
votes
0answers
19 views

How to implement hierarchical labeling classification?

I am currently working on task of eCommerce product name classification, so I have categories and subcategories in product data. I noticed that using subcategories as labels delivers worse results (84%...
2
votes
1answer
53 views

How to extract and classify data from a column in excel?

I have a column in an Excel sheet that contains a lot of data separated by || delimiters. The data can be classified to some classes like Entity, IFSC codes, ...
1
vote
0answers
20 views

Word classification in the context

I'm trying to solve a 'negation-like' classification problem, where I need to classify whether a certain word within the context has negative or positive label. For example, how to identify whether a ...
1
vote
0answers
14 views

Supervisory information through side output in convolutional neural network

I am trying to implement this paper https://ieeexplore.ieee.org/document/7828014 Here they have mentioned text local (edge) and global regions as supervisory information. Side output is generated ...
1
vote
0answers
23 views

What options are out there to extract text from a group of PDFs where each PDF is formatted differently but contains the general same content [closed]

Think insurance/medical forms that come from different companies. There is no standard on formatting. I am trying to extract the text based on each section of a given form. A form might have a ...
0
votes
0answers
12 views

Text segmentation based on most probable classes

I am working on a text classification problem. As training data, I have human annotated text, which was manually segmented into sections and then these sections are labeled with some class. Training ...
0
votes
1answer
17 views

Size of Output vector from AvgW2V Vectorizer is less than Size of Input data

Hi, I have been seeing this problem for quite some time. Whenever I tried vectorizing input text data though avgw2v vectorization technique. The size of vectorized data is less than the size of the ...
0
votes
1answer
23 views

How to use correlation matrix when the dataset contains multiple columns with text data? [closed]

How to use it with Amazon fine food reviews dataset?
2
votes
1answer
53 views

CNN to many outputs

I have a dataset with 100 columns (categorial one-hot encoded) and 1 column with text data (simple sentences) and i want to build a neural network to arround 380.000 outputs labels. I have no idea ...
0
votes
1answer
89 views

Multi-Class Text Classification: Doc2Vec performing very bad compared to Hashing Vector

I have a multi-class text classification problem in hand this is similar to product category mapping where we map products to its correct Category based on the text content provided. I first created ...
0
votes
2answers
67 views

Use text similarity (cosine) instead of machine learning to classify companies into industries

I am building an industry classifier. I.e. classifying companies into industries based on a company's description. Each company can only have one industry. I took 2000 companies and assigned them ...
1
vote
1answer
22 views

Can I treat text review analysis as a regression problem?

I am playing with a dataset that contains tripadvisor restaurant reviews and their labels (either 1, 2, 3, 4 or 5 stars). Initially I was thinking of using it as a classification problem, applying ...
1
vote
0answers
12 views

Ordering quotes in a list based on user input and text analysis

A little bit of context: I have a website that has many quotes. These quotes are organized automatically by Solr into lists of quotes, so e.g. there is a list called 'Smart Quotes' that includes ...
-1
votes
0answers
40 views

Where and when can I find the faster implementation of Google Wavenet?

https://deepmind.com/blog/wavenet-launches-google-assistant This link talks about how Google researchers have found a way to make the original Wavenet 1000 times faster. Where and when can I expect ...
0
votes
2answers
167 views

Street address clustering?

I have a huge dataset of addresses. I have another data stream that contains addresses that I need to match against those in the original dataset. As all the addresses are user-provided, matching them ...
2
votes
1answer
39 views

Changing multiple models into 1 model

I am working for a recruitment company on developing machine learning algorithms to automatically classify job applicants as either to be interviewed or not be interveiwed. The data is highly ...
1
vote
2answers
191 views

Accuracy reduces drastically when using TruncatedSVD with hashingvector

I have around 0.8 million product description with categories. There are around 280 categories. I want to train a model with given dataset so that in future I can predict Category for the given ...
1
vote
0answers
53 views

Where can I find a dataset for long sequence text chunking? [closed]

Context: I have documents with reviews of articles that have the following structure: Introduction: a description of the review, dates and metadata that will be discarded. (avg~180 words, std~30 ...
1
vote
2answers
34 views

Grouping company information

I have 3 different datasets with company information, in all of them I have company name, but is not perfect: For example: Dataset A: Company name: Facebook Dataset B: Company name: Facebook, Inc ...
3
votes
1answer
809 views

Is there any clear tutorial for how to use AutoEncoders with text as input

I have a pandas dataframe that describes some fields of the register. I have used one hot encoding to encode the feature vectors that are not numbers. Finally my dataset now has 4000 rows * 4 columns. ...
2
votes
0answers
324 views

What is the minimum number of times a word needs to appear in word2vec training corpus for quality results?

When training a word2vec model with, eg, gensim, you can specify the minimum times a word needs to be seen (with the parameter min_count). The default value for this seems to be 5. Are there any ...
2
votes
1answer
114 views

Why is spam detection a classification problem and not a class modelling problem

Trying to get my feet wet with machine learning on text. The most common dataset I've seen in this space is the sms dataset with classes ham and spam. And the most common and successful approach ...
1
vote
1answer
2k views

One hot encoding at character level with Keras

I am reading Chollet's book on deep learning at the moment and in the NLP chapter he says: ...
2
votes
1answer
2k views

How does ,the Mutlinomial Bayes's alpha parameter, affects the text classification task?

I would like to know how the alpha parameter, in Multinomial Bayes, affects the text classification task. I know that this parameter is correlated to the algorithm'...
3
votes
1answer
2k views

Which type auto encoder gives best results for text

I did I couple of examples for auto encoders for images and they worked fine. Now I want to do an auto encoder for text that takes as input a sentence and returns the same sentence. But when I try to ...
1
vote
0answers
51 views

Categorize text as Body, Heading in a loosely formatted document

Given a semi-structured document with only texts and images, and some style properties present on the text, What is the best possible way to classify the text present in the document to any formal ...
7
votes
1answer
4k views

How do you apply SMOTE on text classification?

Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique used in an imbalanced dataset problem. So far I have an idea how to apply it on generic, structured data. But is it ...
1
vote
0answers
452 views

Multiclass classification with many classes and wide range of sample sizes

I'm working on a free text classification problem with over 100 classes in the training data. There is huge variation in the sample sizes of the classes: ranging from 1 to around 6000. I am using a ...
1
vote
1answer
380 views

Unsupervised clustering of unstructured text by document type

I have 100,000+ PDF healthcare documents from which I have extracted text. I would like to cluster these documents by type (e.g. pathology report, doctor visit notes, prescription orders, etc.) The ...
0
votes
2answers
46 views

Classify text labels in to a similar category [closed]

I'm trying to classify same kind of text labels in to one category. For example, if I have labels like qty, quantity, qty_no all of them should direct to Quantity. ...
1
vote
2answers
312 views

Text classification problem using Python or R

I am a novice in machine learning and new to NLP. I am looking for ideas on how to solve the below two problems. I have a dataset with two columns, "Titles" and "Description". Titles column has names ...
1
vote
3answers
2k views

Grouping of similar looking text

I have a data frame which has two columns, "Title" and "Description". The title column has a bunch of titles related to clinical lab tests. Unfortunately, most of the titles are a repeat of the same ...
1
vote
1answer
63 views

How to add incorporate meta data into text classification? [closed]

I have a collection of statements which I need to classify into 5 classes. Each statement have meta data in different columns: Author|Editor| date of release| statement | Class How can one use the ...
2
votes
0answers
1k views

Doc2vec to calculate cosine similarity - absolutely inaccurate

I'm trying to modify the Doc2vec tutorial to calculate cosine similarity and take Pandas dataframes instead of .txt documents. I ...
-3
votes
1answer
32 views

Algorithms/services to know an “iPhone case” is not an “iPhone”, in the context of complex item descriptions? [closed]

We are trying to implement a highly accurate search, based on user-entered search terms, into a large product database. For example, if the user searches for "iPhone", then one of these is ...
8
votes
4answers
26k views

Sentence similarity prediction

I'm looking to solve the following problem: I have a set of sentences as my dataset, and I want to be able to type a new sentence, and find the sentence that the new one is the most similar to in the ...
2
votes
2answers
225 views

Discovering string “motifs” in python

I have millions of strings from different sources that tend to exhibit some common patterns. Is there a way to extract these common motifs? For example, in a list (of millions) that includes strings ...
0
votes
1answer
88 views

Where can I find datasets with labeled duplicate text documents?

I'm working on detecting duplicate text documents using a classifier. I am looking for training data - a corpus of text documents and corresponding metadata which lists out pairs of duplicate ...
-1
votes
1answer
63 views

Text standardisation for manually entered data

I am working on a project that involves dealing with manually entered text data. I have a dataset of customs records where the customs officers manually enter the name and address of companies ...
0
votes
1answer
79 views

What methods to create singular content classification from inconsistent inbound info?

I am attempting to aggregate professional profile info from multiple sources, imposing a consistent taxonomy. Specifically, the current problem is how to impose a preferred taxonomy on profiles with ...
-1
votes
1answer
255 views

ML project ideas for dataset [closed]

Not sure if this is the right forum, but currently i have a dataset which contains a list of TV shows. Each record contains pricing between competitors (price in provider 1. Example: Itunes) TV show ...
1
vote
0answers
405 views

Methods for string classifications

I have a list of some 100 millions of strings, each of different length. Examples: nsdgnlnesef ngmrlxkvgrmksefsfnlj <...
0
votes
3answers
288 views

Using training data generated with pure regular expressions - Can machine learning surpass the accuracy of your regular expression?

For text classification with machine learning - If your training data was generated purely with regular expressions, is it possible to train a machine learning model with this training data which will ...
1
vote
1answer
2k views

What are some function/package in R to find similarity of individual words not in the context of sentences?

what are some function/package in R/python to find similarity of individual words not in the context of sentences? As a novice, I searched and found methods like adist, cosine similarity, word2vec, ...
6
votes
1answer
3k views

How to use TFIDF vectors with multinomial naive bayes?

Say we have used the TFIDF transform to encode documents into continuous-valued features. How would we now use this as input to a Naive Bayes classifier? Bernoulli naive-bayes is out, because our ...
0
votes
3answers
2k views

How to use binary text classifier(built using SVM with TF-IDF) to classify new text document?

I have built binary text classifier using SVM on TF-IDF for news articles(Sports: Non-Sports). But I not sure how to classify new document using this model. Since TF-IDF is calculated based on the ...