Questions tagged [text]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
3
votes
0answers
19 views

Data transformations in hierarchical classification

I am building a hierarchical text classifier using the Local Classifier Per Parent Node (LCPN) approach with the 'siblings' policy as described in the PDF: E.g. if we have the classes 1.1, 1.2, 2.1, ...
0
votes
0answers
18 views

How would I define a model that computes “trainable edit distance” \ string similarity for Entity Linking

I want to compute a measure of string similarity based on "edit distance". Classic solutions for edit distance predefine the cost of each editing operations, and use a combination of atomic operations ...
2
votes
1answer
35 views

How to extract keywords from a list of URLs?

I have a bunch of URLs in a text file like- ...
1
vote
2answers
22 views

Text classification for data with multiple labels per observation

I have a dataset of tweets that has been labeled by multiple people. So the columns look something like: Tweet_ID, Coder_1_Classification, Coder_2_Classification, etc. The idea is to build a tweet ...
1
vote
0answers
21 views

PDF/Text to csv table

I have very little python experience but I have been wanting to get into data science and thought I would start with pdf/text mining. Because it's something I need right now anyway. I have a list of ...
0
votes
0answers
15 views

Looking for a good text autoencoder model (for text reconstruction)

I am looking for a simple and good model that can learn to encode and reconstruct text sentences to use it in some downstream task. I tried a tensorflow seq2seq model here, but it doesn't do ...
0
votes
0answers
4 views

Finding relevant pain points in feedbacks(open text)

I have employee feedback and need to find the appropriate pain points out of their feedback. Need help with the approach and analysis. I have provided a couple of examples below. Note: The feedbacks ...
1
vote
0answers
20 views

Embedding representation for a document?

Is averaging sentence embeddings, the right way to get representation for documents. Say I have a list of sentence embeddings representing symptoms. A data point looks like these: x|S1,S2,S3 --> Y|D1,...
1
vote
0answers
15 views

How do I identify specific parts of a PDF document?

I have a bunch of medical records that I have to input manually. I would like to automate this but all of the records are in different formats. What is the best strategy to build a deep learning model ...
1
vote
1answer
66 views

How to match a word from column and compare with other column in pandas dataframe

I have the below dataframe ...
1
vote
1answer
35 views

Technical term for using regular expressions to classify text?

Background I'm helping a researcher programmatically classify ~123,000 US Government court case files stored in plaintext. He wants to classify the claims as either having been "approved", "denied", ...
0
votes
1answer
33 views

Is there any library available for balancing imbalanced text dataset?

I have a text dataset similar to newsgroup dataset, the problem with the dataset is that it is highly imbalanced. So is there any readily built library that will do upsampling or downsampling with a ...
0
votes
1answer
31 views

Do word embeddings help with out of vocab tokens?

I am performing sentiment analysis on a custom dataset of text with Keras but am a little confused about word embeddings. I have been able to train an "Embedding" layer and have also learned to load ...
0
votes
1answer
31 views

How to utilize dictionary data set for text classification?

I have a dataset similar to newsgroup20 for classification. With the training dataset, I have a dictionary data set that explains some jargons in the training dataset. These both are different data ...
0
votes
0answers
311 views

Semantic Similarity in Universal Sentence Encoder

I am currently using Universal Sentence Encoder to embed certain sentences which I would then feed to a deep learning model to do some prediction, but just to test whether the universal sentence ...
3
votes
2answers
64 views

Multiclass classification of textual data

I have a problem statement in which I have to classify the text data into various classes, but the training data is very less (250-300 data points for 4 classes). I am confused about what approach to ...
0
votes
1answer
833 views

nltk's stopwords returns “TypeError: argument of type 'LazyCorpusLoader' is not iterable”

While trying to remove stopwords using the nltk package, the following error occurred: ...
-1
votes
1answer
17 views

Identifying specific words in text

Let's say I have the following text" Is that another kitten playing in the shoes in the top right? I would like my code to extract kitten from that text. Is ...
1
vote
4answers
62 views

Is it OK to train a binary classifier using all the extremely imbalanced data if the majority class is negative?

I'm training a neural network as a binary classifier for text classification. The data is very imbalanced, where the ratio of TRUE:FALSE is approximately 100:10000 Intuitively, it feels like using ...
0
votes
0answers
39 views

Map predictions to real text

I have read the paper "Learning to Read by Spelling" by Gutpa et al. They present a method for visual text recognition without using any paired supervisory data. In chapter 4 they describe how to ...
1
vote
1answer
17 views

Complete IPv4 Address Space

I know this was asked before, but I didn't get a definitive answer. I am trying to download a simple 17GB .txt file that contains the entire IPv4 address space. There must be a file like this already ...
0
votes
1answer
50 views

Complete IPv4 Address List

Is it possible to pull a complete list of all IPv4 addresses and put them into a text file? Since there are 4,294,967,296 IPv4 addresses and each one takes about 4 bytes I would guess the file would ...
0
votes
1answer
374 views

Extracting structure and content from invoices

Lately, I have been largely inspired by this https://rossum.ai/, which is able to extract text from invoice documents. Do you have any ideas on how this could be implemented? It's clear that they ...
0
votes
0answers
37 views

Make use of multiple labels in doc2vec: Setting up the data

I am trying to implement the doc2vec algorithm with a rather small sample size: ca. 120 documents with a total of 25000 unique words. My ...
2
votes
2answers
321 views

Bidirectional Encoder Representations from Transformers in R

Can anybody suggest to me, where I can find example code for R language for BERT neural network for text mining tasks. All I can see are python examples, and I need R. https://github.com/google-...
1
vote
0answers
25 views

Fuzzy matching of author names

We are trying to figure out what is the best approach for us to train a ML model to identify Authors. We have structured metadata of authors (given name, surname. etc) and the task is to train a model ...
0
votes
0answers
13 views

Do double quotes, dots and commas modify the forget weights in LSTM if retained?

I am trying to implement custom NER with LSTM. In the pre processing steps is it required to remove the punctuation marks like double quotes, dots and commas? Do they add any significance if retained? ...
0
votes
0answers
188 views

Implementing back translation as a data augmentation for text classification

Since back translation English->other language -> English seems like quite a useful data augmentation technique , I wanted to experiment with it. E.g. it occurred to me that languages from very ...
1
vote
2answers
54 views

Input data of variable length - two scenarios

I'm trying to figure out how I could train a neural network with inputs that have variable length. This issue comes up in the following 2 scenarios I'm trying to solve. Scenario 1: I have a long list ...
1
vote
0answers
77 views

python - Identify variable in similar sentences

I'm looking to solve the following problem: I have a list of similar sentences as my dataset, and I want to be able to type a new sentence, which is also similar to the sentences in my dataset and ...
2
votes
3answers
50 views

Match a two items from two different receipts

I have two different invoices or receipts. One is a Purchase order one is something like a receipt(acknowledgement). Suppose I have ordered(PO) Wine: White Wine Red Wine Rose Wine And I receive ...
3
votes
0answers
90 views

How to implement hierarchical labeling classification?

I am currently working on task of eCommerce product name classification, so I have categories and subcategories in product data. I noticed that using subcategories as labels delivers worse results (84%...
2
votes
2answers
295 views

How to extract and classify data from a column in excel?

I have a column in an Excel sheet that contains a lot of data separated by || delimiters. The data can be classified to some classes like Entity, IFSC codes, ...
1
vote
0answers
38 views

Word classification in the context

I'm trying to solve a 'negation-like' classification problem, where I need to classify whether a certain word within the context has negative or positive label. For example, how to identify whether a ...
1
vote
0answers
18 views

Supervisory information through side output in convolutional neural network

I am trying to implement this paper https://ieeexplore.ieee.org/document/7828014 Here they have mentioned text local (edge) and global regions as supervisory information. Side output is generated ...
1
vote
0answers
64 views

What options are out there to extract text from a group of PDFs where each PDF is formatted differently but contains the general same content [closed]

Think insurance/medical forms that come from different companies. There is no standard on formatting. I am trying to extract the text based on each section of a given form. A form might have a ...
0
votes
0answers
31 views

Text segmentation based on most probable classes

I am working on a text classification problem. As training data, I have human annotated text, which was manually segmented into sections and then these sections are labeled with some class. Training ...
0
votes
1answer
37 views

Size of Output vector from AvgW2V Vectorizer is less than Size of Input data

Hi, I have been seeing this problem for quite some time. Whenever I tried vectorizing input text data though avgw2v vectorization technique. The size of vectorized data is less than the size of the ...
0
votes
1answer
35 views

How to use correlation matrix when the dataset contains multiple columns with text data? [closed]

How to use it with Amazon fine food reviews dataset?
2
votes
1answer
77 views

CNN to many outputs

I have a dataset with 100 columns (categorial one-hot encoded) and 1 column with text data (simple sentences) and i want to build a neural network to arround 380.000 outputs labels. I have no idea ...
0
votes
1answer
206 views

Multi-Class Text Classification: Doc2Vec performing very bad compared to Hashing Vector

I have a multi-class text classification problem in hand this is similar to product category mapping where we map products to its correct Category based on the text content provided. I first created ...
0
votes
2answers
153 views

Use text similarity (cosine) instead of machine learning to classify companies into industries

I am building an industry classifier. I.e. classifying companies into industries based on a company's description. Each company can only have one industry. I took 2000 companies and assigned them ...
1
vote
1answer
34 views

Can I treat text review analysis as a regression problem?

I am playing with a dataset that contains tripadvisor restaurant reviews and their labels (either 1, 2, 3, 4 or 5 stars). Initially I was thinking of using it as a classification problem, applying ...
1
vote
0answers
13 views

Ordering quotes in a list based on user input and text analysis

A little bit of context: I have a website that has many quotes. These quotes are organized automatically by Solr into lists of quotes, so e.g. there is a list called 'Smart Quotes' that includes ...
0
votes
2answers
368 views

Street address clustering?

I have a huge dataset of addresses. I have another data stream that contains addresses that I need to match against those in the original dataset. As all the addresses are user-provided, matching them ...
2
votes
1answer
48 views

Changing multiple models into 1 model

I am working for a recruitment company on developing machine learning algorithms to automatically classify job applicants as either to be interviewed or not be interveiwed. The data is highly ...
2
votes
2answers
448 views

Accuracy reduces drastically when using TruncatedSVD with hashingvector

I have around 0.8 million product description with categories. There are around 280 categories. I want to train a model with given dataset so that in future I can predict Category for the given ...
1
vote
0answers
108 views

Where can I find a dataset for long sequence text chunking? [closed]

Context: I have documents with reviews of articles that have the following structure: Introduction: a description of the review, dates and metadata that will be discarded. (avg~180 words, std~30 ...
1
vote
2answers
41 views

Grouping company information

I have 3 different datasets with company information, in all of them I have company name, but is not perfect: For example: Dataset A: Company name: Facebook Dataset B: Company name: Facebook, Inc ...
3
votes
1answer
1k views

Is there any clear tutorial for how to use AutoEncoders with text as input

I have a pandas dataframe that describes some fields of the register. I have used one hot encoding to encode the feature vectors that are not numbers. Finally my dataset now has 4000 rows * 4 columns. ...