Questions tagged [natural-language-process]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation. See NLP.

113 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
0answers
45 views

Clustering and graphing similarities of sentence subjects

I have a bunch of sentences. Each sentence is given a weight of how "close" it is to a particular subject. Ex. "I love reading math books" Subjects for the above sentence = ...
1
vote
0answers
131 views

NLP to recognize the meaning of a paragraph

How can I apply NLP to extract the summary of a paragraph,I found this but was wondering if there is a better and easy way before implementing it, as the ans seems to be an old one. Basically what I ...
1
vote
0answers
58 views

How is maximizing L(lambda1, lamda2, lamda3) equivalent to minimizing perplexity?

In language modeling, L(lambda1, lambda2, lambda3) is defined as: Sum(count of trigram(u,v,w) x q(w|u,v)) where u, v, w are words in the corpus and perplexity ...
1
vote
0answers
80 views

How to extract name of objects from technical description (NLP)

I have a list of technical descriptions of mechanical parts such as Relay Compressor A4u8skk Relay-Start-Compressor sdfvsh2 Relay-Start-Compressor-Copelan 05-569-21 Compressor-J4KS0AK-5gss-fj2 ...
1
vote
0answers
104 views

Job Recommendation System

I am building a Job Recommendation System where I have Student Data for different subjects in Machine Learning(Data Viz, Python, Statistics, etc) and their skills from the resume. Need to Recommend ...
1
vote
0answers
39 views

How to learn word embedding from a context on the fly?

Consider the fictional word tahiliuk in the sentence “We found a small, fluffy tahiliuk running around our garden.” While hearing a new word used in context, people are remarkably adept at inferring a ...
1
vote
0answers
28 views

TextRank algorithm for Web content

I am looking for an algorithm that would be able to extract meaningful keyphrases from web articles. Each article has more than 2000 words and information is structured using paragraphs, h1, h2 tags ...
1
vote
0answers
288 views

Audio signal signal processing for background noise reduction or removal

I am performing simple audio recognition using tensor-flow to spot key word or hot word detection. The graph takes the microphone input directly and performs "Audio spectrometer --> mfcc's --> and ...
1
vote
0answers
39 views

Using an ontology to recognize named entities in free text

I'm trying to solve a fairly basic problem in NPL efficiently. What tool or software package would you use to identify the words, or group of words that are part of an given ontology within a free ...
1
vote
0answers
32 views

Common deep learning practices in NLP for text classification

Are there any articles about best modern techniques in text classification (not only for English texts, but in general)? In particular, I'm interested: what kind of text preprocessing techniques are ...
1
vote
0answers
38 views

Check If Answer is Correct by Similarity

I am new to data science and machine learning. Let's say that I have a question, and some correct answers for that question (for example, 10 correct answers). Is there a way to get a new answer as ...
1
vote
0answers
12 views

Grouping prefix and term queries

Consider the following context: You have a website that is offering a search function, which produces a list of matching articles. The search has prefix functionality: if you are looking for "...
1
vote
0answers
198 views

Loss function for Hierarchical Multi-label classification

I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels (e.g., a simple MLP branch inside a ...
1
vote
0answers
85 views

How are natural language generation algorithms given a target

I've started learning about NLP and NLG and I'm fascinated! I've been blown away by the things I've seen from NLP; but I have a few questions about NLG. All my questions boil down to this: Given a ...
1
vote
0answers
19 views

Machine Learning Text Categorization with Data in multiple Languages

I want to label natural text Dokuments in various different categories (arround 400 different categories) using Neural Networks. I have around 50.000 documents already labeled. The problem is around ...
1
vote
0answers
79 views

Build a relevancy scoring model of articles using NLP

I'm really new to Data Science and text mining. I want to build a relevancy scoring model. Suppose I have a bag of words (guns, military, terrorists). I also have a list of articles. I want to find if ...
0
votes
0answers
16 views

Optimal input setup for character-level text classification RNN

I want to classify 500-character long text samples as to whether they look like natural language using a character-level RNN. I'm unsure as to the best way to feed the input to the RNN. Here are two ...
0
votes
0answers
8 views

Research on Product Search Machine Learning @ Amazon

I am looking for any sort of research paper/ reference where we can read more about machine learning principles using for the amazon's main website. Mainly interested in how they do the product search ...
0
votes
0answers
13 views

Converting a string to a recommendation type string

I am trying to build a recommendation system and some of the labels are ...
0
votes
0answers
13 views

How one can match similar sounding features of two items

I am trying to build simple app which will compare two products from provideid URLs based on their features. We will be showing the products in tabular form with columns being each product and rows ...
0
votes
0answers
16 views

Question about Relative-Position-Representation code

In https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/layers/common_attention.py In _relative_attention_inner method, which I think is one of the ...
0
votes
1answer
23 views

Is it possible to classify documents of corpus using labels?

I have a corpus of 23000 documents that need to be classified into 5 different categories. I do not have any labeled data available to me, just freeform text documents and labels(yes, one-word labels, ...
0
votes
0answers
22 views

Are there any deep models that are better than BERT-CRF in NER task?

Named entity recognition (NER) is task that mark tags of the input text sequence. BERT-CRF is a good NER model. I want to find a better NER model. Or I want to improve the BERT-CRF model. What can I ...
0
votes
1answer
35 views

Using BERT for co-reference resolving, what's the loss function?

I'm working my way around using BERT for co-reference resolving. I'm following this highly-cited paper BERT for Coreference Resolution: Baselines and Analysis (https://arxiv.org/pdf/1908.09091.pdf). I ...
0
votes
0answers
30 views

How to insert sentences into training data which has 2 words, 3 words 4 and 5 etc into training data?

I have a set of sentences which each contain 2 words, 3 words, 4 words, 5 words etc. When I am trying to give the training data only the first two words in a sentence it is not accepting it. It is ...
0
votes
1answer
44 views

Prediction using words which were not in training in a CNN with pre-trained word embeddings

In sentence classification using pre-trained embeddings(fasttext) in a CNN, how does the CNN predict the category of a sentence when the words were not in the training set? I think the trained model ...
0
votes
1answer
28 views

I tried loading my saved .h5 model and predicting with that model, i'm getting error list index out of range

I saved my keras model into .h5 format. Again I've loaded that .h5 file into my colab and tried to predict with that model. ...
0
votes
1answer
27 views

how to train custom word2vec embeddings to find related articles?

I am beginner in machine learning. My project is to make search engine based on AI which shows related articles when we search on website. For this i decided to train my own embedding. I found two ...
0
votes
0answers
73 views

Does GPT-2 has pre trained for sentiment analysis?

I tried sentiment analysis with 345M model of GPT-2. But it took a long time to train. So is there any other GPT-2 model available for sentiment analysis?
0
votes
0answers
11 views

Is there any inductive Graph Variational Auto Encoder?

I have been reading about how we can model a Variational AutoEncoder (VAE) into a Graph Variational AutoEncoder (GVAE) where the decoder reconstructs the adjacency matrix. I presume that this approach ...
0
votes
0answers
9 views

Multi class product classification according to description of product

I have some products, along with their description. I wish to assign USPSC code to each product. This would require classification on 4 levels. All the examples I could find online were that of ...
0
votes
0answers
9 views

Retrieving information from 2 or more approaches

I'm trying to extract information from documents. There are two approaches currently which produces the following cases where both approaches ...
0
votes
1answer
84 views
0
votes
0answers
20 views

Looking for a web based tool for natural language to PostgreSQL query for custom database

I need to build a web-based solution that will take a natural language query in English, convert it into an SQL query that I need to execute on a PostgreSQL database. I tried ln2sql/nl2sql but it only ...
0
votes
0answers
24 views

Natural language processing for ecommerce search flow

I'm implementing an ecommerce search using nlp. there is a lack of clear documentation anywhere online about implementation and flow. this is what i'm doing. Search query comes in Do spellcheck Do ...
0
votes
1answer
23 views

Is the number of bidirectional LSTMs in encoder-decoder model equal to the maximum length of input text/characters?

I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/. It ...
0
votes
0answers
24 views

Natural Language Processing: Identifying Words That Are Out of Place?

I want to make it easier to manually (or potentially automatically) correct AI transcription. I've noticed that one significant error that can be very easily picked up by human transcriptionists is ...
0
votes
0answers
322 views

Keras: Vector regression: ValueError:can not squeeze dim[1]

I'm trying to create a NLP model which takes x_train_padded_2 (padded/tokenized text sequences) as input and try to approximate ...
0
votes
1answer
13 views

Gender identification task on instance or user level?

I'm working on a task which is gender identification. Given a user account (e.g. Twitter account) with its documents (e.g. 100 tweets), the user should be classified as a male or a female. The ...
0
votes
0answers
24 views

Named Entity Recognition/Linking with a dictionary of names (e.g., move titles, book titles, company names, people names)

I have a lot of text messages on which I want to perform Named Entity Recognition and Named Entity Linking. For example, I want to detect all movie names and find their IMDB ID. To make things easier,...
0
votes
3answers
44 views

approach to classify text with natural language processing methods

I have a problem with regards to text classification/categorization. The task is bugging me for days already and as I am pretty new to AI and the field of natural language processing (NLP) I am just ...
0
votes
2answers
309 views

BERT classifier with Ktrain API is unable to predict new data

I have trained a classifier for sentiment analysis using BERT architecture. I am able to train the classifier and I am getting a validation accuracy of 87%. But whenever I feed in test data, or some ...
0
votes
0answers
17 views

Known datasets for long document analysis

Are there good baselines datasets or benchmarks for similarities/retrieval of long texts (extremely long documents, books, etc.)? Although significant both academically and applicational, I cannot get ...
0
votes
1answer
67 views

How to implement LSTM using Doc2Vec vectors to get representation?

Hi all. I'm a newbie in ML. I read and found a paper about A Multi-Level Plagiarism Detection System Based on Deep Learning Algorithms and want to implement this model . But I can't find more about ...
0
votes
0answers
9 views

Taking huge time to execute piepeline text classification model using sklearn?

I have created a pipeline model for text classification using python ,Firstly i have tried on 30k records dataset it is working fine got the good results , but when it comes to huge data set like 50k ...
0
votes
0answers
8 views

Looking for suggestions on performing Sementic Analysis of ASR text

Currently I am working on a project where I have ASR on which I am performing semantic analysis to extract meaning out of it. The ASR text contains huge amount of vague conversational text which needs ...
0
votes
1answer
62 views

Train a deep learning model in chunks/sequentially to avoid memory error

How do I train/fit a model in chunks so as to escape the dreaded memory error? ...
0
votes
1answer
189 views

Feeding XLM-R embeddings to neural machine translation?

I’m very new to the field of deep learning. My aim is to make a translation between Catalan to Catalan Sign Language. The grammar of the two languages is different Input: He sells food. Output (...