Questions tagged [natural-language-process]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation. See NLP.

Filter by
Sorted by
Tagged with
1
vote
1answer
20 views

Looking for a causality to effect dataset

I am looking for a causality dataset that would look like this: ...
0
votes
1answer
16 views

I tried loading my saved .h5 model and predicting with that model, i'm getting error list index out of range

I saved my keras model into .h5 format. Again I've loaded that .h5 file into my colab and tried to predict with that model. ...
0
votes
1answer
21 views

how to train custom word2vec embeddings to find related articles?

I am beginner in machine learning. My project is to make search engine based on AI which shows related articles when we search on website. For this i decided to train my own embedding. I found two ...
1
vote
0answers
17 views

Semantic networks and conceptual graphs

I would like to use semantic networks to understand changes in texts. For example when I add/remove some words within a text. ...
2
votes
1answer
48 views

KeyError: Selecting text from a dataframe based on values of another dataframe

I have the following two dataframes badges and comments. I have created a list of 'gold users' from ...
1
vote
1answer
16 views

How can I compare the grammatical complexity between two texts using their sentences dependency length?

This is a continuation to the following thread. I have two texts, common English texts such as news articles and informative texts versus a technical textbook. I want to compare the grammatical ...
1
vote
1answer
64 views

How to generalize comments using NLP

I have list of log comments in CSV file. I want to cluster those log comments using K-Means and after that I want convert each cluster comments into general form. for eg. I have bunch of comments in ...
1
vote
1answer
33 views

Creating a valid dataset for obtaining results

I have created a domain-specific dataset, lets say it is relating to python programming topic posts. I have taken data from various places specific to this topic to create positive examples in my ...
0
votes
0answers
13 views

Does GPT-2 has pre trained for sentiment analysis?

I tried sentiment analysis with 345M model of GPT-2. But it took a long time to train. So is there any other GPT-2 model available for sentiment analysis?
0
votes
0answers
13 views

Neural network for filtering spam/not spam email [closed]

I have some doubts regarding how Neural Networks work. Most of NN are already built in so they might not required a very good knowledge. I am trying to understand how to build an easy NN which ...
-1
votes
1answer
41 views

Identify specify areas in the text

I'd be interested in identifying various areas in the text message. Let's say I have a text containing some introduction, then there is a poem and at the end there are some urls to some web pages. I'...
2
votes
1answer
148 views

Twitter POS and NER: What is state-of-the-art?

What is the current state-of-the-art for pos tagging and named entity recognition for twitter data? Are industrial-strength programs like Spacy and ...
0
votes
0answers
5 views

Is there any inductive Graph Variational Auto Encoder?

I have been reading about how we can model a Variational AutoEncoder (VAE) into a Graph Variational AutoEncoder (GVAE) where the decoder reconstructs the adjacency matrix. I presume that this approach ...
0
votes
0answers
33 views

Semantic network using word2vec

I have thousands of headlines and I would like to build a semantic network using word2vec, specifically google news file. My sentences look like ...
1
vote
1answer
1k views

what is BIO Tags for creating custom NER Named entity recognization?

I want to create custom NER Named entity recognition but im confused with this part of what is BIO Tags. Can anyone please explain the steps for creating NER and about this B, I, O tag.
0
votes
1answer
12 views

Product classification according to description

I have some products, along with their description. I wish to assign USPSC code to each product. I have a really basic doubt here. What exactly is my test file and training file? Eg. Should the ...
0
votes
0answers
7 views

Multi class product classification according to description of product

I have some products, along with their description. I wish to assign USPSC code to each product. This would require classification on 4 levels. All the examples I could find online were that of ...
1
vote
1answer
44 views

Emotional tension score in sentences

I am beginner in natural language processing and my goal is to find a way to score sentences based on their emotional tension. More specifically, I would like to know to what degree a sentence ...
1
vote
0answers
15 views

How to do embedding for nested dictionary with varying size?

I'm working on an RL task in which the agent needs have some observation. Instead using images, I want to use available information of the environment as the observation. The information regarding the ...
0
votes
0answers
9 views

Building quick web app to deploy ML models [migrated]

I've almost completed my DL models for an NLP project. Now I want to make a web app. I created models in PyTorch to detect propaganda text fragments in news articles. The models have poor performance. ...
1
vote
1answer
17 views

Generate text using user-supplied keywords

I've got a use case where I need to generate sentences based on a set of user supplied keywords. Here is an example of what I need: User input: End-User: Data Scientists Region: Middle East ...
0
votes
2answers
180 views

BERT classifier with Ktrain API is unable to predict new data

I have trained a classifier for sentiment analysis using BERT architecture. I am able to train the classifier and I am getting a validation accuracy of 87%. But whenever I feed in test data, or some ...
1
vote
2answers
50 views
1
vote
1answer
31 views

Is there any text similarity databse available for phrases?

I want to train my application for phrase similarity. I want my model to predict similarity score for phrases as shown in below examples. ex- ...
1
vote
2answers
54 views

Extract information using NLP and store it in csv file

I have a text file that stores the pickup, drops, and time. SMS text is a dummy file that is used to train a cab service model. The text is like in this format: ...
2
votes
1answer
49 views

German Chatbot or conversational AI

I want to build a chatbot mostly BERT(Transformer) based in the German Language. But I do not find any German chatbot data set! So does it make sense to use google translator API to translate the ...
0
votes
0answers
9 views

Retrieving information from 2 or more approaches

I'm trying to extract information from documents. There are two approaches currently which produces the following cases where both approaches ...
3
votes
1answer
62 views

How do I visualize data for a natural language processing project?

I am using a question-and-answer dataset. My neural network takes a question and an article content, and outputs where an answer starts (as an integer). To visualize my data, how should I process it ...
1
vote
1answer
16 views

HMM and its competitive alternatives

In Natural language processing, what are the major applications of Hidden Markov Chain (HMM), and what are the alternatives that usually can outperform HMM, is RNN and LSTM always the choice right now?...
0
votes
1answer
27 views

How to identify similar words as input from a dictionary

Let's say I have a CSV file (single column) with a list of words as input. Meaning the file looks like below Input Terms ...
3
votes
0answers
28 views

Dataset availability for automatic text summarization

I'm working on an automatic text summarization NLP problem and looking for a dataset with USA legal case reports similar to the Australian legal case reports dataset in UCI repository. Can you please ...
0
votes
0answers
13 views

Is there a way to find difference between topics in two languages using nlp

I want to analyse queries and their differences between two different languages English and Spanish in this case. I'm aware about topic modelling. I'm in search of any corpus available or any ...
0
votes
0answers
14 views

Looking for a web based tool for natural language to PostgreSQL query for custom database

I need to build a web-based solution that will take a natural language query in English, convert it into an SQL query that I need to execute on a PostgreSQL database. I tried ln2sql/nl2sql but it only ...
1
vote
2answers
50 views

Where can I get Insurance claim data for practicing NLP(Natural Language) processing?

I am looking for specifically Insurance dataset for practicing Machine Learning & NLP, but unable to find much in kaggle, udemy or other websites. Is there a way to get that dataset or any website ...
0
votes
1answer
20 views

What is syntax V and S standing for nominal subject?

I was reading the recent paper https://www.aclweb.org/anthology/P19-1580.pdf and noticed that in section 5.2, the syntactic relation is studied in terms of the "direction between two tokens". In table ...
0
votes
2answers
28 views

Identifying if the sentence if it comprise information about education

Given a sentence I am trying to classify if the sentence contain information about education. For example: ...
17
votes
4answers
25k views

Natural Language to SQL query

I have been working on developing a system "Converting Natural Language to SQL Query". I have read the answers from the similar questions, but was not able to get the information that I was looking ...
1
vote
1answer
30 views

How to figure out if two sentences have the same meaning with AI?

I have two sentences which might be similar in meaning. Are there a useful and successful (machine learning) algorithms, which is able to determine the semantic similarity? Are there any approaches ...
0
votes
2answers
165 views

How do you measure performance for word prediction tasks?

Say I have to predict the next word in a sentence, given the initial few words. Suppose the prefix is "I went to _____". This prefix is common enough that it might appear 10 times in the training ...
1
vote
1answer
11 views

What are some of the available methods for handling multi-label classification for longer sequences of text

I am looking to solve a multi-class classification problem with long sequences of text with some rows having 1000's of tokens. Some of the state of the art methods such as BERT have a token limit and ...
1
vote
1answer
68 views

What are x variable and y variable in word2vec model if it is supervised learning

What are x variable and y variable in word2vec model if it is supervised learning. In both the flavours- CBOW and skip-gram model. Though some blogs have explained it as unsupervised learning. ...
0
votes
1answer
43 views

How to use fine tuning of BERT when i have unlabelled dataset of text documents?

I have gained a basic understanding of using BERT for various NLP/text mining tasks. When it comes to fine-tuning of BERT, I always see that fine-tuning is performed using some classification tasks. ...
0
votes
1answer
13 views

Gender identification task on instance or user level?

I'm working on a task which is gender identification. Given a user account (e.g. Twitter account) with its documents (e.g. 100 tweets), the user should be classified as a male or a female. The ...
1
vote
0answers
18 views

Natural language processing for ecommerce search flow

I'm implementing an ecommerce search using nlp. there is a lack of clear documentation anywhere online about implementation and flow. this is what i'm doing. Search query comes in Do spellcheck Do ...
1
vote
0answers
42 views

How can I find colors in a sentence?

Given a sentence "I like blue jeans", the output should be "blue". I do not have any training data. I'll just be downloading a bunch of tweets related to a hashtag. How do I build a model for this? ...
0
votes
0answers
8 views

How to classify very short text for spend analytics?

I have a very small description of the item, example below- ACCELATOR SHAFT ADAPTOR –EGR PIPE & EGR VALVE(125KVA) ADAPTOR-EGR COOLER AND VALVE-125 KVA ADJUSTING LATCH - ALTERNATOR-125 KVA Some ...
0
votes
0answers
11 views

Is the number of bidirectional LSTMs in encoder-decoder model equal to the maximum length of input text/characters?

I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/. It ...
0
votes
0answers
24 views

Natural Language Processing: Identifying Words That Are Out of Place?

I want to make it easier to manually (or potentially automatically) correct AI transcription. I've noticed that one significant error that can be very easily picked up by human transcriptionists is ...
0
votes
1answer
101 views

*Challenge* Making an algo that learns from a book, and can answer anything about it

I recently took this challenge where I am trying to make a set of algorithms to read any particular book, understand and store the context and subsequently answer any question asked about it. In ways ...

1
2 3 4 5