Questions tagged [natural-language-process]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation. See NLP.

Filter by
Sorted by
Tagged with
1
vote
0answers
28 views

Predicting a new document [closed]

I have a document, (purchase agreement) of approx. 100 pages. This document is sent from buyer to seller several times, and each time there is a negotiation. Negotiation could be anything. For eg. ...
5
votes
1answer
520 views

Is it valid to include your validation data in your vocabulary for NLP?

At the moment, I am following best practices and creating a "bag of words" vector with a vocabulary from the training data. My cross validation (and test) datasets are transformed using this model, ...
1
vote
1answer
379 views

transfer learning with sentiment analysis?

The question is how good and what are some things to keep in mind when sentiment analysis models are tested on different datasets than they are trained on. Say the task is to perform sentiment ...
1
vote
2answers
53 views

How should I format input and output for text generation with LSTMs

I'm attempting to generate a response to an input line of text using an LSTM. I've considered various forms of input, including one-hot encoding each character in the line and passing each input line ...
2
votes
1answer
335 views

Fine-tuning NLP models

In computer vision, if we don't have a large training set, a common method is to start with a pre-trained model for some related task (e.g., ImageNet) and fine-tune that model to solve our problem. ...
1
vote
0answers
18 views

Machine Learning Text Categorization with Data in multiple Languages

I want to label natural text Dokuments in various different categories (arround 400 different categories) using Neural Networks. I have around 50.000 documents already labeled. The problem is around ...
11
votes
3answers
17k views

Natural Language to SQL query

I have been working on developing a system "Converting Natural Language to SQL Query". I have read the answers from the similar questions, but was not able to get the information that I was looking ...
1
vote
1answer
303 views

How to combine sparse text features with user smile for sentiment classification? [closed]

I am trying to perform sentiment classification task where I have some text and some information about whether the user smiled or not. Now when I use count-vectorizer to convert my text to feature ...
2
votes
1answer
73 views

Machine Learning and Natural Language Processing : Project Initiation

I am in the research phase of a long project and am willing to get some useful feedback from your side about the most appropriate project path to take. Current situation: A large team of so called ...
1
vote
0answers
46 views

Build a relevancy scoring model of articles using NLP

I'm really new to Data Science and text mining. I want to build a relevancy scoring model. Suppose I have a bag of words (guns, military, terrorists). I also have a list of articles. I want to find if ...
4
votes
1answer
105 views

Can CBOW model only accept fixed number of words?

I have a question about CBOW prediction. Suppose my job is to use 3 surrounding words w(t-3), w(t-2), w(t-1)as input to predict one target word w(t). Once the model is trained and I want to predict a ...
0
votes
2answers
65 views

How to select features for clustering to detect the number of different unique products in a search result?

I am trying to use clustering to determine the number of products in a search of products. So far I am using kmeans clustering. I have run into a problem where I cannot determine good features to use. ...
1
vote
1answer
78 views

What distance should I use for edges weights in textrank algorithm

I found this python implementation on github with 400+ stars which use levenshtein distance between each nodes. But original paper (page 4) said: Next, all lexical units that pass the syntactic ...
0
votes
1answer
39 views

Should I remove features that occur very rarely to build a model?

I am trying ML techniques in language processing. I have got 3000 short texts and I extract features(words and phrases) from all of them and build a vocabulary. I end up with 6000 od features and most ...
9
votes
2answers
233 views

what machine/deep learning/ nlp techniques are used to classify a given words as name, mobile number, address, email, state, county, city etc

I am trying to generate an intelligent model which can scan a set of words or strings and classify them as names, mobile numbers, addresses, cities, states, countries and other entities using machine ...
0
votes
1answer
595 views

Text analysis - classification, parsing

Excuse if this has been answered before. I need to extract features and parse from a piece of text and run some analysis. For e.g. "Plot the past 5-year sales of Apple" should give me the following ...
2
votes
1answer
1k views

Why are Chunking and IOB tags necessary?

I've just come across chunking and I can't get my head around why is it necessary? I know that it is used for 'named entity recognition'. I have few questions: Why and how is Chunking helpful? Plus ...
1
vote
1answer
652 views

Words as features of a neural networks

I'm new in Machine learning and I'm working on a problem related to text. I know that in ML we can use features as numerical values as input to neural network, but I don't know how to use features as ...
1
vote
0answers
84 views

Sentiment Analysis [closed]

I am currently doing a project in python. This project is Aspect based sentiment analysis. I don't know how to prepare training file for that
2
votes
1answer
135 views

How do we pass data to a RNN?

Let's say we have A1, A2, ... , Am different articles in the corpus and each of them has W1, W2, ....., Ww words. We are training a language model on them. Do we: Scheme 1 Take the first batch of ...
2
votes
2answers
305 views

Training an AI to play Starcraft 2 with superhuman level of performance?

I'm interested in working on challenging AI problems, and after reading this article (https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/) by DeepMind and ...
1
vote
1answer
215 views

GloVe vector representation homomorphism question

In the paper GloVe: Global Vectors for Word Representation, there is this part (bottom of third page) I don't understand: I understand what groups and homomorphisms are. What I don't understand is ...
7
votes
1answer
197 views

Understanding of naive bayes: computing the conditional probabilities

For a task on sentiment analysis, suppose we have some classes represented by $c$ and features $i$. We can represent the conditional probability of each class as: $$P(c | w_i) = \frac{P(w_i|c) \cdot ...
6
votes
1answer
3k views

Why do we need to add START <s> + END </s> symbols when using Recurrent Neural Nets for Sequence-to-Sequence Models?

In the Sequence-to-Sequence models, we often see that the START (e.g. <s>) and END (e.g. </s>) symbols are added to ...