Skip to main content

Questions tagged [stanford-nlp]

Stanford NLP is natural language processing research group at Stanford University.

Filter by
Sorted by
Tagged with
1 vote
0 answers
44 views

Is openAI text generation models an extension of embedding models?

we can creating embeddings using below code ...
Vinay Sharma's user avatar
0 votes
0 answers
13 views

stanford NER breaks a phrase into two for NER

I'm using Stanford NER on text and find out it breaks my phrase into two. Is it possible to control the granularity of the splitting of words? For example, the text ...
Student's user avatar
  • 411
1 vote
0 answers
57 views

Understanding Syntactic divergence

I am trying to read a paper https://arxiv.org/abs/2004.14444 Section 6.1 of the paper describes Syntactic divergence. I have confusion regarding the distribution graphs and split of the dataset. If a ...
jon's user avatar
  • 11
1 vote
0 answers
27 views

Recommendations of NLP for classifying sentance into tense forms

I have a dataset of tweets. I have to classify each tweet into it's tense forms like whether it's about past, present or future. So for that can you please recommend any pretrained NLP model or method ...
Samar Pratap Singh's user avatar
1 vote
1 answer
100 views

weights of coocurrence matrix in glove

I was studying the theory behind glove and was checking out some implementations of it. Before passing the data to its neural networks, I noticed that the weights of the co-occurrence matrix aren't ...
mihael's user avatar
  • 121
1 vote
1 answer
86 views

Reversing a dependency tree into the original sentence

I'm wondering if it is possible to convert a dependency parser such as ...
Carlos Vázquez Monzón's user avatar
1 vote
1 answer
436 views

How to evaluate triple extraction in NLP?

I am current NLP work, I am extracting triples using triple extraction function in Stanford NLP and Spacy libraries. I am looking for a good method to evaluate how good the extraction has been? Any ...
SageMaker's user avatar
  • 185
1 vote
0 answers
113 views

How to extract numerical information from text descriptions

I have an attribute that is the description of an operation (i.e description of a building consent), I need to translate this to a mathematical operation. I need to find out the new number of dwelling ...
Mohsen Sichani's user avatar
1 vote
0 answers
36 views

GloVe dot product optimized for non-comutative data whilst the operation itself being commutative

To my current knowledge, GloVe word vectors dot product are optimized to be the w_i ⋅ w_j = log⁡(P(ⅈ|j)) The probability being computed from a cooccurance matrix. However, dot product is a commutative ...
Arik's user avatar
  • 23
-1 votes
2 answers
57 views

How do I go for NLP based on phrases instead of sentences? [closed]

I have a list of words in this format: chem, chemistry chemi, chemistry chm, chemistry chmstry, chemistry Here, the first column represents the actual word which ...
girl101's user avatar
  • 1,161
0 votes
2 answers
309 views

How to stay up to date in NLP and use the best approaches?

There are many fast advancements in NLP field, BERT, RoBERTa, ALBERT, and XLNe, and no one can check the news or papers daily. Is there any way or site that keeps track of all these new developments ...
Thomas Lee's user avatar
0 votes
0 answers
29 views

Restrict Date parser in certain cases

Sorry if the title wasn't self-explanatory. Here is a detailed version. I created a data parser to parse dates from resumes. The ultimate goal is to find how many years of work experience a candidate ...
Deepak Sharma's user avatar
1 vote
1 answer
406 views

Trying to compress text with NLP

For a university project, I need to send text in Spanish via SMS. As these have a cost, I am trying to compress this text in an inefficient way. This consists of first generating a permutation of ...
Fmkit's user avatar
  • 11
0 votes
1 answer
339 views

How to predict the sentiment of the entities form the tweet?

I have a JSON file (tweets.json) that contains tweets (sentences) along with the name of the author. Objective 1: Get the most frequent entities from the tweets. Objective 2: Find out the sentiment/...
coding_ninza's user avatar
3 votes
1 answer
1k views

How to i get word embeddings for out of vocabulary words using a transformer model?

When i tried to get word embeddings of a sentence using bio_clinical bert, for a sentence of 8 words i am getting 11 token ids(+start and end) because "embeddings" is an out of vocabulary ...
cerofrais's user avatar
  • 131
2 votes
1 answer
53 views

What is the more natural parsing, the one that leads to the preferred reading of the sentence

I have those rules: and those two possible parse trees: I am asked for the next question: What is the more natural parsing, the one that leads to the preferred reading of the sentence? Can anyone ...
Ilya.K.'s user avatar
  • 157
1 vote
0 answers
64 views

For an n-Gram model with n>2, do we need more context at end of each sentence?

Jurafsky's book says we need to add context to left and right of a sentence: Does this mean, for example, if we've a corpus of three sentences: ...
KGhatak's user avatar
  • 123
4 votes
1 answer
472 views

In smoothing of n-gram model in NLP, why don't we consider start and end of sentence tokens?

When learning Add-1 smoothing, I found that somehow we are adding 1 to each word in our vocabulary, but not considering start-of-sentence and end-of-sentence as two words in the vocabulary. Let me ...
KGhatak's user avatar
  • 123
1 vote
1 answer
3k views

TF-IDF for Topic Modeling

Can TF-IDF be used a sole method for Topic Modeling ? (I know there are better methods like LDA , LSA etc) I just want to understand if TF-IDF alone can help us in Topic modeling . If yes , can ...
Bharathi's user avatar
  • 277
0 votes
1 answer
270 views

Closed Domain Question Answering which doesn't answer Questions

I've been exploring Closed Domain Question Answering Implementations which have been trained on SQuAD 2.0 dataset. Ideally, it should not answer questions which the context text corpus doesn't contain ...
Anirban Saha's user avatar
0 votes
2 answers
515 views

How to detect medicine name from the medicine wrapper

I have got medicine wrapper ( Packaging ) of different medicines. I want to detect medicine name out of it. I'm using Google Cloud Vision to extract all the text from the medicine wrapper. Text ...
Åbħísħêk Đêsħkăr's user avatar
0 votes
1 answer
61 views

Why is n-grams language independent?

I don't understand how n-grams are language independent. I've read that by using character n-grams of a word than the word itself as dimensions of a vector space model, we can skip the language-...
Bharathi's user avatar
  • 277
0 votes
1 answer
41 views

How can I extract the reason of the legal compensation from a court report?

I'm working on a project (court-related). At a certain point, I have to extract the reason of the legal compensation. For instance, let's take these sentences (from a court report) Order mister X to ...
Omar Souaidi's user avatar
2 votes
1 answer
347 views

Why is T test reweighting on a word X word co-occurrence matrix so effective?

I am going through Stanford NLP class: http://web.stanford.edu/class/cs224u/ A task in the homework is to implement T-test reweighting on a word X word co-occurrence matrix: https://nbviewer.jupyter....
silkAdmin's user avatar
  • 143
1 vote
1 answer
158 views

What dataset was Stanford NER trained on?

I would like to re-train the Stanford NER library from scratch as a 1 class model. Only 3,4 and 7 class models are available out of the box. Is it possible to obtain the data that the model was ...
tim_xyz's user avatar
  • 187
2 votes
1 answer
2k views

what is sentence embeding and how to do sentence embedding for a sentence and how to use word embedding to create a sentence embedding?

What is sentence embedding? How would you do sentence embedding for a sentence like: "How old are you?" How do you use word embedding to create a sentence ...
star's user avatar
  • 1,471
2 votes
1 answer
2k views

n-gram Model - Why Smoothing?

I am creating an n-gram model that will predict the next word after an n-gram (probably unigram, bigram and trigram) as coursework. I have seen lots of explanations about HOW to deal with zero ...
Chris's user avatar
  • 121
1 vote
1 answer
41 views

Extracting information with corresponding fields

I have large pool of scanned county documents. I need to extract information like document title, borrower name&address, lender name&address etc. The text is like this Eg: the deed of trust,...
user10351235's user avatar
7 votes
2 answers
4k views

Lemmatization Vs Stemming

I have been reading about both these techniques to find the root of the word, but how do we prefer one to the other? Is "Lemmatization" always better than "Stemming"?
ashirwad's user avatar
0 votes
1 answer
735 views

Using the Stanford Named Entity Tagger in R [closed]

I am experimenting with the Stanford Named Entity Tagger here http://nlp.stanford.edu:8080/ner/process and I feel it would be useful in my research. Does anyone know of a example that I could follow ...
Stephen Clark's user avatar
1 vote
0 answers
609 views

Glove supported languages

Recently I started reading more about NLP and following tutorials in Python in order to learn more about the subject. I started experimenting with words embeddings also, and I found some interesting ...
moz_szt's user avatar
  • 75
1 vote
1 answer
105 views

How to extract the positions of employee from raw text [closed]

I have raw text like "Mr John Fullerton is Chief Executive Officer and Managing Director of Australian Rail Track Corporation Ltd, and was appointed to the position in February 2011." I easily ...
Jay Pratap Pandey's user avatar
5 votes
1 answer
9k views

Interpretation of the loss function for word2vec

I am trying to understand the loss function which is used for the word2vec model, but I don't really follow the argumentation behind this video https://www.youtube.com/watch?v=ERibwqs9p38&t=5s, at ...
shaft's user avatar
  • 165
1 vote
0 answers
99 views

How does Stanford CRF encode NER string features?

Most features created by the NERFeatureFactory are strings e.g. from usePrev, useNext, ...
maxbeaudoin's user avatar
5 votes
1 answer
6k views

NLP - How to perform semantic analysis?

I'd like to perform a textual/sentiment analysis. I was able to analyse samples with 3 labels: (positive, neutral, negative) and I used algorithms such as SVM, ...
GG24's user avatar
  • 153
0 votes
1 answer
1k views

ImportError: cannot import name 'StanfordCoreNLPParser'

I've been trying to extract subject-predicate-object triples from sentences and found this awesome API that did just that. However, when it was written, it used the ...
mathdeviant's user avatar
1 vote
1 answer
2k views

Why would you use word embeddings to find similar words?

One of the applications of word embeddings (such as GloVe) is finding words of similar meaning. I just had a look at some embeddings produced by glove on large datasets and I found that the nearest ...
gen's user avatar
  • 95
2 votes
3 answers
2k views

Is there any named entity reconginition algorithm trained for the french language?

I am trying to implement a utility for my mobile application to perform some actions based on user questions. I need an algorithm to extract named entities from a text string (French grammar). I have ...
Karim Manaouil's user avatar
1 vote
2 answers
3k views

Entity Recognition in Stanford NLP using Python

I am using Stanford Core NLP using Python. I have taken the code from here. This is the code: ...
idpd15's user avatar
  • 161
4 votes
1 answer
47 views

Correcting ALL CAPS for human and algorithmic consumption

United States federal tax returns tend to be written in ALL CAPS to facilitate OCR. This practice has persisted even when returns are filed electronically. Thus, much of the text in the IRS 990 ...
David Bruce Borenstein's user avatar
2 votes
0 answers
264 views

Extracting date, relation and noun phrase from text

A sentence (Segmented from a document) as below: ...
Mohammed Ashiq's user avatar
2 votes
0 answers
203 views

NER at sentence level or document level?

Should NER models (LSTM or CRF) take input training data at sentence level or paragraph level? Let's say we have this input text, and we would like to do Named Entity Extraction: GOP Sen. Rand ...
Franklin Dong's user avatar
1 vote
2 answers
1k views

How to extract entities from text using existing ontologies?

I am working on a entity extraction task and I am using Stanford CoreNLP NER. Here, I want to detect entities of type "Animal", "Building", "Imagery", etc., which are not covered in Stanford CoreNLP ...
Swastik Roy's user avatar
2 votes
0 answers
22 views

How are dynamic memory networks employed in sequence to sequence modelling

Dynamic Memory networks are described here . I understand what is going on for question answering task but when it comes to sequence to sequence modeling, they describe it in 4th paragraph of 2.4 ...
figs_and_nuts's user avatar
1 vote
1 answer
2k views

What is a lower bound on the vocabulary size for generating word/sentence embedding using word2vec or skip thought vectors?

I am working on a NLP related task. I have about 150 documents, each few pages long (5/6 pages long on average). After removing stopwords and other unnecessary symbols and digits, I have about 104,000 ...
user62198's user avatar
  • 1,091
1 vote
0 answers
194 views

In practice, is relation extraction or relationship extraction the correct term?

I ask from the practitioner point-of-view, and I hope the answer does not come down to nit-picking, but I would like to settle the matter once and for all in the work I do. One of the components of ...
demongolem's user avatar
3 votes
2 answers
4k views

StanfordTokenizer will be deprecated in version 3.2.5 Warning

I was testing the StanfordNERTagger using the NLTK wrapper and this warning appeared: ...
Anoroah's user avatar
  • 153
1 vote
0 answers
135 views

Chunker/shallow parser for spoken language

I'm trying to extract NPs from transcribed spoken text, such as um it's the bl- it's the blue one in the right no left hand corner which contains e.g. fillers (e.g. um) and disfluencies (e.g. bl-,...
errantlinguist's user avatar
15 votes
2 answers
12k views

Why should the initialization of weights and bias be chosen around 0?

I read this: To train our neural network, we will initialize each parameter W(l)ijWij(l) and each b(l)ibi(l) to a small random value near zero (say according to a Normal(0,ϵ2)Normal(0,ϵ2) ...
cinqS's user avatar
  • 367
1 vote
2 answers
3k views

Agglomerative Clustering without knowing number of clusters

I want to perform agglomerative clustering, but I have no idea of number of clusters before hand. But I want that every cluster has at least 40 data points in it. How can I apply this to sklearn....
Nshn's user avatar
  • 71