Questions tagged [spacy]

Industrial strength Natural Language Processing (NLP) with Python and Cython

Filter by
Sorted by
Tagged with
5
votes
1answer
6k views

How to train a spacy model for text classification?

Can i know the way or steps to train a spacy model for text classification. (binary classification in my case) Please help me with the process and way to approach.
5
votes
1answer
4k views

Is there any way to define custom entities in Spacy

1) I have just started working on NLP the basic Idea is to extract meaningful information from text. For this I am using "Spacy". As far as I have studied Spacy has following entities. ORG PERSON ...
5
votes
2answers
604 views

Converting paragraphs into sentences

I'm looking for ways to extract sentences from paragraphs of text containing different types of punctuations and all. I used SpaCy's ...
5
votes
3answers
122 views

Chunking Sentences with Spacy

I have a lot of sentences (500k) which looks like this: ...
4
votes
1answer
150 views

Does spaCy support multiple GPUs?

I was wondering if spaCy supports multi-GPU via mpi4py? I am currently using spaCy's nlp.pipe for Named Entity Recognition on a high-performance-computing cluster ...
4
votes
2answers
269 views

Is NLP suitable for my legal contract parsing problem?

My company has a product that involves the extraction of a variety of fields from legal contract PDFs. The current approach is very time consuming and messy, and I am exploring if NLP is a suitable ...
3
votes
1answer
3k views

Address parsing using spaCy

I am trying to parse addresses from various documents using spaCy using NER but the results are not so accurate. I know this is bit generic question but it would be a great help if I could get ...
3
votes
2answers
235 views

How to make the spacy 3.0 custom NER model training to optimize on precision rather than recall?

My current custom NER model is good on recall but I want to focus on improving precision, is it possible to change the optimizer metric in config file?
3
votes
1answer
126 views

Smart sentence segmentation not splitting on abbreviations

Sentencer from SpaCy and NLTK does not catch the fact that typical abbreviations (e.g. Mio. for Million in German) and the ...
3
votes
1answer
754 views

SpaCy vs AllenNLP?

I have used a little of both spaCy and allenNLP in my NLP projects. I like them both as they work very well with PyTorch (my DL framework choice!). But, I still cannot decide which one to master in a ...
3
votes
1answer
965 views

Spacy word embeddings for sentence

Spacy offers pre-trained vectors for words. However I have notices that you can get vectors for sentences too: spacy_nlp('hello I').has_vector == True However I ...
3
votes
0answers
59 views

Medical NER for French language

I'm currently exploring the options to extract medical NER specifically for French language. I tried SpaCy's general French NER but it wasn't helpful to the cause (...
3
votes
1answer
255 views

Character-level embeddings in python

I'm working on an NLP task that requires the use of character level embeddings, and I've been trying to use Spacy. However, it seems that spacy uses word-level embeddings for the word vectors, and I ...
3
votes
0answers
46 views

Change the way spacy works - Custom properties for training and prediction

Spacy detects the entities using its predefined algorithm. It parses tokens in text considering position of tokens with respect to tokens surrounding it. It also takes into consideration the POS ...
3
votes
1answer
2k views

Can not install spacy package on windows 10 via pip

I have below environment. OS: Windows 10 Python: Python 3.7.4 PIP: pip 19.3.1 I am trying to install spacy in my windows 10 OS. It gives me below error. ...
2
votes
1answer
686 views

Twitter POS and NER: What is state-of-the-art?

What is the current state-of-the-art for pos tagging and named entity recognition for twitter data? Are industrial-strength programs like Spacy and ...
2
votes
1answer
176 views

Spacy tokenizer that strips HTML/XML keeping the positions

I'm new to Spacy and infact new to Data Science. I would like to process some XML files for NER and then mark that in the original XML. I would like to know how to tokenize the XML for the NER. I ...
2
votes
1answer
67 views

How to go about training a NER model to extract book citations in free-form?

I'm doing a project where I wish to create a graph visualization of free-form citations (not academic style citations) across all my e-books. E.g. David Foster Wallace's essays cite a lot of other ...
2
votes
1answer
337 views

Detect passive voice in headlines

To detect passive voice in sentences, we can use the spacy module to tag each token in the sentence, then build a classifier to classify it as passive or active based on conventional grammatical rules,...
2
votes
1answer
223 views

Difficulty interpreting word embedding vector similarity (spaCy)

I calculate vector similarities like this: ...
2
votes
1answer
132 views

How can I extract numerical information conditional on other information?

Given text data, I'd like to extract numerical information based on the presence of other information. For instance, "There are 119 people in the gym right now. 100 people in the weight room, 19 on ...
2
votes
1answer
24 views

Heauristics for a NER model prediction

I am trying to build and NER model that can name entities in a "Job description." The entities are: Mandatory skills (Must have skills like java, python, c++ etc.) Nicetohave skills (the ...
2
votes
0answers
26 views

Spacy dependencymatcher pattern not returning matches

I am trying to create, add and get results from a pattern using spacy DependencyMatcher. I created a pattern for the sentence: "From Monday to Friday" The full pattern: ...
2
votes
1answer
123 views

Testing Spacy NER model

I've trained an NER model with the use of Spacy, and I would like to test the accuracy on a test dataset. What would be the best way to perform this?
2
votes
0answers
384 views

improve NER model accuracy with spaCy dependency tree

I have search at lot, was not able to find a solution for my problem... I am training a NER model, that should detect two types of words: Instructions and Conditions. This is not the standard use-case ...
1
vote
1answer
40 views

How can i extract words from a single concatenated word?

I'm stuck on this problem and would love some input. I have mulitple words such as getExtention, getPath, someWord or someword and i want to separate each concatinated words into its own words such as:...
1
vote
1answer
77 views

NLP approaches to infer Processes from Text

I would like to use NLP techniques to infer a process out of raw text. For example, if I have a sentence like: Recruitment is about attracting and selecting the right person for the job. To get the ...
1
vote
1answer
120 views

SpaCy string store

I am trying to get the hash values for some English words from the string store of SpaCy. ...
1
vote
1answer
137 views

Extracting Names using NER | Spacy

I'm new to NER and I've been trying to extract names using Spacy. Here's my code: ...
1
vote
1answer
32 views

Spacy - Ignored (essential) entities during labeling - how important is this aspect?

I'm training a custom Spacy model. Does it affect the results whether I leave entities unlabeled? For example, Germany is LOC. In one example (1) I label this. In the other example (2) I ignore it and ...
1
vote
2answers
44 views

How do you distinguish between conversational text and possible news article?

Context When you receive messages in group chats, how do you detect if that message belongs to conversational dialogue or if it is a 'news' article (could be fake or real) that they are sharing? ...
1
vote
1answer
64 views

Organization finder in spaCy

I am trying to find the parties between which the agreement is executed from the statement below. AGREEMENT, dated as of January 10, 2000, is entered into by and between ABC-EFG GROUP Inc. having an ...
1
vote
1answer
1k views

How to replace words in a sentence with their POS tag generated with SpaCy efficiently?

How is it possible to replace words in a sentence with their respective PoS tags generated with SpaCy in an efficient way?
1
vote
0answers
14 views

Adding additional classes in stanford NLP NER or Spacy

For stanford NER 3 class model, Location, Person, Organization recognizers are available. Is it possible to add additional classes to this model. For example : Sports as one class to tag sports names. ...
1
vote
0answers
17 views

Personal Project Classifying Bank Account Data - NLP Noob [closed]

Background: I'm looking to get up to speed with some of the newer ML classification techniques and keep my ML python fresh in my spare time/learn some new skills, so as a challenge I'm trying to see ...
1
vote
1answer
112 views

Phone number tagging with spaCy

I have to do a BIO tagging for a given set of sentences. For example: ...
1
vote
1answer
1k views

Spacy v2.0.1 custom NER: How to improve training of existing model

I implemented custom NER with bellow trained data first time and it gives me good prediction with Name and PrdName. I mentioned code bellow. ...
1
vote
1answer
552 views

Sentiment Analysis of News Headlines

I'm trying to do sentiment analysis of News Headlines about a particular subject mentioned in it. Initially, I used TextBlob library for sentiment analysis to ...
0
votes
1answer
45 views

Meaning of NER Training values using Spacy

I am trying to train custom entities using Spacy. During the training process I am getting number of values of LOSS, score etc. What is the meaning of these values ...
0
votes
1answer
27 views

How to extract details (educational details, exp details etc.) from a resume?

I am trying to build a resume parser which can extract details such as Name, Address, Education details (degree name, college name, university name, course duration), Experience details (designation, ...
0
votes
1answer
29 views

Why do we calculate the vector of a document by averaging the vectors of all the words?

I am trying to build a search engine to query a folder of documents. Tutorials online suggest that we should obtain the vector of a document by averaging the vectors of all the words, then compare ...
0
votes
1answer
120 views

Text Classification on a very small data set with a lot of classes

I have a data set consisting of 455 rows spread over 21 different classes. The data set is imbalanced as well as you can see below. ...
0
votes
1answer
301 views

spaCy - Text Preprocessing - Keeping "Pronouns" in text

I am fairly new to machine learning and NLP in general. I am trying to wrap my head around how to do proper text pre-processing (cleaning the text). I have built a custom text classification model. I ...
0
votes
1answer
311 views

What should return doc.ents if the doc have no entities, in spacy?

I want to answer this question: "How many sentences contain named entities given a doc?" and I have this piece of code as solution ...
0
votes
1answer
98 views

Where does spacy, nltk, prodigy, sklearn fit in the AI project?

Where does tools like spacy, sklearn, prodigy, nltk fit in the below AI project architecture and what are some common alternatives to these:
0
votes
2answers
238 views

How to obtain the word vectors optimally

I have a list of strings as shown sent_list = ["Carrefour is in France", "Apple pie is delicious", "Amazon has just delivered", ...] My code to get word ...
0
votes
1answer
255 views

NLP techniques to label unlabeled data in a dataset [closed]

So I have a .xls file with negative and neutral reviews of a medicine. However, this dataset does not have labels. I converted this .xls into a dataframe and I am using the Spacy Lib. I have to use ...
0
votes
2answers
612 views

Custom POS tags with SpaCy for NER

Quite new to NLP and especially NER. I'm trying to train a NER model on a custom dataset. This is a dataset of houses for sale. As part of the entities I'm training the model to extract are reference ...
0
votes
1answer
368 views

Will the performance of my NER model improve?

I am training a spacy model from scratch by creating a dataset of my own with format spacy needs it to be in, the model is an NER model and the entity i am trying to recognize is Food items. I have ...
0
votes
1answer
2k views

Facing this issue while predicting "CountVectorizer - Vocabulary wasn't fitted"

#these are classifier and vectorizer vectorizer = CountVectorizer(tokenizer = spacy_tokenizer, ngram_range=(1,1)) classifier = LinearSVC() I have created a ...