Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [spacy]

Industrial strength Natural Language Processing (NLP) with Python and Cython

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

Choosing the right number of tokens in a dictionary for an LDA topic modelling analysis

I have a data set of customers with n complaints. As a result, I want to perform topic modelling to find topics that customers talk about in their complaints. For this I use LDA from spacy. I have ...
methus's user avatar
  • 111
0 votes
0 answers
6 views

Semantic grouping and replacement of words to improve topic modelling with LDA

I have a data set of customers with complaints. As a result, I want to perform topic modelling to find topics that customers talk about in their complaints. I use LDA for this. In the results of LDA, ...
methus's user avatar
  • 111
0 votes
0 answers
14 views

Best approach to find whether a scientific research paper has human trials/human testing or no

I want to know the best way to know if a paper has human trials/test subjects/testing. Was thinking of searching for some keywords in the paper like "Human", "Trials" etc. The ...
Chaitanya Malhotra's user avatar
1 vote
2 answers
78 views

Unable to do POS (part-of-speech) tagging using Spacy

I managed to load as nlp = spacy.load("en_core_web_sm-3.5.0") but when i run the 2nd line i.e. ...
user155105's user avatar
1 vote
1 answer
135 views

Why is Spacy sentiment score 0.0 for a sentence?

I'm trying to get a sentence's sentiment score using Spacy and apparently every sentence I pass gets a score of 0.0. Can someone help me understand what's going wrong here? ...
Fardeen Khan's user avatar
0 votes
0 answers
20 views

Error in seq2seq translation when passing predicted output to rnn due to input shape not always being the same

I'm working on a language translator and I'm getting an error I'm unsure about. During the decoding process when using argmax on the predicted output I am sometimes getting an RuntimeError ...
bailey.bailey's user avatar
0 votes
1 answer
52 views

What are the approaches for extracting an injury and its description from a paragraph?

Suppose I have a paragraph which explains the injuries and its descriptions. I want to extract the injuries and its corresponding descriptions from the text. How can I do that? For example, the ...
SRJ577's user avatar
  • 197
3 votes
1 answer
822 views

Sentence tokenization for sentence without punctuation

I wish to perform sentence tokenization for sentence without punctuation, below is the code: ...
william007's user avatar
3 votes
1 answer
256 views

SpaCy textcat_multilabel, how to supply data

Just as I was warned, the documentation in SpaCy is a bit difficult to read. I don't have a software-engineer / CS background, so I'm really struggling with this. I would like to use SpaCy's ...
AyamGorengPedes's user avatar
0 votes
0 answers
12 views

Using NLP output for subsequent model?

I use SpaCy to output a vectorized array of my text field. I'm having issues plugging this output into my random forest and could use some guidance. I label encoded other fields so my pandas dataframe ...
user145603's user avatar
1 vote
0 answers
28 views

How to use/vectorize SpaCy output?

I'm performing preprocessing on a dataset that I'm planning on using as a feature set for an RF model. I don't have too much experience with NLP so just want to make sure I'm processing my text fields ...
user145603's user avatar
0 votes
0 answers
96 views

Creating variations of prompts for ChatGPT

I am developing a fine tune model to emulate a tech support chatbot based on my given information. I am struggling to create a large dataset (aiming for 1000 prompt/completion pairs), does anyone have ...
user624's user avatar
0 votes
1 answer
51 views

Who the article is about

I have a problem that I need to solve. It involves articles about football. I have to determine who is the main protagonist in the article. I already have a solution that I have implemented. Its good ...
sohailziahh's user avatar
0 votes
1 answer
89 views

How to Train Q&A model using Bert for multiple comma seperated values in a given data

I'm using the entire text book data by scraping the information of each chapter. How do I highlight the spacy spancat NER or Bert Q&A based models to train multiple comma separated values in the ...
jason's user avatar
  • 53
1 vote
1 answer
462 views

NLP Question: Where can I find a list of all open compound words (with spaces in them), like "peanut butter" or "high school"? [close

I already have a list of "1-gram" words, which include closed compound words like "skyscraper" or "weatherman." However, I'm also interested in compiling a list of "...
Joh's user avatar
  • 13
0 votes
1 answer
139 views

Possible NLP approaches to extract 'goals' from text

I am planning to take up an interesting NLP project. I want to extract 'goal' statements from lengthy reports. For example, the goals can be We would be reducing our carbon footprint by 50% by 2025 or ...
air cooled's user avatar
0 votes
1 answer
67 views

Best approach for rule-based system in multilabel classification-problem?

I’m new to the world of NLP and am looking for some guidance. I want to create a rule-based system that “grades” text in accordance to some set of criteria. For example, one criteria could be “The ...
Incubu121's user avatar
3 votes
1 answer
462 views

Is there a way to map words to their synonyms in tfidf?

I have the following code: ...
james pow's user avatar
  • 167
1 vote
0 answers
75 views

Issue with Relation Annotation (rel.manual) in Spacy's Prodigy tool

I am trying to build a relation extraction model via spacy's prodigy tool. NOTE: ner.manual, ner.correct, rel.manual are all recipes provided by prodigy. (ner.manual, ner.correct) The first step ...
Deepak Sharma's user avatar
0 votes
1 answer
181 views

Named Entity Recognition using Spacy V3 with imbalance entities

Will the spacy V3 model get affected by imbalanced entities? I have got a dataset annotated in spacy format and if I look into my custom entities the rations are different for different entities. For ...
jeevu94's user avatar
  • 103
5 votes
1 answer
426 views

Spacy custom POS tagging for medical concepts

We are a group of doctors trying to use linguistic features of "Spacy", especially the part of speech tagging to show relationships between medical concepts like: 'Femoral artery ...
Sobath Premaratne's user avatar
0 votes
0 answers
528 views

Dealing with near duplicates using NLP

I have a dataframe like as shown below ...
The Great's user avatar
  • 2,565
0 votes
1 answer
146 views

How to train a spaCy language model from scratch?

I am still quite a beginner with spaCy (although I already do enjoy it). I would like to create a language model for a language still unsupported, that is from scratch. I do have comprehensive text ...
Stanislav Koncebovski's user avatar
1 vote
0 answers
146 views

What approach I should take to extract number entity from dataset

I have the training, validation, and test dataset. The first column has store data and the second column has store numbers. I need to develop an entity extractor model which can extract store numbers ...
Rjones's user avatar
  • 11
2 votes
1 answer
1k views

spacy multi label classification help

I would like to create a multilabel text classification algorithm using SpaCy text multi label. I am unable to understand the following questions: How to convert the training data to SpaCy format i....
Nikhil Jaitak's user avatar
3 votes
1 answer
256 views

Compare Books using book categories list NLP

I have a database of books. Each book has a list of categories that describe the genre/topics of the book (I use Python models). The categories in the list most of the time are composed of 1 to 3 ...
Eitan Rosati's user avatar
1 vote
2 answers
707 views

Train a spaCy model for semantic similarity

I'm attempting to train a spaCy model for the purposes of computing semantic similarity but I'm not getting the results I would anticipate. I have created two text files that contain many sentences ...
motormal's user avatar
0 votes
0 answers
96 views

spaCy text classification based on NER

I'm pretty new to spaCy and NLP in general, and I'm trying to figure out how to classify text. I've already gone through quite a few tutorials, and have figured out how to train my model, based on ...
Thomas's user avatar
  • 1
0 votes
1 answer
6k views

How to extract details (educational details, exp details etc.) from a resume?

I am trying to build a resume parser which can extract details such as Name, Address, Education details (degree name, college name, university name, course duration), Experience details (designation, ...
SRJ577's user avatar
  • 197
0 votes
0 answers
398 views

Open Source Annotation Tools for spaCy's Entity Linker?

According to the Tutorial "Training a custom ENTITY LINKING model with spaCy" (20:33) this is the training data format for spaCy's Entity Linker: ...
gzkhv's user avatar
  • 11
3 votes
1 answer
1k views

How to identify/recognize that a sentence about talks about future?

Brief Introduction: I have a report/paragraph in which there are sentences with reference to future plans/outlooks/expectations for a particular entity. I want to extract all such sentences for now. ...
Krs's user avatar
  • 31
2 votes
1 answer
800 views

Converting to lowercase while creating dataset for NER using spacy

I am trying to make a custom entity model for an NER application using spacy. In several NLP projects, I have converted all the data to lowercase and applied several ML techniques. For NER also should ...
Aniiya0978's user avatar
7 votes
1 answer
3k views

Meaning of NER Training values using Spacy

I am trying to train custom entities using Spacy. During the training process I am getting number of values of LOSS, score etc. What is the meaning of these values ...
Aniiya0978's user avatar
1 vote
1 answer
277 views

Why do we calculate the vector of a document by averaging the vectors of all the words?

I am trying to build a search engine to query a folder of documents. Tutorials online suggest that we should obtain the vector of a document by averaging the vectors of all the words, then compare ...
amber's user avatar
  • 51
0 votes
1 answer
114 views

Traning New Entities in Spacy NER Model

I want to add new entities to python spacy NER module. I have few doubts regarding this. Is it possible to remove some of the presently existing entities and add new entities to the remaining ones. ...
Aniiya0978's user avatar
1 vote
0 answers
28 views

Personal Project Classifying Bank Account Data - NLP Noob [closed]

Background: I'm looking to get up to speed with some of the newer ML classification techniques and keep my ML python fresh in my spare time/learn some new skills, so as a challenge I'm trying to see ...
user2065472's user avatar
3 votes
3 answers
4k views

Phone number tagging with spaCy

I have to do a BIO tagging for a given set of sentences. For example: ...
CK23's user avatar
  • 68
4 votes
2 answers
2k views

Does spaCy support multiple GPUs?

I was wondering if spaCy supports multi-GPU via mpi4py? I am currently using spaCy's nlp.pipe for Named Entity Recognition on a high-performance-computing cluster ...
Jinhua Wang's user avatar
0 votes
0 answers
2k views

ValueError: GPU is not accessible. Was the library installed correctly?

I installed spacy 3 in a venv and tried to execute: spacy.require_gpu() Then I got this as output: ...
SRJ577's user avatar
  • 197
1 vote
0 answers
337 views

How can I retrain existing spacy v3 NER model with additional data?

I have trained Spacy v3 model on 1 million rows and want to train for additional 1 million. ...
Rheatey Bash's user avatar
2 votes
1 answer
38 views

Heauristics for a NER model prediction

I am trying to build and NER model that can name entities in a "Job description." The entities are: Mandatory skills (Must have skills like java, python, c++ etc.) Nicetohave skills (the ...
Deepak Sharma's user avatar
1 vote
1 answer
238 views

How can i extract words from a single concatenated word?

I'm stuck on this problem and would love some input. I have mulitple words such as getExtention, getPath, someWord or someword and i want to separate each concatinated words into its own words such as:...
Mosleh Mahamud's user avatar
1 vote
1 answer
103 views

NLP approaches to infer Processes from Text

I would like to use NLP techniques to infer a process out of raw text. For example, if I have a sentence like: Recruitment is about attracting and selecting the right person for the job. To get the ...
Aquiles Páez's user avatar
3 votes
2 answers
1k views

How to make the spacy 3.0 custom NER model training to optimize on precision rather than recall?

My current custom NER model is good on recall but I want to focus on improving precision, is it possible to change the optimizer metric in config file?
alias_neo92's user avatar
2 votes
0 answers
88 views

Spacy dependencymatcher pattern not returning matches

I am trying to create, add and get results from a pattern using spacy DependencyMatcher. I created a pattern for the sentence: "From Monday to Friday" The full pattern: ...
Nicole Douglas's user avatar
0 votes
1 answer
339 views

How to predict the sentiment of the entities form the tweet?

I have a JSON file (tweets.json) that contains tweets (sentences) along with the name of the author. Objective 1: Get the most frequent entities from the tweets. Objective 2: Find out the sentiment/...
coding_ninza's user avatar
3 votes
1 answer
2k views

Testing Spacy NER model

I've trained an NER model with the use of Spacy, and I would like to test the accuracy on a test dataset. What would be the best way to perform this?
Adnos's user avatar
  • 81
1 vote
1 answer
989 views

Extracting Names using NER | Spacy

I'm new to NER and I've been trying to extract names using Spacy. Here's my code: ...
Luke's user avatar
  • 113
3 votes
0 answers
188 views

Medical NER for French language

I'm currently exploring the options to extract medical NER specifically for French language. I tried SpaCy's general French NER but it wasn't helpful to the cause (...
Van Peer's user avatar
  • 285
1 vote
1 answer
112 views

Spacy - Ignored (essential) entities during labeling - how important is this aspect?

I'm training a custom Spacy model. Does it affect the results whether I leave entities unlabeled? For example, Germany is LOC. In one example (1) I label this. In the other example (2) I ignore it and ...
gzkhv's user avatar
  • 11