Skip to main content

Questions tagged [nlp]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation.

Filter by
Sorted by
Tagged with
2 votes
1 answer
88 views

How does Bert masked language modelling task make sense if half the time the next sentence is wrong context in the sequence passed through the encoder

Bert has two types of tasks that it uses to learn contextual word embeddings: Masked word prediction Next sentence prediction I have read the paper and even there the training details are a little ...
figs_and_nuts's user avatar
0 votes
0 answers
75 views

Fine-tuning MT5 for making it more like ChatGPT

I am trying to fine-tune a model which works like ChatGPT for Punjabi language, using the mt5-base, however I am not sure if I should go ahead with it since it does not even generate text and when I ...
Rukaiya Hasan's user avatar
0 votes
0 answers
43 views

F1 and Exact-Match (EM) Score in Extractive QA NLP

I have a question as to how the F1 should be calculated in NLP and whether the text normalization is optional or not. So I have been working on a project where we created a closed-domain extractive QA ...
tt40kiwi's user avatar
  • 101
1 vote
1 answer
76 views

Unsupervised Machine Translation System Using Variational Autoencoder Models

I want to work on an unsupervised machine translation system using a variational autoencoder. I did a literature review but didn't find any related work, and most of the work is based on denoising ...
kartikeya saraswat's user avatar
0 votes
0 answers
15 views

Do ML model measurements and validation standards (e.g. NIST, ISO) exists for the finance, healthcare, and technology industries? Provide citations

Normally, for example, we talk about splitting datasets into training and test datasets. But. The splitting % per train and test sets happens in a subjective manner. Sometimes. The train is 60% or 70%,...
Full Array's user avatar
0 votes
0 answers
14 views

Best approach to find whether a scientific research paper has human trials/human testing or no

I want to know the best way to know if a paper has human trials/test subjects/testing. Was thinking of searching for some keywords in the paper like "Human", "Trials" etc. The ...
Chaitanya Malhotra's user avatar
0 votes
0 answers
110 views

Sentencepiece Tokenizer training from scrath

To train BPE model on sentencepiece as per given Usage instructions As, it is mentioned in the instructions that --input: one-sentence-per-line raw corpus file. ...
Vinay Sharma's user avatar
0 votes
0 answers
8 views

Systematic way of selecting internet texts for a machine translation corpus / dataset?

I am currently working on a neural machine translation project and want to gather a corpus (or dataset) of internet texts that are written in standard and plain language. In theory, it certainly makes ...
tschomacker's user avatar
0 votes
0 answers
33 views

Why did the Double Metaphone algorithm choose to substitute and merge consonants?

Is there any literature describing the decisions behind why the mappings from input text characters to Metaphone hash consonants were made? Why did they choose to leave out vowels? Why did they merge ...
Lance's user avatar
  • 273
0 votes
0 answers
14 views

Highly unbalnced text data giving very low matrics

I have an unbalanced multi-class banking text data with around 76 classes. Classes are badly distributed such as one class which is combination of 240 other different categories, represents 50% of ...
Remrem's user avatar
  • 1
3 votes
2 answers
118 views

Is there a Language Model that can accept huge corpse of tabular data and answr questions about?

I have been researching Language Models that can work with tabular data. My main goal is to have a model to answer simple questions about my data. An example is having household sales data and asking ...
Shahriyar Mammadli's user avatar
0 votes
0 answers
13 views

【NLP】Is there a model or task that determines contextual similarity?

I am trying to work on an engagement detection task in which I have to determine if a student is engaged in class. I am looking for an NLP approach where I can calculate the similarity score of a ...
Leo's user avatar
  • 1
0 votes
0 answers
66 views

Leverage LLMs to classify sentence similarity

This is intended to be mainly a reference request in the vast world of NLP and LLMs. Context A certain protocol is given in the form of text. This can be, for instance, the general description of a ...
Andrea Gagna's user avatar
0 votes
0 answers
25 views

TFRobertaSequenceClassification for Address Normalization task

I have dataset with two column: one with faulty addresses, and other with correct addresses. I want to train a model such that, I can use it later for correcting all the incoming faulty addresses. I ...
learner_account's user avatar
0 votes
1 answer
115 views

Clustering words with similar meanings

What methods are there to cluster words/word phrases with similar meanings together from a list of words/word phrases?
ros's user avatar
  • 1
0 votes
0 answers
66 views

How to improve GPT2 tokenizer trained from scratch?

I trained a GPT2 Tokenizer on Hindi dataset of size 170 MB from scratch and saved it as new_tokenizer. When I tried the new_tokenizer on a Hindi sentence ...
Vinay Sharma's user avatar
0 votes
0 answers
13 views

Expanding Training Data for Intent and Entity Recognition Model

I have a specific use case where I need to identify both intent and entities within a given statement. For example, given the statement "Book train tickets from Mumbai to Delhi," the intent ...
D.Sunil's user avatar
0 votes
0 answers
42 views

How can I avoid the irrelevant number of sentences in the result?

The nature of the data I have is not arranged; however, I'm trying to extract the appropriate sentences for each query as a sample for ground truth. Also, the most critical problem is that I use the ...
Begnnier's user avatar
0 votes
1 answer
49 views

Understanding Multi-headed Attention from architecture details

I've a conceptual question BERT-base has a dimension of 768 for query, key and value and 12 heads (Hidden dimension=768, number of heads=12). The same is conveyed if we see the BERT-base architecture <...
Namburi Srinath's user avatar
1 vote
0 answers
125 views

In rotary positional embeddings (RoPE), why do we not rotate the values as well?

Actually, the question is all there is As per the paper I see that the rotations are applied only to the keys and the queries. Why are the rotations not applied to the values as well? The reasons for ...
figs_and_nuts's user avatar
0 votes
1 answer
222 views

What is the "Extract" token and how is the final Linear layer applied in GPT?

In the manuscript of GPT, the authors have given the following image: Questions: What is the final "Extract" (token?)? Is it the "END" token? How is the final linear layer ...
figs_and_nuts's user avatar
0 votes
0 answers
19 views

Harford 2023 model: explicit reference

I'm citing in a paper the Wizard-Vicuna Uncensored 30B model of Hartford (2023). But I don't have an exact bib reference other than various web links for that model. Could anyone help?
cel's user avatar
  • 1
0 votes
0 answers
26 views

Semantic Scoring and readability for short sentences

I am working on short sentences for NLP based classification. I wish to make a assessment if a sentence is readable before training the system on it. Now readability scores are not working since ...
Vinay Varahabhotla's user avatar
1 vote
0 answers
44 views

Is openAI text generation models an extension of embedding models?

we can creating embeddings using below code ...
Vinay Sharma's user avatar
1 vote
1 answer
272 views

A question about contextual embeddings in the decoder only transformer architecture (gpt)

I am reading up on the decoder only architecture Relevant excerpts: We can use any model that maps token sequences into contextual embeddings (e.g., LSTMs, Transformers): $$\phi : V^L \to R^{d \times ...
figs_and_nuts's user avatar
0 votes
0 answers
848 views

How to read CSV File into Vector Store

I have a CSV file, and I am using langchain to read it into the vector store FAISS. My question is, since I have a CSV file, is RecursiveTextSplitter required? Put differently, consider the following ...
Karl 17302's user avatar
0 votes
1 answer
30 views

How to use location information as feature?

I have a location feature in a dataset. Some examples are: London, Uk; Sheefield Town, Ohio; UK ; North Carolina. etc. How to encode them into features? Is there any word embeddings suitable for such ...
Alex_ban's user avatar
  • 101
0 votes
0 answers
41 views

To extend or not to extend vocabulary for instruction tuning

I want to fine tune a base llm using an instruction dataset. In order to minimize VRAM footprint I want to use SFTTrainer and QLora. My prompt can take the following structure: ...
tog's user avatar
  • 101
1 vote
2 answers
159 views

Training model using BERT

I have generated dataset using chat gpt. Dataset has 9000 data recodes. It's 6 class sentiment analysis. classes are 0,1,2,3,4,5 I used 3000 recodes for training, 1200 recods for validation and ...
Sandun Tharaka's user avatar
2 votes
1 answer
648 views

what is the difference between window size and context length of language model?

is window size and context length of language model one and the same thing? ******** following text is added as question with ONLY above text was not allowed ***** I am trying to understand how GPT ...
Vinay Sharma's user avatar
1 vote
0 answers
96 views

Product name matching - Entity Resolution or Entity Linkage or both?

Context I am at the start of a project where I would like to map/match/link external product names to the respective internal product names. The goal should be to ingest related external information (...
Elodin's user avatar
  • 145
0 votes
1 answer
20 views

Locating base.py when working on Colab

I have faced an error while working with langchain on colab. There is a post on github which recommends changing some configurations in ...
IwillLearn's user avatar
1 vote
1 answer
191 views

A good set of datasets/models for testing an NLP technique

I am a machine learning researcher who up until this point has primarily worked on Computer Vision problems. However, I have an idea for an NLP technique involving a novel Transformer architecture, ...
Anonymous's user avatar
0 votes
1 answer
75 views

Why do we want to maximize the average log probability in neural language models?

I am currently trying to understand the Paragraph Vector framework by reading the paper "Distributed Representation of Sentences and Documents" by Quoc Le and Thomas Mikolov but I have ...
LeeKed's user avatar
  • 1
0 votes
0 answers
89 views

Help understanding working of KeyBERT for keyphrase extraction

I'm fairly new to reading and understanding research papers, so I wanted to get a second opinion on whether my understanding of KeyBERT was correct. Here is a high level overview of my understanding ...
Prithvi's user avatar
  • 33
0 votes
0 answers
308 views

How to get Llama-2 Rotary Embeddings?

I want to get the Llama-2 rotary embeddings. I do print(model) and get the following output: In the picture I highlight the rotary embeddings. How can get the ...
Christian01's user avatar
2 votes
0 answers
173 views

How does fine-tuning work in question answering for custom documents

I am trying to build a Q&A bot for which I have a bunch of documents like articles (specific domain). I understand I can create a Retrieval-Augmented Generation (RAG) system for this, but I want to ...
Glinty's user avatar
  • 33
0 votes
0 answers
172 views

Store and retrieve multiple documents in free vector stores based on critria

I am new to vector stores, and so far experimented with storing 1 file in Faiss and Pinecone. I am looking for tutorials that teach me how to save multiple files in free versions of any vector store, ...
IwillLearn's user avatar
0 votes
0 answers
238 views

Specifying arguments of HuggingFaceHub

In this tutorial, when specifying llm = HuggingFaceHub( repo_id=repo_id, model_kwargs={"temperature": 0.5, "max_length": 64}) only ...
Karl 17302's user avatar
0 votes
0 answers
63 views

Combining Textual, Categorical and Numerical data for Semantic Search using SentenceTransformers model

I'm building a food semantic search model and I want to use a pre-trained SentenceTransformers model with cosine similarity. I'm using Epicurious dataset for the corpus which consists of textual (&...
Alex's user avatar
  • 1
0 votes
0 answers
24 views

How to deal with short text data using NLP models?

Now I want to use my own domain data to train NLP model like BERT. The following is the details of my data: data length distribution: over 70% of my data has the length shorter than 5 and the largest ...
Jackie Shi's user avatar
0 votes
0 answers
17 views

NLP Decision confidence analysis

I have text from people explaining their decision. I want to extract a confidence score of how confident they are in their decision. I am looking for a pre-trained model. I looked up online but I ...
P.Ung's user avatar
  • 1
1 vote
2 answers
542 views

What is source_column argument in csv loader?

In this tutorial, what is the purpose of source_column argument? Does it act like a primary key in Databases? Thanks in advance. ...
Karl 17302's user avatar
0 votes
0 answers
73 views

Finding the suitable semantic search engine tool for querying an excel/csv file containing data rich in numbers

I have an excel dataset that has the columns: name of the test, user name, data source, total number of records, number of failed records, number of passed records, date of test execution etc. I want ...
Apoorva's user avatar
  • 307
1 vote
1 answer
509 views

How to Use Multiple Adapters with a Pretrained Model in Hugging Face Transformers for Inference?

I have a pretrained Llama-2 model in the models_hf directory and two fine-tuned adapters: a summarization adapter in ...
Aun Zaidi's user avatar
1 vote
1 answer
161 views

Why is Spacy sentiment score 0.0 for a sentence?

I'm trying to get a sentence's sentiment score using Spacy and apparently every sentence I pass gets a score of 0.0. Can someone help me understand what's going wrong here? ...
Fardeen Khan's user avatar
1 vote
0 answers
34 views

"text" parameter in pinecone call from langchain

In this tutorial, I do not understand what "text" refers to vectorstore = Pinecone(index, embeddings.embed_query, "text") Could you please help?...
Karl 17302's user avatar
0 votes
0 answers
149 views

Parsing response from llama2

I want to extract phone numbers from a given text and i am prompting a llama2 model for that ..I want the output in form of a list but i am getting unnecessary output like sure here are the phone ...
Debarshi Dasgupta's user avatar
1 vote
0 answers
192 views

How does supervised fine-tuning work in InstructGPT?

See Figure 2 from the InstructGPT paper: I want to know how Step 1 works. Here is one possible algorithm. Pass the prompt through the model, and compute the negative log of the probability of the ...
jskattt797's user avatar
2 votes
1 answer
70 views

Text preprocessing decreases classifier accuracy

I try to solve a binary text classification problem using sklearn's Tfidf Vecotrizer and a naive bayes classifier. Before I pass the training/test data to the vectorizer I do some text preprocessing. ...
MC Racoon's user avatar

1 2 3
4
5
56