Questions tagged [nlp]
Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation.
2,688
questions
0
votes
0
answers
12
views
Seeking datasets for training a Language Model on U.S. mortgage loan processes
I'm in the process of training a Language Model (LLM) and require datasets that encompass various aspects of the U.S. mortgage loan process. The model's aim is to understand and simulate decision-...
- 1
0
votes
0
answers
12
views
Prefix tuning in LLM uses learnable vectors to fine tune the model
I would like to implement a new architecture for Transformer.
Below description is my thought.
Prefix tuning in LLM uses learnable vectors to fine tune the model.
Is there a way to use the output ...
- 25
0
votes
0
answers
29
views
Some questions from a noob - Chat bot
I'm a T-SQL and .NET developer but I need to understand some basis about a data science / ml project.
I would like to know the guidelines and be directed to the right topics about the steps that need ...
0
votes
0
answers
17
views
Fine-tune zero-shot classification model multi-label
I started a small project where I am trying to fine-tune a zero-shot classification model on a proprietary dataset. I was thinking to use the NLI approach, building contradiction and entailment ...
0
votes
1
answer
35
views
Semantic Search on numeric data
I have a dataset in csv format. Most columns have numbers in them. The data is mostly number based, with little text.
A sample of my dataset:
I want to build a semantic search engine that can answer ...
- 275
0
votes
1
answer
38
views
Purely extractive Language Model
Given an email thread, I am trying to extract the body of the most recent email.
I used to do that with rules. Now I am testing Large Language Models (LLM) to see if I they provide a less ad hoc ...
- 103
0
votes
0
answers
14
views
Performing Multi label text classification
I have a text and it's class, so I have performed single text classification, but now I want to train a multi label classifier, so I tried combining sentence to form a multi label dataset, but the ...
0
votes
0
answers
14
views
NLP approach for classifying webscraped data
I have a challenge in a project of mine where I will be provided with a list of scraped datas from a website. Along with the data i will also be provided some parameters like the tag of scraped ...
- 1
0
votes
0
answers
13
views
Get accurate alias list of people without information on aliases
I'm working on a project where I have a large dataframe of paintings (from the Art500k dataset), each row corresponds with a painting, containing the author's name in the ...
- 13
0
votes
1
answer
19
views
What is the input to an encoder-decoder transformer in next word prediction task?
I'm trying to understand how encoder-decoder architectures are used, or if they are used at all, for generative tasks that do not require an explicit prompt (ie. machine translation, summarization, ...
- 103
0
votes
0
answers
6
views
Is there a tool to do "next speech prediction"?
Given the enormous success of next-token prediction, I was wondering if there are any successful models that try to apply the same principle on speech, i.e., the analogue audio of speech rather than ...
- 129
1
vote
1
answer
50
views
How does Bert masked language modelling task make sense if half the time the next sentence is wrong context in the sequence passed through the encoder
Bert has two types of tasks that it uses to learn contextual word embeddings:
Masked word prediction
Next sentence prediction
I have read the paper and even there the training details are a little ...
- 821
0
votes
0
answers
26
views
Fine-tuning MT5 for making it more like ChatGPT
I am trying to fine-tune a model which works like ChatGPT for Punjabi language, using the mt5-base, however I am not sure if I should go ahead with it since it does not even generate text and when I ...
0
votes
0
answers
10
views
F1 and Exact-Match (EM) Score in Extractive QA NLP
I have a question as to how the F1 should be calculated in NLP and whether the text normalization is optional or not.
So I have been working on a project where we created a closed-domain extractive QA ...
- 101
1
vote
1
answer
46
views
Unsupervised Machine Translation System Using Variational Autoencoder Models
I want to work on an unsupervised machine translation system using a variational autoencoder. I did a literature review but didn't find any related work, and most of the work is based on denoising ...
0
votes
0
answers
13
views
Do ML model measurements and validation standards (e.g. NIST, ISO) exists for the finance, healthcare, and technology industries? Provide citations
Normally, for example, we talk about splitting datasets into training and test datasets. But. The splitting % per train and test sets happens in a subjective manner. Sometimes. The train is 60% or 70%,...
- 156
0
votes
0
answers
14
views
Best approach to find whether a scientific research paper has human trials/human testing or no
I want to know the best way to know if a paper has human trials/test subjects/testing. Was thinking of searching for some keywords in the paper like "Human", "Trials" etc.
The ...
0
votes
0
answers
25
views
Sentencepiece Tokenizer training from scrath
To train BPE model on sentencepiece as per given Usage instructions
As, it is mentioned in the instructions that --input: one-sentence-per-line raw corpus file.
...
- 167
0
votes
0
answers
7
views
Systematic way of selecting internet texts for a machine translation corpus / dataset?
I am currently working on a neural machine translation project and want to gather a corpus (or dataset) of internet texts that are written in standard and plain language. In theory, it certainly makes ...
- 101
0
votes
0
answers
18
views
Why did the Double Metaphone algorithm choose to substitute and merge consonants?
Is there any literature describing the decisions behind why the mappings from input text characters to Metaphone hash consonants were made? Why did they choose to leave out vowels? Why did they merge ...
- 263
0
votes
0
answers
10
views
Highly unbalnced text data giving very low matrics
I have an unbalanced multi-class banking text data with around 76 classes. Classes are badly distributed such as one class which is combination of 240 other different categories, represents 50% of ...
- 1
3
votes
2
answers
85
views
Is there a Language Model that can accept huge corpse of tabular data and answr questions about?
I have been researching Language Models that can work with tabular data. My main goal is to have a model to answer simple questions about my data. An example is having household sales data and asking ...
- 1,128
0
votes
0
answers
9
views
【NLP】Is there a model or task that determines contextual similarity?
I am trying to work on an engagement detection task in which I have to determine if a student is engaged in class.
I am looking for an NLP approach where I can calculate the similarity score of a ...
- 1
0
votes
0
answers
18
views
Leverage LLMs to classify sentence similarity
This is intended to be mainly a reference request in the vast world of NLP and LLMs.
Context
A certain protocol is given in the form of text. This can be, for instance, the general description of a ...
- 101
0
votes
0
answers
16
views
TFRobertaSequenceClassification for Address Normalization task
I have dataset with two column: one with faulty addresses, and other with correct addresses. I want to train a model such that, I can use it later for correcting all the incoming faulty addresses.
I ...
0
votes
1
answer
22
views
Clustering words with similar meanings
What methods are there to cluster words/word phrases with similar meanings together from a list of words/word phrases?
- 1
0
votes
0
answers
28
views
How to improve GPT2 tokenizer trained from scratch?
I trained a GPT2 Tokenizer on Hindi dataset of size 170 MB from scratch and saved it as new_tokenizer. When I tried the new_tokenizer on a Hindi sentence
...
- 167
0
votes
0
answers
10
views
Expanding Training Data for Intent and Entity Recognition Model
I have a specific use case where I need to identify both intent and entities within a given statement. For example, given the statement "Book train tickets from Mumbai to Delhi," the intent ...
- 1
0
votes
0
answers
33
views
How can I avoid the irrelevant number of sentences in the result?
The nature of the data I have is not arranged; however, I'm trying to extract the appropriate sentences for each query as a sample for ground truth. Also, the most critical problem is that I use the ...
- 17
0
votes
1
answer
37
views
Understanding Multi-headed Attention from architecture details
I've a conceptual question
BERT-base has a dimension of 768 for query, key and value and 12 heads (Hidden dimension=768, number of heads=12). The same is conveyed if we see the BERT-base architecture
<...
1
vote
0
answers
35
views
In rotary positional embeddings (RoPE), why do we not rotate the values as well?
Actually, the question is all there is
As per the paper I see that the rotations are applied only to the keys and the queries. Why are the rotations not applied to the values as well?
The reasons for ...
- 821
0
votes
1
answer
62
views
What is the "Extract" token and how is the final Linear layer applied in GPT?
In the manuscript of GPT, the authors have given the following image:
Questions:
What is the final "Extract" (token?)? Is it the "END" token?
How is the final linear layer ...
- 821
0
votes
0
answers
19
views
Harford 2023 model: explicit reference
I'm citing in a paper the Wizard-Vicuna Uncensored 30B model of Hartford (2023). But I don't have an exact bib reference other than various web links for that model. Could anyone help?
- 1
0
votes
0
answers
17
views
Semantic Scoring and readability for short sentences
I am working on short sentences for NLP based classification. I wish to make a assessment if a sentence is readable before training the system on it. Now readability scores are not working since ...
0
votes
0
answers
26
views
Is openAI text generation models an extension of embedding models?
we can creating embeddings using below code
...
- 167
1
vote
1
answer
58
views
A question about contextual embeddings in the decoder only transformer architecture (gpt)
I am reading up on the decoder only architecture
Relevant excerpts:
We can use any model that maps token sequences into contextual embeddings (e.g., LSTMs, Transformers):
$$\phi : V^L \to R^{d \times ...
- 821
0
votes
0
answers
202
views
How to read CSV File into Vector Store
I have a CSV file, and I am using langchain to read it into the vector store FAISS. My question is, since I have a CSV file, is RecursiveTextSplitter required? Put differently, consider the following ...
0
votes
1
answer
25
views
How to use location information as feature?
I have a location feature in a dataset. Some examples are: London, Uk; Sheefield Town, Ohio; UK ; North Carolina. etc. How to encode them into features? Is there any word embeddings suitable for such ...
- 101
0
votes
0
answers
12
views
To extend or not to extend vocabulary for instruction tuning
I want to fine tune a base llm using an instruction dataset. In order to minimize VRAM footprint I want to use SFTTrainer and QLora. My prompt can take the following structure:
...
- 101
1
vote
2
answers
88
views
Training model using BERT
I have generated dataset using chat gpt. Dataset has 9000 data recodes. It's 6 class sentiment analysis. classes are 0,1,2,3,4,5
I used 3000 recodes for training, 1200 recods for validation and ...
1
vote
1
answer
141
views
what is the difference between window size and context length of language model?
is window size and context length of language model one and the same thing?
******** following text is added as question with ONLY above text was not allowed *****
I am trying to understand how GPT ...
- 167
1
vote
0
answers
46
views
Product name matching - Entity Resolution or Entity Linkage or both?
Context
I am at the start of a project where I would like to map/match/link external product names to the respective internal product names. The goal should be to ingest related external information (...
- 145
0
votes
1
answer
17
views
Locating base.py when working on Colab
I have faced an error while working with langchain on colab. There is a post on github which recommends changing some configurations in
...
- 103
1
vote
1
answer
25
views
A good set of datasets/models for testing an NLP technique
I am a machine learning researcher who up until this point has primarily worked on Computer Vision problems. However, I have an idea for an NLP technique involving a novel Transformer architecture, ...
- 13
0
votes
1
answer
40
views
Why do we want to maximize the average log probability in neural language models?
I am currently trying to understand the Paragraph Vector framework by reading the paper "Distributed Representation of Sentences and Documents" by Quoc Le and Thomas Mikolov but I have ...
- 1
0
votes
0
answers
39
views
Help understanding working of KeyBERT for keyphrase extraction
I'm fairly new to reading and understanding research papers, so I wanted to get a second opinion on whether my understanding of KeyBERT was correct. Here is a high level overview of my understanding ...
- 33
0
votes
0
answers
208
views
How to get Llama-2 Rotary Embeddings?
I want to get the Llama-2 rotary embeddings. I do print(model) and get the following output:
In the picture I highlight the rotary embeddings.
How can get the ...
- 101
1
vote
0
answers
52
views
How does fine-tuning work in question answering for custom documents
I am trying to build a Q&A bot for which I have a bunch of documents like articles (specific domain).
I understand I can create a Retrieval-Augmented Generation (RAG) system for this, but I want to ...
- 23
0
votes
0
answers
66
views
Store and retrieve multiple documents in free vector stores based on critria
I am new to vector stores, and so far experimented with storing 1 file in Faiss and Pinecone. I am looking for tutorials that teach me how to save multiple files in free versions of any vector store, ...
- 103
0
votes
0
answers
48
views
Specifying arguments of HuggingFaceHub
In this tutorial, when specifying
llm = HuggingFaceHub(
repo_id=repo_id, model_kwargs={"temperature": 0.5, "max_length": 64})
only ...