Questions tagged [nlp]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

Seeking datasets for training a Language Model on U.S. mortgage loan processes

I'm in the process of training a Language Model (LLM) and require datasets that encompass various aspects of the U.S. mortgage loan process. The model's aim is to understand and simulate decision-...
Anand 's user avatar
0 votes
0 answers
12 views

Prefix tuning in LLM uses learnable vectors to fine tune the model

I would like to implement a new architecture for Transformer. Below description is my thought. Prefix tuning in LLM uses learnable vectors to fine tune the model. Is there a way to use the output ...
jackson's user avatar
  • 25
0 votes
0 answers
29 views

Some questions from a noob - Chat bot

I'm a T-SQL and .NET developer but I need to understand some basis about a data science / ml project. I would like to know the guidelines and be directed to the right topics about the steps that need ...
Pietro Ariano's user avatar
0 votes
0 answers
17 views

Fine-tune zero-shot classification model multi-label

I started a small project where I am trying to fine-tune a zero-shot classification model on a proprietary dataset. I was thinking to use the NLI approach, building contradiction and entailment ...
lucatrovato's user avatar
0 votes
1 answer
35 views

Semantic Search on numeric data

I have a dataset in csv format. Most columns have numbers in them. The data is mostly number based, with little text. A sample of my dataset: I want to build a semantic search engine that can answer ...
Apoorva's user avatar
  • 275
0 votes
1 answer
38 views

Purely extractive Language Model

Given an email thread, I am trying to extract the body of the most recent email. I used to do that with rules. Now I am testing Large Language Models (LLM) to see if I they provide a less ad hoc ...
mirix's user avatar
  • 103
0 votes
0 answers
14 views

Performing Multi label text classification

I have a text and it's class, so I have performed single text classification, but now I want to train a multi label classifier, so I tried combining sentence to form a multi label dataset, but the ...
Gaurav Joshi's user avatar
0 votes
0 answers
14 views

NLP approach for classifying webscraped data

I have a challenge in a project of mine where I will be provided with a list of scraped datas from a website. Along with the data i will also be provided some parameters like the tag of scraped ...
LEO_007's user avatar
0 votes
0 answers
13 views

Get accurate alias list of people without information on aliases

I'm working on a project where I have a large dataframe of paintings (from the Art500k dataset), each row corresponds with a painting, containing the author's name in the ...
me9hanics's user avatar
0 votes
1 answer
19 views

What is the input to an encoder-decoder transformer in next word prediction task?

I'm trying to understand how encoder-decoder architectures are used, or if they are used at all, for generative tasks that do not require an explicit prompt (ie. machine translation, summarization, ...
mehsheenman's user avatar
0 votes
0 answers
6 views

Is there a tool to do "next speech prediction"?

Given the enormous success of next-token prediction, I was wondering if there are any successful models that try to apply the same principle on speech, i.e., the analogue audio of speech rather than ...
Ameer Jewdaki's user avatar
1 vote
1 answer
50 views

How does Bert masked language modelling task make sense if half the time the next sentence is wrong context in the sequence passed through the encoder

Bert has two types of tasks that it uses to learn contextual word embeddings: Masked word prediction Next sentence prediction I have read the paper and even there the training details are a little ...
figs_and_nuts's user avatar
0 votes
0 answers
26 views

Fine-tuning MT5 for making it more like ChatGPT

I am trying to fine-tune a model which works like ChatGPT for Punjabi language, using the mt5-base, however I am not sure if I should go ahead with it since it does not even generate text and when I ...
Rukaiya Hasan's user avatar
0 votes
0 answers
10 views

F1 and Exact-Match (EM) Score in Extractive QA NLP

I have a question as to how the F1 should be calculated in NLP and whether the text normalization is optional or not. So I have been working on a project where we created a closed-domain extractive QA ...
tt40kiwi's user avatar
  • 101
1 vote
1 answer
46 views

Unsupervised Machine Translation System Using Variational Autoencoder Models

I want to work on an unsupervised machine translation system using a variational autoencoder. I did a literature review but didn't find any related work, and most of the work is based on denoising ...
kartikeya saraswat's user avatar
0 votes
0 answers
13 views

Do ML model measurements and validation standards (e.g. NIST, ISO) exists for the finance, healthcare, and technology industries? Provide citations

Normally, for example, we talk about splitting datasets into training and test datasets. But. The splitting % per train and test sets happens in a subjective manner. Sometimes. The train is 60% or 70%,...
Full Array's user avatar
0 votes
0 answers
14 views

Best approach to find whether a scientific research paper has human trials/human testing or no

I want to know the best way to know if a paper has human trials/test subjects/testing. Was thinking of searching for some keywords in the paper like "Human", "Trials" etc. The ...
Chaitanya Malhotra's user avatar
0 votes
0 answers
25 views

Sentencepiece Tokenizer training from scrath

To train BPE model on sentencepiece as per given Usage instructions As, it is mentioned in the instructions that --input: one-sentence-per-line raw corpus file. ...
Vinay Sharma's user avatar
0 votes
0 answers
7 views

Systematic way of selecting internet texts for a machine translation corpus / dataset?

I am currently working on a neural machine translation project and want to gather a corpus (or dataset) of internet texts that are written in standard and plain language. In theory, it certainly makes ...
tschomacker's user avatar
0 votes
0 answers
18 views

Why did the Double Metaphone algorithm choose to substitute and merge consonants?

Is there any literature describing the decisions behind why the mappings from input text characters to Metaphone hash consonants were made? Why did they choose to leave out vowels? Why did they merge ...
Lance's user avatar
  • 263
0 votes
0 answers
10 views

Highly unbalnced text data giving very low matrics

I have an unbalanced multi-class banking text data with around 76 classes. Classes are badly distributed such as one class which is combination of 240 other different categories, represents 50% of ...
Remrem's user avatar
  • 1
3 votes
2 answers
85 views

Is there a Language Model that can accept huge corpse of tabular data and answr questions about?

I have been researching Language Models that can work with tabular data. My main goal is to have a model to answer simple questions about my data. An example is having household sales data and asking ...
Shahriyar Mammadli's user avatar
0 votes
0 answers
9 views

【NLP】Is there a model or task that determines contextual similarity?

I am trying to work on an engagement detection task in which I have to determine if a student is engaged in class. I am looking for an NLP approach where I can calculate the similarity score of a ...
Leo's user avatar
  • 1
0 votes
0 answers
18 views

Leverage LLMs to classify sentence similarity

This is intended to be mainly a reference request in the vast world of NLP and LLMs. Context A certain protocol is given in the form of text. This can be, for instance, the general description of a ...
Andrea Gagna's user avatar
0 votes
0 answers
16 views

TFRobertaSequenceClassification for Address Normalization task

I have dataset with two column: one with faulty addresses, and other with correct addresses. I want to train a model such that, I can use it later for correcting all the incoming faulty addresses. I ...
learner_account's user avatar
0 votes
1 answer
22 views

Clustering words with similar meanings

What methods are there to cluster words/word phrases with similar meanings together from a list of words/word phrases?
ros's user avatar
  • 1
0 votes
0 answers
28 views

How to improve GPT2 tokenizer trained from scratch?

I trained a GPT2 Tokenizer on Hindi dataset of size 170 MB from scratch and saved it as new_tokenizer. When I tried the new_tokenizer on a Hindi sentence ...
Vinay Sharma's user avatar
0 votes
0 answers
10 views

Expanding Training Data for Intent and Entity Recognition Model

I have a specific use case where I need to identify both intent and entities within a given statement. For example, given the statement "Book train tickets from Mumbai to Delhi," the intent ...
D.Sunil's user avatar
0 votes
0 answers
33 views

How can I avoid the irrelevant number of sentences in the result?

The nature of the data I have is not arranged; however, I'm trying to extract the appropriate sentences for each query as a sample for ground truth. Also, the most critical problem is that I use the ...
Begnnier's user avatar
0 votes
1 answer
37 views

Understanding Multi-headed Attention from architecture details

I've a conceptual question BERT-base has a dimension of 768 for query, key and value and 12 heads (Hidden dimension=768, number of heads=12). The same is conveyed if we see the BERT-base architecture <...
Namburi Srinath's user avatar
1 vote
0 answers
35 views

In rotary positional embeddings (RoPE), why do we not rotate the values as well?

Actually, the question is all there is As per the paper I see that the rotations are applied only to the keys and the queries. Why are the rotations not applied to the values as well? The reasons for ...
figs_and_nuts's user avatar
0 votes
1 answer
62 views

What is the "Extract" token and how is the final Linear layer applied in GPT?

In the manuscript of GPT, the authors have given the following image: Questions: What is the final "Extract" (token?)? Is it the "END" token? How is the final linear layer ...
figs_and_nuts's user avatar
0 votes
0 answers
19 views

Harford 2023 model: explicit reference

I'm citing in a paper the Wizard-Vicuna Uncensored 30B model of Hartford (2023). But I don't have an exact bib reference other than various web links for that model. Could anyone help?
cel's user avatar
  • 1
0 votes
0 answers
17 views

Semantic Scoring and readability for short sentences

I am working on short sentences for NLP based classification. I wish to make a assessment if a sentence is readable before training the system on it. Now readability scores are not working since ...
Vinay Varahabhotla's user avatar
0 votes
0 answers
26 views

Is openAI text generation models an extension of embedding models?

we can creating embeddings using below code ...
Vinay Sharma's user avatar
1 vote
1 answer
58 views

A question about contextual embeddings in the decoder only transformer architecture (gpt)

I am reading up on the decoder only architecture Relevant excerpts: We can use any model that maps token sequences into contextual embeddings (e.g., LSTMs, Transformers): $$\phi : V^L \to R^{d \times ...
figs_and_nuts's user avatar
0 votes
0 answers
202 views

How to read CSV File into Vector Store

I have a CSV file, and I am using langchain to read it into the vector store FAISS. My question is, since I have a CSV file, is RecursiveTextSplitter required? Put differently, consider the following ...
Karl 17302's user avatar
0 votes
1 answer
25 views

How to use location information as feature?

I have a location feature in a dataset. Some examples are: London, Uk; Sheefield Town, Ohio; UK ; North Carolina. etc. How to encode them into features? Is there any word embeddings suitable for such ...
Alex_ban's user avatar
  • 101
0 votes
0 answers
12 views

To extend or not to extend vocabulary for instruction tuning

I want to fine tune a base llm using an instruction dataset. In order to minimize VRAM footprint I want to use SFTTrainer and QLora. My prompt can take the following structure: ...
tog's user avatar
  • 101
1 vote
2 answers
88 views

Training model using BERT

I have generated dataset using chat gpt. Dataset has 9000 data recodes. It's 6 class sentiment analysis. classes are 0,1,2,3,4,5 I used 3000 recodes for training, 1200 recods for validation and ...
Sandun Tharaka's user avatar
1 vote
1 answer
141 views

what is the difference between window size and context length of language model?

is window size and context length of language model one and the same thing? ******** following text is added as question with ONLY above text was not allowed ***** I am trying to understand how GPT ...
Vinay Sharma's user avatar
1 vote
0 answers
46 views

Product name matching - Entity Resolution or Entity Linkage or both?

Context I am at the start of a project where I would like to map/match/link external product names to the respective internal product names. The goal should be to ingest related external information (...
Elodin's user avatar
  • 145
0 votes
1 answer
17 views

Locating base.py when working on Colab

I have faced an error while working with langchain on colab. There is a post on github which recommends changing some configurations in ...
IwillLearn's user avatar
1 vote
1 answer
25 views

A good set of datasets/models for testing an NLP technique

I am a machine learning researcher who up until this point has primarily worked on Computer Vision problems. However, I have an idea for an NLP technique involving a novel Transformer architecture, ...
Anonymous's user avatar
0 votes
1 answer
40 views

Why do we want to maximize the average log probability in neural language models?

I am currently trying to understand the Paragraph Vector framework by reading the paper "Distributed Representation of Sentences and Documents" by Quoc Le and Thomas Mikolov but I have ...
LeeKed's user avatar
  • 1
0 votes
0 answers
39 views

Help understanding working of KeyBERT for keyphrase extraction

I'm fairly new to reading and understanding research papers, so I wanted to get a second opinion on whether my understanding of KeyBERT was correct. Here is a high level overview of my understanding ...
Prithvi's user avatar
  • 33
0 votes
0 answers
208 views

How to get Llama-2 Rotary Embeddings?

I want to get the Llama-2 rotary embeddings. I do print(model) and get the following output: In the picture I highlight the rotary embeddings. How can get the ...
Christian01's user avatar
1 vote
0 answers
52 views

How does fine-tuning work in question answering for custom documents

I am trying to build a Q&A bot for which I have a bunch of documents like articles (specific domain). I understand I can create a Retrieval-Augmented Generation (RAG) system for this, but I want to ...
Glinty's user avatar
  • 23
0 votes
0 answers
66 views

Store and retrieve multiple documents in free vector stores based on critria

I am new to vector stores, and so far experimented with storing 1 file in Faiss and Pinecone. I am looking for tutorials that teach me how to save multiple files in free versions of any vector store, ...
IwillLearn's user avatar
0 votes
0 answers
48 views

Specifying arguments of HuggingFaceHub

In this tutorial, when specifying llm = HuggingFaceHub( repo_id=repo_id, model_kwargs={"temperature": 0.5, "max_length": 64}) only ...
Karl 17302's user avatar

1
2 3 4 5
54