Questions tagged [llm]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

How to design a writing assistive system that can do writing style guide check like how Grammarly does for gramma checking

I have a style guide that specifies the use of punctuations, line length, and how to break up a line (e.g. before prepositions). I have a dataset that contains original text and the version of the ...
Wayne's user avatar
  • 21
0 votes
0 answers
17 views

Details about Pre-existing knowledge database in RAG for LLMs

In RAG, As part of retriever model- we are retrieving the relevant information from external knowledge source (i.e. vector database) and this database is always updating with new updates. In ...
Tovlk's user avatar
  • 1
0 votes
1 answer
14 views

How to choose ideal pretrained model for fine-tuning?

I started to work with LLMs lately and want to know how people choose their pre-trained models in their fine-tuning tasks? What is the criteria to choose the base model and which factors affect?
heyula's user avatar
  • 23
0 votes
0 answers
12 views

Can I fine tune MedPaLM model

Is it possible to fine-tune MedPaLM and MedPaLM2; Google's llms trained using PaLM specialized for medical domain. Can we fine-tune these models further to get more specialized models?
heyula's user avatar
  • 23
0 votes
0 answers
12 views

How to Detect and Identify German Compound Words in a Text using Python

I would like to know if there is any python libraries or packages available which can help me detect and extract German compound words. I took a look at nltk and spacy packages and they don't seem to ...
Ali Mazhar Sadat's user avatar
-1 votes
0 answers
18 views

What are some good NLP courses to understand LLMs? Is there any course that also talks about protein language models and embeddings? [closed]

Please help me out here. I'm from Computational Biology background. I know machine learning deep learning basics. have done quite a few projects in them. i want to delve into LLms and there ...
As13's user avatar
  • 75
-1 votes
0 answers
15 views

Getting key insights from chat input using LLM [closed]

I'd like to understand how in a chat powered by chat gpt (or any other LLM based model). Can I get some insights to run requests in the backend. Take for example I need to provide suggestions based on ...
tsar2512's user avatar
  • 181
0 votes
0 answers
12 views

Create a gpt-3.5 API request that determines whether any time range in a list intersects with a given time range

I've created a prompt that should select a requested number of employees from the list. But the step 1 doesn't work properly. Sometimes GPT takes in account only the time range and ignores the date. I ...
pazukdev's user avatar
  • 101
0 votes
0 answers
23 views

How to improve performance of a Retrieval Augmented Generative (RAG) model?

I had implemented a Retrieval Augmented generation (RAG) model on the Healthcare CSV file. The model has to give answers to natural language queries based on the data provided. After implementing the ...
user159579's user avatar
1 vote
1 answer
48 views

Train Reward Model using Llama2:

this is my code that use to train reward model: ...
Sandun Tharaka's user avatar
0 votes
0 answers
100 views

AI tools/LLM models to analyze excel data

I have a bunch of excel files containing information of employee attendance. I want to use NLP-based search to ask questions like: Which employee has taken most leaves?; What dates witness a high ...
Apoorva's user avatar
  • 275
0 votes
0 answers
10 views

using llama2 on windows

does llama2 work by downloading the .bin file locally and running on CPU without install llama.cpp or any other dependencies. I am using theblokes llama2 models
Saket Vempaty's user avatar
0 votes
1 answer
36 views

Getting a free and unknown answer to a question against a fine-tuned text generation model trained on many essays and their few questions and answers

Aim I want to fine-tune a text generation model with essays of changing size and then ask each of these input texts a few questions. I already have a wider range of question-answer pairs at hand for ...
questionto42's user avatar
0 votes
1 answer
44 views

Outdated Transformers TextDataset class drops last block when text overlaps. Replace by datasets Dataset class as input of Trainer train_dataset?

Why I try to replace the transformers TextDataset class with datasets Dataset class I stumbled upon this when I tried to make ...
questionto42's user avatar
0 votes
0 answers
31 views

unable to download llama2 weights because of http error 416

i am trying to download the weights for llama2 7b-chat but i always end up with "Connecting to download.llamameta.net (52.84.205.116:443) wget: server returned error: HTTP/1.1 416 Requested Range ...
Saket Vempaty's user avatar
0 votes
0 answers
17 views

How to perform inference on a finetuned falcon 7b model fine tuned on open assistant dataset

I finetuned a falcon 7b model on the open assistant dataset using the official colab notebook provided by huggingface at https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=...
Saket Vempaty's user avatar
1 vote
1 answer
37 views

Falcon-7B llm giving random output

I am using a falcon 7B model for a chatbot without any finetuning with the following code ...
Saket Vempaty's user avatar
0 votes
0 answers
20 views

Why does Mistral model or in general Large language models have very low percentage of trainable parameters compared to total number of parameters of?

I am using the below function to print the trainable parameters. I am getting this output: trainable params: 262410240 || all params: 7241732096 || trainable%: 3.6235839233122604 ...
SRIVATSA KULKARNI's user avatar
0 votes
1 answer
30 views

How does RAG query affect the similarity search?

I have a RAG pipeline where I want to extract a piece of information called "X" In a regular RAG pipeline, there is a query entered by the user. Then, ...
ahmedmoh123's user avatar
0 votes
0 answers
35 views

RLHF fine-tune llama2 in vertex ai

I have fine tune RLHF with Vertex AI Pipeline. But deployed model not showing in model registry. Why? code i have used: ...
Sandun Tharaka's user avatar
0 votes
2 answers
113 views

How can you get a Huggingface fine-tuning model with the Trainer class from your own text where you can set the arguments for truncation and padding?

I want to find out the role of truncation and padding in Huggingface Transformers pretrained models and any fine-tuning model on top of that. Therefore I played around with these parameters, but I ...
questionto42's user avatar
2 votes
2 answers
368 views

Should you care about truncation and padding in an LLM even if it has a very large tokenizer.max_length so that truncation will never happen?

I want to find out the role of truncation and padding in Huggingface Transformers pretrained models and/or any fine-tuning models on top. Taking a large language model like the German GPT2 shows that ...
questionto42's user avatar
0 votes
1 answer
125 views

How does RAG (Retrieval Augmented Generation ) work around limited context length?

My understanding of the RAG pipeline can be summarized with the following diagram: I understand steps 1-7 splits and vectorizes an external text data source into chunks and steps 8-11 retrieves n ...
Clement's user avatar
0 votes
1 answer
40 views

Why is temperature necessary in the loss function for Model Distillation?

I'm studying the LLMs course on Coursera. In one video on model distillation, the lecturer says (from the transcript): Model Distillation is a technique that focuses on having a larger teacher model ...
Allure's user avatar
  • 275
0 votes
1 answer
23 views

Multilingual sentence generation with Hugging Face

For an application I need to generate some random sentences, i.e. I don't need the output sentences to have any specific link to the prompt other than using the same language. If possible I need this ...
Erwan's user avatar
  • 25.2k
0 votes
0 answers
36 views

How to find proper context in open book question answering?

I want to make an Open Book Question Answering / Retrieval Augmented Generation system. The major concern here is the proper context selection. There are some fundamental issues related to this. For ...
hafiz031's user avatar
  • 121
0 votes
1 answer
30 views

Using activations at a specific layer as an input for an LLM such as OPT-350m

I'm working with the OPT-350m model and aiming to utilize embeddings from different layers as inputs for generating data. I've encountered issues when trying to feed these embeddings back into the ...
Talia Seada's user avatar
1 vote
2 answers
123 views

Using LLM in low memory (CPU) local environnements

Kind of a generic question, but I wonder if there is any solution on using LLM in low-memory environments, that is CPU environments that can't fit the whole model in memory. I am mostly thinking about ...
Lucas Morin's user avatar
  • 2,093
0 votes
0 answers
13 views

Open-source library suggestion: Semantic Role Labeling

Can I be referred to a SOTA model/script/pipeline with a commercially open-source license for Semantic Role Labeling or semantic parsing? I prefer it to be based on the most recent LLM to achieve ...
Arash Vahabpour's user avatar
0 votes
1 answer
58 views

Do we really need a very large dataset to train GPTs?

Do we really need a very large dataset to train GPTs? If this dataset is not big, won't GPT work well? Or will it still work better than conventional learning models in this situation? And is it ...
nisar's user avatar
  • 1
1 vote
1 answer
57 views

How can I leverage machine learning for log analysis?

I am new to data science and trying to find possibilities of using datascience in tasks. I have a set of logs which I want to convert to json. The logs are more or less of same format and I can write ...
SUNITA GUPTA's user avatar
4 votes
1 answer
177 views

Higher level sentence similarity (meaning instead of 'just' embeddings)

I am looking for the correct model / approach for the task of checking if two sentences have the same meaning I know I can use embeddings to check similarity, but that is not what I am after. I ...
Rob Audenaerde's user avatar
0 votes
0 answers
10 views

Decision Metrics for Selecting Promopt Structures: Creating a Prompt-Based System

I'm currently working on developing a system that generates various prompts structures (such as tree of thoughts, chain of thoughts, etc.) in response to user prompts. However, I'm facing a challenge ...
Jay Padhiyar's user avatar
1 vote
0 answers
69 views

Fine-tuning Hugging Face’s Llama Model with Unlabelled Data from PDFs from niche domain

I’m unsure about the next steps. Specifically, I have the following questions: How can I prepare my unlabelled data for the fine-tuning process? What’s the best way to fine-tune the Llama model with ...
Divya Patel's user avatar
0 votes
1 answer
130 views

Open-Source Large Language Models (LLM): Your experience and recommendation

I’m looking for an open-source LLM for a new project. I want to use it for instructions and to fine-tune the model to a specific domain like legal and rights. Some LLMs are open-source, but they didn’...
Christian01's user avatar
0 votes
0 answers
238 views

Why is 0.7, in general, the default value of temperature for LLMs?

I have recently read through a lot of documentation and articles about Large Language Models (LLMs), and I have come to the conclusion that 0.7 is, most of the time, the default value for the ...
jmpion's user avatar
  • 1
2 votes
1 answer
1k views

How to download and fine tune a large language model locally?

I want to fine tune a model locally, not using HuggingFace or any other third party tool. Basically, I want: Download a trained model (Llama-2, Falcon, whatever is easiest). Fine-tune it locally with ...
user's user avatar
  • 1,993
0 votes
0 answers
67 views

Fine-tuning MT5 for making it more like ChatGPT

I am trying to fine-tune a model which works like ChatGPT for Punjabi language, using the mt5-base, however I am not sure if I should go ahead with it since it does not even generate text and when I ...
Rukaiya Hasan's user avatar
0 votes
0 answers
18 views

F1 and Exact-Match (EM) Score in Extractive QA NLP

I have a question as to how the F1 should be calculated in NLP and whether the text normalization is optional or not. So I have been working on a project where we created a closed-domain extractive QA ...
tt40kiwi's user avatar
  • 101
0 votes
1 answer
38 views

Reinforcement learning

I am working on a sentiment analysis project. I used BERT model for training but lack of data it gives huge overfitting. So after i moved LLM approach to do that. Using LLM finally i got good results....
Sandun Tharaka's user avatar
3 votes
2 answers
96 views

Is there a Language Model that can accept huge corpse of tabular data and answr questions about?

I have been researching Language Models that can work with tabular data. My main goal is to have a model to answer simple questions about my data. An example is having household sales data and asking ...
Shahriyar Mammadli's user avatar
0 votes
0 answers
29 views

For Finetuning Llama 2, what form of data is required?

I am working on a customer service chatbot project. I have files of different product's manuals on which i need to train the LLM which is Llama 2. As these are manuals so they not in Q/A form. In this ...
divertiss Blogger's user avatar
0 votes
0 answers
7 views

Is there an online Vicuana model demo that supports plugging in embeddings directly?

I'm doing some work with using multi-modal data with LLMs like LLaMA and Vicuana. For example, something similar to Video-LLaMA, which converts images/videos into embeddings, concatenates it with the ...
Joe C.'s user avatar
  • 101
0 votes
0 answers
22 views

Since LLMs only provide output in text, how can LLMs do 'action' such as search, run code etc?

Since LLMs only provide output in text, I wonder how LLMs can do 'action' such as search, run code etc. I try to delve deep into how langchain agent works and what I got is that the used prompt is ...
Jarun Vantanavijarn's user avatar
0 votes
0 answers
50 views

Leverage LLMs to classify sentence similarity

This is intended to be mainly a reference request in the vast world of NLP and LLMs. Context A certain protocol is given in the form of text. This can be, for instance, the general description of a ...
Andrea Gagna's user avatar
0 votes
0 answers
25 views

To extend or not to extend vocabulary for instruction tuning

I want to fine tune a base llm using an instruction dataset. In order to minimize VRAM footprint I want to use SFTTrainer and QLora. My prompt can take the following structure: ...
tog's user avatar
  • 101
0 votes
0 answers
78 views

Decreasing the summary length from langchain load_summarize_chain?

How can i reduce the output size of the summarization in langchain map reduce method ?
Aadhil Imam's user avatar
0 votes
0 answers
13 views

Pre-trained text classifier to check whether a document is business or personal

I need a pre-trained classifier that checks if a document is either a business or personal document. I don't have time to train one myself so if anyone knows of a classifier that is pre-trained and ...
user155097's user avatar
0 votes
0 answers
25 views

What options do I have when performing RAG on similarly phrased chunks?

I'm building a RAG pipeline to extract real estate phrases from excel documents. These phrases are short (2-5 words) and are often phrased differently. I've manually added in different phrases to each ...
codeananda's user avatar
0 votes
0 answers
26 views

Errors in results after saving model vs using directly from memory

I am trying to save a Fine Tuned model using trainer.save_model() but after I load the saved_model it just responds with the input back again and does not give any ...
Nikhil Verma's user avatar