Questions tagged [text-generation]

The tag has no usage guidance, but it has a tag wiki.

Filter by
Sorted by
Tagged with
0 votes
0 answers
19 views

Details about Pre-existing knowledge database in RAG for LLMs

In RAG, As part of retriever model- we are retrieving the relevant information from external knowledge source (i.e. vector database) and this database is always updating with new updates. In ...
Tovlk's user avatar
  • 1
2 votes
1 answer
43 views

How to select the optimal beam size for beam search?

Most Text Generation Models use beam search to select the optimal output candidate. How does one choose the optimal beam size? It would probably vary from task to task, dataset to dataset, and model ...
Tathagato Roy's user avatar
0 votes
0 answers
23 views

How to improve performance of a Retrieval Augmented Generative (RAG) model?

I had implemented a Retrieval Augmented generation (RAG) model on the Healthcare CSV file. The model has to give answers to natural language queries based on the data provided. After implementing the ...
user159579's user avatar
0 votes
1 answer
36 views

Getting a free and unknown answer to a question against a fine-tuned text generation model trained on many essays and their few questions and answers

Aim I want to fine-tune a text generation model with essays of changing size and then ask each of these input texts a few questions. I already have a wider range of question-answer pairs at hand for ...
questionto42's user avatar
0 votes
1 answer
129 views

How does RAG (Retrieval Augmented Generation ) work around limited context length?

My understanding of the RAG pipeline can be summarized with the following diagram: I understand steps 1-7 splits and vectorizes an external text data source into chunks and steps 8-11 retrieves n ...
Clement's user avatar
0 votes
1 answer
23 views

Multilingual sentence generation with Hugging Face

For an application I need to generate some random sentences, i.e. I don't need the output sentences to have any specific link to the prompt other than using the same language. If possible I need this ...
Erwan's user avatar
  • 25.2k
0 votes
0 answers
67 views

Fine-tuning MT5 for making it more like ChatGPT

I am trying to fine-tune a model which works like ChatGPT for Punjabi language, using the mt5-base, however I am not sure if I should go ahead with it since it does not even generate text and when I ...
Rukaiya Hasan's user avatar
0 votes
0 answers
45 views

How to train a custom sentence completion model using tensorflow?

What I have A small corpus of English sentences (about 60,000). The Task Sentence completion model. Basically, something like Input: "Paris is the" Output: "Paris is the capital of ...
Della's user avatar
  • 315
0 votes
0 answers
21 views

Seeking Solutions for Generating Text Descriptions from Diagrams, Infographics, Charts, etc

I am currently on a quest to find an efficient way to generate meaningful text descriptions (or alt text) from visual representations such as diagrams, infographics, charts, plots and the like. ...
Yann Stoneman's user avatar
0 votes
0 answers
29 views

Pretrained model for RNN Encoder-Decoder?

Our team are implementing a paper called Cold-Start-Reinforcement-Learning-with-Softmax-Policy-Gradient. Although the paper didn't mention. We want to use a pre-trained model, which is a RNN Encoder-...
jackson's user avatar
  • 25
2 votes
1 answer
801 views

Fine-tuning a pre-trained LLM for question-answering

Objective My goal is to fine-tune a pre-trained LLM on a dataset about Manchester United's (MU's) 2021/22 season (they had a poor season). I want to be able to prompt the fine-tuned model with ...
Tom Bomer's user avatar
1 vote
0 answers
196 views

How to monitor training of text generation models?

I'm finetuning a pretrained Huggingface model based on Transformers for a downstream Text Generation task, but I have doubts on how the fine-tuning process should be monitored: In classification, I ...
Ciodar's user avatar
  • 151
2 votes
1 answer
2k views

How do we evaluate the outputs of text generation models?

Evaluation of a wide variety of natural language generation (NLG) tasks is difficult. For instance, for a question answering model, it is hard for a human to quantify how well the model has answered a ...
Greggs's user avatar
  • 121
1 vote
1 answer
142 views

Preprocessing advice for large text corpus in natural language generation (NLG)

I have a large text corpus (i.e. 30 million sentences, all in lowercase in the format of Penn Treebank) that I want to use to train a neural network for natural language generation. What preprocessing ...
postnubilaphoebus's user avatar
1 vote
1 answer
44 views

I have 2 Columns of text, Should I use different vectorizer and Embeddings for each or just one?

I have a dataset with two input columns as text. Should I use same textvectorizer in both columns or different ones? I am asking this because. columns a has average ...
tikendraw's user avatar
  • 135
0 votes
1 answer
75 views

Models that are good for long answer generation given context and question and what datasets would be the best for training?

Basically I am trying to create a context-needing question and long answer model and I was wondering what model would be best for such tasks, currently I am leaning towards T5, or GPT-NeoX-20B. ...
Thamognya Kodi's user avatar
1 vote
2 answers
91 views

How to evaluate Natural Question-Answer Generation pairs?

I am trying to generate Natural Question-Answer for a specific domain. I am using a Large Language Model (LLM). I have only context to generate question-answers but don't have any ground truth. How to ...
Aaditya ura's user avatar
0 votes
1 answer
276 views

Generate similar text based on category or the similar texts

I'm trying to generate the similar text based on the category or to generate text by combining similar texts into the new text. I was checking multiple nlp tasks like question generation, but they don'...
Crn's user avatar
  • 1
0 votes
1 answer
133 views

Is it possible to apply stable diffusion to text?

Is it possible in theory to apply Stable Diffusion to a text domain? I'm trying to generate text using a Seq2Seq approach, and I'm wondering whether or not it's possible to apply stable diffusion by ...
Andrew1024's user avatar
1 vote
1 answer
396 views

Using different tokens for padding, end-of-sentence, and start-of-sentence in autoregressive sequence modeling?

Is there utility in using different tokens for end-of-sentence, start-of-sentence, and padding for autoregressive sequence modeling (i.e. text generation)? Or can I use the same token for all of them?
postnubilaphoebus's user avatar
1 vote
1 answer
60 views

Abstracted text summarisation and generation from weighted keywords

Suppose I have a list of weighted keywords/phrases, such as "solar panel", "rooftop", etc. The weights are in [0,1] with higher weights indicating a stronger preference for ...
Jeff's user avatar
  • 113
0 votes
0 answers
60 views

Dialogue history encoding for multi-turn dialogues using Seq2seq

In single-turn dialogue seq2seq models where the goal is to produce a good answer y to a query x, sentences are usually encoded such that x is fed to the encoder, while the decoder is only given a &...
postnubilaphoebus's user avatar
0 votes
1 answer
110 views

Advantages of different tokenizers for NLP (specifically text generation)

What are the advantages of using different tokenizers? For example, let's take the sentence: "In Düsseldorf I took my hat off. But I can't put it back on." The treebank tokenizer yields: &...
postnubilaphoebus's user avatar
0 votes
1 answer
18 views

Is there a machine learning model that is able to take reviews as input and output a new and unique blog article from them?

I am looking for a machine learning model ideally with inference speeds of no longer than a few minutes that is able to take in n human written reviews and output a blog article from them. The model ...
knowledge_seeker's user avatar
1 vote
1 answer
33 views

Next-word Generation in Tabular Dataset

I'll build next-word generation using Tensorflow to predict address mapping. But, I saw many tutorial, next-word generation use long-text narration for its training dataset. But, I have dataset ...
Mico S's user avatar
  • 41
1 vote
1 answer
190 views

Why do RNN text generation models treat word prediction as a classification task?

In many of the sources I have found regarding text generation with word-based RNN models (LSTM or GRU), the model is trained to perform a classification task across the vocabulary (such as with ...
twiddler's user avatar
  • 113
1 vote
0 answers
46 views

Guide to Natural language Prompt programming for few-shot learning of Pretrained Language Models

I'm currently working on a project with the goal of producing AI content in the space of a content generation like blog writing, Instagram caption generation etc. Found the in-context few-shot ...
vishal singh's user avatar
1 vote
0 answers
126 views

Using LSTM for text generation keeps generating same word

I work on a simple text generation problem using a portion of the Shakespeare dataset that I decided to use LSTM for. I primarily used this tutorial for reference. However, as I ran the below code, I ...
lazarea's user avatar
  • 289
1 vote
0 answers
59 views

Does N-gram language model for text generation are more efficient than Neural Network language models?

I recently build an language model with N-gram model for text generation and for change I started exploring Neural Network for text generation. One thing I observed that the previous model results ...
Tejax's user avatar
  • 11
0 votes
1 answer
675 views

Word-level text generation with word embeddings – outputting a word vector instead of a probability distribution

I am currently researching the topic of text generation for my university project. I decided (ofc) to go with a RNN getting a sequence of tokens as input with a target of predicting the next token ...
czypsu's user avatar
  • 101
0 votes
0 answers
509 views

Is is possible to make a text generator with sklearn?

So recently I made a Tensorflow model using RNN (Recurrent neural networks) and I was wondering if it was possible with sklearn too, through the usage of SVMs or Naive bayes. I searched up many ...
Aryan's user avatar
  • 101
0 votes
0 answers
32 views

How effective is text generation?

I have implemented some basic models like composing a poem using the dataset of poems. But the results were not that good in general. I want to make a model that could write an essay for me. One ...
Dwip Dalal's user avatar
1 vote
0 answers
42 views

Text generation with deep neural network?

For my master's project, I have to build a deep learning model for text generation: the model learns on a set of sentences, then it generates new sentences based on those from which it learned. I ...
zakya's user avatar
  • 11
2 votes
2 answers
138 views

Generation of medical institution names: training corpora?

My question is quite similar to this one: Generation of institution names. I need to be able to produce 'fake' names of medical institutions, specifically to create data for unit tests. Unfortunately, ...
Stanislav Koncebovski's user avatar
1 vote
0 answers
314 views

English to "basic English" translation

I'd like to build something (ideally in Python) that can translate an English sentence into "basic" English. Are there any free/open-source tools/frameworks that can help? If not, what kind ...
Ben's user avatar
  • 141
1 vote
0 answers
655 views

LSTM Text Generation with Pytorch

I am currently trying quote generation (character level) with LSTMs using Pytorch. I am currently facing some issues understanding exactly how the hidden state is implemented in Pytorch. Some details: ...
Apocalyptic Warrior 's user avatar
3 votes
1 answer
3k views

Pytorch: understanding the purpose of each argument in the forward function of nn.TransformerDecoder

According to https://pytorch.org/docs/stable/generated/torch.nn.TransformerDecoder.html, the forward function of nn.TransformerDecoder contemplates the following arguments: tgt – the sequence to the ...
Pablo Messina's user avatar
7 votes
1 answer
9k views

Minimal working example or tutorial showing how to use Pytorch's nn.TransformerDecoder for batch text generation in training and inference modes?

I want to solve a sequence-to-sequence text generation task (e.g. question answering, language translation, etc.). For the purposes of this question, you may assume that I already have the input part ...
Pablo Messina's user avatar
1 vote
0 answers
61 views

Best way to suggest answers given historical question-answer pairs

Many question-answering implementations focus on extracting information from large documents/corpora of text such as Wikipedia. I have access to a full chat log from the customer service of a large ...
Joep's user avatar
  • 11
1 vote
0 answers
25 views

Img to text tensorflow

I read this article: https://www.tensorflow.org/tutorials/text/image_captioning And i understand main concepts of this article. We have label and image in our dataset. We pass image to our model and ...
kali_xyyali's user avatar
1 vote
1 answer
30 views

How to handle like meaning sentences when working on text summarization

Suppose we have a text like Today is a very bad day. Very bad day is today. I wont come to play. What kind of technique should I use to summarize similar texts like ...
DrDoggo's user avatar
  • 11
0 votes
3 answers
51 views

Distinguish randomly generated texts from reasonable for human texts [closed]

I have strings short texts of 2 types: '23jd2032n0d2mn', 'fn830n30rn83', 'fhui29n4ok', 'qn4foml', ... and ...
Marcel Mars's user avatar
1 vote
1 answer
438 views

Based on transformer, how to improve the text generation results?

If I do not pretrain the text generation model like BART, how to improve the result based on transformer like tensor2tensor? What are the improvement ideas for transformer in text generation task?
惊天补扣's user avatar
0 votes
1 answer
25 views

Question of pretraining text-generation task, it seems that pretraining is not work for a small model?

My task is to generate keywords from sentences. I pretrain a text-generation model. I mask the sentences' tokens and predict the whole sentences' tokens. Pretraining batch_size = 8 and step = 1000000 ...
惊天补扣's user avatar
1 vote
2 answers
51 views

Generation of institution names

I have found a number of parsers for the automatic extraction of institution names from texts (e.g. this one). My task is in a sense the inverse one: I want to automatically generate reality-like ...
Alex Konnen's user avatar
2 votes
1 answer
129 views

Generate text using user-supplied keywords

I've got a use case where I need to generate sentences based on a set of user supplied keywords. Here is an example of what I need: User input: End-User: Data Scientists Region: Middle East ...
Sameer Zahid's user avatar
0 votes
1 answer
36 views

Is it a Problem if Training Data and Evaluation Data are Very Similar?

I'm working on a group project where we have decided to use this Microsoft WIP repository as our starting point. It's a project which compares different frameworks and models and their ability to do ...
Philip Kirkbride's user avatar
1 vote
0 answers
12 views

Looking for suggestions on performing Sementic Analysis of ASR text

Currently I am working on a project where I have ASR on which I am performing semantic analysis to extract meaning out of it. The ASR text contains huge amount of vague conversational text which needs ...
Mousam Singh's user avatar
3 votes
2 answers
143 views

How to generate a sentence with exactly N words?

Thanks to GPT2 pretrained model now it is possible to generate meaningful sequence of words with or without prefix. However a sentence should end with a proper endings (.,!,?). I am just wondering how ...
user185597's user avatar
1 vote
0 answers
158 views

Paraphrase Generation - state-of-the-art?

I need to paraphrase short, but abstract sentences, such as: He prefers variety to routine. or I am easily discouraged. I've played around couple of online tools (such as https://quillbot.com/)...
Comfort Eagle's user avatar