Questions tagged [text-generation]
The text-generation tag has no usage guidance, but it has a tag wiki.
84
questions
1
vote
1
answer
392
views
English to "basic English" translation
I'd like to build something (ideally in Python) that can translate an English sentence into "basic" English.
Are there any free/open-source tools/frameworks that can help? If not, what kind ...
0
votes
0
answers
2
views
"Background of the study" for proposal on title "Factors Affecting Students' Academic Achievement in Mathematics in secondary schools of silte zone"
✔️**** Background of the Study
The educational sphere of Silte Zone, a region in the South Nations, Nationalities, and Peoples' Regional State in Ethiopia, faces challenges that hinder academic ...
4
votes
1
answer
85
views
LLMs for text generation
We know that AI is rapidly growing. do we have any large language models (LLMs) to process images, pdf documents directly (fine-tune approach) for text generation tasks?
0
votes
1
answer
288
views
Generate similar text based on category or the similar texts
I'm trying to generate the similar text based on the category or to generate text by combining similar texts into the new text. I was checking multiple nlp tasks like question generation, but they don'...
1
vote
1
answer
399
views
Is it possible to use Word2vec for text paraphrasing?
After reading several papers I am not sure if it is possible to some how generate text with the same meaning (paraphrase it) using only Word2vec.
I found out other approaches that use sequences of ...
0
votes
0
answers
22
views
Details about Pre-existing knowledge database in RAG for LLMs
In RAG, As part of retriever model- we are retrieving the relevant information from external knowledge source (i.e. vector database) and this database is always updating with new updates.
In ...
2
votes
1
answer
58
views
How to select the optimal beam size for beam search?
Most Text Generation Models use beam search to select the optimal output candidate. How does one choose the optimal beam size? It would probably vary from task to task, dataset to dataset, and model ...
0
votes
1
answer
45
views
Getting a free and unknown answer to a question against a fine-tuned text generation model trained on many essays and their few questions and answers
Aim
I want to fine-tune a text generation model with essays of changing size and then ask each of these input texts a few questions. I already have a wider range of question-answer pairs at hand for ...
2
votes
1
answer
867
views
Fine-tuning a pre-trained LLM for question-answering
Objective
My goal is to fine-tune a pre-trained LLM on a dataset about Manchester United's (MU's) 2021/22 season (they had a poor season). I want to be able to prompt the fine-tuned model with ...
0
votes
0
answers
45
views
How to improve performance of a Retrieval Augmented Generative (RAG) model?
I had implemented a Retrieval Augmented generation (RAG) model on the Healthcare CSV file. The model has to give answers to natural language queries based on the data provided. After implementing the ...
0
votes
1
answer
703
views
Word-level text generation with word embeddings – outputting a word vector instead of a probability distribution
I am currently researching the topic of text generation for my university project. I decided (ofc) to go with a RNN getting a sequence of tokens as input with a target of predicting the next token ...
3
votes
1
answer
516
views
Choosing a right algorithm for template-based text generation
I am doing a text generation project -- the task is to basically represent the statistical data in a readable way.
The way I decided to go about this is template-based: each data type has a template ...
0
votes
1
answer
217
views
How does RAG (Retrieval Augmented Generation ) work around limited context length?
My understanding of the RAG pipeline can be summarized with the following diagram:
I understand steps 1-7 splits and vectorizes an external text data source into chunks and steps 8-11 retrieves n ...
0
votes
1
answer
25
views
Multilingual sentence generation with Hugging Face
For an application I need to generate some random sentences, i.e. I don't need the output sentences to have any specific link to the prompt other than using the same language. If possible I need this ...
2
votes
1
answer
154
views
Create an RNN on text sources with different lengths
I want to create an RNN to generate a new text based on many examples of existing texts of a certain format in the training data.
The type of texts in the training data consists of 3 segments, like so:...
0
votes
0
answers
73
views
Fine-tuning MT5 for making it more like ChatGPT
I am trying to fine-tune a model which works like ChatGPT for Punjabi language, using the mt5-base, however I am not sure if I should go ahead with it since it does not even generate text and when I ...
0
votes
0
answers
49
views
How to train a custom sentence completion model using tensorflow?
What I have
A small corpus of English sentences (about 60,000).
The Task
Sentence completion model. Basically, something like
Input: "Paris is the"
Output: "Paris is the capital of ...
0
votes
0
answers
22
views
Seeking Solutions for Generating Text Descriptions from Diagrams, Infographics, Charts, etc
I am currently on a quest to find an efficient way to generate meaningful text descriptions (or alt text) from visual representations such as diagrams, infographics, charts, plots and the like. ...
1
vote
0
answers
214
views
How to monitor training of text generation models?
I'm finetuning a pretrained Huggingface model based on Transformers for a downstream Text Generation task, but I have doubts on how the fine-tuning process should be monitored:
In classification, I ...
0
votes
0
answers
29
views
Pretrained model for RNN Encoder-Decoder?
Our team are implementing a paper called Cold-Start-Reinforcement-Learning-with-Softmax-Policy-Gradient.
Although the paper didn't mention. We want to use a pre-trained model, which is a RNN Encoder-...
1
vote
1
answer
423
views
Using different tokens for padding, end-of-sentence, and start-of-sentence in autoregressive sequence modeling?
Is there utility in using different tokens for end-of-sentence, start-of-sentence, and padding for autoregressive sequence modeling (i.e. text generation)?
Or can I use the same token for all of them?
2
votes
1
answer
2k
views
How do we evaluate the outputs of text generation models?
Evaluation of a wide variety of natural language generation (NLG) tasks is difficult. For instance, for a question answering model, it is hard for a human to quantify how well the model has answered a ...
0
votes
1
answer
142
views
Is it possible to apply stable diffusion to text?
Is it possible in theory to apply Stable Diffusion to a text domain?
I'm trying to generate text using a Seq2Seq approach, and I'm wondering whether or not it's possible to apply stable diffusion by ...
1
vote
1
answer
143
views
Preprocessing advice for large text corpus in natural language generation (NLG)
I have a large text corpus (i.e. 30 million sentences, all in lowercase in the format of Penn Treebank) that I want to use to train a neural network for natural language generation. What preprocessing ...
1
vote
1
answer
49
views
I have 2 Columns of text, Should I use different vectorizer and Embeddings for each or just one?
I have a dataset with two input columns as text. Should I use same textvectorizer in both columns or different ones?
I am asking this because. columns a has average ...
0
votes
1
answer
86
views
Models that are good for long answer generation given context and question and what datasets would be the best for training?
Basically I am trying to create a context-needing question and long answer model and I was wondering what model would be best for such tasks, currently I am leaning towards T5, or GPT-NeoX-20B. ...
1
vote
2
answers
110
views
How to evaluate Natural Question-Answer Generation pairs?
I am trying to generate Natural Question-Answer for a specific domain. I am using a Large Language Model (LLM). I have only context to generate question-answers but don't have any ground truth. How to ...
1
vote
1
answer
62
views
Abstracted text summarisation and generation from weighted keywords
Suppose I have a list of weighted keywords/phrases, such as "solar panel", "rooftop", etc. The weights are in [0,1] with higher weights indicating a stronger preference for ...
0
votes
0
answers
63
views
Dialogue history encoding for multi-turn dialogues using Seq2seq
In single-turn dialogue seq2seq models where the goal is to produce a good answer y to a query x, sentences are usually encoded such that x is fed to the encoder, while the decoder is only given a &...
0
votes
1
answer
112
views
Advantages of different tokenizers for NLP (specifically text generation)
What are the advantages of using different tokenizers? For example, let's take the sentence:
"In Düsseldorf I took my hat off. But I can't put it back on."
The treebank tokenizer yields: &...
0
votes
1
answer
18
views
Is there a machine learning model that is able to take reviews as input and output a new and unique blog article from them?
I am looking for a machine learning model ideally with inference speeds of no longer than a few minutes that is able to take in n human written reviews and output a blog article from them.
The model ...
1
vote
1
answer
33
views
Next-word Generation in Tabular Dataset
I'll build next-word generation using Tensorflow to predict address mapping. But, I saw many tutorial, next-word generation use long-text narration for its training dataset. But, I have dataset ...
0
votes
0
answers
520
views
Is is possible to make a text generator with sklearn?
So recently I made a Tensorflow model using RNN (Recurrent neural networks) and I was wondering if it was possible with sklearn too, through the usage of SVMs or Naive bayes.
I searched up many ...
1
vote
1
answer
199
views
Why do RNN text generation models treat word prediction as a classification task?
In many of the sources I have found regarding text generation with word-based RNN models (LSTM or GRU), the model is trained to perform a classification task across the vocabulary (such as with ...
2
votes
1
answer
50
views
Automatic data summarization with text
I would like to automate periodic report writing based on data. Given one/some data tables, the machine should output texts like ...
1
vote
0
answers
47
views
Guide to Natural language Prompt programming for few-shot learning of Pretrained Language Models
I'm currently working on a project with the goal of producing AI content in the space of a content generation like blog writing, Instagram caption generation etc. Found the in-context few-shot ...
1
vote
0
answers
59
views
Does N-gram language model for text generation are more efficient than Neural Network language models?
I recently build an language model with N-gram model for text generation and for change I started exploring Neural Network for text generation. One thing I observed that the previous model results ...
1
vote
0
answers
128
views
Using LSTM for text generation keeps generating same word
I work on a simple text generation problem using a portion of the Shakespeare dataset that I decided to use LSTM for. I primarily used this tutorial for reference. However, as I ran the below code, I ...
4
votes
1
answer
420
views
NLP - Paraphrase extraction in Python
I am trying to develop a NLP model, which takes something like you have high levels of cholesterol(this will be a tag) as input and has to output something like <...
1
vote
0
answers
2k
views
Distractor Generation for Multiple Choice Questions
I'm currently working on generating distractor for multiple choice questions. Training set consists of question, answer and 3 distractor and I need to predict 3 distractor for test set. I have gone ...
0
votes
1
answer
179
views
predicting next jobtitle
I have a dataset of which has 30M rows each like [current_jobtitles, nextjobtitles].
...
0
votes
0
answers
32
views
How effective is text generation?
I have implemented some basic models like composing a poem using the dataset of poems. But the results were not that good in general. I want to make a model that could write an essay for me. One ...
1
vote
0
answers
43
views
Text generation with deep neural network?
For my master's project, I have to build a deep learning model for text generation: the model learns on a set of sentences, then it generates new sentences based on those from which it learned.
I ...
2
votes
2
answers
150
views
Generation of medical institution names: training corpora?
My question is quite similar to this one: Generation of institution names. I need to be able to produce 'fake' names of medical institutions, specifically to create data for unit tests. Unfortunately, ...
1
vote
0
answers
682
views
LSTM Text Generation with Pytorch
I am currently trying quote generation (character level) with LSTMs using Pytorch. I am currently facing some issues understanding exactly how the hidden state is implemented in Pytorch.
Some details:
...
3
votes
1
answer
3k
views
Pytorch: understanding the purpose of each argument in the forward function of nn.TransformerDecoder
According to https://pytorch.org/docs/stable/generated/torch.nn.TransformerDecoder.html, the forward function of nn.TransformerDecoder contemplates the following arguments:
tgt – the sequence to the ...
7
votes
1
answer
9k
views
Minimal working example or tutorial showing how to use Pytorch's nn.TransformerDecoder for batch text generation in training and inference modes?
I want to solve a sequence-to-sequence text generation task (e.g. question answering, language translation, etc.).
For the purposes of this question, you may assume that I already have the input part ...
2
votes
1
answer
129
views
Generate text using user-supplied keywords
I've got a use case where I need to generate sentences based on a set of user supplied keywords. Here is an example of what I need:
User input:
End-User: Data Scientists
Region: Middle East
...
1
vote
0
answers
62
views
Best way to suggest answers given historical question-answer pairs
Many question-answering implementations focus on extracting information from large documents/corpora of text such as Wikipedia.
I have access to a full chat log from the customer service of a large ...
1
vote
1
answer
30
views
How to handle like meaning sentences when working on text summarization
Suppose we have a text like Today is a very bad day. Very bad day is today. I wont come to play.
What kind of technique should I use to summarize similar texts like ...