Questions tagged [text-generation]
The text-generation tag has no usage guidance, but it has a tag wiki.
79
questions
0
votes
0
answers
21
views
How to train a custom sentence completion model using tensorflow?
What I have
A small corpus of English sentences (about 60,000).
The Task
Sentence completion model. Basically, something like
Input: "Paris is the"
Output: "Paris is the capital of ...
- 315
0
votes
0
answers
13
views
Seeking Solutions for Generating Text Descriptions from Diagrams, Infographics, Charts, etc
I am currently on a quest to find an efficient way to generate meaningful text descriptions (or alt text) from visual representations such as diagrams, infographics, charts, plots and the like. ...
- 101
0
votes
0
answers
19
views
Pretrained model for RNN Encoder-Decoder?
Our team are implementing a paper called Cold-Start-Reinforcement-Learning-with-Softmax-Policy-Gradient.
Although the paper didn't mention. We want to use a pre-trained model, which is a RNN Encoder-...
- 23
1
vote
0
answers
629
views
Fine-tuning a pre-trained LLM for question-answering
Objective
My goal is to fine-tune a pre-trained LLM on a dataset about Manchester United's (MU's) 2021/22 season (they had a poor season). I want to be able to prompt the fine-tuned model with ...
- 13
1
vote
0
answers
101
views
How to monitor training of text generation models?
I'm finetuning a pretrained Huggingface model based on Transformers for a downstream Text Generation task, but I have doubts on how the fine-tuning process should be monitored:
In classification, I ...
- 151
2
votes
1
answer
778
views
How do we evaluate the outputs of text generation models?
Evaluation of a wide variety of natural language generation (NLG) tasks is difficult. For instance, for a question answering model, it is hard for a human to quantify how well the model has answered a ...
- 121
1
vote
1
answer
131
views
Preprocessing advice for large text corpus in natural language generation (NLG)
I have a large text corpus (i.e. 30 million sentences, all in lowercase in the format of Penn Treebank) that I want to use to train a neural network for natural language generation. What preprocessing ...
1
vote
1
answer
23
views
I have 2 Columns of text, Should I use different vectorizer and Embeddings for each or just one?
I have a dataset with two input columns as text. Should I use same textvectorizer in both columns or different ones?
I am asking this because. columns a has average ...
- 135
0
votes
1
answer
55
views
Models that are good for long answer generation given context and question and what datasets would be the best for training?
Basically I am trying to create a context-needing question and long answer model and I was wondering what model would be best for such tasks, currently I am leaning towards T5, or GPT-NeoX-20B. ...
1
vote
2
answers
58
views
How to evaluate Natural Question-Answer Generation pairs?
I am trying to generate Natural Question-Answer for a specific domain. I am using a Large Language Model (LLM). I have only context to generate question-answers but don't have any ground truth. How to ...
- 415
0
votes
1
answer
170
views
Generate similar text based on category or the similar texts
I'm trying to generate the similar text based on the category or to generate text by combining similar texts into the new text. I was checking multiple nlp tasks like question generation, but they don'...
- 1
0
votes
0
answers
37
views
How to generate question based on information given in question or by combining multiple questions?
I have a dataset of question and I want to generate the questions from the given text. For example, if I have a task like "John paid 1500$ for television set. Television set has 25% discount. ...
- 1
0
votes
1
answer
111
views
Is it possible to apply stable diffusion to text?
Is it possible in theory to apply Stable Diffusion to a text domain?
I'm trying to generate text using a Seq2Seq approach, and I'm wondering whether or not it's possible to apply stable diffusion by ...
0
votes
0
answers
14
views
Text generation from noise
What are some general methods for generating text from a trained model and a random key/noise? I am thinking along the lines of a 1 dimension stable diffusion (but there are probably better ways out ...
- 101
1
vote
1
answer
238
views
Using different tokens for padding, end-of-sentence, and start-of-sentence in autoregressive sequence modeling?
Is there utility in using different tokens for end-of-sentence, start-of-sentence, and padding for autoregressive sequence modeling (i.e. text generation)?
Or can I use the same token for all of them?
1
vote
1
answer
38
views
Abstracted text summarisation and generation from weighted keywords
Suppose I have a list of weighted keywords/phrases, such as "solar panel", "rooftop", etc. The weights are in [0,1] with higher weights indicating a stronger preference for ...
- 113
0
votes
0
answers
50
views
Dialogue history encoding for multi-turn dialogues using Seq2seq
In single-turn dialogue seq2seq models where the goal is to produce a good answer y to a query x, sentences are usually encoded such that x is fed to the encoder, while the decoder is only given a &...
0
votes
1
answer
90
views
Advantages of different tokenizers for NLP (specifically text generation)
What are the advantages of using different tokenizers? For example, let's take the sentence:
"In Düsseldorf I took my hat off. But I can't put it back on."
The treebank tokenizer yields: &...
0
votes
0
answers
40
views
is it possible to turn a list of sentences into paragraph?
I have a problem and seeks advise, I have a couple of sentences like:
...
- 1
0
votes
0
answers
25
views
Unstructured Text Prediction
I have a question regarding a real-world problem on medical data. I have an input and output dataset which looks like this:
Input
Output
Patient is sick with fever
Prescribe Panadol and tylenol
...
- 1
0
votes
1
answer
18
views
Is there a machine learning model that is able to take reviews as input and output a new and unique blog article from them?
I am looking for a machine learning model ideally with inference speeds of no longer than a few minutes that is able to take in n human written reviews and output a blog article from them.
The model ...
- 103
1
vote
1
answer
30
views
Next-word Generation in Tabular Dataset
I'll build next-word generation using Tensorflow to predict address mapping. But, I saw many tutorial, next-word generation use long-text narration for its training dataset. But, I have dataset ...
- 41
1
vote
1
answer
128
views
Why do RNN text generation models treat word prediction as a classification task?
In many of the sources I have found regarding text generation with word-based RNN models (LSTM or GRU), the model is trained to perform a classification task across the vocabulary (such as with ...
- 113
1
vote
0
answers
39
views
Guide to Natural language Prompt programming for few-shot learning of Pretrained Language Models
I'm currently working on a project with the goal of producing AI content in the space of a content generation like blog writing, Instagram caption generation etc. Found the in-context few-shot ...
- 11
1
vote
0
answers
113
views
Using LSTM for text generation keeps generating same word
I work on a simple text generation problem using a portion of the Shakespeare dataset that I decided to use LSTM for. I primarily used this tutorial for reference. However, as I ran the below code, I ...
- 289
1
vote
0
answers
57
views
Does N-gram language model for text generation are more efficient than Neural Network language models?
I recently build an language model with N-gram model for text generation and for change I started exploring Neural Network for text generation. One thing I observed that the previous model results ...
- 11
0
votes
1
answer
512
views
Word-level text generation with word embeddings – outputting a word vector instead of a probability distribution
I am currently researching the topic of text generation for my university project. I decided (ofc) to go with a RNN getting a sequence of tokens as input with a target of predicting the next token ...
- 101
0
votes
0
answers
423
views
Is is possible to make a text generator with sklearn?
So recently I made a Tensorflow model using RNN (Recurrent neural networks) and I was wondering if it was possible with sklearn too, through the usage of SVMs or Naive bayes.
I searched up many ...
- 101
0
votes
0
answers
32
views
How effective is text generation?
I have implemented some basic models like composing a poem using the dataset of poems. But the results were not that good in general. I want to make a model that could write an essay for me. One ...
1
vote
0
answers
35
views
Text generation with deep neural network?
For my master's project, I have to build a deep learning model for text generation: the model learns on a set of sentences, then it generates new sentences based on those from which it learned.
I ...
- 11
2
votes
2
answers
102
views
Generation of medical institution names: training corpora?
My question is quite similar to this one: Generation of institution names. I need to be able to produce 'fake' names of medical institutions, specifically to create data for unit tests. Unfortunately, ...
1
vote
0
answers
90
views
English to "basic English" translation
I'd like to build something (ideally in Python) that can translate an English sentence into "basic" English.
Are there any free/open-source tools/frameworks that can help? If not, what kind ...
- 141
1
vote
0
answers
547
views
LSTM Text Generation with Pytorch
I am currently trying quote generation (character level) with LSTMs using Pytorch. I am currently facing some issues understanding exactly how the hidden state is implemented in Pytorch.
Some details:
...
3
votes
1
answer
2k
views
Pytorch: understanding the purpose of each argument in the forward function of nn.TransformerDecoder
According to https://pytorch.org/docs/stable/generated/torch.nn.TransformerDecoder.html, the forward function of nn.TransformerDecoder contemplates the following arguments:
tgt – the sequence to the ...
- 377
7
votes
1
answer
6k
views
Minimal working example or tutorial showing how to use Pytorch's nn.TransformerDecoder for batch text generation in training and inference modes?
I want to solve a sequence-to-sequence text generation task (e.g. question answering, language translation, etc.).
For the purposes of this question, you may assume that I already have the input part ...
- 377
1
vote
0
answers
55
views
Best way to suggest answers given historical question-answer pairs
Many question-answering implementations focus on extracting information from large documents/corpora of text such as Wikipedia.
I have access to a full chat log from the customer service of a large ...
- 11
1
vote
0
answers
25
views
Img to text tensorflow
I read this article: https://www.tensorflow.org/tutorials/text/image_captioning
And i understand main concepts of this article.
We have label and image in our dataset. We pass image to our model and ...
- 51
1
vote
1
answer
28
views
How to handle like meaning sentences when working on text summarization
Suppose we have a text like Today is a very bad day. Very bad day is today. I wont come to play.
What kind of technique should I use to summarize similar texts like ...
- 11
0
votes
3
answers
51
views
Distinguish randomly generated texts from reasonable for human texts [closed]
I have strings short texts of 2 types:
'23jd2032n0d2mn', 'fn830n30rn83', 'fhui29n4ok', 'qn4foml', ...
and
...
- 103
1
vote
1
answer
349
views
Based on transformer, how to improve the text generation results?
If I do not pretrain the text generation model like BART, how to improve the result based on transformer like tensor2tensor?
What are the improvement ideas for transformer in text generation task?
- 661
0
votes
1
answer
22
views
Question of pretraining text-generation task, it seems that pretraining is not work for a small model?
My task is to generate keywords from sentences.
I pretrain a text-generation model. I mask the sentences' tokens and predict the whole sentences' tokens.
Pretraining batch_size = 8 and step = 1000000
...
- 661
1
vote
2
answers
48
views
Generation of institution names
I have found a number of parsers for the automatic extraction of institution names from texts (e.g. this one). My task is in a sense the inverse one: I want to automatically generate reality-like ...
- 111
2
votes
1
answer
109
views
Generate text using user-supplied keywords
I've got a use case where I need to generate sentences based on a set of user supplied keywords. Here is an example of what I need:
User input:
End-User: Data Scientists
Region: Middle East
...
- 141
0
votes
1
answer
36
views
Is it a Problem if Training Data and Evaluation Data are Very Similar?
I'm working on a group project where we have decided to use this Microsoft WIP repository as our starting point. It's a project which compares different frameworks and models and their ability to do ...
- 155
1
vote
0
answers
11
views
Looking for suggestions on performing Sementic Analysis of ASR text
Currently I am working on a project where I have ASR on which I am performing semantic analysis to extract meaning out of it. The ASR text contains huge amount of vague conversational text which needs ...
- 111
3
votes
2
answers
132
views
How to generate a sentence with exactly N words?
Thanks to GPT2 pretrained model now it is possible to generate meaningful sequence of words with or without prefix. However a sentence should end with a proper endings (.,!,?). I am just wondering how ...
- 31
1
vote
0
answers
155
views
Paraphrase Generation - state-of-the-art?
I need to paraphrase short, but abstract sentences, such as:
He prefers variety to routine.
or
I am easily discouraged.
I've played around couple of online tools (such as https://quillbot.com/)...
- 121
1
vote
0
answers
67
views
Pretrained Models for Keyword-Based Text Generation
I'm looking for an implementation that allows me to generate text based on a pre-trained model (e.g. GPT-2).
An example would be gpt-2-keyword-generation (click here for demo). As the author notes, ...
- 121
0
votes
1
answer
172
views
predicting next jobtitle
I have a dataset of which has 30M rows each like [current_jobtitles, nextjobtitles].
...
- 239
4
votes
1
answer
410
views
NLP - Paraphrase extraction in Python
I am trying to develop a NLP model, which takes something like you have high levels of cholesterol(this will be a tag) as input and has to output something like <...
