Questions tagged [text-generation]

The tag has no usage guidance, but it has a tag wiki.

Filter by
Sorted by
Tagged with
1 vote
1 answer
392 views

English to "basic English" translation

I'd like to build something (ideally in Python) that can translate an English sentence into "basic" English. Are there any free/open-source tools/frameworks that can help? If not, what kind ...
0 votes
0 answers
2 views

"Background of the study" for proposal on title "Factors Affecting Students' Academic Achievement in Mathematics in secondary schools of silte zone"

✔️**** Background of the Study The educational sphere of Silte Zone, a region in the South Nations, Nationalities, and Peoples' Regional State in Ethiopia, faces challenges that hinder academic ...
4 votes
1 answer
85 views

LLMs for text generation

We know that AI is rapidly growing. do we have any large language models (LLMs) to process images, pdf documents directly (fine-tune approach) for text generation tasks?
0 votes
1 answer
288 views

Generate similar text based on category or the similar texts

I'm trying to generate the similar text based on the category or to generate text by combining similar texts into the new text. I was checking multiple nlp tasks like question generation, but they don'...
1 vote
1 answer
399 views

Is it possible to use Word2vec for text paraphrasing?

After reading several papers I am not sure if it is possible to some how generate text with the same meaning (paraphrase it) using only Word2vec. I found out other approaches that use sequences of ...
0 votes
0 answers
22 views

Details about Pre-existing knowledge database in RAG for LLMs

In RAG, As part of retriever model- we are retrieving the relevant information from external knowledge source (i.e. vector database) and this database is always updating with new updates. In ...
2 votes
1 answer
58 views

How to select the optimal beam size for beam search?

Most Text Generation Models use beam search to select the optimal output candidate. How does one choose the optimal beam size? It would probably vary from task to task, dataset to dataset, and model ...
0 votes
1 answer
45 views

Getting a free and unknown answer to a question against a fine-tuned text generation model trained on many essays and their few questions and answers

Aim I want to fine-tune a text generation model with essays of changing size and then ask each of these input texts a few questions. I already have a wider range of question-answer pairs at hand for ...
2 votes
1 answer
867 views

Fine-tuning a pre-trained LLM for question-answering

Objective My goal is to fine-tune a pre-trained LLM on a dataset about Manchester United's (MU's) 2021/22 season (they had a poor season). I want to be able to prompt the fine-tuned model with ...
0 votes
0 answers
45 views

How to improve performance of a Retrieval Augmented Generative (RAG) model?

I had implemented a Retrieval Augmented generation (RAG) model on the Healthcare CSV file. The model has to give answers to natural language queries based on the data provided. After implementing the ...
0 votes
1 answer
703 views

Word-level text generation with word embeddings – outputting a word vector instead of a probability distribution

I am currently researching the topic of text generation for my university project. I decided (ofc) to go with a RNN getting a sequence of tokens as input with a target of predicting the next token ...
3 votes
1 answer
516 views

Choosing a right algorithm for template-based text generation

I am doing a text generation project -- the task is to basically represent the statistical data in a readable way. The way I decided to go about this is template-based: each data type has a template ...
0 votes
1 answer
217 views

How does RAG (Retrieval Augmented Generation ) work around limited context length?

My understanding of the RAG pipeline can be summarized with the following diagram: I understand steps 1-7 splits and vectorizes an external text data source into chunks and steps 8-11 retrieves n ...
0 votes
1 answer
25 views

Multilingual sentence generation with Hugging Face

For an application I need to generate some random sentences, i.e. I don't need the output sentences to have any specific link to the prompt other than using the same language. If possible I need this ...
2 votes
1 answer
154 views

Create an RNN on text sources with different lengths

I want to create an RNN to generate a new text based on many examples of existing texts of a certain format in the training data. The type of texts in the training data consists of 3 segments, like so:...
0 votes
0 answers
73 views

Fine-tuning MT5 for making it more like ChatGPT

I am trying to fine-tune a model which works like ChatGPT for Punjabi language, using the mt5-base, however I am not sure if I should go ahead with it since it does not even generate text and when I ...
0 votes
0 answers
49 views

How to train a custom sentence completion model using tensorflow?

What I have A small corpus of English sentences (about 60,000). The Task Sentence completion model. Basically, something like Input: "Paris is the" Output: "Paris is the capital of ...
0 votes
0 answers
22 views

Seeking Solutions for Generating Text Descriptions from Diagrams, Infographics, Charts, etc

I am currently on a quest to find an efficient way to generate meaningful text descriptions (or alt text) from visual representations such as diagrams, infographics, charts, plots and the like. ...
1 vote
0 answers
214 views

How to monitor training of text generation models?

I'm finetuning a pretrained Huggingface model based on Transformers for a downstream Text Generation task, but I have doubts on how the fine-tuning process should be monitored: In classification, I ...
0 votes
0 answers
29 views

Pretrained model for RNN Encoder-Decoder?

Our team are implementing a paper called Cold-Start-Reinforcement-Learning-with-Softmax-Policy-Gradient. Although the paper didn't mention. We want to use a pre-trained model, which is a RNN Encoder-...
1 vote
1 answer
423 views

Using different tokens for padding, end-of-sentence, and start-of-sentence in autoregressive sequence modeling?

Is there utility in using different tokens for end-of-sentence, start-of-sentence, and padding for autoregressive sequence modeling (i.e. text generation)? Or can I use the same token for all of them?
2 votes
1 answer
2k views

How do we evaluate the outputs of text generation models?

Evaluation of a wide variety of natural language generation (NLG) tasks is difficult. For instance, for a question answering model, it is hard for a human to quantify how well the model has answered a ...
0 votes
1 answer
142 views

Is it possible to apply stable diffusion to text?

Is it possible in theory to apply Stable Diffusion to a text domain? I'm trying to generate text using a Seq2Seq approach, and I'm wondering whether or not it's possible to apply stable diffusion by ...
1 vote
1 answer
143 views

Preprocessing advice for large text corpus in natural language generation (NLG)

I have a large text corpus (i.e. 30 million sentences, all in lowercase in the format of Penn Treebank) that I want to use to train a neural network for natural language generation. What preprocessing ...
1 vote
1 answer
49 views

I have 2 Columns of text, Should I use different vectorizer and Embeddings for each or just one?

I have a dataset with two input columns as text. Should I use same textvectorizer in both columns or different ones? I am asking this because. columns a has average ...
0 votes
1 answer
86 views

Models that are good for long answer generation given context and question and what datasets would be the best for training?

Basically I am trying to create a context-needing question and long answer model and I was wondering what model would be best for such tasks, currently I am leaning towards T5, or GPT-NeoX-20B. ...
1 vote
2 answers
110 views

How to evaluate Natural Question-Answer Generation pairs?

I am trying to generate Natural Question-Answer for a specific domain. I am using a Large Language Model (LLM). I have only context to generate question-answers but don't have any ground truth. How to ...
1 vote
1 answer
62 views

Abstracted text summarisation and generation from weighted keywords

Suppose I have a list of weighted keywords/phrases, such as "solar panel", "rooftop", etc. The weights are in [0,1] with higher weights indicating a stronger preference for ...
0 votes
0 answers
63 views

Dialogue history encoding for multi-turn dialogues using Seq2seq

In single-turn dialogue seq2seq models where the goal is to produce a good answer y to a query x, sentences are usually encoded such that x is fed to the encoder, while the decoder is only given a &...
0 votes
1 answer
112 views

Advantages of different tokenizers for NLP (specifically text generation)

What are the advantages of using different tokenizers? For example, let's take the sentence: "In Düsseldorf I took my hat off. But I can't put it back on." The treebank tokenizer yields: &...
0 votes
1 answer
18 views

Is there a machine learning model that is able to take reviews as input and output a new and unique blog article from them?

I am looking for a machine learning model ideally with inference speeds of no longer than a few minutes that is able to take in n human written reviews and output a blog article from them. The model ...
1 vote
1 answer
33 views

Next-word Generation in Tabular Dataset

I'll build next-word generation using Tensorflow to predict address mapping. But, I saw many tutorial, next-word generation use long-text narration for its training dataset. But, I have dataset ...
0 votes
0 answers
520 views

Is is possible to make a text generator with sklearn?

So recently I made a Tensorflow model using RNN (Recurrent neural networks) and I was wondering if it was possible with sklearn too, through the usage of SVMs or Naive bayes. I searched up many ...
1 vote
1 answer
199 views

Why do RNN text generation models treat word prediction as a classification task?

In many of the sources I have found regarding text generation with word-based RNN models (LSTM or GRU), the model is trained to perform a classification task across the vocabulary (such as with ...
2 votes
1 answer
50 views

Automatic data summarization with text

I would like to automate periodic report writing based on data. Given one/some data tables, the machine should output texts like ...
1 vote
0 answers
47 views

Guide to Natural language Prompt programming for few-shot learning of Pretrained Language Models

I'm currently working on a project with the goal of producing AI content in the space of a content generation like blog writing, Instagram caption generation etc. Found the in-context few-shot ...
1 vote
0 answers
59 views

Does N-gram language model for text generation are more efficient than Neural Network language models?

I recently build an language model with N-gram model for text generation and for change I started exploring Neural Network for text generation. One thing I observed that the previous model results ...
1 vote
0 answers
128 views

Using LSTM for text generation keeps generating same word

I work on a simple text generation problem using a portion of the Shakespeare dataset that I decided to use LSTM for. I primarily used this tutorial for reference. However, as I ran the below code, I ...
4 votes
1 answer
420 views

NLP - Paraphrase extraction in Python

I am trying to develop a NLP model, which takes something like you have high levels of cholesterol(this will be a tag) as input and has to output something like <...
1 vote
0 answers
2k views

Distractor Generation for Multiple Choice Questions

I'm currently working on generating distractor for multiple choice questions. Training set consists of question, answer and 3 distractor and I need to predict 3 distractor for test set. I have gone ...
0 votes
1 answer
179 views

predicting next jobtitle

I have a dataset of which has 30M rows each like [current_jobtitles, nextjobtitles]. ...
0 votes
0 answers
32 views

How effective is text generation?

I have implemented some basic models like composing a poem using the dataset of poems. But the results were not that good in general. I want to make a model that could write an essay for me. One ...
1 vote
0 answers
43 views

Text generation with deep neural network?

For my master's project, I have to build a deep learning model for text generation: the model learns on a set of sentences, then it generates new sentences based on those from which it learned. I ...
2 votes
2 answers
150 views

Generation of medical institution names: training corpora?

My question is quite similar to this one: Generation of institution names. I need to be able to produce 'fake' names of medical institutions, specifically to create data for unit tests. Unfortunately, ...
1 vote
0 answers
682 views

LSTM Text Generation with Pytorch

I am currently trying quote generation (character level) with LSTMs using Pytorch. I am currently facing some issues understanding exactly how the hidden state is implemented in Pytorch. Some details: ...
3 votes
1 answer
3k views

Pytorch: understanding the purpose of each argument in the forward function of nn.TransformerDecoder

According to https://pytorch.org/docs/stable/generated/torch.nn.TransformerDecoder.html, the forward function of nn.TransformerDecoder contemplates the following arguments: tgt – the sequence to the ...
7 votes
1 answer
9k views

Minimal working example or tutorial showing how to use Pytorch's nn.TransformerDecoder for batch text generation in training and inference modes?

I want to solve a sequence-to-sequence text generation task (e.g. question answering, language translation, etc.). For the purposes of this question, you may assume that I already have the input part ...
2 votes
1 answer
129 views

Generate text using user-supplied keywords

I've got a use case where I need to generate sentences based on a set of user supplied keywords. Here is an example of what I need: User input: End-User: Data Scientists Region: Middle East ...
1 vote
0 answers
62 views

Best way to suggest answers given historical question-answer pairs

Many question-answering implementations focus on extracting information from large documents/corpora of text such as Wikipedia. I have access to a full chat log from the customer service of a large ...
1 vote
1 answer
30 views

How to handle like meaning sentences when working on text summarization

Suppose we have a text like Today is a very bad day. Very bad day is today. I wont come to play. What kind of technique should I use to summarize similar texts like ...