New answers tagged

0 votes

ChatGPT's Architecture - Decoder Only? Or Encoder-Decoder?

TLDR (simplified): encoder sees into future, decoder predicts transformer sees into future and then predicts, encodes, then decodes gpt doesnt see into future, it only predicts - thats why it's ...
psycho brm's user avatar
1 vote

How to Combine tfidf with LSTM in keras?

Using TfIdf with LSTM is not a common approach, as LSTM networks are generally more suitable for handling sequential data like text sequences. TfIdf, on the other hand, is a technique commonly used ...
lvvittor's user avatar
1 vote

Semantic Search on numeric data

You could feed the LLM a description of the file format and then request it to generate a piece of code to extract the information you want, for instance, in Python. Then, you would run the generated ...
noe's user avatar
  • 23.8k
2 votes
Accepted

Purely extractive Language Model

You can prepend each line of the email with a line number and request the LLM to give you the initial and final line numbers of the most recent email, separated by "-". Then, you can parse ...
noe's user avatar
  • 23.8k
0 votes

Convert natural language text to structured data

This is quite a large project, I can't give an outline for all you asked but I would highly recommend looking into a library called Skweak: https://github.com/NorskRegnesentral/skweak This library is ...
TheAmazingDerik's user avatar
1 vote
Accepted

What is the input to an encoder-decoder transformer in next word prediction task?

If you just want the model for doing next token prediction, then you would not use an encoder-decoder architecture. Instead, you would use a only the decoder, and feed the text you have to it to get ...
noe's user avatar
  • 23.8k
1 vote
Accepted

How does Bert masked language modelling task make sense if half the time the next sentence is wrong context in the sequence passed through the encoder

First, note that the purpose of next sentence prediction objective is not to contribute to the contextual embeddings part, but to allow other downstream tasks like sentence classification and textual ...
noe's user avatar
  • 23.8k
1 vote

Unsupervised Machine Translation System Using Variational Autoencoder Models

First I will answer your questions: The tokenization strategy is constrained by how the alignment is done. If you need word-level representations for the alignment, then the tokenization should be at ...
noe's user avatar
  • 23.8k
0 votes

Is there a Language Model that can accept huge corpse of tabular data and answr questions about?

Yes. Use GPT4 to generate code to print output, then it can read and interpret the output. This is already a product offered by a few companies that people use. I have done exactly what you are ...
BigMistake's user avatar
3 votes

Is there a Language Model that can accept huge corpse of tabular data and answr questions about?

Language models will experience difficulties processing in an analytical way tabular data and answering complex questions about it because of how they work (just predicting the next token). Maybe you ...
Giovanni Amorim's user avatar
0 votes

How can someone evaluate llama index model?

Is there any other way to evaluate model ? Assuming you have a test set with gold answers, you can use a text generation metric. See Evaluation of Text Generation: A Survey. Typical metrics: TF-IDF ...
Franck Dernoncourt's user avatar

Top 50 recent answers are included