New answers tagged nlp
0
votes
ChatGPT's Architecture - Decoder Only? Or Encoder-Decoder?
TLDR (simplified):
encoder sees into future, decoder predicts
transformer sees into future and then predicts, encodes, then decodes
gpt doesnt see into future, it only predicts - thats why it's ...
- 101
1
vote
How to Combine tfidf with LSTM in keras?
Using TfIdf with LSTM is not a common approach, as LSTM networks are generally more suitable for handling sequential data like text sequences. TfIdf, on the other hand, is a technique commonly used ...
- 71
1
vote
Semantic Search on numeric data
You could feed the LLM a description of the file format and then request it to generate a piece of code to extract the information you want, for instance, in Python. Then, you would run the generated ...
- 23.8k
2
votes
Accepted
Purely extractive Language Model
You can prepend each line of the email with a line number and request the LLM to give you the initial and final line numbers of the most recent email, separated by "-". Then, you can parse ...
- 23.8k
0
votes
Convert natural language text to structured data
This is quite a large project, I can't give an outline for all you asked but I would highly recommend looking into a library called Skweak: https://github.com/NorskRegnesentral/skweak
This library is ...
1
vote
Accepted
What is the input to an encoder-decoder transformer in next word prediction task?
If you just want the model for doing next token prediction, then you would not use an encoder-decoder architecture. Instead, you would use a only the decoder, and feed the text you have to it to get ...
- 23.8k
1
vote
Accepted
How does Bert masked language modelling task make sense if half the time the next sentence is wrong context in the sequence passed through the encoder
First, note that the purpose of next sentence prediction objective is not to contribute to the contextual embeddings part, but to allow other downstream tasks like sentence classification and textual ...
- 23.8k
1
vote
Unsupervised Machine Translation System Using Variational Autoencoder Models
First I will answer your questions:
The tokenization strategy is constrained by how the alignment is done. If you need word-level representations for the alignment, then the tokenization should be at ...
- 23.8k
0
votes
Is there a Language Model that can accept huge corpse of tabular data and answr questions about?
Yes. Use GPT4 to generate code to print output, then it can read and interpret the output. This is already a product offered by a few companies that people use.
I have done exactly what you are ...
- 101
3
votes
Is there a Language Model that can accept huge corpse of tabular data and answr questions about?
Language models will experience difficulties processing in an analytical way tabular data and answering complex questions about it because of how they work (just predicting the next token). Maybe you ...
- 326
0
votes
How can someone evaluate llama index model?
Is there any other way to evaluate model ?
Assuming you have a test set with gold answers, you can use a text generation metric. See Evaluation of Text Generation: A Survey. Typical metrics: TF-IDF ...
- 5,630
Top 50 recent answers are included