New answers tagged

1

First I suggest reading the transformers paper. Couple of quick notes is that this model consists of an encoder and a decoder, and the original task the paper is trained on is machine translation. Datasets (benchmarks) they used to train and evaluate this model from scratch were WMT 2014 Engligh-to-German, WMT 2014 English-to-French (section 5.1 of the paper)...


0

I'm not sure there is a need for aggregation, or in other words you may have a pipeline mismatch. BERT sentencepiece tokenization is specifically meant to be passed to some set downstream pipelines, with the aim of the sentencepiece thing being to be able to cater to OOV words. By aggregating the sentencepiece tokens, you might be doing away with the benefit ...


Top 50 recent answers are included