Timeline for How is WordPiece tokenization helpful to effectively deal with rare words problem in NLP?
Current License: CC BY-SA 4.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 12 at 2:10 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jun 6 at 3:02 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jan 31 at 2:08 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Sep 30, 2023 at 14:04 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Aug 25, 2023 at 11:11 | answer | added | Sreejithc321 | timeline score: 0 | |
| Jan 9, 2020 at 7:54 | comment | added | Aditya | Here you go... youtube.com/watch?v=zJW57aCBCTk | I've recently started following this guy. He has a very nice way of explaining such concepts. | |
| Aug 22, 2019 at 14:52 | comment | added | Laura | The URL is broken. Here the corrected one. | |
| Apr 2, 2019 at 12:35 | vote | accept | Harman | ||
| Apr 2, 2019 at 12:35 | comment | added | Harman | This question has been answered here. I'm copying the answer here as well. WordPiece and BPE are two similar and commonly used techniques to segment words into subword-level in NLP tasks. In both cases, the vocabulary is initialized with all the individual characters in the language, and then the most frequent/likely combinations of the symbols in the vocabulary are iteratively added to the vocabulary. Consider the WordPiece algorithm from the [original paper](static.googleusercontent.com/media/research.google.com/en//pubs/… | |
| Mar 27, 2019 at 16:54 | history | asked | Harman | CC BY-SA 4.0 |