I'm novice to deep learning and sorry if these questions may look very basic.
******************* First Question *******************
I need to predict the next k-characters given some initial text input, where the algorithm is learnt on a piece of text (character sequence). I've done some work on predicting only 1-character at a time (using LSTM and Keras wrapper), but my question is how to generate k-characters and not only one. Of course I can generate k-characters recursively using only 1-character prediction algorithm, but I'm wondering if this problem (input text to k-characters) is well known and there exist a state-of-the-art solution.
What I learnt from reading relevant topics, is that this problem is (similar) to "sequence-to-sequence" learning and of course there are many solutions and implementations available (theano, tensorflow, etc). Am I correct?
For example, given a string of letters:
seq_1 = 'aaaaabababcbcbbababbbdcbbbaaabbbaad...'
the algorithm should predict (given a randomly picked input sequence):
aaabbb -> aad
(the last 9 characters from the aforementioned example )
******************* Second Question *******************
I know, that neural network algorithms are very "data hungry" and in order to make meaningful learning and prediction accuracy, one has to provide a lot of data (compared to shallow learning methods) If I have only relatively short strings of letters (50-100 characters), does it make any sense to train seq2seq algorithm (LSTMs network) on such short sequences, but with many bootstrapped random instances?
Are there any theoretical (or some rules of thumb) bounds on the size of the training set and the reliability of the trained algorithm? (overfitting)
I'm quite motivated by the example of learning temporal patterns in alphabet with LSTM RNNs:
The author got very nice results, by random sampling of sequences of varied length and showing them to LSTM RNN. The idea was to randomly generate a relatively large (~1000 instances) data set and then train the network.
Can anyone comment on this way (bootstrapping) of creating a training set from a small sequence? Is it a standard approach? Are there any statistical flaws? Many thanks!!
