1
$\begingroup$

Suppose I have a bunch of sequences of varying lenghts. The absolute majority of them are short, just a few dozens items long. However, very few of them are significantly longer - more than a hundred items long. The question is, how to organize them efficiently as an input to recurrent layer?

Padding doesn't work, since many sequences need to be heavily padded. Limiting batch size is not an option - these sequences are obtained as parts of a larger structure.

Improve this question
$\endgroup$
1
  • $\begingroup$ Could you please tell us more about the problem, you're trying to solve? $\endgroup$
    – Piotr Rarus
    Commented Dec 17, 2019 at 14:25

1 Answer 1

Reset to default
1
$\begingroup$

Sequence bucketing

Depending on the input length of sequences you can dynamically change the padding and speed it up.

Take a look at here Sequence bucketing

Improve this answer
$\endgroup$
2
  • $\begingroup$ I had a glance. It seems like breaking data into smaller batches, long going with long and short going with short, correct? $\endgroup$
    – steam_engine
    Commented Dec 17, 2019 at 15:54
  • $\begingroup$ There are different techniques to solve this, at the end of the day, optimisation problem. What you said is one way, that could be expensive (depending on the data) You can also use arbitrary batches and depending on the size of the longest in every batch padd accordingly. Have a look at here medium.com/@rashmi.margani/… $\endgroup$
    – Noah Weber
    Commented Dec 17, 2019 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.