I've trained a transformer model based on the pytorch tutorial: https://pytorch.org/tutorials/beginner/transformer_tutorial.html, But I found I've difficulties to understant this model's input and output connections. According to its training code, the model's input data is:
t, t, t,...,t[n], and its output target value should be t, t,...,t[n], t[n+1].
input: t, t, t,...,t[n-1], t[n]
output:t, t, t,...,t[n], t[n+1].
And based on my unerstanding, t depends on t, t depends on t, t, and so on, t[n+1] depends on t[n], t[n-1], ..., t.
My question is, since we need cut a long tokens list into multiple segments, and input these segments into the transformer model one by one, let's assume one segment has n tokens, is there any connection exists between two segments? e.g. does any state connection exist between the t[2n+2] and t? Simply to say, is the 2nd segments target value t[2n+2] decided by t, t, ..., t[n], t[n+1], t[n+2]...,t[2n+1]?