# Transformer-based architectures for regression tasks

As far as I've seen, transformer-based architectures are always trained with classification tasks (one-hot text tokens for example). Are you aware of any architectures using attention and solving regression tasks? Could one build a regressive auto-encoder for example? How would normalization fit into this (as LayerNorm destroys some of the information from the input)?