1
$\begingroup$

In the text-to-data(music, image, audio, etc.) generative AI field, one method of encoding text prompts is to use pre-trained language models. Such an approach was used in research on Moûsai [1] and Photorealistic [2] , for example. And in such recent text-to-data generative AI fields, T5 is often used for this pre-trained language model.

But why is T5 often used?

I also wonder if LLMs with a larger number of parameters than T5 could be used to better represent the complexity and composition of music. Therefore, I do not understand why such LLMs are not used.

References:

[1] Schneider, et al., ‘Moûsai: Text-to-Music Generation with Long-Context Latent Diffusion’, Available: https://arxiv.org/abs/2301.11757

[2] Saharia et al., 'Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding', Available: https://arxiv.org/abs/2205.11487

$\endgroup$

1 Answer 1

1
$\begingroup$

Unlike many of the other LLMs, T5 is an encoder-decoder model. (Most are encoder-only like BERT, or decoder-only like the GPT models.)

So it is well suited to sequence to sequence applications. The 11B model is also far from being a small model. Running anything bigger is going to require some serious hardware resources.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.