We do have models that predict the basic color from its description, by basic color I mean red, blue, black etc. But I would like to develop a model that can spit out the RGB or HEX colors by a description of it, an example being, "yellow that is glossy and sorta dark" should give the respective value for the same. Another example would be, "Clear green plastic". This is relative to 3d modelling where I input this text and change the material of the object on the screen.
In fact, you want to translate "yellow that is glossy and sorta dark" by (170,173,11).
A good way to solve this, is by using a neural machine translation model.
Therefore, you can use a encoder/decoder system like many translation models, but with 3 digits as output. To achieve this, you will want to have training data with plenty of text to color translation, but they don't need to be too exhaustive if you cover enough scenarios. The output sequence could be 3 digits 170 173 11. The neural machine translation should adapt itself to the different results automatically, and make good predictions because we have a similar sequence structure as any language.
You have plenty of neural machine translation code examples, here is one among others: https://machinelearningmastery.com/develop-neural-machine-translation-system-keras/