# How do I encode time in high dimensional space?

I have a dataset of form text, text, category, category, time, text and I would like to apply the attention mechanism to it. This requires that all inputs be in the same vector space. I am using a particular encoding method (from BERT) for the text-type data and I can build a custom trainable embedding for the category features. However, I don't have a good way of embedding time data.

Currently, my time feature is normalised on [0,1], and represents when over the time period (one year) the post was created. Naively, I would split this up into month, day of week, and time of day features to do feature engineering, but I don't have a good way to embed this in very high dimensional space (say, 500+ dimensional space).

What's the best option here? I would like to avoid tiling or repeating the same feature set to reach the requisite dimensionality -- is there a better way? I could put a trainable embedding layer on top of those three features, too, but this seems suboptimal.