You are going in the right direction. Dates can contain a lot of information depending on the task you want to learn. A problem with your suggestion of using the day of the year straight up is that the last day of the year and the first day of the following year are very close to each other, while in your representation they are the furthest away. An alternative to this could be using the fact that this is cyclical and map the
day_of_year feature onto a circle and use the coordinates on this circle as features to represent this. A lot of the important features regarding dates are cyclical, like the day of the week, the day of the month, the hour of the day etcetera. An alternative which allows for easier non-linear relationships would be to one-hot encode them as classes. This has the advantage of a more free representation, but the downside that it will cost more features and it cannot generalize based on the fact that two options might be next to each other.
With regards to additional domain knowledge, like knowing that being close to christmas, or close to salary day might be relevant, it's almost always beneficial to add these features to your data. This could be done by calculating absolute distance to christmas, or days before pay day etcetera. Some of these features can be work intensive to create, for example localized holiday features, but they can significantly influence people's decisions and if that is what you are trying to predict, adding these will benefit your performance.