One Hot Encoding of Age

Question

My task is to predict how many years a person has left to live using an MLP. There is one specific feature I'd like to discuss: current age.

Statistically, it's a conditional probability. Example: When 0 years old, I'm expected to live until 70 years old (70 years left). However when I'm 70 years old, I'm expected to live until 83 (13 years left).

In my dataset, I have the true age of when someone died, and it follows a distribution. I have thus augmented my data, if someone died at the age of N, then there will be N datapoints of current age(feature) from 0 -> N and years left to live (target) from N -> 0 accordingly.

Through this augumentation, I'm hoping to model the distribution of current age vs years left to live through the data. I would never want my network to output (after denormalization) that if the current age was 70, the person would have 50 years left.

Now, the best practice of an MLP states that 1-2 hidden layers is enough, and amount of nodes in the hidden layers should be between the amount of input and output layers. However if I just normalized age through feature scaling, I would have one input node, one hidden node and one output node. This is realistically not enough, I'd expect that I need a lot of hidden nodes.

If I used one hot encoding that represents current age from 0 to 100. I'd be able to use 50 hidden nodes through good practice. If I was 70 years today, I would probably never activate the node that will tell me I'd have another 50 years to live.

My question then, is that should I skip the one hot encoding, and just scale the age and use for example 50 hidden nodes?

Theoretically, because age is continuous as not as distinct nor discrete as say "cat" and "robot" then one-hot is not necessary, but is there any drawbacks if I choose to one-hot encode?

Answer 1

The task of predicting how many years a person has left to live is called survival analysis. Survival analysis is a type of time to event analysis. Thus, survival analysis needs a special type of loss function. An appropriate loss function would avoid predictions like 50 years left when the current age is 70. A common survival analysis model is Cox regression model which can be adapted to neural networks.

If survival analysis is used, the current age can be inputted in the model directly in a single input node.

Answer 2

The drawbacks of one-hot encoding is that it generates very sparse matrices. Neural Networks hate sparse data: gradient descent optimizers give their best on continuous variables, while weight updates do not work very good when there is so little variability.

I would never recommend one-hot encoding for ANNs. I use it only when it's absolutely necessary, and avoid it anytime I can. My personal experience as a data scientist tells me sparse data matrices lead to generally bad results.

However, what I think you're going to need is a statistical technique called survival analysis. It actually comes with many names: survival regressions, event-history models, duration models, etc. It's a technique invented by biostatisticians several decades ago, that is meant to estimate the probability of the death of a patient, given how much time passed and external variables. It was later applied to many fields of science that have nothing to do with medicine. It can be generalized as a technique that is meant to estimate the likelihood of a termination event happening. Mathematically speaking, it is estimating a 'survival probability' of a process through time. There is plenty of information you can find just googling. A good start is here, here and here. One of the best books ever written on the topic is this one (it's written for social scientist but it's super useful for anybody.)

Somebody developed Neural Survival models, using ANNs to estimate the survival probability of a process. However I never happened to try these models in practice. You can read some interesting stuff here and here.

This class of models (ANN or not) will let you treat age as a continuous variable.

Answer 3

We know an MLP can compute any function on a compact support (up to any degree of accuracy). I'm not a neural network expert, but my intuition says that you could try to model it with a network with a hidden layer containing say 100 hidden neurons. Then just feed it the data you mentioned: for each age $a$ feed it the training data $\{(0, a),\ (1, a-1), \ \ldots,\ (a, 0)\}$. I'm not sure binning the results would be helpful in this case. It might be helpful to first take a known distribution such as the Gaussian distribution (which of course life expectancy is not), generate some training data from that, and see if you can approximate it.