1
$\begingroup$

My task is to predict how many years a person has left to live using an MLP. There is one specific feature I'd like to discuss: current age.

Statistically, it's a conditional probability. Example: When 0 years old, I'm expected to live until 70 years old (70 years left). However when I'm 70 years old, I'm expected to live until 83 (13 years left).

In my dataset, I have the true age of when someone died, and it follows a distribution. I have thus augmented my data, if someone died at the age of N, then there will be N datapoints of current age(feature) from 0 -> N and years left to live (target) from N -> 0 accordingly.

Through this augumentation, I'm hoping to model the distribution of current age vs years left to live through the data. I would never want my network to output (after denormalization) that if the current age was 70, the person would have 50 years left.

Now, the best practice of an MLP states that 1-2 hidden layers is enough, and amount of nodes in the hidden layers should be between the amount of input and output layers. However if I just normalized age through feature scaling, I would have one input node, one hidden node and one output node. This is realistically not enough, I'd expect that I need a lot of hidden nodes.

If I used one hot encoding that represents current age from 0 to 100. I'd be able to use 50 hidden nodes through good practice. If I was 70 years today, I would probably never activate the node that will tell me I'd have another 50 years to live.

My question then, is that should I skip the one hot encoding, and just scale the age and use for example 50 hidden nodes?

Theoretically, because age is continuous as not as distinct nor discrete as say "cat" and "robot" then one-hot is not necessary, but is there any drawbacks if I choose to one-hot encode?

$\endgroup$
1
$\begingroup$

The task of predicting how many years a person has left to live is called survival analysis. Survival analysis is a type of time to event analysis. Thus, survival analysis needs a special type of loss function. An appropriate loss function would avoid predictions like 50 years left when the current age is 70. A common survival analysis model is Cox regression model which can be adapted to neural networks.

If survival analysis is used, the current age can be inputted in the model directly in a single input node.

| improve this answer | |
$\endgroup$
1
$\begingroup$

The drawbacks of one-hot encoding is that it generates very sparse matrices. Neural Networks hate sparse data: gradient descent optimizers give their best on continuous variables, while weight updates do not work very good when there is so little variability.

I would never recommend one-hot encoding for ANNs. I use it only when it's absolutely necessary, and avoid it anytime I can. My personal experience as a data scientist tells me sparse data matrices lead to generally bad results.

However, what I think you're going to need is a statistical technique called survival analysis. It actually comes with many names: survival regressions, event-history models, duration models, etc. It's a technique invented by biostatisticians several decades ago, that is meant to estimate the probability of the death of a patient, given how much time passed and external variables. It was later applied to many fields of science that have nothing to do with medicine. It can be generalized as a technique that is meant to estimate the likelihood of a termination event happening. Mathematically speaking, it is estimating a 'survival probability' of a process through time. There is plenty of information you can find just googling. A good start is here, here and here. One of the best books ever written on the topic is this one (it's written for social scientist but it's super useful for anybody.)

Somebody developed Neural Survival models, using ANNs to estimate the survival probability of a process. However I never happened to try these models in practice. You can read some interesting stuff here and here.

This class of models (ANN or not) will let you treat age as a continuous variable.

| improve this answer | |
$\endgroup$
0
$\begingroup$

We know an MLP can compute any function on a compact support (up to any degree of accuracy). I'm not a neural network expert, but my intuition says that you could try to model it with a network with a hidden layer containing say 100 hidden neurons. Then just feed it the data you mentioned: for each age $a$ feed it the training data $\{(0, a),\ (1, a-1), \ \ldots,\ (a, 0)\}$. I'm not sure binning the results would be helpful in this case. It might be helpful to first take a known distribution such as the Gaussian distribution (which of course life expectancy is not), generate some training data from that, and see if you can approximate it.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the answer! I didn't mean binning, but rather scalling all ages from say [0,100] to [0,1], and use one input node to represent it, so 10 years old becomes 0.1 as input. If I use a distribution to to generate training data, then my model is only trying to model the distribution, not the "unkown" function. Or do you mean that I do it to test if a normal distributed training data would yield a normal distributed function approxation? Which then I can carry over the logic to say that this unkown distribution will also approximate the unkown function? $\endgroup$ – S.Hua Dec 4 '18 at 14:07
  • $\begingroup$ Yes, something like that. It's pretty straightforward to design an MLP that approximates a function with an input node, a hidden layer, and an output node. You do this by building on step functions. The larger the hidden layer, the more accurate you can approximate the function. But in the constructed case, each couple of neurons only contributes to a local part of the function. I'm not sure how well gradient descent would approximate the function. I will actually give it a go and will report back to you. $\endgroup$ – Joris Weimar Dec 4 '18 at 14:12
  • $\begingroup$ I'm going to attempt this aswell. I have also thought about using RNNs. The chance of someone transitioning from alive to dead increases the longer the person has been alive (expected life becomes lower). RNN should be able to learn this relationship. However its hard to make the input data correct, since if someone was 0 years old, they'd live until 70, while if they were 70, they'd live until 80. (I don't have the factual data, only an approximations through know statistic models). However as you know, I'm trying to beat the statistic models, The only data i have is when they died. $\endgroup$ – S.Hua Dec 4 '18 at 14:38
  • $\begingroup$ @S.Hua, and Joris: you both said a while back you'd try this. How did it turn out? (I guess the question got bumped by Community, and now there are two answers suggesting survival analysis techniques. But it would be nice to hear how your approach(es) turned out.) $\endgroup$ – Ben Reiniger Sep 14 '19 at 15:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.