1
$\begingroup$

I have a neural network that takes in roughly twelve values, and outputs a singly probability. The issue is that the network appears to be smart enough to realize there is a very significant correlation between three of the values and the output, and doesn't seem to be recognizing any correlation with the remainder of the inputs. In other words, the model has figured out that if the and of three specific inputs are non-zero, there is an ~80% chance the output is 1.

This seems like expected behavior, but I am looking to have the network enhance it's 80% prediction with data from the other nine inputs. Is this possible?

My proposed solution requires a bit more information regarding the nature of the data. The twelve inputs can be categorized into four groups of three variables each. For instance, we have three metrics: purchase cost, warranty length and average maintenance costs. These three metrics are then applied to four different appliances in a household: the boiler, washing machine, dishwasher and refrigerator. The network is then supposed to output the probability of the owner of the household moving out in a few months. In this example, there happens to be a 80% correlation between refrigerators and the homeowner moving: if they own a refrigerator, there is an 80% chance they will move out within the next few months.

As mentioned before, the network is smart enough to recognize the correlation between the refrigerator and the moving out, but I am hoping to get it to augment this prediction with the inclusion of the other inputs.

My first strategy for addressing this would be to split the single network into four separate networks, each handling one appliance. These will output four probabilities total, which I can then either weight and compute by hand, or feed into a fifth network for final processing. My worry is that this would result in the same issue occurring - the final network would determine that ignoring the first three inputs and only returning the fridge-based probability is the best way to minimize error.

My second possible solution is to simply up the amount of iterations over the dataset, in the hope that the network happens to locate a slightly better solution by chance, then works to optimize it. This seems unlikely, and overly dependent on the problem.

My third and final possible solution is to initially only train the network on input where the homeowner doesn't own a fridge (all fridge values are zero) such that the network optimizes it's solution without having to worry about the fridges, and then gradually introduce inputs from homeowners with fridges.

Will any of these work? Is there a better solution?

I am using the Tensorflow Estimator API.

Thanks!

$\endgroup$
1
$\begingroup$

This is fairly common with scenarios like predicting fraud when overall cases of fraud are uncommon. The model will frequently just say "no fraud!" and be right a large percent of the time. You need to assign a penalty, cost, or weight so that output 1 isn't so easy.

This article describes that further in a lot of detail:

https://blog.fineighbor.com/tensorflow-dealing-with-imbalanced-data-eb0108b10701

| improve this answer | |
$\endgroup$
  • $\begingroup$ I'll look into that, thank you! I hadn't even considered that option. $\endgroup$ – Cowthulhu Oct 3 '17 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.