I have a neural network that takes in roughly twelve values, and outputs a singly probability. The issue is that the network appears to be smart enough to realize there is a very significant correlation between three of the values and the output, and doesn't seem to be recognizing any correlation with the remainder of the inputs. In other words, the model has figured out that if the and of three specific inputs are non-zero, there is an ~80% chance the output is 1.
This seems like expected behavior, but I am looking to have the network enhance it's 80% prediction with data from the other nine inputs. Is this possible?
My proposed solution requires a bit more information regarding the nature of the data. The twelve inputs can be categorized into four groups of three variables each. For instance, we have three metrics: purchase cost, warranty length and average maintenance costs. These three metrics are then applied to four different appliances in a household: the boiler, washing machine, dishwasher and refrigerator. The network is then supposed to output the probability of the owner of the household moving out in a few months. In this example, there happens to be a 80% correlation between refrigerators and the homeowner moving: if they own a refrigerator, there is an 80% chance they will move out within the next few months.
As mentioned before, the network is smart enough to recognize the correlation between the refrigerator and the moving out, but I am hoping to get it to augment this prediction with the inclusion of the other inputs.
My first strategy for addressing this would be to split the single network into four separate networks, each handling one appliance. These will output four probabilities total, which I can then either weight and compute by hand, or feed into a fifth network for final processing. My worry is that this would result in the same issue occurring - the final network would determine that ignoring the first three inputs and only returning the fridge-based probability is the best way to minimize error.
My second possible solution is to simply up the amount of iterations over the dataset, in the hope that the network happens to locate a slightly better solution by chance, then works to optimize it. This seems unlikely, and overly dependent on the problem.
My third and final possible solution is to initially only train the network on input where the homeowner doesn't own a fridge (all fridge values are zero) such that the network optimizes it's solution without having to worry about the fridges, and then gradually introduce inputs from homeowners with fridges.
Will any of these work? Is there a better solution?
I am using the Tensorflow Estimator API.
Thanks!
