How to separate overlapping curves more effectively?

Question

So, I am trying to create a Neural Network which will effectively separate 2 Gaussian curves with somewhat different means and standard deviations. My basic aim is, for some given input vector the Neural Network will select 2 points from the 2 Gaussian curves (selection based on the input vector) and compare them, and output a value depending on the Gaussian curve which gives a greater value.

Now, if I implement a simple Neural Network with 2 hidden layers the accuracy of my predictions is about 68.75% for 2 Gaussian curves with N(0,1) and N(0,2000) respectively and 100% for N(0,1) and N(500,1) (my NN outputs 0, 1 and hence I can easily measure accuracy by true_positives/total_samples). Clearly more the overlapping curves, less the accuracy as the NN cannot distinguish between the 2 curves.

I have tried increasing the number of hidden layers, changing learning rate but still there is no improvement in accuracy. I am using back-propagation with momentum.

So my question is what structure, learning algorithm and modifications in the Neural Network will help me to effectively separate 2 curves which is kind of overlapping for a better accuracy?

Clarification 1: I have generated the data myself for testing purposes, in real situation I will only have an input/output pair from a black-box, and all I know the black-box follows some Gaussian distribution. Also the input vector has to be arranged in some sort of way, and then if we fit the curve to it, it turns out to be Gaussian.

Clarification 2: I am looking for possible hints. Any experienced person in this field might make a guess for some better methods. I know you cannot have a definite method, so suggestion of methods which might work are welcome.

Answer 1

The problem here isn't your chosen neural network architecture, it's that you're working with literally unseparatable data.

There's isn't any practical way to distinguish observations near the mode of multiple distributions that share a mode. The best you can do is approximate the distance from the mode at which the high variance distribution is more likely than the low variance distribution, and classify observations relative to that. You can do that analytically, and your neural network isn't going to do any better.

Have you tried visualizing your data to better understand what you're tasking your model with here? This is a bit more extreme than "kind of overlapping" distributions.