Why is my MLP with 2 features is doing worse than MLP with 1 feature where the one feature is a combination of feature1*feature2?

Question

I have programmed a MLP for a dataset (~500 rows) containing the length (L) and width (W) of an organism and the output of biomass (the organisms weight in pounds, B).

            mlp = MLPRegressor((5, 5), max_iter=1000)

I have trained the model with features

# Model 1
# Input = Feature 1: Length, Feature 2: Width. Output = Biomass
df = {'length': [60.1, 59.2, 59.4, 58.5], 'width': [15.4, 16.2, 14.9, 15.7], 'weight': [8.34, 7,65, 7.89, 7.14]}


# Model 2
# Input =  Feature 1: Length * Width^2. Output = Biomass
df = {'length*height^2': [60.1, 59.2, 59.4, 58.5], 'weight': [14253.31, 15536.44, 13187.39, 14419.66]}

The overall accuracy of my model with one feature is over 95%, however the accuracy with the features separated is about 85%.

My understanding of an MLP is that Model 1 should do better than model 2 as it will basically find the best combination of length and height to biomass, However my 1 feature model is doing significantly better. I have also tried standardizing the dataset with a scaler with no luck.

scaler = StandardScaler()

Answer 1

$h$ - height
$w$ - width
$p$ - weight

In real world we know that weight is volume times density. And volume is product of height, width and depth. If you assume constant depth and density the weight is just some constant multiplied by width and height i.e. $p = c \times hw$. If you have $hw$ as a feature the NN quickly learns $c$. This gives good performance.

Lets say your NN had identity as activation function i.e. no activation function. All it can learn is linear combination of input features.

So it will just learn $p \approx h w_h + w w_w + b$ where $w_h, w_w$ are weights and $b$ is bias.

If you have activation function like ReLU/Sigmoid it still learn linear combinations of outputs of ReLU/Sigmoid. Inputs to those is just linear combination of $h$ and $w$. In all that there is never a simple $hw$ term. This makes model complicated without having the important feature that is volume(area).

I think that is why the performance of $hw$ model is better.

---

If you have a NN with 2 neurons in the hidden layer with sigmoid activation, and 1 output neuron with no activation, then this is how $hw$ term shows up.

$\frac{1}{1+\exp(hw_{11} +w w_{12} + b_1)} + \frac{1}{1+exp(hw_{21} +w w_{22} + b_2)} = \frac{\ldots}{\ldots + \exp(w_{11} w_{22} hw + w_{12}w_{21}hw) } $

Thats very tricky place to show up if you just want to learn a constant multiplier.