0
$\begingroup$

I am working on a project where I need to model a specific non-linear relationship using a neural network. The relationship is given by $y = 3x_1^2x_2^3 $. The approach involves:

  1. Preprocessing the Inputs: Applying the natural logarithm to the input variables $x_1$ and $x_2$.
  2. Network Design: Using a single layer with one neuron.
  3. Activation Function: Applying an exponential activation function.
  4. Training Parameters: Using Adam optimizer, MAE loss function, and training over 50 epochs with a batch size of 32.

Given this setup, the network should theoretically achieve 100% accuracy with the correct weights and biases:

  • Weights: $[2, 3]$
  • Bias: $ln 3$

However, despite these theoretical expectations, the model does not achieve 100% accuracy in practice. I have experimented with different initializations for weights and biases but still face this issue.

Questions:

  1. Theoretical Feasibility: Is there any theoretical reason why this approach might not achieve 100% accuracy, even with the correct weights and biases?
  2. Practical Considerations: Are there practical aspects of training neural networks that could prevent achieving 100% accuracy in this context? For instance, issues with optimization, numerical precision, or the specific choice of activation and loss functions.
  3. Suggestions for Improvement: What steps can be taken to improve the model's accuracy? Are there alternative methods or modifications to the current approach that could help in achieving the desired accuracy?

Any insights, explanations, or suggestions would be greatly appreciated.


$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.