0
$\begingroup$

How to go about selecting an architecture for a dataset with 80 datapoints and 9 features for a regression model?

Working on the Desarhnais dataset, with "Effort" as the target variable.

Would a simple NN with one hidden layer suffice since the data is not large or overly complicated? Thinking of using ReLU as the activation function since it is a form of linear regression model, but unsure how to select number of neurons in the hidden layer.

Any tips and advice would be helpful.

Improve this question
$\endgroup$
8
  • $\begingroup$ How do you figure that ReLU is a form of linear regression? // I don’t think $80$ points is enough to do any serious work with a neural network, anyway. If you just want to learn the mechanics of coding up your model, though, then that is fine. $\endgroup$
    – Dave
    Commented Nov 24, 2022 at 19:48
  • $\begingroup$ I thought of it like this : in linear regression we find ideal coefficients for each attribute, in a polynomial sense) at least, so the network weights act as coefficients and relu inside a neurons helps in computing the dot product of the weights with the input and passes it on $\endgroup$
    – user17420392
    Commented Nov 24, 2022 at 19:54
  • $\begingroup$ Do you mean that the neural network acts like a linear regression on the final hidden layer? $\endgroup$
    – Dave
    Commented Nov 24, 2022 at 19:56
  • $\begingroup$ Yes, that was kind of my intuition. Like a stacked linear regression model $\endgroup$
    – user17420392
    Commented Nov 24, 2022 at 19:58
  • $\begingroup$ That’s a good way to think of it, but that works for any activation function, not just ReLU. $\endgroup$
    – Dave
    Commented Nov 24, 2022 at 20:00

1 Answer 1

Reset to default
1
$\begingroup$

With such a small dataset, first use the simplest: linear regression. Otherwise, you risk overfitting. If linear regression does not work properly, think of why and choose a different option.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.