Timeline for Why not using linear regression for finetuning the last layer of a neural network?
Current License: CC BY-SA 4.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 4, 2021 at 1:21 | comment | added | grov | @funkwecker Could you say more about why you would want to use linear regression to calibrate the weights in the last layer of the network and later use gradient descent to fine tune globally? It seems possible to change algorithms for different phases of learning but what are the benefits? | |
| Apr 4, 2021 at 1:19 | comment | added | grov | The link to the section in the keras guide with some discussion of the tradeoffs of using another model (e.g. linear regression) vs using gradient descent was added in my newest edit: link | |
| Apr 4, 2021 at 1:16 | history | edited | grov | CC BY-SA 4.0 |
made link more specific
|
| Apr 3, 2021 at 7:34 | comment | added | Funkwecker | Thanks for the answer. If don't find the discussion in the link. Can you be more specific? Also, what confuses me: When I trained the last layer using linear regression, I still can further train the network using gradient descent. I don't see a problem there. | |
| Apr 3, 2021 at 0:41 | history | answered | grov | CC BY-SA 4.0 |