I am testing transfer learning on rather small neural networks with only two hidden layers of 20 neurons on tabular data.

None of my experiments yields any improvement over a basic neural network. Is this expected? Does transfer learning only make sense for very large neural networks consisting of many hidden layers?

Are you aware of some studies that use transfer learning successfully for small networks?


The basic premise of transfer learning is that similar data modalities will hold similar relationships.

If the original data has similar relationships between the data points, then, that can be utilized by the smaller data.

So, the question is whether both the data have similar kind of relationships in them.

Case in point, the vision problems which hold local relationships inside them which are used across the whole domain invariably.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.