So I designed my own CNN with 10 layers of convolutions and no maxpoolings or any other connections. When I ran it on a dataset I got the following loss curve (blue) the other one is accuracy vs number of epochs. What do you think might be the problem here? And how to rectify it?
NOTE: My data-set of size 20 due to lack of computational resources, so the oscillation is probably looking more prominent or happening in the first place.