5
$\begingroup$

This might not be on topic here, but fingers crossed.

There are all manner of neural network architectures out there, everything from convolutional networks to deep recurrent networks (and even deep recurrent convnets). Some network arcitecture development I can kind of see the intuition behind; recurrent networks allow for feedback loop type things to be taken into account. Other ideas seem to come out of left field and change the field. Yolo is a good example of this - I understand how it works, I've implimented it on my own dataset. But how did someone come to the set up that allowed yolo to work.

Is it just creativity, trying random architectures and new ideas to see what sticks? Is there some methodology that I could apply to take some steps towards a new ground-breaking architecture? (I understand that I won't actually make the next new architecture, but it's interesting to understand the process)

$\endgroup$
  • 1
    $\begingroup$ This might be relevant ai.stackexchange.com/questions/6836/… $\endgroup$ – DuttaA Aug 20 '18 at 14:42
  • $\begingroup$ I think it is partially having a good imagination. However, if you don't know linear algebra , imagination won't be helpful. $\endgroup$ – a_a_a Aug 21 '18 at 11:23
  • $\begingroup$ Check this paper that just came out : Neural Architecture Search: A Survey I dont't have read yet but the name and the abstract of the paper seem to be what your are looking for. $\endgroup$ – Adrien D Aug 21 '18 at 12:33
  • $\begingroup$ Which network are you looking for? $\endgroup$ – Media Aug 21 '18 at 13:21
  • $\begingroup$ I'm not looking for details on a specific network. More a general development cycle I suppose. $\endgroup$ – user6916458 Aug 21 '18 at 15:20
1
$\begingroup$

Think of neural networks as math, and ask yourself how is new math discovered? New math comes from a combination of theory, experimentation and feedback from community.

Instead of thinking about it terms of creativity vs randomness to see what new ideas stick; I prefer to think in terms of theory and experimentation.

The current methodology for new neural net discovery is no different than any other applied field of math/engineering.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.