This might not be on topic here, but fingers crossed.

There are all manner of neural network architectures out there, everything from convolutional networks to deep recurrent networks (and even deep recurrent convnets). Some network arcitecture development I can kind of see the intuition behind; recurrent networks allow for feedback loop type things to be taken into account. Other ideas seem to come out of left field and change the field. Yolo is a good example of this - I understand how it works, I've implimented it on my own dataset. But how did someone come to the set up that allowed yolo to work.

Is it just creativity, trying random architectures and new ideas to see what sticks? Is there some methodology that I could apply to take some steps towards a new ground-breaking architecture? (I understand that I won't actually make the next new architecture, but it's interesting to understand the process)

  • 1
    $\begingroup$ This might be relevant ai.stackexchange.com/questions/6836/… $\endgroup$
    – DuttaA
    Commented Aug 20, 2018 at 14:42
  • $\begingroup$ I think it is partially having a good imagination. However, if you don't know linear algebra , imagination won't be helpful. $\endgroup$
    – a_a_a
    Commented Aug 21, 2018 at 11:23
  • $\begingroup$ Check this paper that just came out : Neural Architecture Search: A Survey I dont't have read yet but the name and the abstract of the paper seem to be what your are looking for. $\endgroup$
    – Adrien D
    Commented Aug 21, 2018 at 12:33
  • $\begingroup$ Which network are you looking for? $\endgroup$ Commented Aug 21, 2018 at 13:21
  • $\begingroup$ I'm not looking for details on a specific network. More a general development cycle I suppose. $\endgroup$ Commented Aug 21, 2018 at 15:20

1 Answer 1


Think of neural networks as math, and ask yourself how is new math discovered? New math comes from a combination of theory, experimentation and feedback from community.

Instead of thinking about it terms of creativity vs randomness to see what new ideas stick; I prefer to think in terms of theory and experimentation.

The current methodology for new neural net discovery is no different than any other applied field of math/engineering.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.