What is the time complexity of feed-forward neural networks? I didn't find a book or a reliable source that's talks about this. This link provides one version by breaking the structure of a simple neural network. But I need someone to confirm this, a reviewer asked me to compute a the time complexity of my model. So, I need to be sure.

Also, if two models are trained in parallel, should I sum their time complexity or take the max?

  • $\begingroup$ For the last part of the question: unless the number of models (2) trained in parallel is an input variable, you can ignore it (so take the max for example). Also are you sure that the reviewer isn't just asking you to mention how much computing power is used in your experiment? In this case you wouldn't have to calculate the formal time complexity, just say something like "it takes N hours using M cores of an X processor cluster". $\endgroup$
    – Erwan
    Jun 21, 2020 at 12:44
  • $\begingroup$ "The time complexity of the model has to be addressed" this is what the reviewer said. So, this means the formal time complexity, right? $\endgroup$
    – U. User
    Jun 21, 2020 at 12:46
  • $\begingroup$ It looks so but I'm a bit surprised, it's not a common requirement. Is the paper mostly experimental or proposing some kind of new model? $\endgroup$
    – Erwan
    Jun 21, 2020 at 13:05
  • $\begingroup$ The paper is about a machine learning model and we run experiments on the dataset like most researchers. It's true, we have an algorithm that used to prepare the outputs of the model, but it's not something complicated. $\endgroup$
    – U. User
    Jun 21, 2020 at 13:08
  • $\begingroup$ Ok I don't know, maybe they actually want the formal time complexity indeed. $\endgroup$
    – Erwan
    Jun 21, 2020 at 14:51


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.