1
$\begingroup$

I know that the algorithmic complexity of CNNs and other methods of deep learning can't be fully expressed in simple terms, like big oh complexity. That said, how would one go about comparing the efficiency/complexity of CNNs to standard machine learning methods (say decision trees, LDAs, Naive Bayes, etc.)? Deep learning methods are expensive, but how do we know how expensive they are comparatively?

$\endgroup$
  • 2
    $\begingroup$ Well you can just measure that from runtime surely? Or construct some weird metric like accuracy per training minute? As a data scientist, you should be well aware that it all depends on what you're looking to do! $\endgroup$ – Henry Jul 11 '17 at 20:00
  • $\begingroup$ An interesting read on benchmarking Data Science tools can be found here: github.com/szilard/benchm-ml $\endgroup$ – Jindra Lacko Jul 12 '17 at 9:58
1
$\begingroup$

One deep learning method can apply to multiple problems (for example computer vision, natural language processing etc), the performance of the method may be different depends on the problem that the method handles, but there is "popular dataset" that usually use by deep learning researchers to benchmark their method. Let say you have deep learning method, if you want to know how efficient your method for handwritten classification you can benchmark your method by classifying mnist dataset and compute its accuracy. If you want to compare the performance of your method with another method you can search on the internet another method that uses the same dataset for benchmark, here is an example

| improve this answer | |
$\endgroup$
  • $\begingroup$ OP does not appear to want accuracy, but some measure of resource use (e.g. memory/CPU/training time). The same approach could apply, although finding published benchmarks for comparison of those types of metric may be harder. $\endgroup$ – Neil Slater Jul 12 '17 at 10:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.