Are there any algorithms which were developed using partial differential equations for tackling some of the machine learning problems? Most works I see online are in the field of computer vision and a few bizarre ones in topic modelling. But just curious if someone has used or seen it being used for some decision making process or classification problems?

  • 2
    $\begingroup$ Could you clarify, in what way the equations should be part of the ML problem or solver? Error gradient backpropagation in neural networks is pretty much entirely the chain rule applied repeatedly to a set of partial differential equations. Would that count? $\endgroup$ – Neil Slater Dec 7 '14 at 22:41
  • 1
    $\begingroup$ By PDE, I meant some set of equations which are generally used to describe physical phenomena and using that as a learning/modeling process. The solution to such equations gives us some sort of error estimation to gauge the strength of these models and thereby, reduce the unpredictability in certain situations. $\endgroup$ – Sidhha Dec 9 '14 at 4:20
  • $\begingroup$ @Sidha: That doesn't clarify it for me. Gradient descent, (and the variants of it), is an attempt to find a solution to the minimum of an error function, and uses partial differential equations in its formulation. Please clarify, does that count (because if it does, a large percentage of ML could be said to use PDEs)? If you are looking for a machine learning process that uses PDEs in some other way, it needs to be clearer, because simply "uses some PDEs" applies to anything that can be numerically optimised with gradient descent. $\endgroup$ – Neil Slater Dec 9 '14 at 10:23
  • 1
    $\begingroup$ Gradient descent uses partial derivatives, agreed. What I intended to say was a system of PDE's and not in an optimization problem. Please check the link in the previous comment. This quora discussion mentions some usage in optical flow applications in computer vision. But I am wondering if there are more applications out there using this approach. $\endgroup$ – Sidhha Dec 9 '14 at 15:18

Neil is correct. There are partial derivatives evwrywhere in gradient computation for machine learning models training.

For instance you can look at the gradient descent method used in the backpropagation method for a neural network. The course from AndrewNg on coursera describes it very well.

| improve this answer | |
  • $\begingroup$ It would be nice if you could provide us with an example of gradient application in data mining/machine learning. This would make your answer self contained and more useful to others. $\endgroup$ – Rubens Dec 12 '14 at 23:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.