1
$\begingroup$

I keep hearing that machine learning is just linear algebra.

Does that mean all known (and all possible?) ML algos, from random forest, to support-vector machines, to recursive neural networks, can be written as a sequence of basic matrix and vector operations, such as multiply, add, sum, etc?

Would writing the algorithm in this way be able to capture all aspects of the model, e.g. architecture, activation functions, learning rate, optimization method?

Can advanced features like convolution be expressed as basic matrix operations?

Is there any ML algorithm for which this isn't true?

Is matrix algebra Turing complete?

$\endgroup$
1
$\begingroup$

Linear algebra is very important for ML, but ML is not "just" linear algebra.

Any sequence of linear matrix-vector operations is linear. Therefore any non-linear operation cannot be computed using linear algebra.

A few examples of non-linear operations that are used in ML:

  • The if statement: e.g. (x < 0)
  • The softmax activation
  • The rectified linear unit

Can advanced features like convolution be expressed as basic matrix operations?

This statement is true actually. A single convolution can be expressed as matrix. However, any modern CNN has non-linearities following the convolution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.