# Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

36 questions
Filter by
Sorted by
Tagged with
1answer
936 views

### Deriving backpropagation equations “natively” in tensor form

Image shows a typical layer somewhere in a feed forward network: $a_i^{(k)}$ is the activation value of the $i^{th}$ neuron in the $k^{th}$ layer. $W_{ij}^{(k)}$ is the weight connecting $i^{th}$ ...
0answers
174 views

### Finding linear transformation under which distance matrices are similar

I have n sets of vectors, where each set S_i contains k vectors in ...
3answers
2k views

### How does tensor product/multiplication work?

In Tensorflow, I saw the following example: ...
1answer
172 views

### Closed form solution of linear regression via least squares using matrix derivatives

How is the closed form solution to linear regression derived using matrix derivatives as opposed to using the trace method as Andrew Ng does in his Machine learning lectures. Specifically, I am ...
1answer
190 views

### Mathematical formulation of Support Vector Machines?

I'm trying to learn maths behind SVM (hard margin) but due to different forms of mathematical formulations I'm bit confused. Assume we have two sets of points $\text{(i.e. positives, negatives)}$ one ...
1answer
464 views

### What is the use of additional column of 1s in normal equation?

Currently I am going through Normal Equation in Machine Learning. $$\hat\theta = (X^T \cdot X)^{-1} \cdot X^T \cdot y$$ But when I see how they use this equation, I found they always add an ...
1answer
118 views

### RNN: why Wx + Uh instead of W[x,h]

Traditionally, a state for RNN is computed as $$h_t = \sigma(W\cdot \vec x + U\cdot \vec h_{t-1} + \vec b)$$ For a RNN, why to add-up the terms $(Wx + Uh_{t-1})$ instead of just having a single ...
1answer
25 views

### PCA formulation - Deep Learning book by Ian Goodfellow

I am reading this deep learning book by Ian goodfellow. In the PCA formulation in the first chapter i.e Linear Algebra, he mentions the following: we need to choose the encoding matrix D. To do so,...
0answers
50 views

### Linear algebra library for c++

I have been trying to find the substitute of numpy and perform some linear algebra using c++ and here's a list of libraries I have encountered: Eigen Armadillo Dlib GNU Scientific library Please ...
2answers
47 views

0answers
74 views

### Can we think of neurons as maps between matrices?

Usually when we think about neurons, we imagine that they enact some kind of map between real numbers. For example, a neuron might take in real numbers $x_{i}$ and weight them with parameters $W_{ij}$,...
0answers
12 views

### Structures for incorporating linear functions into a nonlinear optimization problem

I'm working on a problem which naturally involves both linear and nonlinear operations, and I'd like some help understanding the best way to combine these into a neural network framework. To be more ...
2answers
25 views

### Are euclidian vectors and unit vectors same thing? [closed]

Consider this statement : Let the field K be the set R of real numbers, and let the vector space V be the Euclidean space R3. Consider the vectors e1 = (1,0,0), e2 = (0,1,0) and e3 = (0,0,1). Then any ...
0answers
9 views

### Chaining bias terms in backprob

let's say I have a few linear layers $l_1 \dots l_n$: $y=I(\dots I(IX + b_1) + b_2) \dots +b_n)$ where $n$ is sufficiently large and $I$ is the (nonparametric) identity matrix. The gradient for $b_n$...
0answers
9 views

### Performant alternatives to Matrix package in R that requires minimal effort by end users to install

Background I'm going to be releasing a package for R that does calculations involving extremely large sparse matrices (1,000,000 x 1,000,000 is the minimum for what we consider useful). For this ...
1answer
35 views