A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

# Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

42 questions
Filter by
Sorted by
Tagged with
23 views

### optimizing a linear optimization function with linear constarints and binary variables

I am new to optimizations and trying to solve a problem, which I feel falls in the umbrella of optimization. I have an ojective function that needs to be maximized ...
31 views

12 views

### What does sparsely compute mean?

I heard someone say a neural network needs to sparsely compute the output. I get what compute means, I get what a sparse matrix is, but what does sparsely compute mean?
229 views

### Finding linear transformation under which distance matrices are similar

I have n sets of vectors, where each set S_i contains k vectors in ...
14 views

### Chaining bias terms in backprob

let's say I have a few linear layers $l_1 \dots l_n$: $y=I(\dots I(IX + b_1) + b_2) \dots +b_n)$ where $n$ is sufficiently large and $I$ is the (nonparametric) identity matrix. The gradient for $b_n$...
33 views

### I can't understand polynomial in the book

I'm reading a book called Bishop - Pattern Recognition and Machine learning. I came across the following equation, in which I don't understand what $W$ stands for. So, what is $W$?
289 views

### Mathematical formulation of Support Vector Machines?

I'm trying to learn maths behind SVM (hard margin) but due to different forms of mathematical formulations I'm bit confused. Assume we have two sets of points $\text{(i.e. positives, negatives)}$ one ...
39 views

### Eigen Decomposition of Data Matrix for PCA

In PCA we Eigen decompose the covariance matrix, not data matrix, Is it because most data matrices are non-square. If they were, isn't is correct to eigen decompose data matrix than the covariance ...
57 views

### PCA formulation - Deep Learning book by Ian Goodfellow

I am reading this deep learning book by Ian goodfellow. In the PCA formulation in the first chapter i.e Linear Algebra, he mentions the following: we need to choose the encoding matrix D. To do so,...
187 views

### Why does np.linalg.eig produce an opposite-signed eigenvector?

I am learning SVD by following this MIT course. In this video, the lecturer is finding the SVD for $$\begin{pmatrix} 5 & 5 \\ -1 & 7 \end{pmatrix},$$ which involves finding the ...
10 views

### Performant alternatives to Matrix package in R that requires minimal effort by end users to install

Background I'm going to be releasing a package for R that does calculations involving extremely large sparse matrices (1,000,000 x 1,000,000 is the minimum for what we consider useful). For this ...
158 views

2k views

### How to “reshape” into square matrix for numpy.linalg.solve()?

I'm trying to find the intersection of lines $y=a_1x+b_1$ and $y=a_2x+b_2$ using numpy.linalg.solve(). What I can't get my head around is how to correctly make $A$ ...
145 views

### Connection between piecewise linear basis functions and RELU activation function

ReLU activation is defined as follows $$\sigma(x)=\max(0, x).$$ Let's assume that I have deep network of 1 hidden layer, than output from my layer has form $$f(x)= \sigma(Wx +b),$$ where matrix W ...
53 views

### Optimizing vector values for maximum correlation

I'm new to ML, linear algebra, statistics, etc. so bear with me on the terminology... I’m looking to find a vector that produces the maximum correlation for the relationship between 1) all ...
55 views

124 views

### How does “linear algebraic” weight training function work?

This answer shows that linear and polynomial function weights can be trained using this matrix operation: $w = (X^TX)^{-1}X^Ty$ Therefore, algorithms such as gradient descent are not necessary for ...
244 views

297 views

### Is there a quick way to speed up ICP in python using a cached KD-tree

I am currently using ICP to match 2 point clouds. These point clouds evolve in time, so I have to repeat this process many times. I am using a standard KD tree from scipy for my nearest neighbor ...
Image shows a typical layer somewhere in a feed forward network: $a_i^{(k)}$ is the activation value of the $i^{th}$ neuron in the $k^{th}$ layer. $W_{ij}^{(k)}$ is the weight connecting $i^{th}$ ...