# Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

64 questions
Filter by
Sorted by
Tagged with
414 views

### Finding linear transformation under which distance matrices are similar

I have $n$ sets of vectors, where each set $S_i$ contains $k$ vectors in $\mathbb{R}^d$. I know there is some unknown linear transformation $W$ under which the distance matrix $D_i$ (a $k\times k$ ...
6k views

### How does tensor product/multiplication work in TensorFlow?

In Tensorflow, I saw the following example: ...
2k views

### What is the use of additional column of 1s in normal equation?

Currently I am going through Normal Equation in Machine Learning. $$\hat\theta = (X^T \cdot X)^{-1} \cdot X^T \cdot y$$ But when I see how they use this equation, I found they always add an ...
1k views

### Deriving backpropagation equations “natively” in tensor form

Image shows a typical layer somewhere in a feed forward network: $a_i^{(k)}$ is the activation value of the $i^{th}$ neuron in the $k^{th}$ layer. $W_{ij}^{(k)}$ is the weight connecting $i^{th}$ ...
476 views

### Closed form solution of linear regression via least squares using matrix derivatives

How is the closed form solution to linear regression derived using matrix derivatives as opposed to using the trace method as Andrew Ng does in his Machine learning lectures. Specifically, I am ...
546 views

### Mathematical formulation of Support Vector Machines?

I'm trying to learn maths behind SVM (hard margin) but due to different forms of mathematical formulations I'm bit confused. Assume we have two sets of points $\text{(i.e. positives, negatives)}$ one ...
59 views

49 views

### Intuition behind understanding eigenvectors and Machine Learning [closed]

I am struggling to understand linear algebra application in machine learning, and I am not able to answer the following question. Is the model learned in Machine Learning the eigenvector of the ...
21 views

### Possible flaw in the MDS method for dimensionality reduction

The MDS (multidimensional scaling) method is used to solve the problem of dimensionality reduction. Basically, it does the following: given $n$ points $x_1,\cdots,x_n\in\mathbb R^d$, try to find a ...
82 views

### Best linear algebra library for C++?

I have been trying to find the substitute of numpy and perform some linear algebra using C++. Here's a list of the libraries I have encountered: Eigen Armadillo Dlib GNU Scientific library Please ...
131 views

### How to solve Ax = b for A [closed]

Given two know vector x, and b (dimension 3*1 for example), what are the ways to approximate the matrix ...
36 views

53 views

### How to incorporate the uncertainty of the model coefficients in the prediction interval of a multiple linear regression

I'm dealing with the modeling of small experimental data sets. As most experimental work does not generate thousands of samples, but rather a handful, I need to be inventive in how to deal with this ...
49 views

### How does SVD actually provide the recommendations? I seem to get conflicting answers

I am reading a text book that basically says the following: Given a matrix A where A is USERS x ITEMS we can use SVD to decompose the matrix into: $$A = U \times \Sigma \times V^T$$ Then we can take ...
429 views

### Why does np.linalg.eig produce an opposite-signed eigenvector?

I am learning SVD by following this MIT course. In this video, the lecturer is finding the SVD for $$\begin{pmatrix} 5 & 5 \\ -1 & 7 \end{pmatrix},$$ which involves finding the ...
453 views

### Statistics Before Linear Algebra?

I know this is an opinion-based question and will be closed but this is the only place I know that can answer it reasonably and it is a very important matter to me. I am pursuing a machine ...
188 views

### How does “linear algebraic” weight training function work?

This answer shows that linear and polynomial function weights can be trained using this matrix operation: $w = (X^TX)^{-1}X^Ty$ Therefore, algorithms such as gradient descent are not necessary for ...
65 views

76 views

### Can we think of neurons as maps between matrices?

Usually when we think about neurons, we imagine that they enact some kind of map between real numbers. For example, a neuron might take in real numbers $x_{i}$ and weight them with parameters $W_{ij}$,...
13 views

### Structures for incorporating linear functions into a nonlinear optimization problem

I'm working on a problem which naturally involves both linear and nonlinear operations, and I'd like some help understanding the best way to combine these into a neural network framework. To be more ...
43 views

### Are euclidian vectors and unit vectors same thing? [closed]

Consider this statement : Let the field K be the set R of real numbers, and let the vector space V be the Euclidean space R3. Consider the vectors e1 = (1,0,0), e2 = (0,1,0) and e3 = (0,0,1). Then any ...