# Tag Info

## Hot answers tagged linear-algebra

7

The normal equations are designed such that each coefficient in the model has an input of some kind it's being multiplied against. The column of ones is the "input" to the intercept term.

7

Your understandings are right. deriving the margin to be $\frac{2}{|w|}$ we know that $w \cdot x +b = 1$ If we move from point z in $w \cdot x +b = 1$ to the $w \cdot x +b = 0$ we land in a point $\lambda$. This line that we have passed or this margin between the two lines $w \cdot x +b = 1$ and $w \cdot x +b = 0$ is the margin between them which we ...

3

Any scalar multiple of an eigenvector is also an eigenvector. LAPACK (which np.linalg.eig uses under the hood) chooses to return unit-length eigenvectors (good for SVD!), but this still leaves two choices, and there doesn't seem to be a convention for which one to return; it's up to the underlying algorithm (which in turn may depend on the input data). ...

3

I'll give you a small example, if you do the following Kronecker product \begin{equation} \begin{bmatrix} \color{red}{1} \\ \color{green}{5} \\ \color{blue}{10} \end{bmatrix} \otimes \begin{bmatrix} 2 \\ 4 \end{bmatrix} = \begin{bmatrix} \color{red}{1} \begin{bmatrix} 2 \\ 4 \end{bmatrix} \\\\ \color{green}{5} \begin{bmatrix} 2 ...

3

You may want to read the documentation. output[..., i, j] = sum_k (a[..., i, k] * b[..., k, j]), for all indices i, j. For instance, in your example $~~88=1\times12+2\times14+3\times16,~~~94=1\times13+2\times15+3\times17$ $214=4\times12+5\times14+6\times16,~229=4\times13+5\times15+6\times17$

3

Notation matters! The problem starts from: Given $\nabla a_j^{(k+1)} = \frac{\partial E}{\partial a_j^{(k+1)}}$ I don't like your notation! it's wrong in fact, in standard mathematical notation. The correct notation is $$\nabla_{a_j^{(k+1)}} E = \frac{\partial E}{\partial a_j^{(k+1)}}$$ Then, gradient of the error $E$ w.r.t a vector ${\mathbf{a}^{(k)}}$...

2

Latent Semantic Analysis (LSA) relies on linear-algebraic decompositions (e.g. SVD), which in turn involve eigenvectors/values (see here). Not sure if this is quite what your question is driving at, but in general, eigenvectors are useful in data analysis because they define some "natural" direction in the data. For example, the eigenvectors of the ...

2

The hyperbolic trig functions follow the equation for a Rectangular hyperbola, which is something you should be familiar from analytical geometry. The recentgular hyperbola is defined by $x^2 - y^2 = 1$ and if you let $x = cosh(t)$ and $y = sinh(t)$ and plug it into the rectangular hyperbola equation you can verify this fact quite easily. From this idea ...

2

Your goal is to find a $w$ such that $$Xw \approx y$$ and way to model this problem is to minimize the objective function: $$\min_w\|Xw-y\|^2.$$ Differentiating with respect to $w$ and equate it to zero gives us $$2X^T(Xw-y)=0$$ $$X^TXw-X^Ty=0$$ $$X^TXw=X^Ty$$ if $X^TX$ is invertible, then we have $$w=(X^TX)^{-1}(X^Ty)$$ Remark: We tend to avoid ...

2

Theoretically, the formula with two matrices is more clear and self-evident, I think that's the reason why it's used more often. In practice, both approaches are actually used in production and hence are equivalent. It's just a matter of preference. Tensorflow For example, Tensorflow is often optimized for performance. Here's how basic RNN cell is ...

2

Not sure how intense your professor is going to make either course, but assuming it's the hardest possible introductory course, it would be better to take linear algebra before statistics. A lot of statistical operations require fundamental linear algebra concepts. You need to be able to understand abstract notions like vector spaces and fields to know when ...

2

Without reading the book, my guess is $W$ is a vector of weights(polynomial coefficients). Assuming $x$ is the feature vector, I.E $x$ := {$x_1$, $x_2$, ... $x_n$} then W := {$w_1$, $w_2$, ... $w_n$}.

1

Some Intro Eigen-Decompostion does lots of things among them one thing is interesting. Eigen-Decomposition usually captures the characteristic of the matrix it is applied to. For instance is the matrix is the matrix of similarities, the result of eigen decomposition will be a clustering (see Spectral Clustering). PCA The idea behind PCA is to find ...

1

You should formulate your lines as follows to have $(x, y)$ as unknowns: \begin{align} \left.\begin{matrix} a_1x-y=-b_1\\ a_2x-y=-b_2 \end{matrix}\right\} \rightarrow \overbrace{ \begin{bmatrix} a_1& -1\\ a_2& -1 \end{bmatrix} }^{\boldsymbol{a}} \overbrace{ \begin{bmatrix} x\\ y \end{bmatrix} }^{\boldsymbol{x}} = \overbrace{ \begin{bmatrix} -... 1 This is one approach you can follow: Setup a linear regression system, with each match as a row, and each feature corresponding to a team. The unknown feature coefficient for each team is the team 'strength' that we try to determine. The feature values will be one of 0, 1, or -1, depending on whether the team did not play, was the column team, or the row ... 1 In general, linear algebra and basic concepts in statistics (probability distributions, marginalization) and machine learning (you should be familiar with terns like Maximum-A-Posteriori estimation and Maximum-Likelihood-Estimation, maybe Markov chains and SVMs, neural-networks, etc.). Best practice would be to find out who is interviewing you and than look ... 1 I would just repeat my reply under the original issue in case anybody is looking for the answer here. The direct answer to the issue: I can't really recall how I used this code 2 years ago. But I got it working with two steps: Build the shared library with gcc -fPIC -shared st.c -o ts.so. Change the .so path in tangentDistance.py to the absolute path of ... 1 ComOnGetMe's KMeans tangent distance metric looks good, if it didn't work for you initially you should fork it and work on the code. I would contact him directly, he probably would have insight for you. Scikit-learn doesn't have a distance metric for tangent distances but the documentation states you can call a user-defined distance (all be it with overhead).... 1 Can you comment on why this is a very important matter to you? I'm a Ph.D. Candidate in ML and honestly (my opinion, but it is an opinion-based question) I don't think it should be. If you're talking about a first-semester stats course and a first-semester linear algebra course, you'll be fine either way. 1 Tensor multiplication is just a generalization of matrix multiplication which is just a generalization of vector multiplication. Matrix multiplication is defined as: A_i \cdot B_j = C_{i, j} where $i$ is the $i^{th}$ row, $j$ is the $j^{th}$ column, and $\cdot$ is the dot product. Therefore it just a series of dot products. One can then see how this ...

1

In general, you can't. You can find a matrix that will minimize $||\tilde{A}^TB-C||$, and another that minimizes $||\tilde{A}-A||$, but, in general, the two matrices you find won't be the same. In fact, the matrix that solves the second problem is always $\tilde{A} = A$, and, if $||{A}^TB-C||$ is not the minimum of the first function, then your problem doesn'...

1

In least squares regression, the problem that you solve is: $X^T X \beta = X^T y$ where $\beta$ are the parameters, $X$ the regressors and $y$ the predicted variable. If you want, in your notation $A$ to be constant and $b$ to change, you need to have constant regressors and change the predicted variable $y$. I don't think there are applications that the ...

1

$f =X^T Y$ looks like this \begin{equation} f= \begin{pmatrix} \sum_i x_iy_{i1}\\ \sum_i x_iy_{i2}\\ \vdots\\ \sum_i x_iy_{in}\\ \end{pmatrix} \end{equation} where $x_i$ is the $i^{th}$ element of $X$ and $y_{ij}$ is the $(i,j)^{th}$ element of $Y$. Now, to get the gradient w.r.t $X$, is equivalent to deriving each element of $f$ w.r.t each element ...

1

First of all, gradient descent cannot find the global optimum. If your function has just one extremum, it can find it but if it has lots of them there is no guarantee that it finds the best one. If you are familiar with derivative and slope of functions, the Normal equation, tries to find the point which all the derivative is equal to zero for all directions,...

1

Matrix derivatives work a bit different than regular ones. The scalar parallel of $\nabla_{\theta} \theta^{T} X^{T}X\theta$ you make should be more like $\theta x^2 \theta$ which in turn is just $x^2 \theta^2$. (Note that I changed $\alpha$ for $x$ to avoid confusion with $X$.) You do not need to modify $\theta^{T} X^{T}X\theta$ so that the two $\theta$'s ...

1

Try using spatial.cKDTree, in my ICP implementation switching from KDTree to cKDTree 30 iterations went from ~160 seconds to ~50 seconds

Only top voted, non community-wiki answers of a minimum length are eligible