# Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

42 questions
Filter by
Sorted by
Tagged with
48 views

### How do we define a linearly separable problem?

When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not. I am having ...
193 views

19 views

247 views

I know the plot of $\tanh$ activation function looks like. I also know that its output has a range of $[-1, 1]$. Furthermore, I also know the it is defined as follows $$\tanh(x) = \frac{\sinh(x)}{... 0answers 74 views ### Can we think of neurons as maps between matrices? Usually when we think about neurons, we imagine that they enact some kind of map between real numbers. For example, a neuron might take in real numbers x_{i} and weight them with parameters W_{ij},... 0answers 12 views ### Structures for incorporating linear functions into a nonlinear optimization problem I'm working on a problem which naturally involves both linear and nonlinear operations, and I'd like some help understanding the best way to combine these into a neural network framework. To be more ... 1answer 288 views ### Machine learning PhD Interview technical questions [closed] I'm Software Engineer who applied to grad school for Machine Learning/Computer Vision PhD and currently waiting for interview calls. I'm brushing up Linear algebra/ ML topics. What kind of technical ... 2answers 421 views ### How can I implement tangent distance for k-nearest neighbor in python/scikit-learn? My ultimate aim is to have a function which I can feed into scikit-learn's NearestNeighbor class as a custom metric parameter. ... 4answers 157 views ### Statistics Before Linear Algebra? I know this is an opinion-based question and will be closed but this is the only place I know that can answer it reasonably and it is a very important matter to me. I am pursuing a machine ... 2answers 49 views ### Derivates with respect to a vector Suppose I have an equation, f = X^TY + \dots (a few more terms), where X is a vector and Y is a matrix of appropriate dimensions, I want to know how can we take the derivative of f \text{ w.r.t.... 1answer 311 views ### Is there a quick way to speed up ICP in python using a cached KD-tree I am currently using ICP to match 2 point clouds. These point clouds evolve in time, so I have to repeat this process many times. I am using a standard KD tree from scipy for my nearest neighbor ... 1answer 37 views ### On minimizing matrix norm (AB-C) Given A, B and C are matrices with dim(A) = m x n, dim(B) = n x p and dim (C) = m x p, the problem asks to evaluate I need to learn$$\tilde{A}$$such that$$\min_{\tilde{A}}||\tilde{A}^TB-C||$$and ... 1answer 51 views ### Least Squares Regression Ax=b when A is fixed and b is varied The typical setting for least squares regression (or over-determined linear system) for Ax=b is to solve x given A and b. In other words, A and b are fixed when we solve the problem. My ... 2answers 28 views ### Are euclidian vectors and unit vectors same thing? [closed] Consider this statement : Let the field K be the set R of real numbers, and let the vector space V be the Euclidean space R3. Consider the vectors e1 = (1,0,0), e2 = (0,1,0) and e3 = (0,0,1). Then any ... 1answer 328 views ### Closed form solution of linear regression via least squares using matrix derivatives How is the closed form solution to linear regression derived using matrix derivatives as opposed to using the trace method as Andrew Ng does in his Machine learning lectures. Specifically, I am ... 2answers 130 views ### How does “linear algebraic” weight training function work? This answer shows that linear and polynomial function weights can be trained using this matrix operation: w = (X^TX)^{-1}X^Ty Therefore, algorithms such as gradient descent are not necessary for ... 0answers 49 views ### How does SVD actually provide the recommendations? I seem to get conflicting answers I am reading a text book that basically says the following: Given a matrix A where A is USERS x ITEMS we can use SVD to decompose the matrix into:$$A = U \times \Sigma \times V^T Then we can take ...
Image shows a typical layer somewhere in a feed forward network: $a_i^{(k)}$ is the activation value of the $i^{th}$ neuron in the $k^{th}$ layer. $W_{ij}^{(k)}$ is the weight connecting $i^{th}$ ...