Skip to main content
11 votes
Accepted

What is the use of additional column of 1s in normal equation?

The normal equations are designed such that each coefficient in the model has an input of some kind it's being multiplied against. The column of ones is the "input" to the intercept term.
David Marx's user avatar
  • 3,248
7 votes
Accepted

Mathematical formulation of Support Vector Machines?

Your understandings are right. deriving the margin to be $\frac{2}{|w|}$ we know that $w \cdot x +b = 1$ If we move from point z in $w \cdot x +b = 1$ to the $w \cdot x +b = 0$ we land in a ...
Fatemeh Asgarinejad's user avatar
6 votes

How does tensor product/multiplication work in TensorFlow?

You may want to read the documentation. output[..., i, j] = sum_k (a[..., i, k] * b[..., k, j]), for all indices i, j. For instance, in your example $~~88=1\...
user12075's user avatar
  • 2,264
6 votes

How does tensor product/multiplication work in TensorFlow?

I'll give you a small example, if you do the following Kronecker product \begin{equation} \begin{bmatrix} \color{red}{1} \\ \color{green}{5} \\ \color{blue}{10} \end{bmatrix} \otimes \...
Ahmad Bazzi's user avatar
4 votes
Accepted

Dose finding slope/intercept using the formula of m,b gives best fit line always In linear regression?

The formulae you mentioned gives the coefficients of the line of best fit.The values are derived using the least squares method, where the goal is to minimize the sum of squared errors. Following is ...
Ankita Talwar's user avatar
4 votes

What type of technique can be used to solve this question?

The price of each product can be determined by these attributes. I think we can try to make use of this information. Suppose for a start, we assume that price of product $i$, $X_i$, is a linear ...
Siong Thye Goh's user avatar
3 votes
Accepted

Why does np.linalg.eig produce an opposite-signed eigenvector?

Any scalar multiple of an eigenvector is also an eigenvector. LAPACK (which np.linalg.eig uses under the hood) chooses to return unit-length eigenvectors (good for SVD!), but this still leaves two ...
Ben Reiniger's user avatar
  • 11.8k
3 votes
Accepted

How can positional encodings including a sine operation be linearly transformable for any offset?

I elected to ask this question on the Mathematics Stack Exchange and I thought it prudent to add the answer here: https://math.stackexchange.com/q/3119882 From what I have learned from @Servaes, who ...
Stephan Heijl's user avatar
3 votes
Accepted

Deriving backpropagation equations "natively" in tensor form

Notation matters! The problem starts from: Given $\nabla a_j^{(k+1)} = \frac{\partial E}{\partial a_j^{(k+1)}}$ I don't like your notation! it's wrong in fact, in standard mathematical notation. The ...
Ehsan M. Kermani's user avatar
3 votes
Accepted

Understanding the algebra behind a specific partial derivative equation

We know that: (1) $\frac{\partial}{\partial x}\big (f(x) + g(x) \big) = \frac{\partial}{\partial x}f(x) + \frac{\partial}{\partial x}g(x)$ (2) $\frac{\partial}{\partial x}a = 0$ Now, \begin{align*} &...
mujjiga's user avatar
  • 166
2 votes

Eigenvectors and eigenvalues for natural language processing

Latent Semantic Analysis (LSA) relies on linear-algebraic decompositions (e.g. SVD), which in turn involve eigenvectors/values (see here). Not sure if this is quite what your question is driving at, ...
PriceHardman's user avatar
2 votes

What exactly is the "hyperbolic" tanh function used in the context of activation functions?

The hyperbolic trig functions follow the equation for a Rectangular hyperbola, which is something you should be familiar from analytical geometry. The recentgular hyperbola is defined by $x^2 - y^2 = ...
Tophat's user avatar
  • 2,420
2 votes
Accepted

How does "linear algebraic" weight training function work?

Your goal is to find a $w$ such that $$Xw \approx y$$ and way to model this problem is to minimize the objective function: $$\min_w\|Xw-y\|^2.$$ Differentiating with respect to $w$ and equate it ...
Siong Thye Goh's user avatar
2 votes
Accepted

RNN: why Wx + Uh instead of W[x,h]

Theoretically, the formula with two matrices is more clear and self-evident, I think that's the reason why it's used more often. In practice, both approaches are actually used in production and hence ...
Maxim's user avatar
  • 890
2 votes

Machine learning PhD Interview technical questions

In general, linear algebra and basic concepts in statistics (probability distributions, marginalization) and machine learning (you should be familiar with terns like Maximum-A-Posteriori estimation ...
Mark.F's user avatar
  • 2,220
2 votes

How does tensor product/multiplication work in TensorFlow?

Tensor multiplication is just a generalization of matrix multiplication which is just a generalization of vector multiplication. Matrix multiplication is defined as: $$ A_i \cdot B_j = C_{i, j}$$ ...
Daniel's user avatar
  • 246
2 votes
Accepted

How to "reshape" into square matrix for numpy.linalg.solve()?

You should formulate your lines as follows to have $(x, y)$ as unknowns: $$\begin{align} \left.\begin{matrix} a_1x-y=-b_1\\ a_2x-y=-b_2 \end{matrix}\right\} \rightarrow \overbrace{ \begin{bmatrix} ...
Esmailian's user avatar
  • 9,312
2 votes

I can't understand polynomial in the book

Without reading the book, my guess is $W$ is a vector of weights(polynomial coefficients). Assuming $x$ is the feature vector, I.E $x$ := {$x_1$, $x_2$, ... $x_n$} then W := {$w_1$, $w_2$, ... $w_n$}....
yoav_aaa's user avatar
  • 993
2 votes

Statistics Before Linear Algebra?

Not sure how intense your professor is going to make either course, but assuming it's the hardest possible introductory course, it would be better to take linear algebra before statistics. A lot of ...
Kushal Mohnot's user avatar
2 votes
Accepted

Need explanation of a matrix multiplication

Let's take it step by step. $$ f^{(1)}(x)=h=W^Tx \tag{1} $$ $$ f^{(2)}(h)=h^Tw \tag{2} $$ We substitute h. $$ f(x)=(W^Tx)^Tw \tag{3} $$ To make it work, we'...
Piotr Rarus's user avatar
2 votes

When is it useful to measure the Frobenius norm of a matrix?

It is basically a similarity measure for matrices. There are multiple uses for Frobenius Norm, some examples could be classifying covariance matrices for action recognition and low-rank bilinear ...
Pedro Henrique Monforte's user avatar
2 votes

Dose finding slope/intercept using the formula of m,b gives best fit line always In linear regression?

In linear regression you can choose between calculating the optimal weights using the normal equation or try to approximate the optimal weights using gradient descent. Normal equation: The optimal ...
Tim von Känel's user avatar
2 votes
Accepted

Removing constant from the regression model

When you estimate a linear model without constant, you essentially "force" the estimated function to go through the $(0,0)$ coordinates. With an intercept, you estimate a linear function ...
Peter's user avatar
  • 7,466
2 votes
Accepted

Confusion with Notation in the Book on Deep Learning by Ian Goodfellow et al

Let $\mathbf{y} = \mathbf{W}^T \mathbf{x}$ Then, $\mathbf{y}^T =(\mathbf{W}^T \mathbf{x})^T =\mathbf{x}^{T}(W^T)^T = \mathbf{x}^{T}W $. Note that $\mathbf{W}$ does not have to be a square matrix. Let ...
Graph4Me Consultant's user avatar
2 votes
Accepted

How does tree-based algorithms handle linearly combined features?

If the sum of the two feature makes sense on the domain semantically, it might be a good idea. But while trees can handle redundant features pretty well, increasing the number of features without ...
György Móra's user avatar
2 votes

Difference between FDA and LDA

Fisher's Discriminant Analysis (FDA) is Linear Discriminant Analysis (LDA) when there are only two classes. LDA is the direct extension of FDA to two or more classes.
Brian Spiering's user avatar
2 votes
Accepted

3d input for Dense Layer Keras

In a regular fully connected layer (Dense), the computation is done using the following Matrix operation : $$R = A*W + B$$ With all matrixes being vectors (if batch size = 1), exept $W$, which has ...
Ubikuity's user avatar
  • 626
2 votes
Accepted

Why do we use 'T' when we are to say matrix-vector product?

When a vector or matrix has the superscript $T$, it means the matrix/vector is transposed. Transposing a matrix or vector means flipping the matrix along its diagonal, that is, changing the elements ...
noe's user avatar
  • 26.7k
2 votes
Accepted

What are some application of Google's PageRank Algorithm in Data Science

Hubáček et al apply PageRank to football match prediction: PageRank was originally developed for assessing the importance of a website by examining the importance of other websites referring to it. ...
Jonathan's user avatar
  • 5,410
2 votes
Accepted

Why linear model cannot understand the interaction between any two input features?

Imagine our model has two inputs X1, X2 and one output Y Our input variables "interact&...
bogovicj's user avatar
  • 709

Only top scored, non community-wiki answers of a minimum length are eligible