# Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

90 questions
Filter by
Sorted by
Tagged with
16 views

### Can reducing information improve regression prediction?

Variable A is either 0 or 1. It is 0 if the sum of variables a + b + c + d … is less than some constant threshold, and is 1 if the sum of variables a + b + c + d … is greater than some constant ...
19 views

### Calculating the solution of OLS efficiently when adding one feature at a time

We know that the analytical solution for an OLS problem is $𝛽̂ =(𝐗^T𝐗)^{-1}𝐗^𝑇𝐲$. I am looking for an efficient algorithm to solve for $𝛽̂$ when I add one feature at a time. More specifically, ...
82 views

### About the last decoder layer in transformer architecture

So, in the decoder layer of transfomer, suppose I have predicted 3 words till now, including the start token then the last decoder layer will produce 3 vectors of size d-model, and only the last ...
10 views

### Rotate vector anticlockwise

I'm learning mathematics for Machine Learning, and there given a vector [ 1 0 ] and [ 0 1 ]. The vector is packed on a matrix, ...
10 views

### Plus sign inside circle, there seem to be many meaning. Are they all related?

In Linear algebra plus sign inside a circle refers to direct sum. In a paper "Towards Neural Mixture Recommender for Long Range Dependent User Sequences" it seems to refer to concatenation. ...
502 views

### How is weight matrix calculated in a neural network?

Context: I am a pure mathematician trying to understand machine learning. I am studying it from various sources, now focusing on NLP and word embeddings. My question: What is the weight matrix for a ...
19 views

### SVD of multivariate time series interpretation

I have multivariate timeseries data represented in matrix format. I have matrix A of MxN where M is number of timestamp and N is number of sensor. Time series data is sampled at very 1 hour. Each ...
17 views

1 vote
112 views

### What are some application of Google's PageRank Algorithm in Data Science

I came across a topic on computational linear algebra that talks about iterative algorithms to compute eigenvalues. I've worked with power method which is an iterative algorithm that converges a ...
128 views

### Inverting a matrix using a convolutional neural network

Just for a fun exercise, I am trying to invert a matrix, say size 28x28 (or even 5x5) with a neural network. The way I approached this (quite naively) is as follows: I built a fully convolutional ...
57 views

### Why do we use 'T' when we are to say matrix-vector product? [closed]

On the first picture author uses $T$ meaning matrix-vector product But other website do not use $T$, but says that $x$ is a vector, I do not understand if it is important or not
4k views

### 3d input for Dense Layer Keras

Is there any example of how Keras Dense layer handles 3D input. The documentation explains the following: ...
1k views

### Difference between FDA and LDA

I have asked this question in Mathematics Stackexchange, thought however that it might be more fit for here: I am currently taking a Data-Analysis course and I learned about both the terms LDA (Linear ...
77 views

### What is the Intuition behind weight vector W which is normal to the plane? Is the weight vector W same as the W which is normal to the plane π?

In an interview, I was asked the intuition behind the weight vector. I told the weight vector is a vector which we try to minimize to a local minima with the help of regulariser so we don't overfit. ...
1 vote
613 views

### How does the equation "dW = - (2 * (X^T ).dot(Y - Y_hat)) / m" comes in Linear Regression (using Matrix + Gradient Descent)?

I was trying to code the Linear Regression in Python using Matrix Multiplication method using Gradient Descent and followed a code where there was no mention what is the loss but just a code as Per ...
31 views

### How does tree-based algorithms handle linearly combined features?

While I am aware that tree-based algorithms (e.g., DT, RF, XGBoost) are 'immune' to multi-collinearity, how do they handle linearly combined features? For example, is there is any additional value or ...
136 views

### Geometric classification models

In class we have been presented with a Geometric classification model such that the goal is to construct a linear decision boundary $\bf{w} \cdot \bf{x} = t$; where $\bf{w}$ is the vector from the ...
1 vote
1k views

### Dot product and linear regression

I'm studying PCA and my professor said something about finding the linear regression by doing the dot product of both axis. Could someone explain to me why? The dot product returns a number. What's ...
66 views

### Why do we usually have fully connected layers of same sizes in CNNs?

Is there any specific reason that we observe in CNNs, the fully connected layers usually have the same sizes? You can verify this for many CNNs. I'm aware that if, for instance, we have a vector of ...
1 vote
223 views

### Hypothesis vs Hyperplane in Machine Learning

I am finding it hard to understand the clear difference between Hypothesis and Hyperplane. I know that Hypothesis is a candidate model that maps inputs to outputs after training. And, Hyperplane is ...
1 vote
168 views

1 vote
54 views

### What's wrong with my backpropagation through time (BTT) calculation or how to multiple a scaled vector and a matrix without matching dimensions?

I am trying to make a pretty simple RNN from scracth, using only Numpy library of Python. At this moment I am having troubles with BTT as I do not know how to proceed with situation when a ...