Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

27 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
21 views

Possible flaw in the MDS method for dimensionality reduction

The MDS (multidimensional scaling) method is used to solve the problem of dimensionality reduction. Basically, it does the following: given $n$ points $x_1,\cdots,x_n\in\mathbb R^d$, try to find a ...
3
votes
0answers
71 views

Best linear algebra library for c++?

I have been trying to find the substitute of numpy and perform some linear algebra using c++ and here's a list of libraries I have encountered: Eigen Armadillo Dlib GNU Scientific library Please ...
2
votes
0answers
18 views

Deriving vectorized form of linear regression

We first have the weights of a D dimensional vector $w$ and a D dimensional predictor vector $x$, which are all indexed by $j$. There are $N$ observations, all D dimensional. $t$ is our targets, i.e, ...
2
votes
0answers
49 views

How does SVD actually provide the recommendations? I seem to get conflicting answers

I am reading a text book that basically says the following: Given a matrix A where A is USERS x ITEMS we can use SVD to decompose the matrix into: $$A = U \times \Sigma \times V^T$$ Then we can take ...
1
vote
1answer
32 views

Hypothesis vs Hyperplane in Machine Learning

I am finding it hard to understand the clear difference between Hypothesis and Hyperplane. I know that Hypothesis is a candidate model that maps inputs to outputs after training . And , Hyperplane is ...
1
vote
0answers
36 views

Intuition behind understanding eigenvectors and Machine Learning

I am struggling to understand linear algebra application in machine learning, and I am not able to answer the following question. Is the model learned in Machine Learning the eigenvector of the ...
1
vote
1answer
35 views

Why transpose of independent feature matrix is necessary in case of linear regression?

I can follow classical linear regression steps: $Xw=y$ $X^{-1}Xw=X^{-1}y$ $Iw=X^{-1}y$ $w=X^{-1}y$ However, on implementing in Python, I see that instead of simply using ...
1
vote
2answers
32 views

Linear regression with a fixed intercept and everything is in log

I have a set of values for a surface (in pixels) that becomes bigger over time (exponentially). The surface consists of cells that divide over time. After doing some modelling, I came up with the ...
1
vote
1answer
30 views

Backpropagation with a different sized training set?

I'm trying to create a NN whose input is a (length m) array of 3d vectors $$\vec{x}_i = [x_{i,1},x_{i,2},x_{i,3}], \hspace{5mm}i=1:m $$ and whose output is a similarly sized array: $$\vec{h}_{\theta,...
1
vote
0answers
19 views

What's wrong with my backpropagation through time (BTT) calculation or how to multiple a scaled vector and a matrix without matching dimensions?

I am trying to make a pretty simple RNN from scracth, using only Numpy library of Python. At this moment I am having troubles with BTT as I do not know how to proceed with situation when a ...
1
vote
1answer
48 views

How to incorporate the uncertainty of the model coefficients in the prediction interval of a multiple linear regression

I'm dealing with the modeling of small experimental data sets. As most experimental work does not generate thousands of samples, but rather a handful, I need to be inventive in how to deal with this ...
1
vote
0answers
16 views

How effective is Moore Penrose for solving regression problems with overdetermined system of equations?

For regression problems with #Predictors > #observations, I recently read about Moore Penrose (pseudo inverse method) which solves the problem of non invertible matrix in OLS for regression problems. ...
1
vote
0answers
19 views

How to add extra constraints to an equation?

Background: I have an equation which looks like as follows: $W \times P = R$ $\left[\begin{array} &{1}&{0}&{0}&-\frac{w_{1}}{w_{o1}} &\dots &{0} &-\frac{w_{1}}{w_{0} } \\...
1
vote
0answers
175 views

Connection between piecewise linear basis functions and RELU activation function

ReLU activation is defined as follows $$\sigma(x)=\max(0, x).$$ Let's assume that I have deep network of 1 hidden layer, than output from my layer has form $$ f(x)= \sigma(Wx +b), $$ where matrix W ...
1
vote
0answers
76 views

Can we think of neurons as maps between matrices?

Usually when we think about neurons, we imagine that they enact some kind of map between real numbers. For example, a neuron might take in real numbers $x_{i}$ and weight them with parameters $W_{ij}$,...
1
vote
0answers
13 views

Structures for incorporating linear functions into a nonlinear optimization problem

I'm working on a problem which naturally involves both linear and nonlinear operations, and I'd like some help understanding the best way to combine these into a neural network framework. To be more ...
0
votes
0answers
15 views

SVD Kernel and Linear Algebra Kernel, is there a conceptual difference?

Is the term kernel used in Sklearn to execute the SVD machine learning algorithm conceptually related to the notion of a kernel in linear algebra ( null space )? Or do they happen to use this same ...
0
votes
0answers
34 views

Human intuition behind SVD in case of recommendation system

I checked the SVD for recommendation engine thread but it does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to ...
0
votes
0answers
10 views

Predicting a variably-placed value in a vector

I have $m$ vectors in $\mathbb{R}^n$, where $m >> n$, and I want to train a model to impute a value $x_i$ in $\mathbf{x}$, where $1 \leq i \leq n$ (and can vary by vector). For instance, I may ...
0
votes
0answers
8 views

Projecting 3D Chemical Data Onto a 2D Plane in Motion

I'm trying to model the rotation of two hydrogen atoms about a carbon atom. Say I have a conceptual wheel on an axle that is attached to my car. The axle is described by two points in 3D space, as ...
0
votes
0answers
41 views

How to calculate latent vector for a user in ALS based on some new input?

So I have an ALS trained in pyspark but then I get some interactions from a new user that wasn't in the training set. I want to give recommendations to that new user without retraining the ALS based ...
0
votes
0answers
39 views

Workng of LME model used for a set of category variable(s) and a continuous variable?

LME models are being used to analyze the effect of continuos data and category data. Is this model appropriate for checking the effect of two independent variables - one with continuous values and ...
0
votes
0answers
45 views

How do I get confidence intervals for an ElasticNet in sklearn?

I need to produce a row for the confidence interval for every field that I am calculating coefficients and scores off of. So here is my code so far- ...
0
votes
1answer
416 views

Gradient descent formula implementation in python

So I recently started with Andrew Ng's ML Course and this is the formula that Andrew lays out for calculating gradient descent on a linear model. $$ \theta_j = \theta_j - \alpha \frac{1}{m} \sum_{i=1}...
0
votes
0answers
16 views

What does sparsely compute mean?

I heard someone say a neural network needs to sparsely compute the output. I get what compute means, I get what a sparse matrix is, but what does sparsely compute mean?
0
votes
1answer
45 views

Can I use regression to solve a multiple equation problem

I'm working on a problem which is a multiple equation. I have a group of people and each person in the group is working on different tasks (e.g. n tasks in total). Each person in this group is working ...
-1
votes
1answer
23 views

What are some theoretical approches to study Machine Learning/Deep Learning?

I've recently studied "basics of machine learning" (chapter 5) in Deep Learning by Ian Goodfellow. They're explaining ML with most of statistical and probabilitical methods. But I wish to ...