Questions tagged [mathematics]

Mathematics in a data science or machine learning context refers to the mathematical underpinnings for algorithms, optimization, statistics, and linear algebra etc.

Filter by
Sorted by
Tagged with
0
votes
1answer
14 views

Reliable source for Mathematics [closed]

I am curious what books are out there that cover the topics for the following in detail: Machine Learning Deep Learning AI in general I did not searh for books due to i don't know what has good or ...
1
vote
0answers
8 views

How do you calculate how many coefficients are necessary in polynomial regression?

So I can't seem to find much on this by searching so I came here. Let's say I had 3 variables $x_1,x_2,x_3$ and the let's say the degree of the polynomial was $d=2$, I can define the length of a ...
1
vote
1answer
32 views

What is a distribution-wise asymmetric measure?

I was trying to understand KL-Divergence, $$D_{KL} \langle P(X) \Vert P(Y) \rangle,$$ and was going through its Wikipedia article. It says the following In contrast to variation of information, it is ...
1
vote
0answers
15 views

Normalise the scores by small and large numbers division

I am working on a scoring problem use-case where I will score each task based on few aspects and take an average of scores of tasks in each experiment. For each experiment I will have 1 to N number of ...
1
vote
1answer
46 views

Use heading in Neural Network model

I am working on a prediction model where I must find out the destination of a boat based on its actual coordinates and heading (compass direction) : ...
2
votes
1answer
140 views

Find periodicity of a signal using python

I have a dataset that contains occurrences of the Kettle in a single-occupancy house for the duration of a month. In this dataset, ‘ts’ column indicates the unix-timestamp (this can be converted to ...
0
votes
2answers
99 views

Python - accessing dictionary values for math operations [closed]

I have this dictionary: stocks = {'FB': 255, 'AAPL': 431, 'TSLA': 1700} and this script: ...
1
vote
0answers
43 views

AdaBoost.R2 learning rate from scikit learn

AdaBoost.R2 (regression), is presented in the paper "improving regressors with boosting techniques" from Drucker and is freely available on Scholar. The implementation of regression for ...
0
votes
0answers
65 views

How does Batch normalization help optimization? Proof

I am reading the paper How Does Batch Normalization Help Optimization found here. $\newcommand{\norm}[1]{\left\lVert#1\right\rVert}$ But I am having trouble understanding the proof of the paper. It's ...
1
vote
1answer
50 views

Why Adaboost SAMME needs f to be estimable?

I am trying to understand the mathematics behind SAMME AdaBoost: At some stage, the paper adds a constraint for f to be estimable: I do not understand why this is ...
1
vote
1answer
24 views

Mathematics: Writing down a three-class classifier confusion matrix

Confusion matrix 2A three-class classifier is evaluated on a test set of 900 samples which containsall three classes in equal proportions. ...
1
vote
1answer
29 views

Fully-Connected DNN: Compute the numbers of free parameter in a DNN

A fully-connected DNN has layer sizes of 3-3-4-2, where the first layer size represents the input layer. We assume that all layers are affine ones (no ReLU). Give the dimensions of all weight matrices ...
1
vote
1answer
43 views

Mathematics: Can the result of a derivative for the Gradient Descent consist of only one value?

I have a problem of a task using the formula of the Gradient Descent: Perform two steps of the gradient descent towards a local minimum for the function given below, using a step size of 0.1 and an ...
1
vote
1answer
14 views

Explanation on some steps of AdaBoost.R2

I am trying to understand AdaBoost.R2 in order to implement it and apply it to a regression problem. In this circumstances I need to understand it perfectly, however there's some step i don't really ...
0
votes
0answers
9 views

Projecting 3D Chemical Data Onto a 2D Plane in Motion

I'm trying to model the rotation of two hydrogen atoms about a carbon atom. Say I have a conceptual wheel on an axle that is attached to my car. The axle is described by two points in 3D space, as ...
4
votes
2answers
177 views

How propagate the error delta in backpropagation in convolutional neural networks (CNN)?

My CNN has the following structure: Output neurons: 10 Input matrix (I): 28x28 Convolutional layer (C): 3 feature maps with a 5x5 kernel (output dimension is 3x24x24) Max pooling layer (MP): size 2x2 ...
0
votes
0answers
14 views

Determining the degree of freedom in ridge regression | Interpretation of the head matrix

right now, I am diving into statistical learning and stumbled over the so-called "head-matrix" and the determination of the degree of freedom. I am referring to ridge regression: So the ...
0
votes
0answers
21 views

Flow System - Analysing Measurement Errors

I have a closed system with multiple flow measurement points as shown; where in theory (A1 + A2 + A3) = (B1 + B2 + B3) = C = (D1 + D2) and D1 = (E1 + E2 + E3 + E4) D2 = (F1 + F2 + F3 + F4) Some of ...
2
votes
1answer
493 views

Different activation function in same layer of a Neural network

My question is that what will happen if I arrange different activation functions in the same layer of a neural network and continue the same trend for the other hidden layers. Suppose I have 3 ...
0
votes
0answers
12 views

How to compute Fourier Transform complexity?

I would like to know what is wrong with my reasoning: 1D Transform Let suppose we have row vector with N rows I apply Fourier transform for single row I get O(log N) Then after stacking N rows to ...
1
vote
0answers
47 views

What is the underlying difference between linear and non-linear relationship?

Study on linear correlation coefficient and nonlinear correlation coefficient in mathematical statistics WANG Ting, ZHANG Shiqiang Studies in Mathematical Sciences 3 (1), 58-63, 2011 From the above ...
0
votes
0answers
42 views

logarithmic exponential function python

The code below (I found from googling) does the trick in terms of outcome. What I need to work out is the formula for this, so for any Y point we can get corresponding X. This is part of a larger ...
0
votes
0answers
21 views

What are some modern algorithms\analysis that use Taylor Series or Lagrange multipliers?

What are some modern machine learning algorithms\analysis that use Taylor Series or Lagrange multipliers? I spent a good chunk of my weekend reading about SQP, aka SLSQP. SciPy has an implementation, ...
0
votes
1answer
105 views

How to compute Hessian matrix for log-likelihood function for Logistic Regression

I am currently studying the Elements of Statistical Learning book. The following equation is in page 120. It calculates the Hessian matrix for the log-likelihood function as follows \begin{equation} ...
1
vote
2answers
68 views

Is there a possibility that there is no relationship between some inputs and outputs?

I have a general question that comes to my mind, I'm doing machine learning projects and I took a look at many datasets and worked with, mostly there are already famous datasets that everyone uses. ...
1
vote
1answer
31 views

How to adaptively sample n-dimensional data and build an optimum training set

My input space is at least 10 dimensional (after reducing it by various component analysis such as PCA) and the output space is 4 dimensional. I am building a neural network that works something like ...
0
votes
0answers
11 views

What is the difference between “letter”-dimensional vector types?

What is the difference between D-dimensional vectors and S-dimensional vectors? Are there any other "letter"-dimensional vectors which one should be aware of? I am reading about occupancy modeling ...
2
votes
0answers
26 views

LONG SHORT-TERM MEMORY Hochreiter's paper BPTT

I was trying to read paper about LSTM, and I am stuck with mathematical problem. http://www.bioinf.jku.at/publications/older/2604.pdf page 4. see, |$f'_l$$_m$($net_l$$_m$)$w_l$$_m$ $_l$$_m$$_-$$_1$...
3
votes
0answers
98 views

How does one derive the modified tanh activation proposed by LeCun?

In "Efficient Backprop" (http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf), LeCun and others propose a modified tanh activation function of the form: $$ f(x) = 1.7159 * tanh(\frac{2}{3}*x) $$ ...
1
vote
1answer
33 views

What level of math is required for machine learning research

There are several levels of math understanding: Know the math Know the intuitions behind math concepts Know the intuitions and proofs of math concepts Know the intuitions, proofs of math concepts and ...
2
votes
0answers
30 views

Generative Model for learning Periodic solutions 3-body/N-body problems

I am tasked with finding research where a GAN or any other generative model is used to generate new shapes of 3-bodies moving under the influence of each others' gravitational pull, in a periodic ...
0
votes
1answer
26 views

intuitive understanding of (1+parameter)/parameter

When looking at a function that contains term (1+parameter)/parameter, should there be an intuitive understanding for why this term is included. For example if a function for fishing mortality ...
2
votes
0answers
61 views

Computational Complexity of Tucker decomposition

I am currently doing background reading for my Masters Thesis. I am working with tensor decompositions, whereby tensor I simply mean a multi-dimensional array. The aim of my master's project is to ...
1
vote
1answer
683 views

How regularization helps to get rid of outliers?

I have heard regularization helps to get rid of outliers, how so? 'My intuition is, regularization shrinks parameter or even make it zero, and hence large value will have less effect on overall result'...
2
votes
0answers
23 views

Is there a universal convergence rate when stacking models/experts?

It's fairly common to see people stacking different models when chasing marginal gains in contexts such as Kaggle competitions or the Netflix challenge. I would like to know about the mathematics ...
0
votes
2answers
100 views

ML, Statistics and Mathematics

I have just started getting my hands wet in ML and every time I try delving deeper into the concepts/code, I face the challenges of the mathematics and its cryptic notations. Coming from a Computer ...
0
votes
1answer
41 views

How do I know how to construct the layers of my CNN

I've done a CNN project with Keras and OpenCV, and I've got roughly 65% accuracy. And now I have to present this work in my University, but I'm afraid if the teachers ask me for how do I knew how to ...
2
votes
1answer
226 views

Help in understanding the maths behind Logistic Regression

I am following the lecture notes available https://www.stat.cmu.edu/~cshalizi/uADA/12/lectures/ch12.pdf I cannot understand how Eqs 12.4 and 12.5 come, why the Bernoulli probability has $1-p(x)$ in ...
0
votes
1answer
47 views

Kernel Trick and Inner Product Preservation

I understand that the point of using the kernel trick is to project the problem onto a higher dimensional space, where the problem is linearly separable. In this explanation, https://www.quora.com/...
3
votes
3answers
951 views

How do I calculate the range of a F1-score from a confusion matrix of 3 class,A,B,C

Is there any support function to calculate the average F1-score range?
2
votes
0answers
48 views

Max/Min of neurons at the hidden layer of a neural network

We have a neural network with inputs $x_1,...,x_n$, a single hidden layer with neurons $y_1,\ldots,y_m$, and outputs $z_1,...,z_v$. There are no activation functions (i.e., the activation functions ...
7
votes
1answer
140 views

bias and variance trade off related question

I am having difficulty to understand the expected squared errors formula in this website: $y=f(x)+e$ true regression line $\hat{y}=\hat{f}(x)$ your estimated regression line $error(x)=\bigg(\...
0
votes
2answers
627 views

Solving Bayes Theorem equation -> I can't calculate proper result

I am solving questions for an edx course on Machine Learning. One particular question is giving me a problem: Assume a patient comes into the doctor’s office to test whether they have a particular ...
13
votes
1answer
418 views

Finding linear transformation under which distance matrices are similar

I have $n$ sets of vectors, where each set $S_i$ contains $k$ vectors in $\mathbb{R}^d$. I know there is some unknown linear transformation $W$ under which the distance matrix $D_i$ (a $k\times k$ ...
0
votes
2answers
83 views

How to learn certain Maths to understand machine Learning papers?

I have done the deeplearning.ai course on deep learning. But I cannot Understand equations like minGmaxDV(D,G)=Ex∼pdata(x)[logD(x)]+Ez∼pz(z)[log(1−D(G(z)))] ...
1
vote
1answer
42 views

Choose points to maximize volume of convex hull

Suppose I have N points (labeled 1, 2, ..., k, ..., N) in D dimensions. I'd like to choose ...
1
vote
0answers
101 views

Updating Weight Using Updates on Related Data

Suppose $$ x=Ay $$ The $x$ is $M\times 1$, $y$ is $N \times 1$ and $A$ is $M\times N$ We have the data $x$ and would like to know what $y$ is. However, the matrix $A$ is too large for pseudo-...
16
votes
0answers
295 views

Formal proof of vanilla policy gradient convergence

So I stumbled upon this question, where the author asks for a proof of vanilla policy gradient procedures. The answer provided points to some literature, but the formal proof is nowhere to be included....
2
votes
1answer
94 views

Elements of Statistical Learning - question on p. 18

In the following expression: What do $E_{X}$ and $E_{Y|X}$ mean, respectively? In case helpful, I note that the authors derived that expression from the following: Based on the above, how would one ...
6
votes
0answers
142 views

When is the sum of models the model of the sum?

The response variable in a regression problem, $Y$, is modeled using a data matrix $X$. In notation, this means: $Y$ ~ $X$ However, $Y$ can be separated out into different components that can be ...