Questions tagged [mathematics]
Mathematics in a data science or machine learning context refers to the mathematical underpinnings for algorithms, optimization, statistics, and linear algebra etc.
230
questions
1
vote
1
answer
24
views
Which machine learning models are rational to use on NP-hard and NP-complete "theoretical" problems?
Time and time again I run into "surprising" NP-hard problems that seem naturally simpler than they are. I recently worked on a weighted graph theoretical problem where the point is to ...
- 13
0
votes
1
answer
150
views
Why both ChatGPT and Bard can't get a simple matrix calculation right?
I asked the following question to both ChatGPT 4 and Bard to see if they can get a simple matrix calculation right (after all Bill Gates said he was impressed by ChatGPT's math ability).
So I asked,
<...
- 113
0
votes
0
answers
15
views
Standardizing my target vs not standardizing
I've heard from multiple sources that it depends on whether I should standardize or not. Most of the time, people would say it doesn't make sense to do so, some would say it's better if I standardize ...
0
votes
1
answer
38
views
Suggestions to learn the Machine Learning models in greater depth?
I've been learning machine learning for the past few weeks from books and online courses. The books I've been reading, and currently still reading is "Hands-On Machine Learning with Scikit-Learn ...
0
votes
0
answers
32
views
RetNet Paper Multi Scale Retention dimemsion question
From the paper:
https://arxiv.org/pdf/2307.08621.pdf
But since X is of size n by $d_{model}$.
How can we compute $XW_Q$? Since the row length of X which is $d_{model}$ is not the same as the column ...
1
vote
1
answer
32
views
How in the heck should I tackle this classification problem? I'm not even sure if it's classification or regression
So, I'm currently a third year student in electrical engineering and I'm currently enrolled in a Mathematical Modelling and Machine Learning class and we're currently tasked to classify or use ...
- 11
0
votes
1
answer
42
views
Question regarding Lecture 5 CS229
I was at the part where we are using one covariance matrix and two mean vectors for fitting our Gaussian. I understood that we are using one covariance matrix and we can use different ones that would ...
0
votes
0
answers
8
views
Derive Keras Cosine Restarts Scheduler Formula from Source Code
I used Keras implementation of the cosine restarts algorithm. I need to know the exact formula of the algorithm for my LaTeX manuscript. The documentation is here. The implementation differs (in terms ...
- 101
0
votes
0
answers
13
views
When training deep learning model which is better, training with sampled data Vs. training on shorter epoch
I am running multiple hyperparameter optimization trials therefore trying to find a way to reduce time consumption. Two ways that I could think of are
search hyperparameter on subset of data.
search ...
- 457
0
votes
0
answers
32
views
How to split a range of numbers considering other variables as well?
Let's say that I have a vector of numbers and I'd like to split it into 3 most optimal ranges for example, then I suppose I can use k-means or Jenks natural breaks. If I'd like to do the same thing ...
0
votes
0
answers
13
views
How to prove equation (7)(8) in paper A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning?
I couldn't derive the formula (7)(8) in paper
https://arxiv.org/pdf/2110.01515.pdf
They didn't seem obvious to me.
- 1
1
vote
1
answer
41
views
Handling error when returning from log transform
Before training the model, I convert a target value to the log scale since range of the target is quite large.
After training the model, the Absolute Mean error was estimated as, for instance, 0.5.
If ...
- 13
0
votes
1
answer
30
views
Problem for a math formula in Weight Uncertainty in Neural Network
I am studying the paper https://arxiv.org/pdf/1505.05424.pdf and there is a formula I don't get page 4:
I don't understand how they obtain this formula. Moreover, with chain rule, I get $\frac{\...
- 1
0
votes
0
answers
17
views
How do you appropriately measure the real mean squared error of a box cox transformed linear regression model?
My understanding is that it can make sense to transform the outcomes of a linear regression model to make them more normally distributed. That's because it could 1) help me find more linear ...
- 101
0
votes
0
answers
28
views
Bayesian Neural Network Inference
In a paper about Bayesian Neural Network, I saw the following algorithm:
Algorithm 1 Inference procedure for a BNN.
Define $p(\theta\mid D)$;
for $i = 0$ to $N$ do
Draw $θ_i \sim p(\theta\mid D)$;
$...
- 1
0
votes
0
answers
6
views
Generalization error of a simple perceptron for a student-teacher network
I have been asked to prove the following expression given the following density probability function for a student-teacher
$ P(x,y) = \frac{1}{2\pi\sqrt{Q-R^2}} \cdot \exp\left(-\frac{1}{2}\left(\frac{...
- 1
1
vote
0
answers
32
views
Advice in career path [closed]
I'm currently in Argentina without any type of degree and knowledge in the field of data, maths, programming. My career goal is to get a starting job from here (Argentina) and eventually with some ...
0
votes
0
answers
21
views
What is the name of this window function?
Is there a common name for this window function?
I made it to replace a Hann window used in loading an FFT.
It is basically a wide lobe cosine tapered window, or negative Blackman window. Is there a ...
- 1
0
votes
0
answers
45
views
What is degree assortativity coefficient of a complete undirected graph?
Because it computes the correlation coefficient of degrees and the correlation coefficient of constant arrays is not defined, networkx library returns ...
- 3
1
vote
0
answers
24
views
Influence functions on neural networks: Help with understanding of result and derivation
I'm working through a paper titled "Understanding Black-box Predictions via Influence Functions" where they introduce the notion of influence functions from robust statistics to approximate ...
- 11
0
votes
0
answers
10
views
calculation for population vs for group
When performing statistical calculations like variance, mean squared error, and other equations, the approach differs depending on whether the data represents a sample from a population or the entire ...
- 51
0
votes
1
answer
68
views
Understanding the ResNet paper
I am having trouble understanding the mathematical meanings behind the notations in the ResNet paper:
I believe that the function we are trying to optimize is the residual which is denoted as $\...
- 1
0
votes
0
answers
17
views
Chi-Sqare test and Continuous values
How does chi-square test work for continuous variables. I see that it is used in most papers to test dependencies between continuous explanatory variables and the target variable. Please how does this ...
0
votes
0
answers
20
views
Fast Fourier Transform in computer vision
Can someone explain me how does FFT works in computer vision, please. I know something about FFT as an algorithm of competitive programming but I can't understand how it perform an image in computer ...
- 101
1
vote
0
answers
32
views
In WGAN paper, why does clipping weights approximate Lipschitz function?
In Wasserstein GAN, it's explained that maximizing a certain formula over a set of K-Lipschitz functions approximates the 1-Wasserstein distance and they model the functions as NNs. That much I ...
- 11
1
vote
1
answer
92
views
Trying to extrapolate info from a partial data set - statistical inference
I am wondering if my logic is OK here or not.
98% of a group without a device has an event occur
2% of group with device has an event occur
Since we know that correlation isn't causation I can't say ...
- 63
0
votes
0
answers
15
views
Why do you need more subjects than levels in RM ANOVA?
Honestly, I'm hoping I even asked this question appropriately. It's one of those things I never quite grasped, but just accepted to be true, that when running a within-subjects ANOVA, you have to have ...
- 1
0
votes
0
answers
10
views
Rotate vector anticlockwise
I'm learning mathematics for Machine Learning, and there given a vector [ 1 0 ] and [ 0 1 ]. The vector is packed on a matrix, ...
- 11
0
votes
0
answers
16
views
Matrix factorization approximate products to solve math solution
Problem
Matrix factorization for approximating products how do we solve such that Z approximates products N, M. How to define the math formula for solve for Z approximtaes the products of N,M?
...
0
votes
0
answers
10
views
Plus sign inside circle, there seem to be many meaning. Are they all related?
In Linear algebra plus sign inside a circle refers to direct sum.
In a paper "Towards Neural Mixture Recommender for Long Range Dependent User Sequences" it seems to refer to concatenation.
...
- 457
0
votes
0
answers
77
views
How to replicate MMCV conv layer with integrated batch norm using PyTorch
Question
How do I replicate the following convolution layer from MMCV using PyTorch? I cannot find any reference in the MMCV docs on how norm_cfg works.
...
- 46
0
votes
0
answers
16
views
Can I describe likelihood of prediction model using P(D | theta)?
Generally, to optimize model using MLE, you should find $\theta$ that maximizes $P(D | \theta)$.
In prediction model that gets input and predicts output, you should find $\theta$ that maximizes $P(Y|X,...
- 101
0
votes
1
answer
44
views
Can't find this book [closed]
I came across the most comprehensive ML Mathematics book (700+ pages) with sections on Probability, Calculus, Linear Algebra and Mathematical Foundations of the famous tricks in Deep Learning on ...
- 183
0
votes
0
answers
36
views
Mathematics for Machine Learning by A. Aldo Faisal is too abstract for machine learning?
I'm currently studying mathematics to jump into machine learning courses, and I'm taking the mathematics for machine learning specialization on Coursera while simultaneously reading the mentioned book....
2
votes
3
answers
76
views
Are there deterministic problems a neural network can't learn?
I have a software which takes 5 input numbers and outputs a number, deterministically. I want to try and mimic this software precisely with a neural network, but I am finding it very difficult. The ...
- 31
0
votes
1
answer
79
views
How does dropout behave like model averaging?
It is claimed Srivastava, Hinton, et al. that "dropout can be effectively applied in the hidden layers as well and that it can be interpreted as a form of model averaging" and that "...
- 1
1
vote
0
answers
35
views
A mathematician from the outside looking in
I am wondering if anybody could give a survey of applications of approximation theory to data science. One application I am familiar with are, for example, wavelet neural networks. Does anybody know ...
- 111
1
vote
1
answer
101
views
Is it possible to extract mathematical expression of an trained ML Model?
In Python & R, Linear Regression model gives the mathematical representation after learning the training data, typically in the form of intercept, coefficients of variables, and the p-value/t-...
- 11
0
votes
0
answers
13
views
What is the difference between classification and regression metrices such as accuracy and MSE? [duplicate]
What is the difference between classification metrics such as accuracy and regression metrics such as MPE? & Why can’t accuracy be used as regression metric?
0
votes
1
answer
66
views
What is the difference between accuracy in percentage (classification metric) and MPE (regression metric) as evaluation metrices?
What is the difference between accuracy in percentage (classification metric) and MPE (regression metric) as evaluation metrices?
0
votes
1
answer
43
views
what are the hot areas/future in computer science/machine learning in the next decade? [closed]
what are the hot areas in computer science/machine learning in the next decade ? I am interested in knowing this since deep learning/machine learning which dominated in the last decade has saturated ...
user141819
-1
votes
1
answer
88
views
How is Probability Used in Data Science? [closed]
This is my first Question so apologies if I do not stick to the standards.
What I want to understand is how is all of the following topics:
Probability
Different Probability Distributions.
Baye's ...
- 11
0
votes
1
answer
30
views
Journals to publish a proof of a math result used for neural-network algorithms
I would like to know which journal is an appropriate outlet for the results described below.
I recently came across a particular neural-network training algorithm. The algorithm is based on a result ...
- 1
1
vote
1
answer
32
views
Why there is so few research on neural code of artificial neural networks and are there alternatives to the neural code approach?
I feel that the neural code/neural coding (how neurons or biases enode the symbolic concepts of the chains of concepts, e.g. each feature is chain of symbolic functions and their parameters) is the ...
- 141
1
vote
0
answers
68
views
What is the meaning about the $\alpha$ in TD3 algorithm
I am study the paper with TD3 algorithm.
I am curious about the meaning of $\alpha$ while the paper prove that overestimation will be happened in a critical situation.
The contents about mathematical ...
- 25
2
votes
0
answers
33
views
Discretize interval into fixed number of bins while minimizing error
Background
I have been working on an optimization problem related to the following function:
$$ y = f(x) = c_{1}(1-e^{-c_{2}(x-x_{0})})^{2}$$
where $c_{1}$, $c_{2}$ and $x_{0}$ are constants. Assume ...
1
vote
0
answers
21
views
Resources for Symbolic Regression
What are some good resources/recommendations related to symbolic regression? (In terms of predicting analytic mathematical equations from neural networks)
- 45
1
vote
1
answer
28
views
How do we derive our loss function from the gradient objective?
I've been dwelling through RL theory and practice and one particular part I find hard to properly understand is the relation between the practical loss function and ...
- 111
0
votes
1
answer
15
views
How can I transform or plot my data to see power consumption more easily?
I have two heating tapes installed in my setup and they provide heat to maintain the reaction at a certain setpoint temperature. Basically, the heating tapes go on a cycle of on/off to maintain the ...
1
vote
1
answer
21
views
Problem understanding the forward algorithm for HMMs
I found a recursive version of the forward algorithm on wikipedia, however I don't understand the notation given in the pseudocode:
What means $$x_{t-1}$$ under the summation sign? What do I need to ...
- 131