# Tag Info

Accepted

GELU function We can expand the cumulative distribution of $\mathcal{N}(0, 1)$, i.e. $\Phi(x)$, as follows: $$\text{GELU}(x):=x{\Bbb P}(X \le x)=x\Phi(x)=0.5x\left(1+\text{erf}\left(\frac{x}{\sqrt{2}}... 33 votes Accepted ### What does it mean when we say most of the points in a hypercube are at the boundary? Speaking of '99\% of the points in a hypercube' is a bit misleading since a hypercube contains infinitely many points. Let's talk about volume instead. The volume of a hypercube is the product of ... 16 votes ### Do you actually need math for your data science job? Having a solid mathematical background is crucial for data science. Someone without solid mathematical background will always use the algorithms as black box models. Mathematical reasoning is needed ... 10 votes Accepted ### Beginner math books for Machine Learning Although you need book, I recommend the following courses respectively for understanding statistics which are used for machine learning and other tasks in data science. They are free. Learn ... 10 votes ### What does it mean when we say most of the points in a hypercube are at the boundary? You can see the pattern clearly even in lower dimensions. 1st dimension. Take a line of length 10 and a boundary of 1. The length of the boundary is 2 and the interior 8, 1:4 ratio. 2nd dimension. ... 10 votes ### What is GELU activation? First note that$$\Phi(x) = \frac12 \mathrm{erfc}\left(-\frac{x}{\sqrt{2}}\right) = \frac12 \left(1 + \mathrm{erf}\left(\frac{x}{\sqrt2}\right)\right)$$by parity of \mathrm{erf}. We need to show ... 6 votes Accepted ### Estimating the value of \pi with a Monte Carlo dartboard: < or \leq? Short answer: Both formulations lead to the same answer. Mathematical explanation: In order to understand that let us look at two similar problems. Imagine we want to integrate a function f(x)=x^2 ... 5 votes Accepted ### Help in understanding the maths behind Logistic Regression This feels like a bit of a convoluted way to introduce the concept, but alright :D Let me start at a slightly different point. Maybe in Machine Learning or in other places you have encountered the ... 5 votes Accepted ### Which is meant by +/-9.2e18 years in timespan? From the documentation you referred: "The length of the span is the range of a 64-bit integer times the length of the date or unit." 64 bit integer has values from ... 5 votes Accepted ### Why both ChatGPT and Bard can't get a simple matrix calculation right? In my understanding, LLMs are, very simplified speaking, probabilistic solvers. Math problems such as matrix multiplications are, on the other hand, deterministic in nature. Thus, using a an LLM for ... 4 votes Accepted ### How are the channels handled in CNN? Is it independently processed or fused? Let n be a convolutional layer with dimensions w' \times h' \times c'. Then each of its c' filters is connected to all c filters (or channels*) of the previous layer. I find it helpful to ... 4 votes Accepted ### Is there such thing as linear and non-linear data? Naming linear data or non-linear data is a bit misleading and wrong I would say. Instead, there is linear relation and non-linear relation between variables would be better and correct naming. It can ... 4 votes ### Do you actually need math for your data science job? Statistical knowledge or statistical thinking is useful or necessary to: Understand, evaluate and pick appropriate metrics to use to evaluate the performance of models. You need to understand the ... 4 votes ### Do you actually need math for your data science job? No, you don't need mathematics for data science in the same way that you need it for physics. As a data scientist, you won't be integrating a stress-energy tensor, or even solving a differential ... 3 votes Accepted ### Neural networks, optimization math intuition Note that \frac{\partial L}{\partial \theta} is different from \frac{\partial \theta}{\partial L}. What you tried to describe seems to be \frac{\partial L}{\partial \theta} where \theta is a ... 3 votes ### What does the term "proportional to" mean in Bayes Equation? It means that P(\theta | y ) = kP(\theta) P(y | \theta), where k is a constant that does not depend on \theta. In fact, the Bayes Theorem states P(\theta | y ) = \frac{P(\theta) P(y | \theta)}... 3 votes Accepted ### Knn and euclidean distance You can think of examples as vectors in \mathbb{R}^p, where p is the number of features. Two examples will be very similar if the distance between them is close to 0 (in the extreme case, if two ... 3 votes Accepted ### Recreating the sum symbol using python In most cases, I would go for NumPy. Implement a Python function f(t) that calculates the t-th summand. Then run ... 3 votes ### Beginner math books for Machine Learning Introduction to Linear Algebra is a good starting point. Make sure you are good with probability theory, linear algebra, and statistics. A very in depth knowledge may not be necessary, but having a ... 3 votes Accepted ### SVM hyperplane margin After we have$$w^Tx + b = \pm \delta$$We can always divide everything by \delta,$$\left( \frac{w}{\delta}\right)^Tx + \left( \frac{b}{\delta}\right)=\pm1Now, we can set \tilde{w}=\frac{w}... 3 votes Accepted ### Understanding the algebra behind a specific partial derivative equation We know that: (1) \frac{\partial}{\partial x}\big (f(x) + g(x) \big) = \frac{\partial}{\partial x}f(x) + \frac{\partial}{\partial x}g(x) (2) \frac{\partial}{\partial x}a = 0 Now, \begin{align*} &... 2 votes ### Beginner math books for Machine Learning Before doing my master in Analytics, I was suggested by my seniors to go through these couple of books to know more about Machine Learning and Statistics. Namely: Discovering statistics with SPSS/R -... 2 votes Accepted ### How can positional encodings including a sine operation be linearly transformable for any offset? I elected to ask this question on the Mathematics Stack Exchange and I thought it prudent to add the answer here: https://math.stackexchange.com/q/3119882 From what I have learned from @Servaes, who ... 2 votes ### Comparison between addition and multiplication function in deep neural network? The observation is very interesting you report, since concatenation and addition are practically the same. A nice explanation can be found in https://distill.pub/2018/feature-wise-transformations/ . 2 votes Accepted ### Meaning of this notion in 0-1 loss? Your understanding is correct. This is known as the indicator function. The indicator function of a subset A of a set X is a function1_A(x)= \begin{cases}1, & x \in A \\ 0, & x \...
Accepted

No, the logartihm doesn't disappear. From the equation , When you want to calculate , it essentially means calculating , Now , So , as .
Accepted

### Can a single-layer ANN get XOR wrong?

Assuming the implementation is as simple as possible, with no advanced concepts, is it likely for something like this to happen or is it definitely an error in the implementation? In my experience, ...
Accepted

### What is the range of values of the expected percentile ranking?

What is $t$? It means observed $r_{ui}$ in the one-week test set (page 6-left). Is my understanding of the metric correct? First two examples are correct. Assuming user-item relation $r_{ui}^t$ ...
A simple and direct answer is that skewness and kurtosis are both defined in terms of the $Z-$values, $Z = (X-\mu)/\sigma$ : Skewness = $E(Z^3)$ and Kurtosis = $E(Z^4)$. When talking about a data set, ...