# Tag Info

36

GELU function We can expand the cumulative distribution of $\mathcal{N}(0, 1)$, i.e. $\Phi(x)$, as follows: $$\text{GELU}(x):=x{\Bbb P}(X \le x)=x\Phi(x)=0.5x\left(1+\text{erf}\left(\frac{x}{\sqrt{2}}\right)\right)$$ Note that this is a definition, not an equation (or a relation). Authors have provided some justifications for this proposal, e.g. a ...

32

Speaking of '$99\%$ of the points in a hypercube' is a bit misleading since a hypercube contains infinitely many points. Let's talk about volume instead. The volume of a hypercube is the product of its side lengths. For the 50-dimensional unit hypercube we get $$\text{Total volume} = \underbrace{1 \times 1 \times \dots \times 1}_{50 \text{ times}} = 1^{50} =... 15 Having a solid mathematical background is crucial for data science. Someone without solid mathematical background will always use the algorithms as black box models. Mathematical reasoning is needed when you debug your models but also when you want to come up with a creative solution to a problem at hand. Someone without mathematical background will have a ... 10 Although you need book, I recommend the following courses respectively for understanding statistics which are used for machine learning and other tasks in data science. They are free. Learn Statistics - Intro to Statistics Course Intro to Descriptive Statistics Inferential Statistics: Learn Statistical Analysis If I want to recommend a book, I would ... 10 You can see the pattern clearly even in lower dimensions. 1st dimension. Take a line of length 10 and a boundary of 1. The length of the boundary is 2 and the interior 8, 1:4 ratio. 2nd dimension. Take a square of side 10, and boundary 1 again. The area of the boundary is 36, the interior 64, 9:16 ratio. 3rd dimension. Same length and boundary. The volume ... 10 First note that$$\Phi(x) = \frac12 \mathrm{erfc}\left(-\frac{x}{\sqrt{2}}\right) = \frac12 \left(1 + \mathrm{erf}\left(\frac{x}{\sqrt2}\right)\right)$$by parity of \mathrm{erf}. We need to show that$$\mathrm{erf}\left(\frac x {\sqrt2}\right) \approx \tanh\left(\sqrt{\frac2\pi} \left(x + a x^3\right)\right)$$for a \approx 0.044715. For large values ... 6 Short answer: Both formulations lead to the same answer. Mathematical explanation: In order to understand that let us look at two similar problems. Imagine we want to integrate a function f(x)=x^2 on two intervals I_1=[0,1] (including both bounds) and I_2=(0,1) (excluding both bounds). For I_1 we have$$\int_0^1 x^2~dx=1/3.$$For the second ... 5 This feels like a bit of a convoluted way to introduce the concept, but alright :D Let me start at a slightly different point. Maybe in Machine Learning or in other places you have encountered the sigmoid function:$$ sigmoid(S) = \frac{e^S}{1+e^S} = \frac{1}{1+e^{-S}} $$The sigmoid has the nice property to map any real number S to a number between 0 ... 5 From the documentation you referred: "The length of the span is the range of a 64-bit integer times the length of the date or unit." 64 bit integer has values from -2^63 to 2^63-1, which is the same as from -9.2e18 to 9.2e18. So, the time span column shows you which dates would you cover if use only the corresponding units. Note, i.e. that time ... 4 Naming linear data or non-linear data is a bit misleading and wrong I would say. Instead, there is linear relation and non-linear relation between variables would be better and correct naming. It can easily be explained by example from real life. Non-linear relation: Any kind of relationship that is not linear can be put forward as an example (e.g. ... 4 Statistical knowledge or statistical thinking is useful or necessary to: Understand, evaluate and pick appropriate metrics to use to evaluate the performance of models. You need to understand the real-world cost of prediction errors and how each metric relates to this. Explore and understand the data e.g. to help inform future models or other business ... 3 Note that \frac{\partial L}{\partial \theta} is different from \frac{\partial \theta}{\partial L}. What you tried to describe seems to be \frac{\partial L}{\partial \theta} where \theta is a variable. If \theta is high dimensional, sometimes we just use the \nabla notation. Gradient descent is$$\theta_{n+1}=\theta_n-\gamma \nabla L(\theta_n)...

3

It means that $P(\theta | y ) = kP(\theta) P(y | \theta)$, where $k$ is a constant that does not depend on $\theta$. In fact, the Bayes Theorem states $P(\theta | y ) = \frac{P(\theta) P(y | \theta)}{P(y)}$. This proportion has to be added in order that $\sum_{\theta}P(\theta | y) = 1$ is satisfied (that is, $P(\theta | y)$ has to be a probability).

3

You can think of examples as vectors in $\mathbb{R}^p$, where $p$ is the number of features. Two examples will be very similar if the distance between them is close to $0$ (in the extreme case, if two examples are equal their euclidean distance is $0$). One way to measure the distance is using euclidean distance, but other distances can be used, as cosine ...

3

In most cases, I would go for NumPy. Implement a Python function f(t) that calculates the $t$-th summand. Then run import numpy as np result = np.array([f(t) for t in range(1,m+1)]).sum() This will be very fast, unless $m$ is so large that [f(t) for t in range(1,m+1)] does not fit into memory. In this case, I would follow your approach and use a for-loop: ...

3

Introduction to Linear Algebra is a good starting point. Make sure you are good with probability theory, linear algebra, and statistics. A very in depth knowledge may not be necessary, but having a good knowledge is required.

3

After we have $$w^Tx + b = \pm \delta$$ We can always divide everything by $\delta$, $$\left( \frac{w}{\delta}\right)^Tx + \left( \frac{b}{\delta}\right)=\pm1$$ Now, we can set $\tilde{w}=\frac{w}{\delta}$ and $\tilde{b}=\frac{b}{\delta}$. $$\tilde{w}^Tx+\tilde{b}=\pm1$$ This is as if we have set $\delta=1$ from the beginning. The derivation of the ...

3

We know that: (1) $\frac{\partial}{\partial x}\big (f(x) + g(x) \big) = \frac{\partial}{\partial x}f(x) + \frac{\partial}{\partial x}g(x)$ (2) $\frac{\partial}{\partial x}a = 0$ Now, \begin{align*} &\frac{\partial}{\partial w_{12}^{1}} (w_{11}^{1}h_1^{2} + w_{12}^{1}h_2^{2} + w_{13}^{1}h_3^{2} + b_1^{1}) = & \text{[using (1)]}\\ &\frac{\partial}{\...

3

No, you don't need mathematics for data science in the same way that you need it for physics. As a data scientist, you won't be integrating a stress-energy tensor, or even solving a differential equation. What you do need is good quantitative reasoning and critical thinking skills. NotThatGuy's answer gives good examples of things you might think of as &...

2

Before doing my master in Analytics, I was suggested by my seniors to go through these couple of books to know more about Machine Learning and Statistics. Namely: Discovering statistics with SPSS/R - Andy Field R Beginner and R for Everyone Predictive Analytics - The Power to Predict Who Will Click, Buy, Lie or Die Data Science for Business and many more ...

2

Your understanding is correct. This is known as the indicator function. The indicator function of a subset $A$ of a set $X$ is a function $$1_A(x)= \begin{cases}1, & x \in A \\ 0, & x \notin A \end{cases}$$

2

No, the logartihm doesn't disappear. From the equation , When you want to calculate , it essentially means calculating , Now , So , as .

2

Assuming the implementation is as simple as possible, with no advanced concepts, is it likely for something like this to happen or is it definitely an error in the implementation? In my experience, using the simplest possible network, and simplest gradient descent algorithm, then yes this happens relatively frequently. It is an accident of the starting ...

2

What is $t$? It means observed $r_{ui}$ in the one-week test set (page 6-left). Is my understanding of the metric correct? First two examples are correct. Assuming user-item relation $r_{ui}^t$ is constant $a$ for all items in the test set, and predicted ranks are uniform across $[0, 1]$, then, the third one would be: \overline{\text{rank}} = \frac{\...

2

As Elias mentioned, the expectations are related to random variables and you would be good to go if you know about conditional probability, multivariate probability, joint and marginal distributions. I would suggest you take a course that has a syllabus on the lines of https://secure.oregonstate.edu/ap/cps/documents/view/134169.

2

A simple and direct answer is that skewness and kurtosis are both defined in terms of the $Z-$values, $Z = (X-\mu)/\sigma$ : Skewness = $E(Z^3)$ and Kurtosis = $E(Z^4)$. When talking about a data set, you can just replace the expectation operator "$E$" with the ordinary average. Since raising a number to either the third or fourth power amplifies values ...

2

I think the intuition here is that you want to push the negative loglikelihood (NLL) * the reward to be as negative as possible. Since our reward is often not differentiable because it is obtained through sampling, we can only change the NLL. For actions with high reward, we have greater "pressure" / gradient to push the NLL down, than if the ...

2

Yes, the line indicates that KNN is weighted and that the weight is the inverse of the distance. All of this can easily be found in scikit-learn's documentation Also, pro-tip, you can find an object's documentation using the help function. In this case: from sklearn.neighbors import KNeighborsClassifier print(help(KNeighborsClassifier)) As to how ...

2

You're using the Mean Square Error $\Sigma\frac{1}{N}(y-(Wx+b))^2$as the loss function, if you take the derivative, you will have the $2$. In some materials, we will use $\frac{1}{2}\Sigma\frac{1}{N}(y-(Wx+b))^2$ as the loss function to cancel out the $2$. In fact, this doesn't matter at all and it has no impacts on params optimization.

2

There can be no objective answer to this question. Obviously the more one understands the better, but the field of ML is vast, quite specialized and ranges from very theoretical to very applied research, so it's perfectly reasonable to publish in ML without a strong background in maths. A better way to estimate your own ability to publish papers in a ...

Only top voted, non community-wiki answers of a minimum length are eligible