What's the deal with Deno? We talk with a major contributor to find out. Listen now.

Hot answers tagged

30

Speaking of '$99\%$ of the points in a hypercube' is a bit misleading since a hypercube contains infinitely many points. Let's talk about volume instead. The volume of a hypercube is the product of its side lengths. For the 50-dimensional unit hypercube we get $$\text{Total volume} = \underbrace{1 \times 1 \times \dots \times 1}_{50 \text{ times}} = 1^{50} =...


10

You can see the pattern clearly even in lower dimensions. 1st dimension. Take a line of length 10 and a boundary of 1. The length of the boundary is 2 and the interior 8, 1:4 ratio. 2nd dimension. Take a square of side 10, and boundary 1 again. The area of the boundary is 36, the interior 64, 9:16 ratio. 3rd dimension. Same length and boundary. The volume ...


8

Although you need book, I recommend the following courses respectively for understanding statistics which are used for machine learning and other tasks in data science. They are free. Learn Statistics - Intro to Statistics Course Intro to Descriptive Statistics Inferential Statistics: Learn Statistical Analysis If I want to recommend a book, I would ...


6

Short answer: Both formulations lead to the same answer. Mathematical explanation: In order to understand that let us look at two similar problems. Imagine we want to integrate a function $f(x)=x^2$ on two intervals $I_1=[0,1]$ (including both bounds) and $I_2=(0,1)$ (excluding both bounds). For $I_1$ we have $$\int_0^1 x^2~dx=1/3.$$ For the second ...


3

It means that $P(\theta | y ) = kP(\theta) P(y | \theta)$, where $k$ is a constant that does not depend on $\theta$. In fact, the Bayes Theorem states $P(\theta | y ) = \frac{P(\theta) P(y | \theta)}{P(y)}$. This proportion has to be added in order that $\sum_{\theta}P(\theta | y) = 1$ is satisfied (that is, $P(\theta | y)$ has to be a probability).


3

After we have $$w^Tx + b = \pm \delta$$ We can always divide everything by $\delta$, $$\left( \frac{w}{\delta}\right)^Tx + \left( \frac{b}{\delta}\right)=\pm1$$ Now, we can set $\tilde{w}=\frac{w}{\delta}$ and $\tilde{b}=\frac{b}{\delta}$. $$\tilde{w}^Tx+\tilde{b}=\pm1$$ This is as if we have set $\delta=1$ from the beginning. The derivation of the ...


3

Note that $\frac{\partial L}{\partial \theta}$ is different from $\frac{\partial \theta}{\partial L}$. What you tried to describe seems to be $\frac{\partial L}{\partial \theta}$ where $\theta$ is a variable. If $\theta$ is high dimensional, sometimes we just use the $\nabla$ notation. Gradient descent is $$\theta_{n+1}=\theta_n-\gamma \nabla L(\theta_n)...


3

You can think of examples as vectors in $\mathbb{R}^p$, where $p$ is the number of features. Two examples will be very similar if the distance between them is close to $0$ (in the extreme case, if two examples are equal their euclidean distance is $0$). One way to measure the distance is using euclidean distance, but other distances can be used, as cosine ...


3

In most cases, I would go for NumPy. Implement a Python function f(t) that calculates the $t$-th summand. Then run import numpy as np result = np.array([f(t) for t in range(1,m+1)]).sum() This will be very fast, unless $m$ is so large that [f(t) for t in range(1,m+1)] does not fit into memory. In this case, I would follow your approach and use a for-loop: ...


3

We know that: (1) $\frac{\partial}{\partial x}\big (f(x) + g(x) \big) = \frac{\partial}{\partial x}f(x) + \frac{\partial}{\partial x}g(x)$ (2) $\frac{\partial}{\partial x}a = 0$ Now, \begin{align*} &\frac{\partial}{\partial w_{12}^{1}} (w_{11}^{1}h_1^{2} + w_{12}^{1}h_2^{2} + w_{13}^{1}h_3^{2} + b_1^{1}) = & \text{[using (1)]}\\ &\frac{\partial}{\...


2

Introduction to Linear Algebra https://math.mit.edu/~gs/linearalgebra/ is a good starting point. Make sure you are good with Probability Theory, Linear Algebra and Statistics. A very in depth knowledge may not be necessary but having a good knowledge is required.


2

Before doing my master in Analytics, I was suggested by my seniors to go through these couple of books to know more about Machine Learning and Statistics. Namely: Discovering statistics with SPSS/R - Andy Field R Beginner and R for Everyone Predictive Analytics - The Power to Predict Who Will Click, Buy, Lie or Die Data Science for Business and many more ...


2

No , log doesn't disappear . From the equation , When you want to calculate , it essentially means calculating , Now , So , as .


2

Assuming the implementation is as simple as possible, with no advanced concepts, is it likely for something like this to happen or is it definitely an error in the implementation? In my experience, using the simplest possible network, and simplest gradient descent algorithm, then yes this happens relatively frequently. It is an accident of the starting ...


2

Your understanding is correct. This is known as the indicator function. The indicator function of a subset $A$ of a set $X$ is a function $$1_A(x)= \begin{cases}1, & x \in A \\ 0, & x \notin A \end{cases}$$


2

What is $t$? It means observed $r_{ui}$ in the one-week test set (page 6-left). Is my understanding of the metric correct? First two examples are correct. Assuming user-item relation $r_{ui}^t$ is constant $a$ for all items in the test set, and predicted ranks are uniform across $[0, 1]$, then, the third one would be: $$\overline{\text{rank}} = \frac{\...


2

A simple and direct answer is that skewness and kurtosis are both defined in terms of the $Z-$values, $Z = (X-\mu)/\sigma$ : Skewness = $E(Z^3)$ and Kurtosis = $E(Z^4)$. When talking about a data set, you can just replace the expectation operator "$E$" with the ordinary average. Since raising a number to either the third or fourth power amplifies values ...


2

As Elias mentioned, the expectations are related to random variables and you would be good to go if you know about conditional probability, multivariate probability, joint and marginal distributions. I would suggest you take a course that has a syllabus on the lines of https://secure.oregonstate.edu/ap/cps/documents/view/134169.


2

I think the intuition here is that you want to push the negative loglikelihood (NLL) * the reward to be as negative as possible. Since our reward is often not differentiable because it is obtained through sampling, we can only change the NLL. For actions with high reward, we have greater "pressure" / gradient to push the NLL down, than if the ...


2

You're using the Mean Square Error $\Sigma\frac{1}{N}(y-(Wx+b))^2$as the loss function, if you take the derivative, you will have the $2$. In some materials, we will use $\frac{1}{2}\Sigma\frac{1}{N}(y-(Wx+b))^2$ as the loss function to cancel out the $2$. In fact, this doesn't matter at all and it has no impacts on params optimization.


2

There can be no objective answer to this question. Obviously the more one understands the better, but the field of ML is vast, quite specialized and ranges from very theoretical to very applied research, so it's perfectly reasonable to publish in ML without a strong background in maths. A better way to estimate your own ability to publish papers in a ...


2

Let $n$ be a convolutional layer with dimensions $w' \times h' \times c'$. Then each of its $c'$ filters is connected to all $c$ filters (or channels*) of the previous layer. I find it helpful to look at the number of weights here: A single filter of that convolutional layer $n$ with kernel size $k'\times k'$ will have $c \times k' \times k'$ weights. And ...


2

Reading the article they state that The two vertical lines represent the $L^2$ norm of the error, or what is known as the sum-of-squares error (SSE) Otherwise known as the Euclidean norm, Euclidean length, $L^2$ distance, $ℓ^2$ distance, $L^2$ norm, $ℓ^2$ norm, $2$-norm, or square norm. Also speaking from my experience, most of the time when a norm is ...


2

Check out the indicator function $1_S(\cdot)$. In your case it would be fined as $$1_S: \{1, \ldots, d\} \rightarrow \{0, 1\}, j \mapsto \begin{cases} 1 & j \in S \\ 0 & \, \text{else} \end{cases}.$$ Multiplying this function with the respective values should give you what you are looking for. Edit: If I understand your comment correctly, the ...


2

Weights at first layer will depend on the Dimension of the input too. Let's assume, This is your Network. Input has 5 Features $\hspace{6cm}$ Image credit - ML Visuals by dair.ai Per layer weights and biases \begin{array} {|r|r|} \hline &Weights &Biases\\ \hline Layer-1 &5*3 = 15 &3\\ \hline Layer-2 &3*3 = 9 &3\\ \hline Layer-3 &...


2

He is referring to Lagrangian Multipliers, an optimization technique for problems with equality constraints.


1

By the looks of it you just flipped the conditional probabilities when build the first table. P(test+|sick) = 0.85 according to the description. In your table and equation, however, you take it to be 0.90. Because this felt too short for an answer, I reworked it out : P(test+|sick)P(sick) P(sick|test+) = -------------------- ...


1

Bayes theorem: $P(A|B) = \frac{P(B|A) P(A)}{P(B)}$ In our case: $P(disease|test_+) = \frac{P(test_+|disease) P(disease)}{P(test_+)}$ Given: $P(test_+|disease) = 0.85 \\ P(test_-|healthy) = 0.9 \\ P(disease) = 0.02$ Since a test can be positive while the patient has a disease and it also can be positive while the patient is healthy, the overall ...


1

The actual loss is supposed to be $m.log\_prob(action) * reward$ without the negative sign. The default optimizer in Pytorch uses gradient descent methods, while the REINFORCE assumes gradient ascent update rule. To account for this the loss is made negative. This is clearly mentioned in the documentation. To get the intuitive understanding, you can remove ...


1

Apart from your tattoo, in gradient descent, the loss function needs to be minimised which is our objective function in this case. The Gradient Descent update rule states that, $\Large \theta_{ij} = \theta_{ij} - \frac{\partial L}{\partial \theta_{ij}}$ Where $\theta$ is the parameter which needs to be optimised. This is the fundamental equation of ...


Only top voted, non community-wiki answers of a minimum length are eligible