Skip to main content
Expanded comment on formula
Source Link
Skiddles
  • 998
  • 4
  • 12

In your example, you should calculate the probability that the value goes up, and use that as your $p$.

That said, I think the calculation should be $\sigma = \sqrt{p(1-p)}$ otherwise, your standard deviation is inflated by a factor tied to $n$. By using probabilities, you are essentially taking $n$ into account because: $$p = {Number\ of\ "Up"\ instances \over n}$$ Where $n$ is the total number of observations.

Given the nature of the data, that is, it is binary, we would expect the $\sigma$ to be between 0 and 1 and not particularly sensitive to the number of observations.

That said, if we look at a few calculations, the value of $n$ in your formula significantly impacts the $\sigma$.

Assume for simplicity that $p$ is 0.5, that is an equal chance of going up or down. If you have three different values of $n$ you get three vastly different values for $\sigma$.

Example 1: $n$ = 1 (Or not included) $$\sigma = \sqrt{0.5 * (1-0.5)},\ \therefore \sigma = 0.5$$

Example 2: $n$ = 100 $$\sigma = \sqrt{0.5 * 100 * (1-0.5)},\ \therefore \sigma = 5$$

Example 3: $n$ = 1000 $$\sigma = \sqrt{0.5 * 1000 * (1-0.5)},\ \therefore \sigma = 15.8$$

Of the three, only example 1 really makes any sense intuitively. The mean of all instances would be 0.5 and you would expect the observed values to be 0.5 away from the mean.

In your example, you should calculate the probability that the value goes up, and use that as your $p$.

That said, I think the calculation should be $\sigma = \sqrt{p(1-p)}$ otherwise, your standard deviation is inflated by a factor tied to $n$. By using probabilities, you are essentially taking $n$ into account because: $$p = {Number\ of\ "Up"\ instances \over n}$$ Where $n$ is the total number of observations.

In your example, you should calculate the probability that the value goes up, and use that as your $p$.

That said, I think the calculation should be $\sigma = \sqrt{p(1-p)}$ otherwise, your standard deviation is inflated by a factor tied to $n$. By using probabilities, you are essentially taking $n$ into account because: $$p = {Number\ of\ "Up"\ instances \over n}$$ Where $n$ is the total number of observations.

Given the nature of the data, that is, it is binary, we would expect the $\sigma$ to be between 0 and 1 and not particularly sensitive to the number of observations.

That said, if we look at a few calculations, the value of $n$ in your formula significantly impacts the $\sigma$.

Assume for simplicity that $p$ is 0.5, that is an equal chance of going up or down. If you have three different values of $n$ you get three vastly different values for $\sigma$.

Example 1: $n$ = 1 (Or not included) $$\sigma = \sqrt{0.5 * (1-0.5)},\ \therefore \sigma = 0.5$$

Example 2: $n$ = 100 $$\sigma = \sqrt{0.5 * 100 * (1-0.5)},\ \therefore \sigma = 5$$

Example 3: $n$ = 1000 $$\sigma = \sqrt{0.5 * 1000 * (1-0.5)},\ \therefore \sigma = 15.8$$

Of the three, only example 1 really makes any sense intuitively. The mean of all instances would be 0.5 and you would expect the observed values to be 0.5 away from the mean.

Expanded comment on formula
Source Link
Skiddles
  • 998
  • 4
  • 12

In your example, you should calculate the probability that the value goes up, and use that as your $p$.

That said, I think the calculation should be $\sigma = \sqrt{p(1-p)}$ otherwise, your standard deviation is inflated by a factor tied to $n$. By using probabilities, you are essentially taking $n$ into account because: $$p = {Number\ of\ "Up"\ instances \over n}$$ Where $n$ is the total number of observations.

In your example, you should calculate the probability that the value goes up, and use that as your $p$.

In your example, you should calculate the probability that the value goes up, and use that as your $p$.

That said, I think the calculation should be $\sigma = \sqrt{p(1-p)}$ otherwise, your standard deviation is inflated by a factor tied to $n$. By using probabilities, you are essentially taking $n$ into account because: $$p = {Number\ of\ "Up"\ instances \over n}$$ Where $n$ is the total number of observations.

Source Link
Skiddles
  • 998
  • 4
  • 12

In your example, you should calculate the probability that the value goes up, and use that as your $p$.