2
$\begingroup$

I am reading pattern Recognition and machine learning by Bishop and in the chapter about probability, "noise in the observed data" is mentioned many times. I have read on the internet that noise refers to the inaccuracy while reading data but I am not sure whether it is correct. SO, what actually is noise in observed data? And what is additive noise and Gaussian noise ?

$\endgroup$
6
$\begingroup$

When you have sensors, the values you receive change even if the signal that was recorded didn't change. This is one example of noise.

When you have a model of the world, it abstracts from the real relationships by simplifying things which are not too important. To take into account for the simplification, you model the error as noise (e.g. in a Kalman filter).

But noise sources can be anything. For example, in a image classification problem, where you have to assign labels (like cat, dog, car, plane) to images, you could model errors of the human labelers as noise. In general, I'd say noise are (undesirable) effects which you can't describe deterministic in a simple enough way, so you describe it with random variables.

And what is additive noise?

Suppose your system equation is

$$z = H \cdot x$$

where $z \in \mathbb{R}^{n_m}$ is the observation, $x \in \mathbb{R}^{n_x}$ is the state you're interested in and $H \in \mathbb{R}^{n_m \cdot n_x}$ is a transformation matrix. Then the noise could interact with your system in any way. But most of the time it is logical and practical that the noise is additive, meaning your model is

$$z = H \cdot x + r$$

where $r$ is sampled from a random variable of any distribution.

What is (additive) Gaussian noise?

$$r \sim \mathcal{N}(\mu, \sigma^2)$$

See normal distribution

$\endgroup$
  • 1
    $\begingroup$ I think it is $r\sim N(0,\sigma^2)$. $\endgroup$ – L.V.Rao Jan 1 '17 at 6:56
  • $\begingroup$ @L.V.Rao Usually, yes (+1). The point is, if $r$ has a mean $\mu \neq 0$, then you can add this term in your system equation and thus make $r$ have zero mean. So it all depends on your model. $\endgroup$ – Martin Thoma Jan 1 '17 at 10:53
  • $\begingroup$ "I'd say noise are (undesirable) effects which you can't describe deterministic in a simple enough way". I think this is not correct. We have stochastic and deterministic noise. The definition matches to stochastic noise but not to deterministic noise $\endgroup$ – Code Pope Jun 15 '18 at 6:23
  • $\begingroup$ @CodePope Coud you give an example for stochastic noise? Where does your definition come from? $\endgroup$ – Martin Thoma Jun 15 '18 at 6:30
  • 1
    $\begingroup$ If the noise is random like in your example, we are talking about stochastic noise. If the noise is due to the complexity of the data so some part cannot be modeled then we have deterministic noise. Wikipedia: en.wikipedia.org/wiki/Deterministic_noise $\endgroup$ – Code Pope Jun 15 '18 at 7:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.