# What is “noise” in observed data?

I am reading pattern Recognition and machine learning by Bishop and in the chapter about probability, "noise in the observed data" is mentioned many times. I have read on the internet that noise refers to the inaccuracy while reading data but I am not sure whether it is correct. SO, what actually is noise in observed data? And what is additive noise and Gaussian noise ?

When you have sensors, the values you receive change even if the signal that was recorded didn't change. This is one example of noise.

When you have a model of the world, it abstracts from the real relationships by simplifying things which are not too important. To take into account for the simplification, you model the error as noise (e.g. in a Kalman filter).

But noise sources can be anything. For example, in a image classification problem, where you have to assign labels (like cat, dog, car, plane) to images, you could model errors of the human labelers as noise. In general, I'd say noise are (undesirable) effects which you can't describe deterministic in a simple enough way, so you describe it with random variables.

$$z = H \cdot x$$

where $z \in \mathbb{R}^{n_m}$ is the observation, $x \in \mathbb{R}^{n_x}$ is the state you're interested in and $H \in \mathbb{R}^{n_m \cdot n_x}$ is a transformation matrix. Then the noise could interact with your system in any way. But most of the time it is logical and practical that the noise is additive, meaning your model is

$$z = H \cdot x + r$$

where $r$ is sampled from a random variable of any distribution.

$$r \sim \mathcal{N}(\mu, \sigma^2)$$
• I think it is $r\sim N(0,\sigma^2)$. – L.V.Rao Jan 1 '17 at 6:56
• @L.V.Rao Usually, yes (+1). The point is, if $r$ has a mean $\mu \neq 0$, then you can add this term in your system equation and thus make $r$ have zero mean. So it all depends on your model. – Martin Thoma Jan 1 '17 at 10:53