I am reading pattern Recognition and machine learning by Bishop and in the chapter about probability, "noise in the observed data" is mentioned many times. I have read on the internet that noise refers to the inaccuracy while reading data but I am not sure whether it is correct. SO, what actually is noise in observed data? And what is additive noise and Gaussian noise ?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
6
When you have sensors, the values you receive change even if the signal that was recorded didn't change. This is one example of noise.
When you have a model of the world, it abstracts from the real relationships by simplifying things which are not too important. To take into account for the simplification, you model the error as noise (e.g. in a Kalman filter).
But noise sources can be anything. For example, in an image classification problem, data compression can distort an image. Images can have different resolution; low resolution signals are harder to classify than high resolution figures. Aliasing effects can also distort images.
And what is additive noise?
Suppose your system equation is
$$z = H \cdot x$$
where $z \in \mathbb{R}^{n_m}$ is the observation, $x \in \mathbb{R}^{n_x}$ is the state you're interested in and $H \in \mathbb{R}^{n_m \cdot n_x}$ is a transformation matrix. Then the noise could interact with your system in any way. But most of the time it is logical and practical that the noise is additive, meaning your model is
$$z = H \cdot x + r$$
where $r$ is sampled from a random variable of any distribution.
What is (additive) Gaussian noise?
$$r \sim \mathcal{N}(\mu, \sigma^2)$$
answered Dec 26, 2016 at 10:25
-
1
-
$\begingroup$ @L.V.Rao Usually, yes (+1). The point is, if $r$ has a mean $\mu \neq 0$, then you can add this term in your system equation and thus make $r$ have zero mean. So it all depends on your model. $\endgroup$ Jan 1, 2017 at 10:53
-
$\begingroup$ "I'd say noise are (undesirable) effects which you can't describe deterministic in a simple enough way". I think this is not correct. We have stochastic and deterministic noise. The definition matches to stochastic noise but not to deterministic noise $\endgroup$ Jun 15, 2018 at 6:23
-
$\begingroup$ @CodePope Coud you give an example for stochastic noise? Where does your definition come from? $\endgroup$ Jun 15, 2018 at 6:30
-
1$\begingroup$ If the noise is random like in your example, we are talking about stochastic noise. If the noise is due to the complexity of the data so some part cannot be modeled then we have deterministic noise. Wikipedia: en.wikipedia.org/wiki/Deterministic_noise $\endgroup$ Jun 15, 2018 at 7:04