I highly recommend you take a look at this anomaly detection survey paper. It not only provides with a great review but also categorises the different approaches focusing in different types of data or domains where these techiques have been applied.
If you go through Section 1.3, you will find a series of interesting references which go as far as 1985. So you could have a look at those to make sense of the historic evolution of these approaches from numerical, statistical or ML standpoints.
Also, since the paper is quite dated already, I suggest a reading about new methods such as deep learning methods to complement your knowledge. A nice starting point could be this deep learning survey paper, which I think is another great more recent survey paper.