18
$\begingroup$

I have found other requests for references here. In particular in: Where to start, which books and Books about the "Science" in Data Science?

I have given a glance to:

  • Artificial Intelligence: A Modern Approach (Russel & Norvig)
  • Machine Learning: The Art and Science of Algorithms that Make Sense of Data (Flach)
  • Learning From Data (Abu-Mostafa et al.)
  • Introduction to Statistical Learning (James et al.)
  • Elements of Statistical Learning (Hastie et al.)
  • Pattern Recognition and Machine Learning (Bishop)

Now it is difficult to evaluate if they would fit my needs because only a few pages are generally available online. However my first impression is that they do not. In the appendices of Artificial Intelligence: A Modern Approach I can read:

Mathematicians define a vector as a member of a vector space, but we will use a more concrete definition: a vector is an ordered sequence of values.

This is exactly the kind of approach I am not looking for.

I'm looking for a book which assumes the reader has a good understanding in set theory, abstract algebra, measure and probability theory, statistics, topology, graph theory, complexity theory, etc and a preference for formal and axiomatic explanations rather than lenghty and so-called "intuitive" approaches based on basic mathematical objects and examples. Furthermore I don't want something that looks like a recipe book from the very beginning. I want a book that formalizes the abstract and common shape of all data science methods as well as their common aim first. Only after that it can start to explain the different categories by explicitely stating which further hypotheses each category is assuming and which cases/problems/domains they are known to handle efficiently or not.

At last, to be clear, I have no problem with being shown concrete examples and their treatment via a specific programming language for example. I just want this to come second as an illustration for the conceptual explanation, not as a substitute.

Improve this question
$\endgroup$
3
  • $\begingroup$ The free Library Genesis is a great help in book search. libgen.is $\endgroup$
    – László Szilágyi
    Jan 9, 2020 at 10:57
  • 1
    $\begingroup$ You may find this blog post (and other posts there) amusing: thenewflesh.net/2019/01/09/dl4m-supervised-learning.html $\endgroup$
    – Ben Reiniger ♦
    Feb 14, 2020 at 21:15
  • 1
    $\begingroup$ Hey Burakumin, did you find what you were looking for? I am having the exact same problem. Having a very hard time finding a good book that would provide a somewhat complete overview of the machine learning field written "with a mathematician's mindset". I would immensely appreciate your input since your post seems to match my needs perfectly. The answers below appear unsatisfactory. $\endgroup$
    – user332582
    Feb 28, 2021 at 19:47

4 Answers 4

Reset to default
9
$\begingroup$

First of all, data only comes in so many forms that it might make sense to stick to a more "concrete definition". Data Science is necessarily practical. But here are a few other books with a more theoretical grounding. Others will certainly know many more...

However, research in machine learning is mostly found in journals and papers. It will be hard to find one or a few books that cover everything you want to know.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Your first sentence is a bit surprising to me. This is precisely because data "comes in so many forms" that it would be helpful to explain what it means to "process/learn/predict" from them in the broadest sense and why there is now a common field dedicated to these tasks. Diversity of cases is precisely the raison d'etre of abstraction: extracting the common essence of all specific examples rather than focusing on accidental details. This way you can provide a more encompassing understanding and make comparisons meaningful. $\endgroup$
    – Contactomorph
    Oct 6, 2016 at 16:27
  • 1
    $\begingroup$ I agree. My point was rather that data science is not performed with pen and paper but using a computer. And the ways data is usually represented are not that manifold. You can probably get a long way with "ordered sequences of values". But, again, I agree with you that a unifying mathematical framework is very helpful for a broader understanding of data science concepts. I'm interested in what references others recommend. $\endgroup$
    – oW_ ♦
    Oct 6, 2016 at 17:20
3
$\begingroup$

Hastie et al is at the mathematical level you require - being written by statistics academics with strong mathematical pedigree (Hastie is currently a mathematics professor, for example) - and the complete text is available for free online via the authors' website. It is probably about the best general survey of machine learning for people with mathematical and statistical background at the graduate student level. That said, it is still a survey, and individual topics will require follow up elsewhere, though useful recommended reading is provided.

Bishop also assumes a reasonable degree of mathematical maturity, although the table of contents may make the content appear more simple than it is, for example by listing a review of probability distributions including the Gaussian as Chapter 2.

Russel & Norvig isn't about machine learning or data science, but rather the wider field of artificial intelligence, in which it includes machine learning as smallish subset, and data science effectively not at all. For example, it discusses a number of different kinds of systems of pre-programmed AI approaches - the exact opposite of machine learning. It is interesting if you want to understand the wider world of automation but will do little to help you understand ML.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Hastie et al: > We have not attempted to write a comprehensive catalog of learning methods, but rather to describe some of the most important techniques. Equally notable, we describe the underlying concepts and considerations by which a researcher can judge a learning method. We have tried to write this book in an intuitive fashion, emphasizing concepts rather than mathematical details. > While some mathematical details are needed, we emphasize the methods and their conceptual underpinnings rather than their theoretical properties. $\endgroup$
    – Contactomorph
    Oct 18, 2016 at 13:21
  • $\begingroup$ "Elements of Statistical Learning" explicitely chooses an intuitive approach rather than a formal one. Furthermore it starts with a long so-called "overview" which mainly consists in opposing least squares and nearest neighbors. Again this is not what I am looking for. I want something formal not example-oriented. I will have a look at Bishop but until now it seems to me that @Ow_'s "Devroye, Györfi, Lugosi" would suits a mathematician expectiations far better (even if it intentionally focuses on a more narrow set of cases). $\endgroup$
    – Contactomorph
    Oct 18, 2016 at 13:37
  • $\begingroup$ I take your point on the additional mathematical formality of Devroye et al, and yes Hastie et al is not at this level.At the same time I think that using a book published in 1996 as a guide to a fast moving field also comes with its own pitfalls. It certainly gives a pause to read in 2016 that there is little value in a neural network with greater than 2 hidden layers... $\endgroup$
    – Robert de Graaf
    Oct 19, 2016 at 0:53
  • $\begingroup$ Sure a more recent reference would be more valuable. If someone knew a 2016 equivalent of Devroye that would be perfect. I'll try to give Hasting a new try. But it has been a bit of a pain last time. A lot of things must be guessed and generalized on the reader's side rather than being formally stated in the book. $\endgroup$
    – Contactomorph
    Oct 19, 2016 at 9:17
2
$\begingroup$

I refer you to a previous question I asked on CrossValidated for a similar request and still stand by my answer to this question.

Per @Coffee's recommendation, I would recommend the text Machine Learning: A Bayesian and Optimization Perspective by Sergios Theodoridis along with Pattern Recognition by the same author.

These two texts combined are 2,000 pages total and cover everything from undergrad-level probability to linear models, and (as far as I can tell) everything covered by Elements of Statistical Learning, in addition to time series, probabilistic graphical models, deep learning, and Monte Carlo methods.

The author makes an excellent effort to make all notation clear and consistent (thank you for bolding all of your vectors!) and seems to have used carefully chosen exercises.

Having a background in probability as well as stats at the level of Casella and Berger would be extremely helpful to have before pursing these texts. There is some discussion of UMVUEs in here.

Improve this answer
$\endgroup$
-1
$\begingroup$

I would recommend

Linear Algebra and Learning from Data by Gilbert Strang

as a nice introduction for someone with an undergrad math background.

It's not particularly wide in scope and contains some probably unnecessary summaries of linear algebra (insightful nonetheless) and probability (very basic), but the portions on data are nice introductions.

You can see some sample chapters at https://math.mit.edu/~gs/learningfromdata/

Improve this answer
$\endgroup$
1
  • $\begingroup$ This is exactly not what my question is asking for $\endgroup$
    – Contactomorph
    Oct 8 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.