What are the books about the science and mathematics behind data science? It feels like so many "data science" books are programming tutorials and don't touch things like data generating processes and statistical inference. I can already code, what I am weak on is the math/stats/theory behind what I am doing.

If I am ready to burn $1000 on books (so around 10 books... sigh), what could I buy?

Examples: Agresti's Categorical Data Analysis, Linear Mixed Models for Longitudinal Data, etc... etc...

  • $\begingroup$ Asking about "good" books will attract opinion-based answers and so this is off-topic. Flagged. $\endgroup$
    – Spacedman
    Jun 11, 2014 at 14:32
  • 3
    $\begingroup$ I've changed it so I am just looking for books. Nothing opinion-based. $\endgroup$
    – Anton
    Jun 11, 2014 at 14:34
  • $\begingroup$ It's spelled S-t-a-t-i-s-t-i-c-s :) Stick with something pragmatic that focuses on prediction rather than inference. Both Elements of Statistical Learning and An Introduction to Statistical Learning are on most people's lists. $\endgroup$ Jun 11, 2014 at 15:17
  • $\begingroup$ I can't add a comment yet, but just FYI ESL is available for free online as a pdf $\endgroup$
    – idclark
    Jun 20, 2014 at 20:44
  • 1
    $\begingroup$ I think this question should be marked as community wiki. $\endgroup$ May 25, 2015 at 15:04

4 Answers 4


If I could only recomend one to you, it would be: The Elements of Statistical Learning and Prediction by Hastie, Tibshirani and Friedman. It provides the math/statistics behind a lot of commonly used techniques in data science.

For Bayesian Techniques, Bayesian Data Analysis by Gelman, Carlin, Stern, Dunson, Vehtari and Rubin is excellent.

Statistical Inference by Casella and Berger is a good graduate-level textbook on the theoretical foundation of statistics. This book does require a pretty high level of comfort with math (probability theory is based on measure theory, which is not trivial to understand).

With respect to data generating processes, I don't have a recommendation for a book. What I can say is that a good understanding of the assumptions of the techniques used and ensuring that the data was collected or generated in a manner that does not violate those assumptions goes a long way towards a good analysis.


Other answers recommended a good set of books about the mathematics behind data science. But as you mentioned, its not just mathematics and activities like data collection and inference from data has their own rules and theories, even if not being as rigorous as mathematical backgrounds (yet).

For theses parts, I suggest the book Beautiful Data: The Stories Behind Elegant Data Solutions which contains twenty case-study like chapters written by people really engaged with real world data analysis problems. It does not contain any mathematics, but explores areas like collecting data, finding practical ways of using data in analyses, scaling and selecting the best solutions very well.

Another really interesting book is Thinking with Data: How to Turn Information into Insights, which is not technical (=programming tutorial) either, but covers important topics on how to really use the data science power in decision making and real world problems.


I like Amir Ali Akbari's suggestions, and I'll add a few of my own, focusing on topics and skills that are not adequately covered in most machine learning and data analysis books that focus on math and/or programming.

Data Cleaning:

Bayesian Data Analysis (alternative to Fisher-style Null Hypothesis Significance Testing):

Inference in the face of uncertainty, incompleteness, contradictions, ambiguity, imprecision, ignorance, etc.:



Expert elicitation, probabilistic estimation:


Not the answer you're looking for? Browse other questions tagged or ask your own question.