Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
What are the books about the science and mathematics behind data science? It feels like so many "data science" books are programming tutorials and don't touch things like data generating processes and statistical inference. I can already code, what I am weak on is the math/stats/theory behind what I am doing.
If I am ready to burn $1000 on books (so around 10 books... sigh), what could I buy?
Examples: Agresti's Categorical Data Analysis, Linear Mixed Models for Longitudinal Data, etc... etc...
Introductory:
Digging deeper:
Some special interest examples:
Broader reference works on machine learning (not really what you asked for, but for completeness):
Bonus paper:
If I could only recomend one to you, it would be: The Elements of Statistical Learning and Prediction by Hastie, Tibshirani and Friedman. It provides the math/statistics behind a lot of commonly used techniques in data science.
For Bayesian Techniques, Bayesian Data Analysis by Gelman, Carlin, Stern, Dunson, Vehtari and Rubin is excellent.
Statistical Inference by Casella and Berger is a good graduate-level textbook on the theoretical foundation of statistics. This book does require a pretty high level of comfort with math (probability theory is based on measure theory, which is not trivial to understand).
With respect to data generating processes, I don't have a recommendation for a book. What I can say is that a good understanding of the assumptions of the techniques used and ensuring that the data was collected or generated in a manner that does not violate those assumptions goes a long way towards a good analysis.
Other answers recommended a good set of books about the mathematics behind data science. But as you mentioned, its not just mathematics and activities like data collection and inference from data has their own rules and theories, even if not being as rigorous as mathematical backgrounds (yet).
For theses parts, I suggest the book Beautiful Data: The Stories Behind Elegant Data Solutions which contains twenty case-study like chapters written by people really engaged with real world data analysis problems. It does not contain any mathematics, but explores areas like collecting data, finding practical ways of using data in analyses, scaling and selecting the best solutions very well.
Another really interesting book is Thinking with Data: How to Turn Information into Insights, which is not technical (=programming tutorial) either, but covers important topics on how to really use the data science power in decision making and real world problems.
I like Amir Ali Akbari's suggestions, and I'll add a few of my own, focusing on topics and skills that are not adequately covered in most machine learning and data analysis books that focus on math and/or programming.
Data Cleaning:
Bayesian Data Analysis (alternative to Fisher-style Null Hypothesis Significance Testing):
Inference in the face of uncertainty, incompleteness, contradictions, ambiguity, imprecision, ignorance, etc.:
Experiments:
Simulation:
Expert elicitation, probabilistic estimation:
