I was reading Modern Optimization with R (Use R!) and wondering if a book like this exists in Python too? To be precise something that covers stochastic gradient descent and other advanced optimization techniques. Many thanks!
You should be able to translate code written in one language -- even pseudo-code -- to another, so I see no reason to avoid books for R. If you want one specifically for python, there's Machine Learning in Action by Peter Harrington.
One of scikit-learn's core committers is a releasing a book in October: Introduction to Machine Learning with Python: A Guide for Data Scientists.
There is a tutorial Modern Optimization in Python and a corresponding video Modern Optimization in Python | SciPy 2017 Tutoria. Although they are not books, I think they may be helpful to you.