I am looking for a website or book where several practical examples are given step by step, explaining how they choose the relevant features, the model selection procedure, etc...
If you want an application-oriented book, consider Christopher Bishop's Model-Based Machine Learning. He has more technical books that are well regarded.
If you are looking for lots of code, Probabilistic Programming & Bayesian Methods for Hackers is an option.
Another introductory book with a more statistical bent is An Introduction to Statistical Learning with Applications in R. Again, the authors have a well-regarded technical version of the book.
I had the same question a few weeks ago.
I personally found O'Reilly's Python for Data Analysis very useful in learning the basics. The book assumes you have some python programming experience, but it also has an appendix in the back to go through the basics.
The author gives you a wide variety of real world (not Monty Python) examples in the beginning you can create within the first few chapters, then goes into detail about each thing as the book goes on, building your knowledge.
I found the instructions very easy and step by step. My professor who is my guide in all this was impressed how quickly I learned.
I also have heard good things about Kaggle.
Data Science in the Cloud with Microsoft Azure Machine Learning and R is a free textbook which works through an example in great detail. Don't be put off by the particular tools used as you don't need them to get some benefit out of the book.
Another one which I enjoyed is Programming Collective Intelligence which also goes through a number of projects in detail, including the web scraping part which most books gloss over.
I can recommend this collection of Ipython Notebooks which includes Data Science, Statistics and Machine Learning commented notebooks.
One place you might find some interesting step-by-step explainations is the Kaggle tutorial and winner's interviews. Often people will post a detailed summary of their approach.
One of the best book I have came across is Machine Learning in Python from Sebastian Raschka. Easy examples, step by step explanation and just right amount of math.
The structure of book covers the whole process from data cleaning to ensembling and evaluation.
Have a look at :
This has a step by step tutorial which will give you an idea of the entire process of Data Exploration, Data Analysis and Building a predictive model.
Explanation regarding Data Exploration and Feature Engineering ( how to choose relevant features ) is here :
See the first 5 Datasets here which have tutorials and work on them to get practical experience:
Also have a look at :
where he uses multiple models on a single dataset which will give you a basic level understanding of different models.
For understanding more about the model selection have a look at this :
The above link has answers given by people working in the field.
To get insights on different datasets you can always log into kaggle and go through competitions and have a look at the wide range of datasets, where you have access to people's code in kernels. The forums in Kaggle are helpful as people discuss about using different models for a problems and their approach.