In experimental science there is the 'scientific method', in mathematics there is the 'proof.' What are the standards for knowledge production in data science?
I've gathered that there a few different philosophies one can take; some of them are similar to the scientific method (introduce a falsifiable model and see if the data fits), others seem more hodge-podge (introduce some kind of learning machine and hope that the regularization prevents over fitting, and that cross validation catches it if not).
(There are some loose connections between those two, like how falsifiability fits into VC entropy and learning rates; this is explained in Vapnik's book on the Foundations of Statistical Learning.)
So, supposing that I have some data and I want to extract knowledge from it. What are the community standards for doing so? Where can I learn about the epistemological philosophy behind these standards, and their mathematical expression?