1
$\begingroup$

In experimental science there is the 'scientific method', in mathematics there is the 'proof.' What are the standards for knowledge production in data science?

I've gathered that there a few different philosophies one can take; some of them are similar to the scientific method (introduce a falsifiable model and see if the data fits), others seem more hodge-podge (introduce some kind of learning machine and hope that the regularization prevents over fitting, and that cross validation catches it if not).

(There are some loose connections between those two, like how falsifiability fits into VC entropy and learning rates; this is explained in Vapnik's book on the Foundations of Statistical Learning.)

So, supposing that I have some data and I want to extract knowledge from it. What are the community standards for doing so? Where can I learn about the epistemological philosophy behind these standards, and their mathematical expression?

$\endgroup$
1
$\begingroup$

Interesting question.

The way I see it, the justification comes from both mathematics and experimental science.

If you suggest a new algorithm, you should state your assumptions and provide a proof, just like in math. However, you should also demonstrate it on real data, showing that your assumptions are reasonable and that your algorithm can deliver value.

If you suggest a model, you should justify it based on its performance on the observations, just like in experimental sciences. This is true no matter how did you derive the model. However, examining the model logic itself and the way it was derived will help to gain confidence in it and trace where it needs modification.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.