0
$\begingroup$

I am currently working on a research project in the field of machine learning where I am attempting to reproduce the results of a specific paper. To ensure accuracy and transparency, I have utilized the code provided by the original authors for this purpose.

However, I am encountering a dilemma regarding which experiment results to include in my paper: should I present the reproduced results obtained from the code provided by the original authors, or should I rely solely on the results as reported in the original paper?

$\endgroup$
0

1 Answer 1

2
$\begingroup$

Either can be justified:

  • if your results are better than the original paper (e.g. due to more extensive hyperparameter search), then it is common to include those, which will present a fairer like-for-like comparison to your new approach and sets a new benchmark for the previous method.
  • if your results are comparable but not identical, just use those from the original paper. This is still presenting fair comparison of results and readers will recognise those results from the previous paper, so things make sense.
  • if your results are significantly worse than the original, you want to try to understand why. If the work is honest and the code legitimate then you may want to tweak hyperparameters etc (check to original paper/repo) to reach one of the cases above. If that doesn't work you can try to contact the paper authors for advice as most authors want their work to be extended and cited. (Worst case, those results are not reproducible, which can happen.)

One thing to avoid is presenting weaker baselines than the original paper that make your results look better, but perhaps because your new method has been more finely tuned than those baselines.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.