1
$\begingroup$

Recently I am seeing the topic of Conformal Prediction to be very trendy on social media and research. Awesome Conformal Prediction

But what is the main difference between conformal prediction and uncertainty estimation?

$\endgroup$
1
  • $\begingroup$ It seems to me that conformal prediction is a simple distribution-free form of uncertainty estimation. $\endgroup$ Jan 17, 2023 at 15:47

2 Answers 2

1
$\begingroup$

The main difference is that Conformal Prediction is a much better framework because it is:

  1. distribution free
  2. works with any model without ...
  3. provided mathematical guarantees of validity for final samples of any size.

In comparison Bayesian frameworks produce posterior in the wrong place (unless using synthetic data where the data distribution is exactly known), interfere with the underlying models and do not have any mathematical guarantees of validity.

$\endgroup$
0
$\begingroup$

Conformal Prediction is a distribution-free statistically rigorous uncertainty uncertainty quantification framework in the form of prediction sets for classification and prediction intervals for regression (Vovk, V.; Gammerman, A.; and Shafer, G. 2005. Algorithmic Learning in a Random World. Berlin, Heidelberg:Springer-Verlag. ISBN 0387001522., and Shafer 2005).

Bayesian method provides a natural and principled way for modeling uncertainty. In Deep learning, epistemic uncertainty concerns the probabilistic estimation of model parameters. There are number of approaches have been developed to measure uncertainty in neural networks, such as approx. Bayesian deep learning techniques such as Monte-Carlo dropout (Gal, Y.; and Ghahramani, Z. 2016), DropWeights (Ghoshal, B.; and Tucker, A. 2019), and an ensemble of deep models (Lakshminarayanan, Pritzel, and Blundell 2017).

$\endgroup$
3
  • 2
    $\begingroup$ I dont see how this answers the question. First part is the definition using both terms. second is unrelated $\endgroup$ Feb 4, 2023 at 8:27
  • $\begingroup$ Probabilistic approach is considered as the most principled approach to uncertainty estimation. Conformal prediction is a distribution-free uncertainty estimation. In classification task conformal prediction generates predictive set instead of single prediction whereas approx. Bayesian deep learning, UQ estimates aleatoric and epistemic uncertainty in neural network. $\endgroup$ Feb 12, 2023 at 10:52
  • $\begingroup$ This comment is better than the answer. Do you want to update the answer? $\endgroup$ Feb 13, 2023 at 11:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.