Questions tagged [uncertainty]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

Convert Regression output to classification based on prospective performance of model

Let’s say I have a regression model that predicts experimental value in the range of 4 to 8 of an object based on a set of features. I am aiming to design an oBject with an experimental value of great ...
ChemDev's user avatar
  • 101
1 vote
0 answers
79 views

Model uncertainty quantification

I'm reading a paper about model uncertainty quantification. Specifically, it says epistemic uncertainty is a kind of uncertainty due to lack of knowledge about a particular region in the input space. ...
piero's user avatar
  • 61
2 votes
2 answers
135 views

Using conformal predictors to estimate uncertainty?

I read this interesting book on conformal predictors: https://arxiv.org/abs/2107.07511. Conformal predictors are a way to choose a set that's guaranteed to include the true labels with some pre-chosen ...
user151434's user avatar
1 vote
0 answers
149 views

What is the best way to combine cross-validation and bootstrapping for one application?

We intend to model data with non-parametric covariate splines and we would like to understand the uncertainty of the parameter estimates/response estimates. Currently, we use cross-validation to model ...
Stan Tendijck's user avatar
0 votes
0 answers
14 views

Adressing uncertainty of a spatio-temporal multivariate timeseries with random temporal gaps

Imagine there are multiple locations of interest from where water samples are gathered manually. Each sample is immediately analyzed, converted to a numerical value (a real number) and fed into a ...
Laurimann's user avatar
1 vote
2 answers
870 views

What is the difference between conformal prediction and uncertainty estimation

Recently I am seeing the topic of Conformal Prediction to be very trendy on social media and research. Awesome Conformal Prediction But what is the main difference between conformal prediction and ...
Carlos Mougan's user avatar
1 vote
0 answers
42 views

Is there a way to quantify uncertainty in classification?

I'm thinking of a way to build an extension to a binary classifier (actually I will get the output probabilities like in logistic regression, so technically you should call this regression) that ...
Tom's user avatar
  • 65
0 votes
1 answer
68 views

Is it ok to use MC-dropout technique to estimate uncertainty without putting dropout after every weight layer?

In the paper by Kendall and Gal (What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?), dropout is being set after every convolutional layer. However, is it still legit to ...
tMan27's user avatar
  • 1
1 vote
1 answer
40 views

Can a simple distance to a few nearest data points be used a measure of the uncertainty of a prediction?

One of the 'selling points' of the Gaussian process regression is that it provides not only the model but also the uncertainty estimate of a prediction. Then usually a picture is shown with a curve ...
Vladislav Gladkikh's user avatar
5 votes
2 answers
64 views

How is uncertainty evaluated for results obtained via machine learning techniques?

As machine learning (in its various forms) grows ever more ubiquitous in the sciences, it becomes important to establish logical and systematic ways to interpret machine learning results. While modern ...
PerplexedDimension's user avatar
0 votes
0 answers
25 views

uncertainties in non-convex optimization problems (neural networks)

How do you treat statistical uncertainties coming from non-convex optimization problems? More specifically, suppose you have a neural network. It is well known that the loss is not convex; the ...
Dave's user avatar
  • 13
0 votes
1 answer
73 views

How can I store sources, effective dates, and confidence for every property in a knowledge graph?

What I am wanting to do is ensure that every property in a knowledge base comes from at least one source. I would like to ensure that every edge is spawned (or at least explained) by some event, like ...
AJAr's user avatar
  • 103
1 vote
0 answers
73 views

How to forecast time series with negative trend in test set and big uncertainty? (uncertainty due to Covid and Ukraine crisis)

Recently I started to create a machine learning model for a European customer for around 800 product time series. The goal is to produce a monthly forecast for the 6 months ahead. Since this customer ...
LGe's user avatar
  • 145
0 votes
1 answer
42 views

Conventional way of representing uncertainty

I am calculating metrics such as F1 score, Recall, Precision and Accuracy in multilabel classification setting. With random initiliazed weights the softmax output (i.e. prediction) might look like ...
Kevin's user avatar
  • 163
0 votes
1 answer
108 views

What are some state of art computer vision models for anomaly detection that can learn continuously and build classes for detected anomalies?

I'm looking forward to build a model that: Detect anomalies Improve over user feedback Build classes for the anomalies based on user feedback Since a schema is worth a thousand words: Do you know ...
Xiiryo's user avatar
  • 111
0 votes
1 answer
21 views

How can I distribute samples optimally to fit a model?

I'm trying to fit a model to a low number (~5-10) of data points. I might be able to suggest the optimal distribution of the data points beforehand, knowing a bit about the data and the model I ...
Matthias's user avatar
  • 101
1 vote
1 answer
607 views

Confidence intervals for evaluation on test set

I'm wondering what the "best practise" approach is for finding confidence intervals when evaluation the performance of a classifier on the test set. As far as I can see, there are two ...
Tom's user avatar
  • 43
3 votes
2 answers
178 views

A deployed model has epistemic or aleatoric uncertainty?

Aleatoric uncertainty refers to the notion of randomness that there is in the outcome of an experiment that is due to inherently random effects. Epistemic uncertainty refers to the ignorance of the ...
Carlos Mougan's user avatar