Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [confidence]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

Calculating prediction confidence from a sequence of token-level confidences

I am working with OCSR (optical chemical structure recognition) models, and they output a sequence of token-level confidences. I am looking for a method of summarising these token-level confidences ...
finlay morrison's user avatar
1 vote
1 answer
35 views

Is there a model that can predict continuous data while also providing a level of confidence in the prediction?

The problem with Bayesian neural network seems to be that it is primarily working for classification problems. Is it possible to adjust this neural network, or even use a different model if one exists,...
tds's user avatar
  • 13
0 votes
0 answers
10 views

Computing empirical coverage in binary classification confidence estimation

I am trying to compute confidence interval empirical coverage. I have computed the confidence intervals for binary probability predictions. For example, the model estimates that there is probability ...
Tiago Melo's user avatar
0 votes
0 answers
26 views

How to compute confidence interval xgboost regressor?

I have time series data to predict values for the next 6 months. I have an xgboost model that predicts the six individual months, for the business what is important is that the cumulative value of ...
tailsrockc's user avatar
1 vote
0 answers
15 views

How to compare 55 models using AUC bootstrap confidence intervals?

I want to check if there is a difference in the confidence intervals of 55 models and select just one model. What should I do?
JAE's user avatar
  • 13
0 votes
0 answers
16 views

Understanding the code for calculating the 95% confidence interval of AUC using bootstrapping

It's really embarrassing, but I lack statistical knowledge. I would like to find the confidence interval for AUC at 95%. Actually, I got the code from here(https://stackoverflow.com/questions/52373318/...
JAE's user avatar
  • 13
0 votes
1 answer
157 views

Understand and compute confidence interval and coefficient of variation for regression model

I would like to better understand the concepts of: coefficient of variation and confidence interval. Trivially taking the definitions from wikipedia: confidence interval (CI) In frequentist ...
Cata's user avatar
  • 1
1 vote
0 answers
26 views

Correctness of derivation for binary F1 variance for F1 confidence intervals

I'm developing a python library for confidence intervals for common accuracy metrics, with both analytic and bootstrap computations. Following this paper, I implemented the Macro and Micro F1 scores ...
Jacob G's user avatar
  • 121
1 vote
1 answer
306 views

Precision vs probability

Say I have a model which predicts a class $C_i$ from an input $X$, with a probability of 0.95 i.e $P(C_i| X)=0.95$. That would mean that if we do this over and over, then 95/100 times we would be ...
CutePoison's user avatar
1 vote
0 answers
49 views

Is there a way to quantify uncertainty in classification?

I'm thinking of a way to build an extension to a binary classifier (actually I will get the output probabilities like in logistic regression, so technically you should call this regression) that ...
Tom's user avatar
  • 65
0 votes
0 answers
11 views

How to find confidence sets for a bunch of regression functions?

Unlike confidence intervals where we are interested in indicating a range of beta values in which the true parameter lies 95% of all times, I would like to understand how confidence sets are ...
TFT's user avatar
  • 135
0 votes
1 answer
24 views

Statistical significance in multiple groups

What is the most appropriate test to validate significance in the case I'm comparing multiple different groups? e.g. I have different groups of ages (teenager, young, adult, elderly) and a conversion ...
simon's user avatar
  • 133
1 vote
1 answer
74 views

Overall acurracy +/- E (with 90% C.I.)

I am assessing the accuracy of my classification model. I performed a 4-folds cross-validation and I obtained the following Overall Accuracy: OA = (0.910, 0.920, 0.880, 0.910). So, the average OA is 0....
sermomon's user avatar
1 vote
1 answer
92 views

How to test the confidence for a rule based system?

I have a multi-class dataset and I generated based on it rules. That is, if certain features are seen then it must be a certain class. I chose only rules with precision 1 (with respect to the whole ...
greenButMellow's user avatar
1 vote
1 answer
404 views

How to use confidence labels?

I have 2 sets of training data in csv files. The training data have class labels, 1 for memorable, and 0 for not memorable. In addition, there is also a confidence label for each sample. The class ...
user avatar
1 vote
0 answers
57 views

Evaluate Dendrogram Statistical Significance

I have N=21 objects and each one has about 80 possible not NaN descriptors. I carried out a hierarchical clustering on the objects and I obtained this dendrogram. I want some kind of 'confidence' ...
Mirko's user avatar
  • 111
1 vote
2 answers
72 views

How are the confidence intervals of a model interpreted?

I am doing some work with R and after obtaining the confusion matrix I have obtained the following metrics corresponding to a logistic regression: ...
PicaR's user avatar
  • 314
1 vote
0 answers
214 views

Association rules - Find 100% confidence rules [closed]

Suppose there are 100 items, numbered 1 to 100, and also 100 baskets, also numbered 1 to 100. Item i is in basket b if and only ...
AskSmart's user avatar
1 vote
1 answer
657 views

Confidence intervals for evaluation on test set

I'm wondering what the "best practise" approach is for finding confidence intervals when evaluation the performance of a classifier on the test set. As far as I can see, there are two ...
Tom's user avatar
  • 43
0 votes
1 answer
38 views

Excluding data via confidence score: Is it a good idea?

Let's say I have a model which has a binary classification task (Two classes of 0 and 1) and therefore, it outputs a number between 0 and 1, if it is greater than 0.5 we consider it to be class 1 and ...
Amirhossein Rezaei's user avatar
1 vote
1 answer
26 views

How to State The Confidence of Accuracy/Inaccuracy?

Consider that I have a dataset automatically acquired by a machine that returns the following measurements: [111, 121, 114, 154, 149, 150] I then go and manually ...
mmarion's user avatar
  • 111
0 votes
1 answer
167 views

How to predict only those values that our model is 95% sure of?

I have 5 classes. I made a XGBoost Classification model and used model.predict(test) to predict the classes of test dataset. Out of all those values predicted by my ...
Adarsh Wase's user avatar
1 vote
1 answer
734 views

Plotting confidence intervals

For the following dataframe, I am trying to plot the means of a sample of 5 random rows . And also plot their respective confidence intervals using errorbars. I am unable to figure how to plot the ...
freshman_2021's user avatar
0 votes
1 answer
104 views

Producing a confidence output to use in a weighted average layer

I am trying to solve a problem where the input consists of an unordered set of observations (variable size), and the desired output is a single value describing some property of the observations as a ...
No idea what I'm doing's user avatar
1 vote
1 answer
59 views

Confidence score for all observations is between 0.50 - 0.55

Hello Data Science Stack Exchange Community, This question will appear to be open-ended, however any answers or thought will be much appreciated. I am trying to go-through a pre-trained Random Model ...
Aman Raparia's user avatar
0 votes
0 answers
511 views

Narrow confidence Interval of forecast

I am new to data science so please accept my apology in advance. I am trying to predict the value using ARIMA. I have got weekly value for the current year. Based on the available weekly values, I ...
user2293224's user avatar
1 vote
0 answers
61 views

Confidence in the rewards for a RL task

For a RL task that I am trying to solve, for which I train once per day, I have the rewards stored for each of those days, so that I can see the progress on daily basis. In the beginning of the ...
ilirosmanaj's user avatar
1 vote
1 answer
67 views

Interpreting confidence interval results for datasets

I have created a dataset automatically and wanted to clarify my interpretation of the amount of noise using the confidence interval. I selected a random sample and manually annotated the sample and ...
dmnte's user avatar
  • 23
2 votes
2 answers
221 views

Confidence rating for regression tasks

In classification tasks, we can interpret the output vector as how "confident" the model is that the input has a certain label. For example, ...
Mossmyr's user avatar
  • 123
2 votes
1 answer
140 views

Relation between Cross Validation and Confidence Intervals

I've read from a source which I forgot where that 'In cross validation, the model with best scores at 95% confidence interval is picked'. But according to my stat knowledge, in order for CI (...
Wong's user avatar
  • 103
1 vote
1 answer
36 views

Understand how to simulate a statistics [closed]

This solution describes how to simulate statistics to find a confidence interval. A journalist called 1000 people in town to ask who will they be voting for out of candidates A and B. The observed ...
maindola's user avatar
2 votes
2 answers
4k views

Predict_proba for Binary classifier in Tensorflow

I'm working in binary classifier problem, where I have used Tensorflow low level API's. Last layer wrapped with Sigmoidal Function and just returning a single value. For my prediction I just set a ...
vipin bansal's user avatar
  • 1,272