Questions tagged [confidence]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
24 views

Correctness of derivation for binary F1 variance for F1 confidence intervals

I'm developing a python library for confidence intervals for common accuracy metrics, with both analytic and bootstrap computations. Following this paper, I implemented the Macro and Micro F1 scores ...
Jacob G's user avatar
  • 121
0 votes
1 answer
134 views

Precision vs probability

Say I have a model which predicts a class $C_i$ from an input $X$, with a probability of 0.95 i.e $P(C_i| X)=0.95$. That would mean that if we do this over and over, then 95/100 times we would be ...
CutePoison's user avatar
0 votes
0 answers
37 views

Is there a way to quantify uncertainty in classification?

I'm thinking of a way to build an extension to a binary classifier (actually I will get the output probabilities like in logistic regression, so technically you should call this regression) that ...
Tom's user avatar
  • 53
0 votes
0 answers
11 views

How to find confidence sets for a bunch of regression functions?

Unlike confidence intervals where we are interested in indicating a range of beta values in which the true parameter lies 95% of all times, I would like to understand how confidence sets are ...
TFT's user avatar
  • 35
0 votes
1 answer
19 views

Statistical significance in multiple groups

What is the most appropriate test to validate significance in the case I'm comparing multiple different groups? e.g. I have different groups of ages (teenager, young, adult, elderly) and a conversion ...
simon's user avatar
  • 133
1 vote
1 answer
73 views

Overall acurracy +/- E (with 90% C.I.)

I am assessing the accuracy of my classification model. I performed a 4-folds cross-validation and I obtained the following Overall Accuracy: OA = (0.910, 0.920, 0.880, 0.910). So, the average OA is 0....
sermomon's user avatar
1 vote
1 answer
65 views

How to test the confidence for a rule based system?

I have a multi-class dataset and I generated based on it rules. That is, if certain features are seen then it must be a certain class. I chose only rules with precision 1 (with respect to the whole ...
greenButMellow's user avatar
1 vote
1 answer
322 views

How to use confidence labels?

I have 2 sets of training data in csv files. The training data have class labels, 1 for memorable, and 0 for not memorable. In addition, there is also a confidence label for each sample. The class ...
user avatar
1 vote
0 answers
41 views

Evaluate Dendrogram Statistical Significance

I have N=21 objects and each one has about 80 possible not NaN descriptors. I carried out a hierarchical clustering on the objects and I obtained this dendrogram. I want some kind of 'confidence' ...
Mirko's user avatar
  • 11
1 vote
2 answers
59 views

How are the confidence intervals of a model interpreted?

I am doing some work with R and after obtaining the confusion matrix I have obtained the following metrics corresponding to a logistic regression: ...
PicaR's user avatar
  • 304
1 vote
0 answers
163 views

Association rules - Find 100% confidence rules [closed]

Suppose there are 100 items, numbered 1 to 100, and also 100 baskets, also numbered 1 to 100. Item i is in basket b if and only ...
AskSmart's user avatar
1 vote
1 answer
500 views

Confidence intervals for evaluation on test set

I'm wondering what the "best practise" approach is for finding confidence intervals when evaluation the performance of a classifier on the test set. As far as I can see, there are two ...
Tom's user avatar
  • 43
0 votes
1 answer
37 views

Excluding data via confidence score: Is it a good idea?

Let's say I have a model which has a binary classification task (Two classes of 0 and 1) and therefore, it outputs a number between 0 and 1, if it is greater than 0.5 we consider it to be class 1 and ...
Amirhossein Rezaei's user avatar
1 vote
1 answer
23 views

How to State The Confidence of Accuracy/Inaccuracy?

Consider that I have a dataset automatically acquired by a machine that returns the following measurements: [111, 121, 114, 154, 149, 150] I then go and manually ...
mmarion's user avatar
  • 111
0 votes
1 answer
102 views

How to predict only those values that our model is 95% sure of?

I have 5 classes. I made a XGBoost Classification model and used model.predict(test) to predict the classes of test dataset. Out of all those values predicted by my ...
Adarsh Wase's user avatar
1 vote
1 answer
556 views

Plotting confidence intervals

For the following dataframe, I am trying to plot the means of a sample of 5 random rows . And also plot their respective confidence intervals using errorbars. I am unable to figure how to plot the ...
freshman_2021's user avatar
0 votes
1 answer
88 views

Producing a confidence output to use in a weighted average layer

I am trying to solve a problem where the input consists of an unordered set of observations (variable size), and the desired output is a single value describing some property of the observations as a ...
No idea what I'm doing's user avatar
1 vote
1 answer
52 views

Confidence score for all observations is between 0.50 - 0.55

Hello Data Science Stack Exchange Community, This question will appear to be open-ended, however any answers or thought will be much appreciated. I am trying to go-through a pre-trained Random Model ...
Aman Raparia's user avatar
0 votes
0 answers
483 views

Narrow confidence Interval of forecast

I am new to data science so please accept my apology in advance. I am trying to predict the value using ARIMA. I have got weekly value for the current year. Based on the available weekly values, I ...
user2293224's user avatar
1 vote
0 answers
52 views

Confidence in the rewards for a RL task

For a RL task that I am trying to solve, for which I train once per day, I have the rewards stored for each of those days, so that I can see the progress on daily basis. In the beginning of the ...
ilirosmanaj's user avatar
1 vote
1 answer
61 views

Interpreting confidence interval results for datasets

I have created a dataset automatically and wanted to clarify my interpretation of the amount of noise using the confidence interval. I selected a random sample and manually annotated the sample and ...
dmnte's user avatar
  • 23
2 votes
2 answers
144 views

Confidence rating for regression tasks

In classification tasks, we can interpret the output vector as how "confident" the model is that the input has a certain label. For example, ...
Mossmyr's user avatar
  • 123
2 votes
1 answer
127 views

Relation between Cross Validation and Confidence Intervals

I've read from a source which I forgot where that 'In cross validation, the model with best scores at 95% confidence interval is picked'. But according to my stat knowledge, in order for CI (...
Wong's user avatar
  • 103
1 vote
1 answer
34 views

Understand how to simulate a statistics [closed]

This solution describes how to simulate statistics to find a confidence interval. A journalist called 1000 people in town to ask who will they be voting for out of candidates A and B. The observed ...
maindola's user avatar
2 votes
2 answers
4k views

Predict_proba for Binary classifier in Tensorflow

I'm working in binary classifier problem, where I have used Tensorflow low level API's. Last layer wrapped with Sigmoidal Function and just returning a single value. For my prediction I just set a ...
vipin bansal's user avatar
  • 1,252