1
$\begingroup$

I have a multi-class dataset and I generated based on it rules. That is, if certain features are seen then it must be a certain class. I chose only rules with precision 1 (with respect to the whole dataset)

It is worth mentioning the dataset is highly imbalance. The major class has 60K samples and other classes can have around 1K samples.

Now, consider a rule that applies to, let's say, 17 endpoints. My question is, which test should I apply to check if I can be confident about this rule? I guess the size of the class will have an effect on it.

$\endgroup$

1 Answer 1

1
$\begingroup$

The confidence interval is the main indicator to test confidence, and it is measurable thanks to the data volume indeed, but also whether or not the endpoints are reliable enough.

Therefore, testing data is necessary at each endpoint or set of endpoints: The percentage of success from a specific confidence interval is defined either manually thanks to the known business limitations or automatically through a Gaussian function.

$\endgroup$
2
  • $\begingroup$ Thanks. Can you help me describe the test? $\endgroup$ Jul 4, 2022 at 16:19
  • $\begingroup$ Could you create in your question a small but representative example? $\endgroup$ Jul 4, 2022 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.