I'd like to know if this is a sensible idea and if there exist any already formed methods to do this (I'm new to the data science area).

Essentially, I have used Naive Bayes to accurately classify three types of food, based on their nutritious value (fat, salt, sugar, protein, and carbohydrates as my features).

Now that I can accurately classify these foods, Is there a method which uses the Naive Bayes to reverse this approach, and find the extreme values these features can be to be still classified as a type of food?

E.g: The max fat food1 can be, to still be considered food1.

I realize that these values will change, as other nutrient variables are changed, but I wondered if an optimized set of equations could be obtained in 5 dimensions?


What you are looking for is called "decision boundary", which is the set of (extreme) points laying on the boundary between classes.

Decision boundary of NB is a set of points $\boldsymbol{x}$ that satisfy at least one of these $K(K-1)/2$ conditions $$i\neq j \in [1, K]:{\Bbb P}(\boldsymbol{x}, C_i)={\Bbb P}(\boldsymbol{x}, C_j) \overset{\forall k}{\geq} {\Bbb P}(\boldsymbol{x}, C_k)$$ Generally, Naive Bayes does not learn an explicit decision boundary (specially when categorical features are involved), however, for example here is the derivation of NB boundary for continuous variables when ${\Bbb P}(\boldsymbol{x}|C_i)$ is chosen from exponential family (e.g. a Gaussian). Nonetheless, you could still select a set of random points, then assign a class to each, and finally plot them using something like t-SNE to get a sense of decision boundaries. For example, a tool like this interactive t-SNE plot would be helpful:

t-SNE explorer

Or a library like mlxtend.plotting that carries out a similar procedure:

On the contrary, models like multi-class logistic regression and SVM both have explicit decision boundaries, which are equations of learned parameters $\boldsymbol{w}$ such as $$w_0+w_1\text{fat}+w_2\text{salt}+w_3\text{sugar}=0$$ that you can access via common libraries.

  • 1
    $\begingroup$ This is great, thank you very much $\endgroup$ – Dan Savage May 3 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.