As you pointed out, a null covariance does not guarantee that variables are independent. You can have strongly dependent variables which show a covariance equal to 0. See a famous plot taken from Wikipedia page on Pearson correlation coefficient

A standard way to uncorrelate variables is to perform principal component analysis on the dataset, and use all components which are, by definition, uncorrelated. But please note that the Naive Bayes assumption is conditional independence, so independence for each class. In the general case, PCA projection would not be the same depending on the selected class.

Regarding the performance of Naive Bayes, I see no reason that a simple transformation should improve the model. This is supported by some reasonably well-received answers on similar questions, for instance this one on Stack Overflow, or this other one on Cross Validated.

However, I have read several reports of people getting increased accuracy when performing PCA prior to Naive Bayes. See here for instance, for a few exchanges on the subject, within a community of which trustworthiness is unknown to me; or search "PCA naive bayes" through your web search engine.

So in the end, I have no mathematical evidence to support this, but my feeling would be that the model performance variation before/after transformation could depend on the problem, especially the directions of principal components for each class.

Perhaps, if you perform some tests, you could share the result here.

As you pointed out, a null covariance does not guarantee that variables are independent. You can have strongly dependent variables which show a covariance equal to 0. See a famous plot taken from Wikipedia page on Pearson correlation coefficient

A standard way to uncorrelate variables is to perform principal component analysis on the dataset, and use all components which are, by definition, uncorrelated. But please note that the Naive Bayes assumption is conditional independence, so independence for each class. In the general case, PCA projection would not be the same depending on the selected class.

Regarding the performance of Naive Bayes, I see no reason that a simple transformation should improve the model. This is supported by some reasonably well-received answers on similar questions, for instance this one on Stack Overflow, or this other one on Cross Validated.

However, I have read several reports of people getting increased accuracy when performing PCA prior to Naive Bayes. See here for instance, for a few exchanges on the subject, within a community of which trustworthiness is unknown to me; or search "PCA naive bayes" through your web search engine.

So in the end, I have no mathematical evidence to support this, but my feeling would be that the model performance variation before/after transformation could depend on the problem, especially the directions of principal components for each class.

Perhaps, if you perform some tests, you could share the result here. Otherwise, I think you could get answers with more mathematical background on Cross Validated.

As you pointed out, a null covariance does not guarantee that variables are independent. You can have strongly dependent variables which show a covariance equal to 0. See a famous plot taken from Wikipedia page on Pearson correlation coefficient

If you try to apply a transformation to the input space, you may create variables which, when plotted, look independent, but their dependency still lies within the transformation. For instance, a bivariate Gaussian variable with non diagonal covariance matrix can easily be rotated so that the transformed variables look like they are independent, but the dependency lies within the rotation angle.

You should however note that, even if variables are not independent, naive Bayes can perform quite well anyway.

As you pointed out, a null covariance does not guarantee that variables are independent. You can have strongly dependent variables which show a covariance equal to 0. See a famous plot taken from Wikipedia page on Pearson correlation coefficient

A standard way to uncorrelate variables is to perform principal component analysis on the dataset, and use all components which are, by definition, uncorrelated. But please note that the Naive Bayes assumption is conditional independence, so independence for each class. In the general case, PCA projection would not be the same depending on the selected class.

Regarding the performance of Naive Bayes, I see no reason that a simple transformation should improve the model. This is supported by some reasonably well-received answers on similar questions, for instance this one on Stack Overflow, or this other one on Cross Validated.

However, I have read several reports of people getting increased accuracy when performing PCA prior to Naive Bayes. See here for instance, for a few exchanges on the subject, within a community of which trustworthiness is unknown to me; or search "PCA naive bayes" through your web search engine.

So in the end, I have no mathematical evidence to support this, but my feeling would be that the model performance variation before/after transformation could depend on the problem, especially the directions of principal components for each class.

Perhaps, if you perform some tests, you could share the result here.