You can't calculate the conditional distribution because that distribution is defined over the entire population and assuming a probabilistic model which you do not know.
In real life you have only a sample of the population. More often than not, in real life you do not have the structure of the model also. You do not know if it is a linear model, a polynomial or another type.
The probabilistic approach is to assume a model, consider some assumptions for that model and fit the model's parameters using data. That is not the Bayes classifier, it's only an approximation.
The trick is 'all models are wrong, some models are useful'. There are many approaches to learn this approximation. You can estimate the joint probability (generative classifiers), you can estimate directly the conditional probability (discriminative classifiers) or you can use a different approach (like SVM, for example) and twist it somehow to approximate the conditional probability. But all of them are approximations. The real one, THE Bayes classifier will remain unknown.
Looking at the Wikipedia page for Bayes classifier it states that: "The Bayes classifier is a useful benchmark in statistical classification.". My opinion is that this statement is confusing, if not dead wrong. Probably this is related with the wrong usage of expression "Bayes classifier" to designate the Naive Bayes classifier. If one search on google for "Bayes classifier", he will see a lot of materials related with Naive Bayes. Personally I do not agree with the statement from wikipedia. Bayes classifier is that state of nature that we do not know but we aim to approximate as well as possible.