I found the following very surprising. I trained different machine learning classifiers on a data set that includes $15$ attributes using $k$-fold cross validation. Then test on a different validation data set. The result were fine, accuracy was about $81\%$ and the f1-measure was about $83\%$.
Out of curiosity, I tried to train and test on one single attribute, and I found that there is one attribute that outperform the previous result, accuracy was about $85\%$ and the f1-measure was about $86\%$.
This results of the one attribute was always better than all attribute when using different binary classifiers.
How common is this? Is there any interpretations for the reasons behind this?