Let's use the following notation : $(x_1, ...x_9)$ are the first 9 variables out of which you try to predict $y$, your tenth variable.
I will try to address what looks like a PCA misunderstanding, then give you some ways to predict which variables $ x_i $ matter most to predict $y$
Principal Component Analysis
The returned features you get from the PCA are not the original one!
The PCA transforms $(x_1, ..., x_9)$ into $(x'_1, ..., x'_9)$ where each $x'_i$ is a particular combination of the $(x_1, ..., x_9)$ such that for any $i, i'$, $x_i$ and $x_i'$ are linearly uncorrelated (ie orthogonal). Moreover, it is done such that each $x_i$ explains a certain part of the training dataset variance. In fact, most of the implementation sort the $x'_1, ..., x'_9$ such that the first one is the one explaining the variance the most, then the second, ...
Thus you end up having your first variable $x'_1$ explaining a lot your prediction $y$. But this $x'_i$ is a particular combinaison of $(x_1, ..., x_9)$.
Getting the feature importance
As to assert which variable $x_i$ explains $y$ the best, one might use different ways. Just to mention that the feature importance is not absolute ; it relies on the technique (or estimator) you use to address this question. Here is a non exhaustive list of possibilities :
- The linear correlation is a good start
- With the
RandomForestClassifier (or RandomForestRegressor depending on $y$) of sklearn.ensemble, you can use feature_importances_ method to get which one is used the most at tree nodes. Note : it works with any decision tree estimator.
- With standard linear or logistic regression (same with Lasso, Ridge, ...), you can check which variable has the higher coefficient (do not forget to normalize your input variables $(x_1, ..., x_9)$ )
I don't know how it is done using neural networks but would be glad if anyone has an hint.
