From the documentation:
coeff = pca(X)
returns the principal component coefficients, also known as loadings, for the $n$-by-$p$ data matrix X
. Rows of X
correspond to observations and columns correspond to variables. The coefficient matrix is $p$-by-$p$. Each column of coeff
contains coefficients for one principal component, and the columns are in descending order of component variance. By default, pca
centers the data and uses the singular value decomposition (SVD) algorithm.
The values in coef
represent the transformation from the original features (rows of coef
) to the principal components (columns of coef
). You'll want to keep only the first $k$ columns, then multiply your data matrix by this matrix.
Your features are the dimensions that your data lives in, so number of features and dimension are the same. Very roughly speaking, PCA rotates the the feature axes to align to the most significant directions rather than the original feature directions, then selects only the most significant directions to keep, thus reducing the dimensionality of your problem. But it also means your new columns won't represent pure features anymore, but linear combinations of them.
If you keep all of the new columns, you won't reduce dimensionality; but your new features will be ordered in such a way that the first ones capture the most variance of your data.