I want to know the do not want to how to use library
I will denote a $n\times p$ data matrix by $X$, where $n<p$. That is, each row of $X$ is one sample data with $p$ feature variables.
By using singular vector decomposition method, I can decompose $X$ into $A$, $B$, and $C$ such that $X=ABC$, where $A$ is a $n\times n$ matrix satisfying $A^TA=I_{[n\times n]}$, $B$ is a $n\times p$ matrix whose diagonal elements are all non-negative and real but non-diagonal elements are all zeros, and $C$ is a $p\times p$ matrix such that $P^TP = I_{[p\times p]}$. Without the loss of generality, I assume that $B$ is obtained satisfying that the diagonal elements of $B$ is descending order, i.e., $b_{i,i} > b_{j,j}$ for $i<j$.
The score matrix, $S$, is obtained by $S=XC$, whose dimension is $n\times p$.
In order to reduce the dimension of variables, we do as follows: Denoting the $i$-th column of $C$ by $c_i$, which is a loading vector with respect to $i$-th principal component. Hence, deleting $c_{k+1}$ to $c_{p}$, I can obtain $C_k = [c_1~c_2~\cdots~c_k]$. The dimension-reduced score matrix, $S$, is obtained by $S_k=XC_k$.
In this situation, how can I run KNN? Actually, I already made my own KNN code, whose input is trainX, trainY, and testX, and whose output is evaluatedY. I want to compare (i) basic KNN and (ii) dimension-reduction-applied KNN.
I obtained error rate of (i) basic KNN using 5-fold cross validation method.
yhat = myKNN(trainX, trainY, testX);
errRate = myErr(y, yhat);
To obtain error rate of (ii) dimension-reduction-applied KNN, is the following correct?
trainS= myDimensionReduction(trainX, k)
testS= myDimensionReduction(testX, k)
yhat = myKNN(trainS, trainY, testS);
errRate = myErr(y, yhat);
where myDimensionReduction is a function that converts a data matrix $(n\times p)$ into a reduced data matrix $(n\times k)$.
