As it stands, NN-based approaches are the current state of the art. It outperforms the method described below, but in the spirit of answering the question, here it goes...
The trick for not using CNNs is to find a good representation of images with faces on them. CNNs are great because they learn good features. If you don't want to use it, you can use CNNs, you can use Eingenfaces as your features.
The main assumption is that most face images lie on a low-dimensional subspace determined by the first k (k<d)
directions of maximum variance.
You can Use PCA to determine the vectors or “eigenfaces” u1, ... ,uk
that span that subspace.
Then you can represent all face images in the dataset as linear combinations of eigenfaces.
For example, given these images for training:

You can learn the following eigenfaces:

These eigenfaces highlight different features in a person's face. If you average all of them you find a mean face, which would be the most common features among all people in your training set.
Then, this face:

Can be reconstructed by a linear combination of the mean face and some other eigenfaces:

Now, the weights of each component can be used as features for your classification algorithm. Which at this point can be anything, like KNN or SVM.