4
$\begingroup$

I am trying to find which classification methods, that do not use a training phase, are available.

The scenario is gene expression based classification, in which you have a matrix of gene expression of m genes (features) and n samples (observations). A signature for each class is also provided (that is a list of the features to consider to define to which class belongs a sample).

An application (non-training) is the Nearest Template Prediction method. In this case it is computed the cosine distance between each sample and each signature (on the common set of features). Then each sample is assigned to the nearest class (the sample-class comparison resulting in a smaller distance). No already classified samples are needed in this case.

A different application (training) is the kNN method, in which we have a set of already labeled samples. Then, each new sample is labeled depending on how are labeled the k nearest samples.

Are there any other non-training methods?

Thanks

Improve this question
$\endgroup$
1
  • 3
    $\begingroup$ Other keywords that might be useful for what you are looking are clustering and the more general unsupervised learning. $\endgroup$
    – vefthym
    Jul 4, 2014 at 7:24

3 Answers 3

Reset to default
7
$\begingroup$

What you are asking about is Instance-Based Learning. k-Nearest Neighbors (kNN) appears to be the most popular of these methods and is applicable to a wide variety of problem domains. Another general type of instance-based learning is Analogical Modeling, which uses instances as exemplars for comparison with new data.

You referred to kNN as an application that uses training but that is not correct (the Wikipedia entry you linked is somewhat misleading in that regard). Yes, there are "training examples" (labeled instances) but the classifier doesn't learn/train from these data. Rather, they are only used whenever you actually want to classify a new instance, which is why it is considered a "lazy" learner.

Note that the Nearest Template Prediction method you mention effectively is a form of kNN with k=1 and cosine distance as the distance measure.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ So, kNN is not training but lazy learning, because it does not abstract and generalize but, in case of classification, for each new observation it learns from the nearest k-labeled observations. While NTP is kNN-1 between an observation and the signature (that can be considered a pseudo-observation). Am I right? $\endgroup$
    – gc5
    Jul 2, 2014 at 14:57
  • 2
    $\begingroup$ While the kNN classifier is referred to as a "lazy learner", that term is somewhat misleading and it is more accurately described as a "lazy classifier". With regard to your comment, it doesn't actually learn when you give it a new observation to classify - it simply calculates the appropriate class. If you give it that same observation later, it would perform the same computation the second time because it doesn't actually learn from the first observation. In short, kNN doesn't produce/learn a model - it simply stores training data, then uses those data when it is time to classify. $\endgroup$
    – bogatron
    Jul 2, 2014 at 16:48
4
$\begingroup$

nsl- I'm a beginner at machine learning, so forgive the lay-like description here, but it sounds like you might be able to use topic modelling, like latent dirichlet analysis (LDA). It's an algorithm widely used to classify documents, according to what topics they are about, based on the words found and the relative frequencies of those words in the overall corpus. I bring it up mainly because, in LDA it's not necessary to define the topics in advance.

Since the help pages on LDA are mostly written for text analysis, the analogy I would use, in order to apply it to your question, is: - Treat each gene expression, or feature, as a 'word' (sometimes called a token in typical LDA text-classification applications) - Treat each sample as a document (ie it contains an assortment of words, or gene expressions) - Treat the signatures as pre-existing topics

If I'm not mistaken, LDA should give weighted probabilities for each topic, as to how strongly it is present in each document.

Improve this answer
$\endgroup$
1
$\begingroup$

Probably kNN and naive Bayes classifier. kNN is very fast, but NBC can break down a lot. Linear regression is also one-step solution that does not involve gradient-based learning, so that might help. LDA is your next choice.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.