How do linear learning systems, such as the simple "closest to the class average" algorithm or SVMs, classify datapoints that fall on the hyperplane?
Linear, binary classifiers can choose either class (but consistently) when the datapoint which is to classify is on the hyperplane. It just depends on how you programmed it.
Also, it doesn't really matter. This is very unlikely to happen. In fact, if we had arbitrary precision computing and normal distributed features, there would be a probability of 0 (exactly, not rounded) that this would happen. We have IEEE 754 floats, so the probability is not 0, but still so small that there are much more important factors to worry about.