How to treat undefined input space in supervised learning algorithms?

Question

In a supervised learning problem (like a neural network) with binary output, we supply input data for which the output values are defined 0 and 1 (indicating probabilities of occurrence of an event). But the input data not necessarily cover the entire input space. How does the algorithm treat the rest of the input space?

For example, for an annular dataset in the figure, there are two sets of data in the 2-D input space annotated as 0s and 1s. But there are a lot of input space which is not defined (neither 0 nor 1). What output can be expected at the prediction stage for an input that does not lie around the input data clouds?

Answer 1

It depends on the machine learning technique. The following link gives a really good example comparing many classifiers and their segregation area. The same happens with clustering methods.

Classifier comparison