Why deep feedforward neural networks are restricted to solve classification problems?

Question

I want to use deep neural networks for regression problems, but as far as I've read through, it's mainly for classification. I was wondering why can't a regular convolutional network, or a multilayer perception work as a linear regression, just with multiple layers.

I tried connecting two convolutional layers with max pooling on each, and a fully connected layer together, without activation function, with 10000 MNIST images, 100 epochs, and with 100 batch size. Used mean squared error as the loss function, and regular stochastic gradient descent as the optimizer. Instead of one hot encoding, I used the actual expected numbers as labels.

The results are very promising even though I haven't played optimizing the parameters. The loss after the training is under 1, and I've got R2 values over 0.9 at testing.

So the question is, why neural networks are mainly used for classification while it could handle regression as well?

Answer 1

There is regression by neural networks. For example, have a look at this analysis (Deep Regression), RBF neural networks, and General regression neural networks. Also, linear regression can be implemented using a neural network.

If using neural networks for interpolation, this may work very good. However, if regression is used to extrapolate, results may be unsatisfactory.

Note also that the universal approximation theorems say only that a contineous function can be approximated within a compact set (thus in particular bounded set!).