Machine Learning on Curved Background

Ok, maybe this is a really dumb question, but has anyone ever considered extending some of the statistical learning methods to live in some space other that $\mathbb R^n$? My guess is that you're just shuffling parameters around, but I wonder if there's any intuition to be gained.

This question came up as I was working through some course notes on neural nets, and realized that computer scientists may be even more cavalier with tensor indices than physicists :)

• Look into manifold learning, information geometry, natural gradient methods, tensor methods, group theoretical methods, complex valued neural networks, etc. It's all there if you look. – Emre May 30 '17 at 7:12
• Thanks for the refs @Emre, just learning the details of neural networks. – BenDundee May 30 '17 at 13:52

Essentially, each non-linear layer in a neural network is a map from $\mathbb{R}^n$ input to a $\mathbb{R}^m$ output. There is no requirement for these dimensions to be separate and uncorrelated, nor for the map to be reversible.