Dimension reduction techniques in R that do not use the full distance matrix

Question

I try to apply non-linear dimension reduction in R. As usual in machine learning I have a large data set (100 K rows). I tried the packages RDRToolbox and vegan:

library( RDRToolbox )    
res.lle = LLE(as.matrix(temp), dim = 3 , k = 10)
res.iso = Isomap(as.matrix(temp), dim = 3 , k = 10)

library( vegan )
isomap(dist(temp), ndim=10)

The first algoritms use the distance matrix internally and crash as they want to allocate 32GB. The second approach needs the distance matrix and crashes too.

Are there any tricks to do non-linear dimension reduction either with these algorithms? Or are there packages that do not use the raw and full distance matrix? Such approaches have to crash in my setting on my machine.

The only algorithm that works for me is Rtsne that appearantly does not use the whole distance matrix. largeVis was too slow so far to let it finish.

Answer 1

You can try using an auto-encoder which also is a non-linear dimension reduction technique. It uses a neural network framework to find the most efficient transformation from $p$ dimensions down to whatever you choose. It then finds how well it can reconstruct the origianl $p$ variables, and keeps tuning until it can optimally recreate the original values as "good" (lowest MSE) as possible.

In this case the input has 5 variables, and it reduces it down to 2 variables using non-linear transformations. Once you get it down to the reduced variables (the light blue nodes) it then tries using non-lienar transformations to recreate those original five inputs using a similar nn structure. The right-five red nodes are then compared to the left-five original inputs to see how much information is encoded in those 2 middle light blue reduced variables.

I've found the h2o implementation is pretty easy to use, but finding the correct parameters (number of layers, nodes, learning rate, etc) is still tough to select depending on your data.

Heres a little blog post that goes through the h2o autoencoder with the MNIST data set.