What is Extreme Learning Machine? Why tuning of weights is not required?

Question

The wikipedia says:

Extreme learning machines are feedforward neural network for classification, regression, clustering, sparse approximation, compression and feature learning with a single layer or multi layers of hidden nodes, where the parameters of hidden nodes (not just the weights connecting inputs to hidden nodes) need not be tuned.

How is it possible to assign values to links without actually seeing the Training set?

Till now from my research to understand this, I've made a lead, but not satisfactory because I can't anything from this below paragraph.

For any nonconstant piecewise continuous function that is used as the activation function, if the parameters of the hidden neurons are tuned, then the function can make the SLFNs approximate any target continuous function f (x). Then, according to any continuous distribution probability, the function sequence {h subscript i (x)} where i goes from 1 to L can be randomly generated, and it has the universal approximation capability, which means that $$\lim_{L→ \infty} ||L i=1 βi hi (x) − f (x)|| = 0$$ holds with probability of one with appropriate output weights $β$.

SOURCE: A Parallel Multiclassification Algorithm for Big Data Using an Extreme Learning Machine, IEEE transactions on neural networks 2017

Can anyone prove/explain this with an example please?

Also any extra knowledge on extreme learning machine is very welcomed.

Answer 1

ELM is actually solving a generalized linear problem :

\begin{equation} H\beta = Y \end{equation}

where $H$ is the matrix of the hidden layer output, $\beta$ is the weight vector (or matrix) connecting the hidden layer and the output layer and $Y$ symbolizes the target of your training.

The goal is to find $\beta$ using the pseudo-inverse of H.

Solving this problem is equivalent of finding the $\beta$ achieving the minimum of the Least-square problem $||H\beta - T ||$ : \begin{equation} \beta = \underset{\beta}{\text{min}} \left \vert\left \vert H\beta - T \right\vert\right\vert \end{equation} To answer your question

How is it possible to assign values to links without actually seeing the Training set?

Since $H$ depends on the activation functions, on the initialized weight matrix linking Input layer and hidden layer and on each hidden node bias, it encompasses all the input information and ensures a learning by tuning $\beta$ coherently.

You can find lot more details in ELM Theory and Application or in this ELM presentation for example.