Understanding the ELM (Extreme Learning Machine)

An ELM is a neural network in which only the last layer is fitted. The hidden layer is drawn at random and then frozen; the output weights come from a single linear solve, with no gradient descent. On the example of this page, with 6 inputs and 25 neurons: one-hour-ahead nRMSE of 0.196, against 0.206 for a linear model (AR) and 0.328 for persistence. You can also paste your own data.

Synthetic series, computed in your browser. The recipe is that of the open code PV_ELM_pred. Version française.

Try it

Three settings: the number of inputs, the number of hidden neurons, the Ridge regularization. Fix them, or let the applet optimise them. At each trial, 100 hidden layers are drawn at random, scored in validation, and the best one is kept.

The applet needs JavaScript.

The recipe, in five steps

  1. The inputs. The last measured values (1 to 24 hours), centred and scaled, plus four clock inputs, which can be unticked: sine and cosine of the hour, sine and cosine of the day of the year. This vector is written x.
  2. Chance. Each hidden neuron receives weights drawn uniformly between −1 and 1 and a bias drawn between 0 and 1. They never change afterwards. The neuron outputs form the matrix H (one row per hour, one column per neuron):H=σ(XW⊤+b),σ(a)=11+e−a
  3. Training. Only the output weights β are fitted. The target is centred and scaled (z), then β is the exact minimum of ‖Hβ − z‖² + λ‖β‖²:β̂=(H⊤H+λI)−1H⊤zThis is the closed form: one linear solve, no learning rate, no local minimum. The term λ (Ridge) penalizes large weights; with λ close to 0, this is ordinary least squares.
  4. The optimisation. The draw matters: two random layers are not equally good. We draw 100 of them, and for each one we try λ = 1, 10 and 25. Each pair is scored by temporal cross-validation: fit on the beginning of the training set, validate on the hours that follow, four times with a longer and longer fitting window. The pair with the smallest mean validation error is kept, then refitted on the whole training set. To optimise the number of inputs and of neurons as well, this search is repeated for each combination: this is the grid search.
  5. The test. Training and optimisation use the first 80% of the series (32 days). The remaining 20% (8 days) are used only once, at the end. Negative forecasts are set to 0 and the night is set to 0. The nRMSE is the RMSE of daytime hours divided by the mean of daytime measurements.

Six trials to run

SettingWhat you seeWhat it teaches
6 inputs, 5 neuronsValidation error from 77.3 to 203.6 W depending on the drawWith few neurons, chance weighs heavily: searching among 100 candidates is essential.
6 inputs, 100 neuronsFrom 61.3 to 64.9 W; test nRMSE 0.198, against 0.196 with 25 neuronsWith many neurons, the draws look alike. But four times more neurons do not improve the test.
1 input, 100 neurons, no RidgeTraining nRMSE 0.200, test nRMSE 0.476 (persistence: 0.328)Without regularization, the model learns noise: very good on training data, poor on the test set. With optimised Ridge: 0.187.
24 inputs, 5 neuronsTest nRMSE 0.330 (persistence: 0.328)More inputs are not enough: 5 neurons cannot summarize 28 inputs. The linear AR model reaches 0.208 with these same inputs.
6 inputs, 25 neurons, without the 4 clock inputsTest nRMSE 0.229, against 0.196 with themThe sines and cosines give the model the time of day: it no longer has to guess it from the last values.
All three on “optimise”25 combinations; kept: 12 inputs, 100 neurons, λ = 1; test nRMSE 0.198The grid picks the best validation (61.0 W, against 62.7 W for 6 inputs and 25 neurons). On the test set the difference is negligible: at equal performance, keep the smallest model.

Why this model

  • Closed form. The result is the exact minimum of the criterion: two people starting from the same draw get the same weights.
  • Simplicity. Three settings: the number of inputs, the number of neurons, λ. The code fits in a few lines.
  • Efficiency. One trial (100 candidates, 3 values of λ, 4 validations, that is 1,201 linear solves) takes less than a second in a browser; the full grid of the three settings (30,001 solves), 15 to 30 seconds depending on the device.
  • Frugality. Few fitted coefficients: one per neuron. In studies on real data, the same recipe is used with 500 neurons and compared with a linear model, which often stays close: see compare forecasts.

The words of this page

Hidden layer
The neurons placed between the inputs and the output. Each one computes a weighted sum of the inputs, then applies the function σ, which maps it between 0 and 1.
Closed form
A solution given by a formula, computed in one go, as opposed to a solution approached step by step.
Ridge, λ
Regularization that adds λ‖β‖² to the criterion. The larger λ, the smaller the weights and the smoother the forecast.
Candidate
One hidden layer drawn at random. We compare 100 of them.
Grid search
Every combination of a list of values for each setting is tried, and the best one in validation is kept.
nMAE, nMBE, R²
Mean absolute error and mean bias (forecast minus measurement), divided by the mean of the measurements; share of the variance of the measurements explained by the forecast.
Temporal cross-validation
Validation always uses hours later than those of the fit, so the future is never used.
Persistence
Reference forecast: the next value will equal the last measured one.
AR (autoregressive model)
Linear reference: the forecast is a weighted sum of the same inputs, plus a constant, fitted by least squares. What the ELM gains over AR measures the contribution of the non-linear layer.
nRMSE
Root mean square error (RMSE) divided by the mean of the measurements of the period. Written as a fraction: 0.22 means 22% of the mean. Two periods with different means cannot be compared by their nRMSE: on the example, RMSE is 85.4 W in training and 85.5 W in test, but nRMSE goes from 0.234 to 0.196, because the test days are sunnier.

Assumptions and limits

  • The series is synthetic: 40 hourly days of a fictitious 1 kWp plant (clear-sky curve multiplied by a cloud factor with memory). The figures on this page illustrate the method; they are not research results.
  • A single horizon (one time step, that is 1 hour for the example) and small sizes (5 to 100 neurons, 32 training days), so that the computation stays fast.
  • The 80% / 20% split is a teaching choice of this page; the PV_ELM_pred code splits 50% / 50%.
  • Your data: from 200 to 2,000 values with a regular time step and no missing value. The series is assumed to start at midnight (for the clock features). Every value is scored, with no night mask; forecasts are bounded at 0 only if no value is negative. Nothing is sent: the computation runs in your browser.
  • “No Ridge” uses λ = 10⁻⁶ and not exactly 0, so that the system remains solvable.
  • Otherwise the recipe follows the PV_ELM_pred code (same random laws, same validation, same refit, same metrics). The random number generator differs. The applet is checked value by value against a Python reference.
  • The solar forecasting course (in French) presents a variant (tanh function, 20 draws); the principle is the same.
  • An ELM does not always beat a linear model. The gain is measured, on the test set, against persistence and an autoregressive model.

Sources

Cite this page

APA

Voyant, C. (2026). Extreme Learning Machine (ELM) explained, with a live demo. www.cyrilvoyant.com. https://doi.org/10.5281/zenodo.24136645. https://www.cyrilvoyant.com/tools/extreme-learning-machine/

BibTeX — the page, then the founding paper of the method.

@misc{voyant2026elm,
  author = {Voyant, Cyril},
  title = {{Extreme Learning Machine (ELM) explained, with a live demo}},
  year = {2026},
  howpublished = {www.cyrilvoyant.com},
  doi = {10.5281/zenodo.24136645},
  url = {https://www.cyrilvoyant.com/tools/extreme-learning-machine/},
  note = {Accessed [date]}
}

@article{Huang2006ELM,
  author = {Huang, Guang-Bin and Zhu, Qin-Yu and Siew, Chee-Kheong},
  title = {Extreme learning machine: Theory and applications},
  journal = {Neurocomputing},
  year = {2006},
  volume = {70},
  pages = {489--501},
  doi = {10.1016/j.neucom.2005.12.126}
}