Methods, decisions and limits

Every dataset on this site is synthetic. There are no people, no personal information and no real measurements. Six time courses and six spatial maps are generated from fixed rules, mixed, and buried in Gaussian noise drawn from a seeded random generator.

The generator is a TypeScript port of numpy's legacy RandomState, so seed 30034 reproduces the noise of the 2021 notebook bit for bit (decision record DR-002). The parity tests compare the port against the files the 2021 notebook wrote to coursework/stage/. The 2026 statistics are checked against numpy, scipy and scikit-learn through scripts/export_stats_reference.py.

The repository also keeps a set of noise drawn in R with the same seed. The final notebook did not use it. A test recomputes the 2021 pipeline on it, and the methods rank in the same order with numbers that differ in the second or third decimal place.

The data are a product of temporal and spatial sources, each with its own noise, with N = 240 time points and V = 441 pixels:

Expanding the product shows which terms act as signal and which as error:

The regression model is $X = DA + E$ with $D = \mathrm{TC}$ known. Each column of X is then standardised, so the estimand is the map on the standardised scale rather than the 0/1 map that generated it.

• Gaussian, independent noise. Every entry of Γ_t and Γ_s is drawn independently from a normal distribution with mean 0 and variance 0.25 (temporal) or 0.015 (spatial), and the two noise matrices are independent of each other and of the sources.
• Known regressors. The estimators are given the true time courses. Real source separation has to estimate them too.
• Standardisation choices. Time courses are standardised with the sample standard deviation (ddof = 1, as pandas does). Columns of X are standardised with the population standard deviation (ddof = 0, as scikit-learn's StandardScaler does). The spatial maps are not standardised. A constant column of X is centred and left unscaled.
• Non-overlapping patches. Each pixel belongs to at most one source, which is why the noise-free target is the true map scaled by $\sqrt{N/(N-1)}$.
• Penalties as submitted. Ridge applies λ to $D^\top D$ without the brief's scaling by V, and the 2021 lasso is a single thresholded step. Both quirks are kept so the report's numbers reproduce.

All five estimators return maps  and time courses D̂ = XÂᵀ.

Least squares and ridge have closed forms:

The lasso as submitted takes one soft-thresholded gradient step from zero with step size $s = 1/(1.1\,\lVert D^\top D\rVert_F)$ and threshold $t = N\rho s$, then scales the survivors by 1 + t (the (1 / 1 + thr) quirk):

The converged lasso (added in 2026, decision record DR-003) solves the objective the 2021 code was approximating, pixel by pixel, by cyclic coordinate descent with $G = D^\top D$ and $c = D^\top x_v$:

Principal component regression runs the 2021 lasso on the left singular vectors $Z = U$ of $\mathrm{TC} = U\Sigma W^\top$ with ρ = 0.001. The singular values of TC run from 24.20 down to 6.05.

Metrics. Reconstruction error is $\lVert X - \hat D \hat A\rVert_F^2 / NV$, the criterion Question 2.3 used to choose ρ. Recovery is the absolute correlation between a true source and a retrieved one. The 2021 sums Σc_T and Σc_S pair sources by index. The 2026 per-source scores take the maximum over retrieved sources, as the brief defines them, and report how often that maximum is the matching index. Bias² and variance are measured against the noise-free target $A^\ast = (D^\top D)^{-1}D^\top X_0$ and divided by $\lVert A^\ast\rVert^2$.

Realisations and seeds. Every Monte-Carlo draws from numpy's generator with the seed shown on the page (30034 by default). In the 2026 analyses, realisation 1 is the 2021 dataset itself. On /tune the as-submitted design continues the 2021 stream, so realisations 1 to 10 are the ten behind the report.

Paired comparisons. Whenever two estimators or two penalties are compared, they are fitted to the same datasets (common random numbers). The site reports the mean of the per-realisation differences with a 95% interval, the standardised effect size $d_z = \bar\Delta / s_\Delta$, how often one beats the other with a Wilson interval, and an exact two-sided sign test. The p-value is never reported alone. Because paired differences can be almost constant, $d_z$ measures how consistent a difference is, not how large it is, so the size of each difference is also given relative to the comparison method's mean, and differences under 1% are called negligible. A comparator is fixed in advance wherever possible. On /tune the cost of ρ = 0.625 is measured against ρ = 0.60, the lowest point of the 2021 curve, rather than against the best ρ of the same run, which is picked from the same datasets and so makes the gap look larger than it is (DR-004).

Intervals. From 30 realisations up, means use 95% percentile-bootstrap intervals over realisations with 2,000 resamples and resampling seed 30034. The resampling indices come from the same numpy generator, so each interval can be reproduced exactly in Python. Below 30 realisations the percentile bootstrap of a mean runs narrow: in a simulation with normal data it covered the true mean 90% of the time at n = 10 and 92% at n = 20. There, means use the Student-t interval $\bar x \pm t_{n-1,\,0.975}\, s/\sqrt n$ instead, and captions say so (DR-005). Where the best ρ lands is always a bootstrap, because it is not a mean. Proportions use the Wilson score interval. Every helper lives in web/src/lib/stats/ and is tested against numpy and scipy values.

Sensitivity. The noise grid restarts the seeded stream in every cell, so all cells share the same standard-normal draws scaled to their own variances. Differences between cells then come from the noise level, not from a different random draw.

• Synthetic setting. Boxcar time courses, square patches and Gaussian noise are much cleaner than real fMRI. Nothing here says how these estimators behave on real recordings.
• Multicollinearity by design. TC4, TC5 and TC6 overlap in time, with correlations up to 0.77. The condition number of TC is 4.0, which inflates the variance of least squares and is what the penalties are meant to fix.
• Known time courses. Every estimator is given D = TC. In practice the time courses are unknown, and methods such as independent component analysis would be needed.
• What the intervals mean. The intervals describe Monte-Carlo uncertainty for this simulation design. They are not confidence intervals for any single real dataset, and they shrink as you add realisations. On step 05 only the MSE carries an interval; its two parts, bias² and variance, are shown as point estimates.
• The tuning criterion. The 2021 MSE reconstructs X from itself (D̂ = XÂᵀ), so it rewards fitting X and only indirectly measures recovery of the sources.
• Plug-in bias². The plug-in bias² is too high by about the variance divided by R − 1, because the average over R realisations is itself noisy. At 50 realisations that is 0.033 for least squares on the maps, two thirds of its plug-in bias² of 0.050, so step 05 also shows a debiased bias² (0.018 here). For ridge and both lassos the correction is 0.005 or less.
• Two scoring conventions. In 2021 the spatial maps were scored against a copy in a different pixel order from the one X was built with. Step 02 keeps that, so it reproduces the report. The 2026 analyses use the matching order, so their spatial sums are higher.

• Choose ρ with a paired design, more realisations and an interval on the minimiser, and refine the grid near the minimum instead of taking a midpoint (DR-001).
• Tune penalties for the quantity of interest, recovery of the true sources, rather than a reconstruction of X. The converged lasso's map error is lowest near ρ = 0.40, not 0.625.
• Use a converged lasso from the start and test it against a reference solver (DR-003).
• Apply ridge with the brief's scaling, λ̃ = λV. The bias-variance view puts the optimum near λ ≈ 300.
• Let the penalty scale with the noise level, so it still works when the noise changes.
• Write comparators and the interval method into an analysis plan before running a simulation (DR-004, DR-005).
• Save the exact noise next to the outputs, and generate data in one language only (DR-002).

What AI does here. One optional feature, “Explain this figure”, on steps 03, 05 and 06. It sends a short numeric summary of the figure on screen to a model and shows back a plain-English reading with caveats. The site is complete without it.

What it never does. It never computes, changes or replaces a number, a figure or a conclusion on the site. Every statistic comes from the deterministic code described above. Its prompt never contains the underlying data matrices, other pages or any personal data. It never runs without a click, and it never uses a key belonging to this site, because there is none.

Your key. You bring your own Anthropic or OpenAI key. It is kept in this browser's sessionStorage, or in localStorage only if you tick “remember on this device”, and a “forget key” button removes it. The browser sends it straight to the provider you chose. It is never sent to this site's server, never logged and never written to the audit log. The default Anthropic model is Claude Haiku 4.5 (claude-haiku-4-5), with Claude Sonnet 5.5 as an option. For OpenAI the model id is editable (default gpt-5-mini).

Data sent to the provider. Fixed instructions, plus a JSON summary of at most a few kilobytes holding the figure's title, a fixed description, and rounded numbers such as means, intervals, sample sizes and seeds. You can read the exact text that would be sent, instructions and summary, under each button before sending it. Because the request goes straight from your browser, the provider also receives the usual request metadata: your IP address, your browser's user agent and this site's origin. The call is billed to and logged against your own provider account, and the provider's terms govern what happens to it there.

Checks and human review. Answers must match a fixed JSON schema, which is validated in the browser. An automatic check compares every number in the answer with the numbers in the summary and flags any that do not match. Counts such as “7 of 50” and percentages must match a count or share in the summary exactly. The check shows that a number appears in the summary, not that it is attached to the right quantity, and it does not check years or the 95% confidence level. Every answer is labelled “AI-generated” and asks you to accept, edit or reject it.

Audit trail. Each call is recorded in this browser's IndexedDB with the exact request sent (endpoint, API version, model, token limit, output schema, instructions and summary, but never the key), the output, latency, token usage, check results and your decision. Failed and cancelled calls are recorded too, and when the model replies in a form that fails validation its raw reply is kept. An explanation is shown only next to the figure it was generated for; if the figure changes, it is hidden until the figure matches again. You can read and export it as JSON or CSV at /ai-log, and it never leaves your device unless you export it.

Frameworks. The design is informed by the Australian Government's policy for the responsible use of AI in government (Digital Transformation Agency), the transparency principles of the EU AI Act and the NIST AI Risk Management Framework. It does not claim compliance with, or certification under, any of them.