Choosing the lasso penalty ρ by the mean MSE over realisations

• Status: Accepted in 2021. Reviewed in 2026 with the evidence below, not superseded.
• Date recorded: 2026-10-06 (the decision itself was made in September 2021)
• Decision in one line: I chose ρ = 0.625, the midpoint of 0.60 and 0.65, from the mean reconstruction MSE over ten noise realisations, and the revival keeps that value wherever it reproduces the report.

Context

Question 2.3 asked for a lasso penalty ρ chosen by simulation. The notebook fitted the lasso at 21 values, ρ = 0, 0.05, …, 1, and repeated this ten times. Every pair of realisation and ρ drew fresh noise Γt and Γs, rebuilt and standardised X, and recorded the reconstruction error $\lVert X - DA\rVert_F^2 / (NV)$ with $D = XA^\top$. The mean over the ten realisations was lowest at ρ = 0.60 (0.6194), with 0.55 at 0.6368 and 0.65 at 0.7493.

The chosen ρ then fed Question 2.4, where the lasso had to beat ridge on both correlation sums, $\Sigma c_{T,\mathrm{LR}} > \Sigma c_{T,\mathrm{RR}}$ and $\Sigma c_{S,\mathrm{LR}} > \Sigma c_{S,\mathrm{RR}}$.

Decision

I read the plot as having its lowest region between 0.60 and 0.65 and took the midpoint, ρ = 0.625. The notebook says so in the cell that sets r = 0.625.

In 2026 the revival keeps ρ = 0.625 on every page that reproduces the report. It adds an uncertainty analysis around the choice on /tune and does not replace it.

Options considered

1. Take the grid minimum, ρ = 0.60. This is the literal answer to "at what value of ρ do you find the minimum MSE".
2. Take the midpoint of the two lowest grid points, ρ = 0.625. This is what I did. It avoided committing to a single grid point.
3. Refine the grid around the minimum. Rerunning on a finer grid between 0.5 and 0.7 would have located the minimum more precisely.
4. Use more realisations and a rule that accounts for noise, such as the one-standard-error rule. This needs a standard error for the mean curve, which ten realisations give only roughly.
5. Choose ρ by recovery of the sources instead of reconstruction error. The data are simulated, so the true maps are known and could be the target.

Why

The brief asked for the ρ that minimises the mean MSE and suggested ten realisations, so I worked within that design. Averaging over realisations protected the choice from one lucky dataset. The midpoint felt like a cautious way to hedge between two neighbouring grid points that both looked low on the plot.

What happened

The revival reran the same procedure with more realisations. With the as-submitted design and seed 30034, realisations 1 to 10 are exactly the 2021 ones, and the next 40 continue the same random stream.

• With 50 realisations the lowest mean MSE moves to ρ = 0.55 (0.6169, 95% bootstrap interval 0.6073 to 0.6281). ρ = 0.60 is close behind at 0.6189 (0.6154 to 0.6225).
• Across 2,000 bootstrap resamples of the realisations, the minimum of the mean curve falls at 0.55 in 65% of resamples and at 0.60 in 35%. It never falls at 0.65.
• One realisation of the as-submitted procedure cannot say which ρ is best for a dataset, because each of its 21 values comes from a different noisy dataset. That question needs a paired design, where one noisy dataset is reused for every ρ, here on a grid with steps of 0.025. Across 50 such datasets the best ρ is 0.575 for 38, 0.60 for 7 and 0.55 for 5. Only 7 of 50 fall in my bracket of 0.60 to 0.65 (14%, Wilson 95% interval 7% to 26%), and none has its best ρ at 0.625.
• In the paired design the minimum of the mean curve is at 0.575 in every bootstrap resample. On the same 50 datasets, ρ = 0.625 gives a higher MSE than ρ = 0.575 in all 50 (exact sign test p < 0.001), by 0.089 on average (95% interval 0.081 to 0.096, $d_z$ = 3.2). Measured against each dataset's own best ρ, an optimistic oracle that no fixed ρ can reach, the gap is 0.092 (0.084 to 0.100), about 16% above that best.

The weak point is the midpoint rule. The curve is not symmetric around its minimum. It rises much faster to the right of 0.6 than to the left, so the midpoint of 0.60 and 0.65 lands on the steep side. The 2021 conclusion in Question 2.4 still holds at ρ = 0.625. Across 100 realisations the as-submitted lasso beats ridge on both correlation sums in every realisation (/recovery). So the choice of ρ did not change the qualitative answer, but the stated reason for this specific number does not survive more data.

Two caveats limit what this shows. The MSE criterion reconstructs X from itself ($D = XA^\top$), so it rewards fitting X and says little about recovering the true sources directly. The lasso in this criterion is also the single-step 2021 version (see DR-003), so the criterion was computed on an estimator that had not converged.

What I'd change

• Use common random numbers, so every ρ is compared on the same datasets, and report differences between values of ρ as paired comparisons.
• Report the minimiser with an interval, for example from a bootstrap over realisations, instead of a point read off a plot.
• Refine the grid where the curve is lowest instead of interpolating between two grid points.
• Choose ρ by the quantity the analysis cares about. On /bias-variance the converged lasso's error against the true maps is lowest at ρ = 0.40.
• Fix the solver first (DR-003), because a tuning criterion is only as good as the estimator it evaluates.