Skip to content
Source Separation Lab

Step 04Question 2.5 · principal component regression

When the regressors lose their shape

Principal component regression is the usual answer to correlated regressors, and TC4, TC5 and TC6 are strongly correlated. Here it backfires: the components are orthogonal, but none of them looks like a source any more.

The thin singular value decomposition of the standardised time courses gives orthonormal regressors Z=UZ = U:

TC=U Σ W⊤U∈R240×6,Σ=diag(σ1,…,σ6)\begin{gathered} \mathrm{TC} = U\,\Sigma\,W^\top \\ U \in \mathbb{R}^{240\times 6},\quad \Sigma = \mathrm{diag}(\sigma_1,\dots,\sigma_6) \end{gathered}

All six components are kept, and the same lasso as before is fitted with a very small penalty, ρ = 0.001. Singular vectors are only defined up to sign. The port flips them to match the signs LAPACK returned in 2021, so the plots below match the originals.

Using the 2021 parameters. Results match the report.

Figure 4.1

Singular values of the time courses

Bar chart of the six singular values: 24.20, 16.76, 15.83, 14.08, 9.06, 6.05051015202524.20PC116.76PC215.83PC314.08PC49.06PC56.05PC6Principal component
The six singular values of TC from np.linalg.svd, which the original plotted as eigenvalues (the eigenvalues of TCᵀTC are their squares). The sixth component is the weakest (σ₆ = 6.050, matching the 2021 value 6.050).

Figure 4.2

Principal components Z beside the source time courses

  • Regressor Zᵢ (principal component)
  • Source TCᵢ

  1. Z1

    Principal component 1 of the time courses−0.0500.050120240

    TC1

    Source time course 1−1010120240

  2. Z2

    Principal component 2 of the time courses−0.100.10120240

    TC2

    Source time course 2−1010120240

  3. Z3

    Principal component 3 of the time courses−0.100.10120240

    TC3

    Source time course 3−1010120240

  4. Z4

    Principal component 4 of the time courses−0.100.10120240

    TC4

    Source time course 4010120240

  5. Z5

    Principal component 5 of the time courses00.10120240

    TC5

    Source time course 5−1010120240

  6. Z6

    Principal component 6 of the time courses−0.100.10120240

    TC6

    Source time course 6−100120240
Each principal component is a weighted mix of all six time courses, chosen to capture variance rather than to match any one source. The boxcar shapes are lost: the regressors in Z no longer look like the sources they are meant to explain.
Lasso penalty ρ on Zρ = 0.0010
Σ cT PCR
0.362
Σ cT lasso (ρ = 0.625)
5.428
Zero coefficients
121 / 2646

Figure 4.3

What PCR retrieves

  • Ground truth
  • Retrieved (PCR)

time courses shown as z-scores

  1. PC1 / S1

    cS 0.05

    cT 0.077

    Source 1: true and retrieved time courses−202080160240

  2. PC2 / S2

    cS 0.11

    cT 0.187

    Source 2: true and retrieved time courses−202080160240

  3. PC3 / S3

    cS 0.05

    cT 0.044

    Source 3: true and retrieved time courses−202080160240

  4. PC4 / S4

    cS 0.10

    cT 0.028

    Source 4: true and retrieved time courses−202080160240

  5. PC5 / S5

    cS 0.19

    cT 0.022

    Source 5: true and retrieved time courses−202080160240

  6. PC6 / S6

    cS 0.26

    cT 0.004

    Source 6: true and retrieved time courses−202080160240
Lasso on the principal components (ρ = 0.0010): each row shows APCR and DPCR for one component, against source i for reference. With almost no penalty the maps are dense, and because each component blends several sources they light up more than one patch: the false positives the report described.

Next · Step 05

Bias–variance

Split each estimator's error into bias² and variance as the penalty grows.