Hardened Balancing Principle (HBP)

Definition

Hardened Balancing Principle (HBP) is an automatic Tikhonov regularization parameter $\lambda$ selection method proposed by Bauer 2007 on top of the Balancing Principle (BP) of Bauer & Hohage 2005. In the Caltech DSCOVR project, Zhenyu (1) carried out a complete symbolic derivation starting from ${\psi }^{+}=1$ (Tikhonov), (2) generalized to the exponential concentration assumption, (3) systematically stress-tested HBP on 4 benchmarks against L-curve / GCV / Morozov discrepancy / Kawahara Bayesian, and (4) adopted it as the primary selector for the DSCOVR automatic pipeline.

HBP’s core advantage over BP: BP’s Lepskij-type stopping rule requires (i) prior knowledge of noise level $\delta$, (ii) tuning constant $\kappa$; HBP, via the construction of “minimum difference regularization parameter”, needs neither.

Bauer 2007 Definition 5.1 (HBP construction skeleton):

• “minimum difference regularization parameter”: $n^{+}=\min \{n\in [0,N]:D(n,k)>D(n-1,k)\text{ for some }k\}$
• Balancing functional (simplified): $b(n,m)=\frac{\Vert x_n-x_m\Vert }{\Phi (N,m)-\Phi (N,n)}$
• HBP picks ${\lambda }_{n^{+}}$

Core Arguments

1. Exponential concentration derivation starting from ${\psi }^{+}=1$ (Tikhonov)

Zhenyu starts from the Tikhonov case ${\psi }^{+}=1$ and uses Bauer’s exponential concentration assumption to derive the HBP ${\lambda }_{n^{+}}$ automatic selection formula. The complete 90-paragraph derivation contains:

• Generalization of the Lepskij-type stopping rule
• Mathé-Pereverzev 2003 geometric framework
• Step-by-step simplification from Tikhonov → BP → HBP
• Verification method for the exponential concentration assumption (minimum selection of the difference function)

[Future expansion] Full mathematical derivation (formula-level depth) reserved for a follow-up post.

2. Systematic stress-test results

Zhenyu ran HBP simultaneously with 4 traditional methods (L-curve / GCV / Morozov discrepancy / Kawahara Bayesian cost) on 4 benchmarks:

Benchmark	Configuration	HBP performance
Hansen gravity problem	500-instance randomization	Automatic, most robust, stably converges
Golub 1979 Laplace	Condition number 2.88e28 (Zhenyu’s measurement) vs the paper’s original 1.54e5 — a stress test more ill-conditioned than the original paper	Automatic, still works
3-point linear regression	Cross-validated via Prof. King-Fai Li × Zhenyu Bayesian closed-form solution	Consistent
DSCOVR real PC2 time series	2-year EPIC PC2, real viewing geometry W (DSCOVR orbit + Earth rotation + pixelation + sun-illumination)	HBP automatic λ convergence range matches Siteng Fan’s manual $\lambda \approx 10^{-3}$ benchmark

3. Pipeline primary selector decision logic (motivated decision after 4-method stress-test)

Method	DSCOVR performance	Selected?	Reason
L-curve	Visual elbow inconsistent with theoretical curvature (axis-scale bug, see L-Curve Axis-Scale Ambiguity (Hansen σ² vs σ⁴ bug catch))	❌	Unreliable unless axis-scale corrected
Kawahara Bayesian cost	Case-by-case unstable on Hansen toy (5/5 fail at $x_{sol}=[10,1]$) + fail under DSCOVR W	❌	Not robust for arbitrary W
GCV	Long flat region → sensitive to noise direction + underestimates best λ	❌	Statistical bias
Morozov discrepancy	Systematic error (always biased to right of L-curve elbow) + requires $\delta$	❌	Systematic bias + prior info requirement
HBP	Automatic, requires neither $\delta$ nor $\kappa$, performance on DSCOVR matches Siteng’s manual result	✅	Most robust and automatic

→ HBP’s adoption is not an arbitrary choice but a motivated decision after systematic stress-testing of 4 textbook methods from Hansen.

Different Perspectives / Comparison with other regularization methods

vs L-curve (the classical method described in Inverse Problems & Regularization):

• L-curve requires “corner identification” — either by naked-eye (a scale-dependent artifact, see L-Curve Axis-Scale Ambiguity (Hansen σ² vs σ⁴ bug catch)) or by computing curvature (sensitive to noise + axis-scale dependent)
• HBP requires no geometric interpretation, is fully algebraic → better suited for pipelines

vs GCV (Golub 1979):

• GCV works on the original paper example of Golub 1979 (condition 1.54e5), but when Zhenyu reproduced it pushing condition number to 2.88e28, noise-direction dependence was first discovered (the 4th of 5 instances had a completely different ${\lambda }_{GCV}$) → later systematized as DSCOVR contribution (d) “Direction of Noise Vector Influences Regularization”
• HBP is more robust against this geometry-dependence

vs Morozov Discrepancy Principle:

• Discrepancy requires $\Vert A{x}_{\lambda }-d\Vert =\delta$, which requires $\delta$ known a priori; HBP is fully data-driven, no $\delta$ needed

vs Kawahara 2016 Bayesian cost:

• Kawahara’s evidence maximization works on specific toy models, but when |x_{sol}| is large (e.g., $[10,1]$), it fails 5/5 on Hansen toy; also fails under the DSCOVR W
• HBP is the only method in Zhenyu’s benchmarks that works on all 4 benchmarks

Downstream influence (methodological influence)

Subsequent collaborators applied HBP to extended regimes (low / high clouds), demonstrating methodology transfer beyond the original PC2 land/ocean retrieval.

Open Questions

• Does HBP remain valid under nonlinear forward operators? Bauer 2007’s original framework is linear Tikhonov; nonlinear extension would require re-deriving the exponential concentration property
• Practical comparison of HBP’s computational cost in high-dimensional regimes (n > 10000) vs. GCV (which requires the trace of the hat matrix)
• Bayesian interpretation: does HBP correspond to a specific hyperprior in the Bayesian framework? Can it be unified with the Type-II MLE in Gaussian Process Bayesian Inversion?

[Future expansion] HBP study notes (full math, Bauer 2007 reading notes, Mathé-Pereverzev 2003 framework derivation) reserved for a follow-up post.

Sources

• research_wiki/projects/mcmc_retrieval/overview.md §Contribution (a) Hardened Balancing Principle (HBP) application + math expansion (2021-09-27 + paper draft 90-paragraph section)

• research_wiki/projects/mcmc_retrieval/overview.md §🌟 Contribution (a) extension: HBP systematic test on the DSCOVR case

• research_wiki/projects/mcmc_retrieval/overview.md §Contribution (a) extension Table (L-curve / Kawahara / GCV / Discrepancy / HBP decision logic)

• HBP mathematical writeup with collaborator acknowledgments (working manuscript, not publicly submitted).

• Bauer 2007 — original HBP method paper (Definition 5.1)

• Bauer & Hohage 2005 — Balancing Principle predecessor

• Mathé & Pereverzev 2003 — geometric framework

• [to be added] HBP study notes in Zhenyu’s research-angle independent wiki (after merge, this page can be expanded to full derivation depth)

Related Pages

• MCMC & Bayesian Inversion — skill page (contains 4 original Caltech DSCOVR contributions subsection)
• Inverse Problems & Regularization — parent concept (HBP is its automatic parameter selection branch)
• L-Curve Axis-Scale Ambiguity (Hansen σ² vs σ⁴ bug catch) — sibling concept (DSCOVR technical discoveries established in the same batch)
• Gaussian Process Bayesian Inversion — Bayesian dual perspective (HBP’s Bayesian interpretation is an open question)
• DSCOVR Inverse Problem + Regularization Methods @ Caltech / UCR — project experience page (split off from PKU-Undergraduate-Research)
• Yuk L. Yung — PI (Caltech)
• King-Fai Li — co-advisor (UCR)
• — HBP adoption decision is a pivotal moment of motivated stress-testing