Research Note

Gaussian Process Bayesian Inversion

Zhenyu He · Jobs Stroustrup 3 min read

Definition

Bayesian Inversion recasts the inverse problem as posterior inference: $p (x ∣ d) \propto p (d ∣ x) \cdot p (x)$ with likelihood $p (d ∣ x)$ , prior $p (x)$ , and the posterior $p (x ∣ d)$ as the deliverable (not a point estimate, but full uncertainty quantification).

Gaussian Process Prior: $x \sim G P (0, K)$ . Treat the unknown as a random function over some index space (sphere, space-time grid); the kernel $K$ encodes “how similar neighboring points should be.”

Conjugacy: with Gaussian likelihood + GP prior, the posterior is Gaussian with closed-form mean and covariance: $\overset{x}{^} = K_{x y} (K_{y y} + Σ_{d})^{- 1} d, Cov (x ∣ d) = K_{xx} - K_{x y} (K_{y y} + Σ_{d})^{- 1} K_{y x}$ This closed-form posterior mean is equivalent to the Tikhonov solution when $K = α^{- 1} I$ — the Bayesian dual of classical regularization.

Core Arguments

1. Tikhonov ≡ posterior mean under isotropic Gaussian prior. $λ = σ^{2} α$ = noise precision / prior precision. Choosing $λ$ = choosing prior strength.

2. GP kernels = structured priors. Isotropic Gaussian is the weakest. RBF kernel $K (p, q) = exp (- ∥ p - q ∥^{2} /2 ℓ^{2})$ gives “spatial smoothness.” In DSCOVR retrieval:

Spatial: RBF on HEALPix sphere, $ℓ$ = “how large a scale of structure do we expect on Earth”
Temporal: separate RBF $K_{T}$
Separable $K = K_{S} \otimes K_{T}$ to avoid $O (N^{3})$

3. Hierarchical Bayes — hyperparameters as random variables. Either Type-II MLE (marginal likelihood point estimate) or Fully Bayesian: MCMC over $θ$ . Zhenyu chose MCMC in .

4. Practical notes on MCMC over hyperparameters

Affine-invariant ensemble (emcee) for moderate dims
Log-space reparameterization (log10_alpha, log10_sigma) for scale-spanning params
Mixed integration (Rao-Blackwell): MCMC samples $θ^{(k)}$ ; given each, $x ∣ d, θ^{(k)}$ is closed-form Gaussian — mixture approximation for final posterior
Numerical stability: Cholesky (scipy.linalg.solve(..., assume_a="pos")), not inv; slogdet, not log(det)

Different Perspectives

Frequentist vs Bayesian: point $λ$ (fast) vs full posterior (slow, honest UQ)
MCMC vs VI: true posterior slowly vs approximate posterior quickly
GP scaling: $O (N^{3})$ Cholesky is the ceiling. Separable kernel is one escape; inducing points / random features / KISS-GP / GPyTorch LazyTensor are modern directions

Kawahara / Aizawa Exoplanet Spin-Mapping Lineage

Zhenyu’s DSCOVR Earth retrieval borrows directly from the exoplanet line:

Cowan & Agol 2008: feasibility of retrieving surface texture from exoplanet light curves
Kawahara & Fujii 2010 / Kawahara 2016: SOT (Spin-Orbit Tomography)
Aizawa 2020 ApJ 896 22 (PDF in repo): Dynamic SOT — time-varying maps
Zhenyu’s transfer: apply SOT to Earth viewed from L1 (DSCOVR) — a perfect testbed

Applications

Light-curve → exoplanet/Earth surface maps; multi-parameter atmospheric retrieval UQ; climate-observation data assimilation; any problem needing smooth solution + UQ.

Open Questions

Non-conjugate likelihoods (Poisson photon counts, Student-t heavy tails) — no closed-form posterior, need more involved MCMC
Prior sensitivity: how much does a wrong $ℓ$ distort results? Systematic sensitivity study?
Deep-learning integration: NTK view — infinite-width NN ≡ GP; deep structured GP priors?
Real-time retrieval: sequential Bayesian update (Kalman-family) rather than batch MCMC each hour?

Sources

— code + Kawahara/Aizawa reproduction + DSCOVR experiments
Undergraduate Research @ Peking University & Caltech/UCR — project background (Yuk Yung / King-Fai Li)
Textbook (raw): Bishop PRML 2006 — Ch. 6 GP
Paper (raw): Aizawa 2020 ApJ 896 22 — Dynamic SOT
Implied papers: Kawahara & Fujii 2010, Cowan & Agol 2008
[Future expansion] Kawahara series papers, derivation details, Zhenyu’s own Chinese .docx study notes

Inverse Problems & Regularization — frequentist dual view
MCMC & Bayesian Inversion — skill page
Python Scientific Computing — emcee / scipy / healpy
Satellite Remote Sensing & Data Processing — DSCOVR data pipeline
Climate Physics & Atmospheric Science — application domain