Probabilistic Machine Learning and Generalization Theory in Modern Deep Learning

This opportunity focuses on probabilistic machine learning for understanding and improving generalization in modern deep learning. It brings together several recent contributions in distribution-dependent generalization bounds, PAC-Bayesian theory, large-deviation ideas, and Bayesian inference for neural networks, as developed in my recent work.

Based on our research on distribution-dependent PAC-Chernoff bounds , we study how rate functions and smoothness—core objects from Large Deviation Theory—provide an oracle perspective on why certain interpolating models generalize exceptionally well while others fail. This theory explains the role of ℓ₂-regularization, distance-from-initialization, data augmentation, invariant architectures, and over-parameterization in navigating toward smoother models.

Complementing this, our work on PAC-Bayes–Chernoff bounds for unbounded losses extends classical Cramér-Chernoff ideas to randomized predictors and Gibbs posteriors, enabling tighter and more model-dependent control of generalization. These bounds naturally exploit differences in cumulant generating functions (CGFs) across models inside the same hypothesis class—revealing why some deep networks are statistically more stable than others.

On the Bayesian side, our analysis of the Cold Posterior Effect (CPE) provides a probabilistic explanation of how tempered posteriors (T<1) mitigate underfitting caused by likelihood or prior misspecification. This connects Bayesian deep learning, probabilistic inference, and temperature-based regularization through a unified probabilistic lens.

Possible collaboration topics include:
• Distribution-dependent generalization bounds for deep or probabilistic models
• Rate functions, Cramér transforms, and smoothness measures
• PAC-Bayes analysis of deep networks and unbounded losses
• Bayesian deep learning, tempered posteriors, and underfitting diagnostics
• Uncertainty quantification in high-dimensional or over-parameterized models
• Theoretical foundations of SGD, implicit regularization, and interpolation
• Applications in vision, NLP, spatio-temporal modelling, or scientific ML

This opportunity suits fellows interested in: theoretical ML, probability theory, Bayesian methods, generalization, statistical learning, or the mathematics behind modern AI.