Jean Barbier

Physics of learning + Math of high dimensions

Specialization in deep networks

Research 

We use the statistical physics of disordered systems, random matrix theory and information theory to sharply describe inference and learning in high dimension.

Highlights

Informal accounts of some of the group’s milestones.

For a wider audience

Introductions for non-specialists.

Selected papers

Selection of papers. The full list is on Scholar. Group members are highlighted.

Deep neural networks beyond the linear regime

Most exact theories of neural networks either linearize them or take a width so large that features barely move. We go beyond by studying networks trained in the practical regime where the number of samples is comparable to their number of parameters. There we characterize when and how they learn features through sharp specialization transitions, and how this shapes their generalization.

Training, fine-tuning and representations

Optimal-performance results say what is possible, not what training finds. We also study what gradient-based training does, when pre-training helps or hurts, or how noise shapes the information stored in learned representations.

Matrix inference and random matrices

Many inference and learning problems reduce to recovering a large matrix from noisy observations. We compute their information-theoretic limits, including in the hard regime where the rank grows with the dimension or rotational invariance is lost, and design algorithms that reach them.

Rigorous foundations of high-dimensional inference

Much of the statistical physics we use rests on non-rigorous methods. A long-standing line of our work turns replica predictions into theorems: adaptive interpolation methods, concentration, and rigorous analyses of approximate message passing algorithms, for estimation, generalized linear models and time series.

  • Information-theoretic limits and approximate message-passing for high-dimensional time series

    Daria Tieplova, Samriddha Lahiry, Jean Barbier

    IEEE Transactions on Information Theory, 2026; IEEE International Symposium on Information Theory, 2025 arXivJournal

  • Performance of Bayesian linear regression in a model with mismatch

    Jean Barbier, Wei-Kuo Chen, Dmitry Panchenko, Manuel Sáenz

    Information and Inference: A Journal of the IMA, 2025 arXivJournal

  • Strong replica symmetry in high-dimensional optimal Bayesian inference

    Jean Barbier, Dmitry Panchenko

    Communications in Mathematical Physics, 2022 arXivJournal

  • Overlap matrix concentration in optimal Bayesian inference

    Jean Barbier

    Information and Inference: A Journal of the IMA, 2021 arXivJournal

  • Mutual information and optimality of approximate message-passing in random linear estimation

    Jean Barbier, Nicolas Macris, Mohamad Dia, Florent Krzakala

    IEEE Transactions on Information Theory, 2020 arXivJournal

  • Optimal errors and phase transitions in high-dimensional generalized linear models

    Jean Barbier, Florent Krzakala, Nicolas Macris, Léo Miolane, Lenka Zdeborová

    Proceedings of the National Academy of Sciences, 2019; Conference on Learning Theory, 2018 arXivJournalICTP news

  • The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference

    Jean Barbier, Nicolas Macris

    Probability Theory and Related Fields, 2019 arXivJournal

Coding and communication

Part of our earlier work is on error-correcting codes, and in particular sparse superposition codes for communication over noisy channels. Using statistical physics and then proofs of threshold saturation, we showed that these codes, with spatial coupling and approximate message-passing decoding, achieve capacity on any memoryless channel. More recently we studied structured coding matrices beyond the i.i.d. Gaussian ensemble.

Combinatorial optimization

Statistical physics also describes hard combinatorial optimization problems on random instances, such as finding a maximum independent set in a random graph. Combining rigorous large-graph asymptotics with numerics, we find that algorithms do reach the limits predicted by asymptotic theory, but can converge to them very slowly: at the sizes met in practice, local algorithms do much better than these limits suggest.