Jean Barbier

Physics of learning + Mathematics of high dimensions

Research

We use the statistical physics of disordered systems, random matrix theory and information theory to sharply describe inference and learning in high dimension. Below is a selection of papers. The full list is on Google Scholar. Group members are highlighted.

Deep neural networks beyond the linear regime

Most exact theories of neural networks either linearize them or take a width so large that features barely move. We go beyond by studying networks trained in the practical regime where the number of samples is comparable to their number of parameters. There we characterize when and how they learn features through sharp specialization transitions.

Training, fine-tuning and representations

Optimal-performance results say what is possible, not what training finds. We also study what gradient-based training does, when pre-training helps or hurts, or how noise shapes the information stored in learned representations.

Matrix inference and random matrices

Many inference and learning problems reduce to recovering a large matrix from noisy observations. We compute their information-theoretic limits, including in the hard regime where the rank grows with the dimension or rotational invariance is lost, and design algorithms that reach them.

Rigorous foundations of high-dimensional inference

Much of the statistical physics we use rests on non-rigorous methods. A long-standing line of our work turns replica predictions into theorems: adaptive interpolation methods, concentration, and rigorous analyses of approximate message passing algorithms, for estimation, generalized linear models and time series.

Coding and communication

Part of our earlier work is on error-correcting codes, and in particular sparse superposition codes for communication over noisy channels. Using statistical physics and then proofs of threshold saturation, we showed that these codes, with spatial coupling and approximate message-passing decoding, achieve capacity on any memoryless channel. More recently we studied structured coding matrices beyond the i.i.d. Gaussian ensemble.

Combinatorial optimization

Statistical physics also describes hard combinatorial optimization problems on random instances, such as finding a maximum independent set in a random graph. Combining rigorous large-graph asymptotics with numerics, we find that algorithms do reach the limits predicted by asymptotic theory, but can converge to them very slowly: at the sizes met in practice, local algorithms do much better than these limits suggest.

For a wider audience