Cardiff Mathematics of Deep Learning reading group

The Mathematics of Deep Learning reading group ran in the academic year 2022-2023 at Cardiff University School of Mathematics. A variety of topics related to machine learning were covered, from rigorous mathematical studies to contemporary methods and models.

Foundations of Statistical Learning, Alexei Stepanenko
2nd November 2022
Reference(s):
  1. Berner, Julius, et al. "The modern mathematics of deep learning." (2021)

Neural Tangent Kernels, Bertrand Gauthier
(Part 1) 09 November 2022
(Part 2) 23 November 2022
Reference(s):
  1. Jacot, Arthur, Franck Gabriel, and Clément Hongler. "Neural tangent kernel: Convergence and generalization in neural networks." Advances in neural information processing systems 31 (2018)

Neural Sheaf Diffusion, Álvaro Torras Casas
(Part 1) 07 December 2022
(Part 2) 12 December 2022
Reference(s):
  1. Bodnar, Cristian, et al. "Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in gnns." Advances in Neural Information Processing Systems 35 (2022): 18527-18541
  2. T. N. Kipf, M. Welling. Semi-Supervised Classification with Graph Convolutional Networks. ICLR (2017)

Probabilistic neural networks and PAC-Bayes bounds, Alexei Stepanenko
21st April 2023
Reference(s):
  1. Pérez-Ortiz, María, et al. "Tighter risk certificates for neural networks." The Journal of Machine Learning Research 22.1 (2021): 10326-10365
  2. Alquier, Pierre. "User-friendly introduction to PAC-Bayes bounds." (2021) 10326-10365

Self-Attention and Transformers, Alexei Stepanenko
5th May 2023
Reference(s):
  1. Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017)

The Adam optimisation algorithm, Matthew Hutchings
28th June 2023
Reference(s):
  1. Kingma, Diederik P., and Jimmy Ba. "Adam: A method for stochastic optimization." (2014)