Simon Leglaive
Assistant Professor
CentraleSupélec, IETR (UMR CNRS 6164)
simon.leglaive@centralesupelec.fr
+33 (0)2 99 84 45 82
CentraleSupélec - Rennes Campus
Avenue de la Boulaie
CS 47601
F-35576 Cesson-Sévigné Cedex
Google Scholar
Twitter
Bayesian methods for machine learning
This page includes the material of the "Bayesian methods for machine learning" elective course for 2nd-year engineering students at CentraleSupélec.
Agenda
| Lecture | Fundamentals of Bayesian modeling and inference |
| Lecture | Fundamentals of machine learning |
| Lecture | Bayesian networks and inference in latent variable models |
| Lecture | Factor analysis |
| Practical | Gaussian mixture model |
| Lecture | Variational inference |
| Practical | Bayesian linear regression |
| Lecture | Markov chain Monte Carlo |
| Practical | Sparse Bayesian linear regression |
| Lecture | Deep generative models |
| Lecture | Revisions |
Fundamentals of Bayesian modeling and inference
Key concepts:
- Latent and observed variables
- Modeling, inference, learning
- Prior, likelihood, marginal likelihood and posterior
- Point estimate, predictive prior, predictive posterior
- Gaussian model with latent mean / variance
Material:
- Slides
- Exercices
- Handwritten solution to exercices
- Jupyter notebook - Gaussian model with latent mean
Fundamentals of machine learning
Key concepts:
- Supervised learning
- Empirical risk minimization
- Underfitting and overfitting
- Bias-variance trade-off
- Maximum likelihood estimation
- Multinomial logistic regression
Material:
Bayesian networks and inference in latent variable models
Key concepts:
- Bayesian networks
- Conditional independence
- D-separation
- Evidence lower-bound
- Expectation-maximization algorithm
Material:
Factor analysis
Key concepts:
- Modeling, inference, and learning with the EM algorithm for factor analysis
- The multivariate Gaussian distribution
Material:
Gaussian mixture model
Material:
Variational inference
Key concepts:
- The problem of intractable posterior
- Kullback-Leibler divergence
- Variational inference
- Mean-field approximation
Material:
- Slides
- Jupyter notebook - Variational inference for the Gaussian model with latent parameters
- Solution variational inference with the mean-field approximation
- Handwritten solution to exercices
Bayesian linear regression
Material:
Markov chain Monte Carlo
Key concepts:
- The Monte Carlo method
- Sampling methods
- Markov chains
- Markov chain Monte Carlo methods
Material:
Sparse Bayesian linear regression
Material:
Deep generative models
Key concepts:
- Deep generative modeling
- Variational autoencoder
- Diffusion models
Material: