Bayesian Model Comparison Using Cross-Validation
Workshop 13 - Winter Semester 2025-26
Overview
This workshop introduces Bayesian model comparison techniques using cross-validation methods for robust model selection.
Learning Objectives
- Understand cross-validation principles in Bayesian context
- Apply LOO-CV and WAIC for model comparison
- Interpret model comparison metrics
- Implement techniques in R with brms
Topics Covered
- Model comparison fundamentals
- Leave-one-out cross-validation (LOO-CV)
- Widely applicable information criterion (WAIC)
- Practical implementation in R
Prerequisites
- Basic knowledge of Bayesian statistics
- Familiarity with R programming
- Understanding of regression models
Materials
We are using the same repository as last time: brms-ws.
The main notebook for this session is on model comparison with cross-validation: - View the rendered notebook: 05_loo.html
We will be using examples from two online books, specifically the chapters on the loo_compare() function:
- Kurz, A. S. (2023). Statistical rethinking with brms, ggplot2, and the tidyverse: Second edition. https://bookdown.org/content/4857/ (chapter 7 "Ulysses' Compass" is a good introduction)
- Nicenboim, B., Schad, D. and Vasishth, S. (2025). Introduction to Bayesian Data Analysis for Cognitive Science. https://bruno.nicenboim.me/bayescogsci/ (chapter 14 "Cross-validation")
Additional materials will be available after the workshop.
Instructor
Job Schepens
Project S, SFB 1252
University of Cologne
Session Details
Date: 17 December 2025
Time: 14:00 - 15:30
Location: House of Prominence, Attic, Luxemburger Str. 299, Cologne