SMLP 2026: Summer School Recap & Discussion

Overview

Session Agenda

SMLP
Participants
Keynotes
5 Classes
“Maximal Models”
Contrast Coding
CRC 1252

1. SMLP

10-Years

Founded by Shravan Vasishth (Uni Potsdam, 2017).
Origin: Funded as part of SFB 1287 (Limits of Variability in Language).
Target audience: cognitive scientists and linguists.
Impact: Over 1,000 participants over the years.

2026 Edition

5 parallel streams.
Keynotes: Lisa DeBruine & Dale Barr.
Many CRC 1252 Participants:

Who Was Where?

Which stream did you attend?

Track 1: Intro to Bayesian (brms)
Track 2: Advanced Bayesian (Stan)
Track 3: Intro to R & Frequentist Stats
Track 4: Foundational LMMs (lme4)
Track 5: Advanced LMMs with Julia

Keynotes attended?

Lisa DeBruine (Power & Simulation)
Dale Barr (PsyTeachR & Open Science)

Primary data in your CRC project:

RTs (eye-tracking, Self paced reading, etc.)
Acoustic measurements (f0, duration, intensity)
Acceptability judgments (binary, Likert, ordinal)
Corpus frequencies & counts (prominence markers)
Electrophysiology (EEG, ERPs, time-series)

2. Keynote 1: Lisa DeBruine

Power and Simulation

Analytic Power Fails: Classical formulas (e.g. $G*\text{Power}$) assume independent data. They break down for crossed random effects (subjects $\times$ items) (DeBruine & Barr, 2021).
Post-Hoc Power Fallacy: Calculating power after the fact from observed effect size is just a noisy transformation of the $p$-value (Vasishth et al., 2018).
The Modern Solution: Simulation-based power analysis.

Core Principle

“If you can simulate a dataset that captures your design and hypothesized effect, you can calculate power for any model, no matter how complex.”

Open Resources:
- faux R package (factorial data simulation)
- data-sim-workshops

Simulation-Based Power Pipeline

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '10px'}}}%%
flowchart TD
    A["Step 1: Specify Generative Model<br/>(Effects, SDs, Correlations)"] --> B["Step 2: Generate N Datasets<br/>(Synthetic Trials, Subjects, Items)"]
    B --> C["Step 3: Fit Model to Each Dataset<br/>(lme4 / brms / MixedModels)"]
    C --> D["Step 4: Evaluate Evidence Rate<br/>(Rate reaching p &lt; .05 or ROPE)"]
    D -->|Refine N / Design| A

Relevance for CRC 1252:

Preregistration: Justifying sample sizes rather than arbitrary $N=30$.
Random effects: Simulations reveal how item variance and random slopes diminish power for subtle prominence interactions.

Previous CRC Workshop

Detailed in Workshop 18: Power Analysis & the interactive Sequential Simulation Report.

3. Keynote 2: Dale Barr

4. Track 1: Intro to Bayesian Analysis

Instructors: Julia Haaf & Nicole Cruz

Book: Bayesian Data Analysis for Cognitive Science (Nicenboim et al., 2024).
Workflow: Prior $\to$ Likelihood $\to$ Posterior.

Syntax: Familiar lme4-style syntax via brms:

brm(rt ~ prominence * focus + 
      (1 + prominence | subj) + 
      (1 + focus | item),
    data = dat, family = lognormal(),
    prior = priors)

Model Validation Workflow

Prior Checks: Plausible RTs before fitting (02_prior_checks).
Diagnostics: $\hat{R} \le 1.01$, $\text{ESS} > 400$ (08_convergence).
Posterior Checks: Test fit with pp_check (03_posterior_checks).
Inference: Credible intervals and ROPE (06_rope · Workshop 14).

Track 1: Regularizing Priors

Prior Specification in `brms`

priors <- c(
  # Baseline intercept ~ 400 ms
  prior(normal(6, 0.5), class = "Intercept"),
  
  # Plausible effect size bounds
  prior(normal(0, 0.1), class = "b"),
  
  # Regularize random slope variances
  prior(exponential(2), class = "sd"),
  
  # Discourage extreme +/- 1 correlations
  prior(lkj(2), class = "cor")
)

Why Priors help

Singularity in lme4: estimated correlations between intercepts and slopes often get to $\pm 1.00$.
The LKJ Prior: Setting $\text{LKJ}(\eta = 2)$ concentrates probability mass around 0, shrinking extreme correlations and stabilizing estimation (Nicenboim et al., 2024).

Prior Sensitivity Tutorial

Compare flat vs. regularizing priors in brms-ws 04_comparing_priors.

5. Track 2: Advanced Bayesian Modeling

Instructors: Bruno Nicenboim & Johan Hennert

Beyond brms: Writing custom probabilistic programs in native Stan.
Cognitive Process Models:
- Drift-Diffusion Models (DDM): Jointly model decision bounds, drift rate, and non-decision time.
- Log-Normal Race Models: Competing accumulators for lexical or syntactic selection.

Principled Model Comparison

Bayes Factors (Bridge Sampling):
- Tests point nulls ($H_0: \theta = 0$). Prior sensitive!
- See Workshop 15: Bayes Factors & 07_bayes_factors.
PSIS-LOO-CV (loo package):
- Out-of-sample predictive accuracy.
- See Workshop 13: Cross-Validation & 05_loo.

Decision Frameworks Comparison

Explore ROPE vs. Bayes Factor vs. LOO.

6. Track 4: Frequentist Foundations & LMMs

“Keep It Maximal” (Barr et al., 2013)

Include all random intercepts and slopes for within-unit experimental factors.
Goal: Rigorously protect against inflated Type I error ($\alpha > .05$).
Trade-off: Frequently results in overparameterization, singular fits, and optimizer convergence errors.

“Parsimonious Models” (Matuschek et al., 2017)

Select random effects via backward elimination with Likelihood Ratio Tests (LRT).
Finding: Over-parameterized maximal models cause substantial loss of power.
Recommendation: Keep only random slopes supported by the empirical data.

Contrast Coding

R Default (Treatment Coding)

# Default R: contr.treatment
fit <- lmer(RT ~ Prominence * Focus + 
              (1 | Subj), data = dat)

Prominence coefficient is NOT the main effect of prominence; it is the difference between the reference level and the other level.
Interactions make the interpretation of lower-order terms more complex (Schad et al., 2020).

Sum Coding

# Sum contrasts
contrasts(dat$Prominence) <- contr.sum(2)
contrasts(dat$Focus) <- contr.sum(2)

Intercept ($\beta_0$) = Grand mean.
Slopes ($\beta_1, \beta_2$) = True main effects averaged across all levels of the other factors.

Contrast Coding Overview

Contrast Scheme	R Command	Intercept ($\beta_0$)	Slope ($\beta_1, \beta_2, \dots$)
Treatment (Default)	`contr.treatment(k)`	Reference group mean	Difference: Group $i$ minus Reference
Sum	`contr.sum(k)`	Grand mean across groups	Deviation: Group $i$ minus Grand mean
Successive Difference	`MASS::contr.sdif(k)`	Grand mean across groups	Difference between adjacent successive levels ($\mu_i - \mu_{i-1}$)
Helmert	`contr.helmert(k)`	Grand mean across groups	Level $i$ compared to mean of prior levels

Good Practice

Check contrasts(dat$my_factor) before fitting models to check coding.

7. Track 5: Advanced LMMs with Julia

Instructors: Reinhold Kliegl & Phillip Alday

The Engine: MixedModels.jl (Kliegl et al., 2024).
Why Julia?
- No two-language overhead (pure Julia).
- High-performance sparse matrix algebra.
- 10x to 100x faster than lme4.
Ideal For: Massive linguistic corpora and multi-million token datasets.

Standout Capabilities

Random Effects PCA (rePCA): Diagnoses the exact mathematical dimensionality of variance components, exposing zero-variance slopes.
Fast Parametric Bootstrap: Generates 10,000 bootstrap replicates in minutes for accurate non-asymptotic CIs and $p$-values.

Interoperability: Calling Julia from R

library(JuliaCall)
julia_setup()
julia_library("MixedModels")
julia_library("DataFrames")

# Transfer R data frame to Julia in-memory
julia_assign("df", crc_data)

# Fit model at compiled C-speed
julia_command("m1 = fit(MixedModel, 
  @formula(RT ~ Prom * Ctx + (1 + Prom | Subj)), 
  df)")

# 1,000 bootstraps in seconds
boot <- julia_eval("parametricbootstrap(1000, m1)")

Seamless R/Julia Workflow

Wrangle & clean in R (tidyverse)
Fit & bootstrap in Julia (MixedModels.jl)
Plot & report in R (ggplot2)

No need to rewrite your entire project workflow in Julia!

8. Tool Comparison?

Feature	R: `lme4`	R: `brms` (Stan)	Julia: `MixedModels.jl`
Paradigm	Frequentist (REML/ML)	Bayesian (MCMC/NUTS)	Frequentist (REML/ML)
Speed	Moderate	Slow (minutes to hours)	Fast (seconds)
Singular Fits	Frequent warning	Regularized by priors	Diagnosed via `rePCA`
Distributions	Gaussian, Binomial, Poisson	Any	Continuous, Binomial, Poisson
Inference	Wald, LRT, Kenward-Roger	Credible intervals, ROPE, BF	Parametric Bootstrap
Setup Cost	Standard R	Requires C++ toolchain	Requires Julia install

Resolving Singular Fits in Practice

%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '10px'}}}%%
flowchart TD
    A["Singular Fit Warning in lme4"] --> B{"Overparameterized<br/>correlation?"}
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px', 'fontFamily': 'Albert Sans, sans-serif'}}}%%
flowchart LR
    A["Singular Fit Warning<br/>in lme4"] --> B{"Overparameterized<br/>correlation?"}
    B -->|Yes| C["Zero-Correlation Model:<br/>(1 + X || Group) in lme4"]
    B -->|Yes| D["Bayesian Prior:<br/>prior(lkj(2), class='cor') in brms"]
    A --> E{"Unsupported<br/>random slopes?"}
    E -->|Yes| F["Parsimonious Model:<br/>Backward LRT selection"]
    E -->|Inspect| G["Julia rePCA():<br/>Diagnose zero-variance axes"]

Practical Decision Guide

Don’t ignore it: A singular fit means estimates are on the boundary of parameter space.
Correlation near $\pm 1$: Use || syntax in lme4 or lkj(2) in brms.
Variance near 0: Drop unsupported slopes to recover power.

Testing Null Effects

For determining if an effect is practically absent, see Workshop 14: ROPE Testing & 06_rope.

Shared Bayesian Core: SMLP Book vs. RDM Workshops

The foundational brms methods taught at SMLP directly mirror our SFB 1252 RDM series:

Core Bayesian Topic	SMLP Book Chapters (Nicenboim et al.)	Relevant SFB 1252 RDM Workshops & Notebooks
Bayesian Workflow & Priors	Ch. 4: Regression Models · Ch. 5: Hierarchical	Workshop 11: Regression with brms · 01_setting_priors
Prior & Posterior Checks	Ch. 4: Prior Checks (§4.2) · Ch. 4: Posterior Checks (§4.4)	Workshop 11 · 02_prior_checks · 03_posterior_checks
MCMC Convergence & Diagnostics	Ch. 3: Computational BDA (§3.3) ($\hat{R}$, ESS, divergences)	Workshop 11 · 08_convergence
Model Comparison (LOO-CV)	Ch. 12: Comparison · Ch. 14: Cross-Validation	Workshop 13: Cross-Validation · 05_loo
Bayes Factors & Point Hypotheses	Ch. 13: Bayes Factors (§13.1) (Bridge sampling / Savage-Dickey)	Workshop 15: Bayes Factors · 07_bayes_factors
Simulation-Based Power	Ch. 3: Power & Sample Size (§3.5) (Generative models)	Workshop 18: Power Analysis · Sequential Simulation Report

SMLP Bayesian Topics Not in RDM Yet

SMLP covers advanced response families, custom syntax, and cognitive process models:

Advanced `brms` Distributions & Syntax

Shifted Lognormal & Ex-Gaussian RTs:
- Estimate non-decision time ($ndt$) alongside mean/variance: family = shifted_lognormal() (Book Ch. 7).
Ordinal & Cumulative Models:
- Directly model Likert / acceptability scales: family = cumulative("logit").
Non-Linear Syntax (bf()):
- Fit non-linear curves & saturation functions.
Missing Data Imputation:
- Model missing predictors via brm(y ~ mi(x) + ...).

From `brms` to Stan & Cognitive Models

Inspecting Generated Stan Code:
- make_stancode() and make_standata() to inspect what brms compiles (Book Ch. 8).
Custom Families (custom_family()):
- Define user-specified likelihoods in brms.
Computational Cognitive Models (Track 2):
- Drift-Diffusion Models (DDM): Joint choice & RT.
- Race Models: Competing accumulators (Book Ch. 18).
- Mixture Models: Latent subpopulations (Book Ch. 17).

Future Bayesian Topics for Project S

RDM Bayesian Topics Not Covered in SMLP

Our local RDM series emphasizes practical equivalence testing, sequential designs, and post-processing tools:

The ROPE Decision Framework

Practical Equivalence Testing:
- Evaluate whether effects are practically zero, negligible, or present (Workshop 14: ROPE Testing).
HDI + ROPE Decision Rules:
- Three-way verdicts (PRESENT, ABSENT, UNDECIDED) with bayestestR (06_rope).
Defensible Boundaries:
- Setting domain-justified equivalence limits in linguistics rather than relying only on point nulls.

Sequential Designs & Post-Processing

Benchmarking Decision Frameworks:
- Direct side-by-side comparison of ROPE vs. Bayes Factor vs. LOO-CV in sequential testing (Workshop 15, 06_sequential_testing).
Post-Hoc Contrasts with emmeans:
- Pairwise comparisons and marginal predictions on brms fits with emmeans and marginaleffects.
Containerized brms Workflows:
- Ready-to-run Docker & Ramses HPC setups in brms-ws to bypass local C++ compiler setup.

Complementary Bayesian Perspectives

SMLP focuses on Bayes Factors; RDM focuses on practical equivalence (ROPE)

References

Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. https://doi.org/10.1016/j.jml.2012.11.001

DeBruine, L. M., & Barr, D. J. (2021). Understanding mixed-effects models through data simulation. Advances in Methods and Practices in Psychological Science, 4(1), 2515245920965119. https://doi.org/10.1177/2515245920965119

Kliegl, R., Alday, P. M., & Bates, D. (2024). Embrace uncertainty: Mixed-effects models with julia. Online book. https://www.embraceuncertaintybook.com/

Matuschek, H., Kliegl, R., Vasishth, S., Baayen, H., & Bates, D. (2017). Balancing type i error and power in linear mixed models. Journal of Memory and Language, 94, 305–315. https://doi.org/10.1016/j.jml.2017.01.001

Nicenboim, B., Schad, D. J., & Vasishth, S. (2024). An introduction to bayesian data analysis for cognitive science. Chapman; Hall/CRC. https://vasishth.github.io/bayescogsci/

Schad, D. J., Vasishth, S., Hohenstein, S., & Kliegl, R. (2020). How to capitalize on a priori contrasts in linear (mixed) models: A tutorial. Journal of Memory and Language, 110, 104038. https://doi.org/10.1016/j.jml.2019.104038

Vasishth, S., Mertzen, D., Jäger, L. A., & Gelman, A. (2018). The statistical significance filter leads to overoptimistic expectations of replicability. Journal of Memory and Language, 103, 151–175. https://doi.org/10.1016/j.jml.2018.07.004

SMLP 2026: Summer School Recap & Discussion

Overview

Session Agenda

1. SMLP

10-Years

2026 Edition

Who Was Where?

Which stream did you attend?

Keynotes attended?

Primary data in your CRC project:

2. Keynote 1: Lisa DeBruine

Power and Simulation

Simulation-Based Power Pipeline

Relevance for CRC 1252:

3. Keynote 2: Dale Barr

4. Track 1: Intro to Bayesian Analysis

Instructors: Julia Haaf & Nicole Cruz

Model Validation Workflow

Track 1: Regularizing Priors

Prior Specification in brms

Why Priors help

5. Track 2: Advanced Bayesian Modeling

Instructors: Bruno Nicenboim & Johan Hennert

Principled Model Comparison

6. Track 4: Frequentist Foundations & LMMs

“Keep It Maximal” (Barr et al., 2013)

“Parsimonious Models” (Matuschek et al., 2017)

Contrast Coding

R Default (Treatment Coding)

Sum Coding

Contrast Coding Overview

7. Track 5: Advanced LMMs with Julia

Instructors: Reinhold Kliegl & Phillip Alday

Standout Capabilities

Interoperability: Calling Julia from R

8. Tool Comparison?

Resolving Singular Fits in Practice

Practical Decision Guide

Shared Bayesian Core: SMLP Book vs. RDM Workshops

SMLP Bayesian Topics Not in RDM Yet

Advanced brms Distributions & Syntax

From brms to Stan & Cognitive Models

RDM Bayesian Topics Not Covered in SMLP

The ROPE Decision Framework

Sequential Designs & Post-Processing

References

Prior Specification in `brms`

Advanced `brms` Distributions & Syntax

From `brms` to Stan & Cognitive Models