Writing

Preprints of my work are posted on the arXiv as much as possible. Highlights include a long but comprehensive introduction to statistical computing and Hamiltonian Monte Carlo targeted at applied researches and a more theoretical treatment of the geometric foundations of Hamiltonian Monte Carlo.

I’ve also been experimenting with a nontraditional introduction to some of the more formal aspects of probability theory and statistical computation, although fair warning that this draft is long overdue for a reorganization. Still, comments are always welcome.

Recently I have been taking advantage of notebook environments such as knitr and Jupyter to develop case studies demonstrating important concepts in statistical workflow and modeling with Stan. If you’re new to statistical inference then I recommend starting with Probability Theory (For Scientists and Engineers), moving on to Robust Statistical Workflow with RStan, and then taking a look at A Principled Bayesian Workflow.

A Principled Bayesian Workflow

Given the posterior distribution derived from a probabilistic model and a particular observation, Bayesian inference is straightforward to implement: inferences, and any decisions based upon them, follow immediately in the form of posterior expectations. Building such a probabilistic model that is satisfactory in a given application, however, is a far more open-ended challenge. In order to ensure robust analyses we need a principled workflow that guides the development of a probabilistic model that is consistent with both our domain expertise and any observed data while also being amenable to accurate computation. In this case study I introduce a principled workflow for building and evaluating probabilistic models in Bayesian inference.

View (HTML)
betanalpha/knitr_case_studies/principled_bayesian_workflow (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2018). A Principled Bayesian Workflow. Retrieved from https://betanalpha.github.io/assets/case_studies/principled_bayesian_workflow.html.

Probability Theory (For Scientists and Engineers)

Formal probability theory is a rich and complex field of mathematics with a reputation for being confusing if not outright impenetrable. Much of that intimidation, however, is due not to the abstract mathematics but rather how they are employed in practice. In this case study I attempt to untangle this pedagogical knot to illuminate the basic concepts and manipulations of probability theory and how they can be implemented in practice. Our ultimate goal is to demystify what we can calculate in probability theory and how we can perform those calculations in practice.

View (HTML)
betanalpha/knitr_case_studies/probability_theory (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2018). Probability Theory (For Scientists and Engineers). Retrieved from https://betanalpha.github.io/assets/case_studies/probability_theory.html.

Bayes Sparse Regression

In this case study I’ll review how sparsity arises in frequentist and Bayesian analyses and discuss the often subtle challenges in implementing sparsity in practical Bayesian analyses.

View (HTML)
betanalpha/knitr_case_studies/bayes_sparse_regression (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2018). Bayes Sparse Regression. Retrieved from https://betanalpha.github.io/assets/case_studies/bayes_sparse_regression.html.

Fitting the Cauchy

In this case study I review various ways of implementing the Cauchy distribution, from the nominal implementation to alternative implementations aimed at ameliorating these difficulties, and demonstrate their relative performance.

View (HTML)
betanalpha/knitr_case_studies/fitting_the_cauchy (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2018). Fitting the Cauchy. Retrieved from https://betanalpha.github.io/assets/case_studies/fitting_the_cauchy.html.

Robust Gaussian Processes in Stan, Part 3

This series of case studies introduces the very basics of Gaussian processes, how to implement them in Stan, and how they can be robustly incorporated into Bayesian models to avoid subtle but pathological behavior.

View (HTML)
betanalpha/knitr_case_studies/gaussian_processes/gp_part3 (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2017). Robust Gaussian Processes in Stan, Part 3. Retrieved from https://betanalpha.github.io/assets/case_studies/gp_part3/part3.html.

Robust Gaussian Processes in Stan, Part 2

This series of case studies introduces the very basics of Gaussian processes, how to implement them in Stan, and how they can be robustly incorporated into Bayesian models to avoid subtle but pathological behavior.

View (HTML)
betanalpha/knitr_case_studies/gaussian_processes/gp_part2 (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2017). Robust Gaussian Processes in Stan, Part 2. Retrieved from https://betanalpha.github.io/assets/case_studies/gp_part2/part2.html.

Robust Gaussian Processes in Stan, Part 1

This series of case studies introduces the very basics of Gaussian processes, how to implement them in Stan, and how they can be robustly incorporated into Bayesian models to avoid subtle but pathological behavior.

View (HTML)
betanalpha/knitr_case_studies/gaussian_processes/gp_part1 (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2017). Robust Gaussian Processes in Stan, Part 1. Retrieved from https://betanalpha.github.io/assets/case_studies/gp_part1/part1.html.

The QR Decomposition for Regression Models

This case study reviews the QR decomposition, a technique for decorrelating covariates and, consequently, the resulting posterior distribution in regression models.

View (HTML)
betanalpha/knitr_case_studies/qr_regression (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2017). The QR Decomposition for Regression Models. Retrieved from https://betanalpha.github.io/assets/case_studies/qr_regression.html.

Robust PyStan Workflow

This case study demonstrates a proper PyStan workflow that ensures robust inferences with the default dynamic Hamiltonian Monte Carlo algorithm.

View (HTML)
betanalpha/jupyter_case_studies/pystan_workflow (GitHub)
Dependences: Python, Jupyter, PyStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2017). Robust PyStan Workflow. Retrieved from https://betanalpha.github.io/assets/case_studies/pystan_workflow.html.

Robust RStan Workflow

This case study demonstrates a proper RStan workflow that ensures robust inferences with the default dynamic Hamiltonian Monte Carlo algorithm.

View (HTML)
betanalpha/knitr_case_studies/rstan_workflow (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2017). Robust RStan Workflow. Retrieved from https://betanalpha.github.io/assets/case_studies/rstan_workflow.html.

Diagnosing Biased Inference with Divergences

This case study discusses the subtleties of accurate Markov chain Monte Carlo estimation and how divergences can be used to identify biased estimation in practice.

View (HTML)
betanalpha/knitr_case_studies/divergences_and_bias (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2017). Diagnosing Biased Inference with Divergences. Retrieved from https://betanalpha.github.io/assets/case_studies/divergences_and_bias.html.

Identifying Bayesian Mixture Models

This case study discusses the common pathologies of Bayesian mixture models as well as some strategies for identifying and overcoming them.

View (HTML)
betanalpha/knitr_case_studies/identifying_mixture_models (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2017). Identifying Bayesian Mixture Models. Retrieved from https://betanalpha.github.io/assets/case_studies/identifying_mixture_models.html.

How the Shape of a Weakly Informative Prior Affects Inferences

This case study reviews the basics of weakly-informative priors and how the choice of a specific shape of such a prior affects the resulting posterior distribution.

View (HTML)
betanalpha/knitr_case_studies/weakly_informative_shapes (GitHub)
Dependences: R, knitr, RStan
Code License: BSD (3 clause)
Text License: CC BY-NC 4.0
Cite As: Betancourt, Michael (2017). How the Shape of a Weakly Informative Prior Affects Inferences. Retrieved from https://betanalpha.github.io/assets/case_studies/weakly_informative_shapes.html.