Bayesian Modeling and Probabilistic Programming
Bayesian modeling treats every unknown quantity as a probability distribution and updates those distributions as new evidence arrives. The output is not a single best estimate but a full description of what we believe and how confidently we believe it. That makes Bayesian methods the natural fit when decisions matter, when data are sparse, and when uncertainty itself is part of the answer.
Probabilistic programming is the engineering discipline that makes Bayesian models writable as code. A probabilistic programming language lets us declare random variables, encode their relationships, and hand the resulting model to a sampler or variational engine that returns the posterior. The interpreter, not the modeller, handles the inference algebra. This separation has done for Bayesian inference what TensorFlow and PyTorch did for deep learning: it has moved the bottleneck from mathematics to model design.
The connection to Causal Inference and Causal Machine Learning is direct. Causal questions are interventional, not predictive. Probabilistic models naturally express the causal structure (which variables influence which others), and Bayesian methods carry uncertainty about the causal effect through the analysis rather than collapsing it to a point estimate. Tools like CausalPy bring the Bayesian and causal stacks together, so a practitioner can write a difference-in-differences or synthetic-control analysis as a generative model and read off both the effect and its uncertainty in one workflow.
The practical reference stack in 2026 is built around PyMC. The core library is open source and battle-tested, the consulting work of PyMC Labs anchors industry adoption, and CausalPy provides the causal-inference layer on top.
Frameworks and tools
- PyMC — Python probabilistic programming library, the de facto Bayesian modelling stack
- CausalPy — PyMC-backed causal inference toolkit (synthetic control, regression discontinuity, difference-in-differences, instrumental variables)
- PyMC Labs — consulting and applied research group behind PyMC; useful as a window into how Bayesian methods are being used commercially
Related notes
- Probabilistic Machine Learning — the broader ML lineage that uses probability theory
- Bayesian Optimisation — a specific Bayesian technique applied to expensive black-box optimisation problems
- Causal Inference — the methodology this work plugs into
- Causal Machine Learning — the ML-flavoured intersection of causal and predictive modelling
- Counterfactual Inference — the “what would have happened” question Bayesian causal methods answer