Monte Carlo Methods

Spring 2026

EN.553.433/633 at Johns Hopkins: Monte Carlo estimation, sampling, MCMC and their applications.
Published

January 20, 2026

Syllabus
(PDF)
Course notes
the online textbook for the course
Class notes
(OneNote, written in class)
Final project
guidelines, project ideas and past projects

About the course

Computer simulation and related Monte Carlo methods are widely used in engineering and scientific work. Simulation provides a powerful tool for the analysis of real-world systems when they are not amenable to traditional analytical approaches. This course introduces upper-level undergraduates and graduate students to fundamental tools for designing, conducting and interpreting Monte Carlo studies.

This is a rigorous mathematics course. The focus is on the mathematical analysis of algorithms: their theoretical foundations, convergence properties and error analysis. Programming assignments, submitted as Jupyter notebooks, are an essential component, but this is not a programming course.

Prerequisites: linear algebra, and probability and some statistics at the advanced undergraduate level; some programming experience in Python.

Recommended textbooks

  • R. Y. Rubinstein and D. P. Kroese, Simulation and the Monte Carlo Method, 3rd edition, Wiley, 2017.
  • R. P. Dobrow, Introduction to Stochastic Processes With R, Wiley.

Course outline

The course has four units. Each week has a homework assignment and a short quiz.

  1. Monte Carlo estimation
  2. Sampling techniques
  3. Markov chain Monte Carlo
  4. Applications

The semester ends with a group final project in place of a final exam.

Final project

In place of a final exam, students complete a final project in groups of two or three. Every project balances three elements: learning a method or application beyond the syllabus, analysis that tests its boundaries, and research, a creative modification of the model or algorithm that is then evaluated.

Timeline
Milestone When
In-class group work sessions April 21 and 23
Project proposal End of class on April 23
Presentations May 11
Final submission May 11

During the two in-class work sessions I give an overview of the deliverables, answer questions about the requirements, and hand out templates and rubrics for the proposal. Each group develops its proposal, splits up roles and responsibilities, and fills in a check-in sheet summarizing its discussion. The proposal is a concrete project plan with each member’s responsibilities and tasks.

Presentation

Slides are assessed on four criteria:

  • Narrative clarity: does the presentation tell a clear, logical story that assumes only the knowledge covered in the course?
  • The “delta”: is the group’s specific modification or research contribution clearly identified and explained?
  • Visual communication: are the slides clean and professional, using effective visualizations rather than walls of text?
  • Technical command: does the group show a solid understanding of its methods, stay within the 15-minute limit and answer questions well?

The final presentation is a chance to share findings with the community built over the semester. The criteria keep everyone accountable, but the goal is a low-stress setting: if you have done the work and can explain it clearly, you will do well.

Project archive

The final submission is a zip file containing a Jupyter notebook and everything it needs to run. The notebook is the heart of the project: it is where the technical rigor and the creative “delta” live. It must run from top to bottom in a clean environment (with numpy, scipy, matplotlib and seaborn installed, plus any packages listed in a requirements.txt or environment.yml), use relative paths, and include all of its data files. It is assessed on:

  • Reproducibility: the notebook executes from top to bottom in a clean environment without manual troubleshooting.
  • Literate programming: Markdown cells give a clear narrative explaining the mathematical intuition behind the code.
  • Algorithmic correctness: the methods and modifications are technically sound and mathematically accurate.
  • The “delta”: the creative modification is clearly coded, tested, and compared against a baseline or standard method.
  • Data visualization: plots and tables demonstrate convergence, error analysis or comparative results.
  • Code quality: intuitive variable names, and complex logic broken into readable, well-commented functions.
  • Rigorous analysis: the method’s boundaries are tested, for example its stability, parameter sensitivity or convergence rate.

Project ideas

Many of these were completed by students in previous years. They are starting points: the best projects usually come from your own research interests or curiosity, so modify them, combine them or propose something entirely new.

Monte Carlo tree search for game AI
Implement Monte Carlo tree search to train an agent for a game (Tic-Tac-Toe, Connect Four or Go). The algorithm builds a search tree by simulating random playouts and uses the results to guide exploration of promising moves.
Connection to the course: Metropolis-Hastings, importance sampling
Monte Carlo methods for option pricing
Price financial derivatives (European, Asian or exotic options) by simulating asset price paths under models such as geometric Brownian motion or stochastic volatility.
Connection to the course: Monte Carlo integration, variance reduction, stochastic processes
Kalman filtering for object tracking
Track moving objects in video, or simulate tracking of vehicles with noisy sensor measurements (GPS and velocity data). Predict future positions and update estimates as new observations arrive.
Connection to the course: Sequential Monte Carlo, Bayesian filtering
Spectral analysis of random graphs
Generate random graphs (Erdős-Rényi or preferential attachment) and analyze spectral properties of their adjacency or Laplacian matrices, such as eigenvalue distributions.
Connection to the course: Markov chains, mixing times
Expander graphs and mixing times
Study expander graphs and their rapid mixing by simulating random walks and estimating mixing times. Compare convergence rates across graph structures.
Connection to the course: Markov chains, mixing times
Metropolis-Hastings for image denoising
Denoise images by sampling from a posterior distribution that combines a prior on image smoothness with the observed noisy pixels.
Connection to the course: Metropolis-Hastings, Bayesian inference
Show 13 more
Approximating Max-Cut and NP-hard graph problems
Use simulated annealing or MCMC to find approximate solutions to NP-hard graph problems like Max-Cut or graph coloring. Compare solution quality and computational cost.
Connection to the course: Simulated annealing, MCMC optimization
Extended Kalman filters for nonlinear systems
Extend the Kalman filter to nonlinear dynamics with an extended Kalman filter, applied to tracking with nonlinear measurement models or estimating pendulum dynamics.
Connection to the course: Sequential Monte Carlo, particle filters
Collapsed Gibbs sampling for latent variable models
Implement collapsed Gibbs sampling for a latent variable model such as latent Dirichlet allocation or a Gaussian mixture model, marginalizing some variables analytically to improve mixing.
Connection to the course: Gibbs sampling, variance reduction
Markov switching models for time series
Analyze time series with regime changes, such as financial returns or economic indicators. Estimate hidden states and transition probabilities using Monte Carlo methods.
Connection to the course: Hidden Markov models, sequential Monte Carlo, Gibbs sampling
Hamiltonian Monte Carlo for Bayesian inference
Sample from a complex posterior with Hamiltonian Monte Carlo and compare its convergence and efficiency with Metropolis-Hastings or Gibbs sampling.
Connection to the course: Metropolis-Hastings, mixing time, convergence diagnostics
Adaptive MCMC methods
Implement adaptive MCMC algorithms (such as adaptive Metropolis) that tune their proposal distributions during sampling, and compare them with fixed proposals.
Connection to the course: Metropolis-Hastings, proposal distributions, convergence diagnostics
Metropolis-adjusted Langevin algorithm (MALA)
Sample from complex posteriors using gradient information with a Metropolis-Hastings correction step, and compare with standard Metropolis-Hastings.
Connection to the course: Metropolis-Hastings, gradient-based sampling, proposal distributions
Simultaneous localization and mapping
Implement particle-filter SLAM, where a robot estimates its position while building a map of an unknown environment from noisy sensor data.
Connection to the course: Particle filters, sequential importance sampling, resampling
Monte Carlo EM algorithm
Use MCMC in the E-step of the EM algorithm to approximate intractable expectations in maximum likelihood estimation for latent variable models.
Connection to the course: Importance sampling, Gibbs sampling, Monte Carlo estimation
Feynman-Kac formula for solving PDEs
Solve PDEs such as the heat equation or the Black-Scholes PDE by averaging over Monte Carlo sample paths of related diffusion processes.
Connection to the course: Monte Carlo integration, stochastic simulation, variance reduction
Multilevel Monte Carlo methods
Reduce the cost of estimating expectations involving stochastic differential equations or nested simulations by combining samples from several discretization levels.
Connection to the course: Monte Carlo integration, variance reduction
Reversible jump MCMC for model selection
Perform Bayesian model selection across models with different numbers of parameters, such as the number of components in a mixture model or the degree of a regression polynomial.
Connection to the course: Metropolis-Hastings, variance reduction
Perfect sampling with the Propp-Wilson algorithm
Generate exact samples from a Markov chain’s stationary distribution with coupling from the past, with no burn-in, for example for an Ising model or random colorings.
Connection to the course: Markov chain sampling, stationary distributions

Student projects

Spring 2026
  • Monte Carlo tree search for multiple sequence alignment
  • Kalman filter estimation of IMU measurements: free fall and 2D pendulum dynamics
  • Building an Othello engine with Monte Carlo tree search
  • Extending MMIL with Monte Carlo EM for cell-state inference in single-cell RNA sequencing
  • Simultaneous localization and mapping via extended Kalman filters
  • Game AI for Connect Four with Monte Carlo tree search, and its statistical properties
  • Feynman-Kac and diffusion Monte Carlo for solving PDEs
  • Monte Carlo simulation of photon transport through multi-layered tissue
  • Adaptive sample allocation in multilevel Monte Carlo for option pricing
  • A comparison of variance reduction techniques for European and Asian options
  • Estimating extreme credit portfolio risk with importance sampling
Fall 2025
  • Kalman filtering for object tracking: alternative models for diver tracking
  • Monte Carlo tree search for Othello
  • Extended Kalman filters versus particle filters
  • Spectra fitting and parameter selection with adaptive Metropolis-Hastings
  • Hamiltonian Monte Carlo and its comparison with Metropolis-Hastings
  • The Propp-Wilson algorithm: methodology and benchmark comparisons
  • Diffusion models from first principles: a Monte Carlo and Markov chain approach to image generation
  • Simultaneous localization and mapping with particle filters
  • How graph structure and walk design affect the mixing of random walks
Spring 2025
  • Expander graphs and fast-mixing random walks
  • Using Monte Carlo methods to approximate PageRank
  • Monte Carlo tree search: an introduction and applications
  • A Monte Carlo algorithm for Min-Cut
  • Adaptive rejection sampling
  • Monte Carlo localization: particle filters for robust robot positioning
  • Gibbs sampling for Markov switching models
  • Gibbs sampling for basketball data
  • Evaluating the statistical randomness of LLM-generated numbers
  • Cryptographically secure random number generation
  • Gibbs sampling for Gaussian mixture models (two projects)
  • Monte Carlo tree search methods
  • Strategy optimization for Go with Monte Carlo tree search
  • Hamiltonian Monte Carlo
  • Estimation in linear Gaussian state space models

Other topics suggested to past cohorts: Langevin dynamics, applications of stochastic differential equations in finance, the Ising model on general graphs, connections to spectral graph theory, the Propp-Wilson algorithm, cryptography using Monte Carlo methods, and the Metropolis-adjusted Langevin algorithm.