Adaptive π
ππ Advent of Open Source β Day 16/24: A Python package for efficient, intelligent sampling in scientific computing.
(See my intro post)
During my PhD, I was running quantum transport simulations on our 800-core computational cluster. Traditional parameter sweeps using dense grids were hitting the cluster’s limits - wasting precious compute time on uninteresting regions while potentially missing important features. Most parameter spaces we explored were “boring” with occasional interesting regions that required higher sampling density.
π Origin Story
Initially, in 2016, I had a collection of one-off adaptive sampling scripts. Together with Joseph Weston and the support of my PhD advisor Anton Akhmerov, we turned it into a full-fledged Python package “Adaptive”. What began as an internal tool to make our cluster usage more efficient slowly gained traction in the wider scientific computing community.
π§ Technical Highlights
- Multiple adaptive sampling strategies (1D, 2D, ND, averaging stochastic functions)
- Parallel execution support (local, MPI clusters, distributed systems)
- Live plotting in Jupyter notebooks with real-time feedback
- Customizable loss functions for different sampling priorities
- Minimal dependencies
π Impact & Applications
- 1164 GitHub stars and thousands of daily downloads
- Cited in 20+ scientific publications
- Dependency of many packages, most notably: Orange Quantum System’s Quantify software for controlling real quantum computers!
Used in cutting-edge research across fields:
- Quantum Computing: Simulating and controlling superconducting qubits and trapped ion systems
- Quantum Physics: Exploring topological quantum systems and novel materials
- Materials Science: Studying graphene and other 2D materials
- Engineering:
- Optimizing radar system performance
- Fluid dynamics simulations
- Chemistry: Molecular modeling and vibrational spectroscopy
- Most Unexpected Use: Found in a paper on predicting energy usage of wartime bungalows!
This range of applications shows how a tool built for quantum transport simulations can find surprising uses across disciplines, from fundamental physics to everyday buildings.
π― Challenges and Solutions
- Making parallel execution truly scalable while maintaining sampling intelligence
- Creating intuitive visualization tools for complex parameter spaces
- Handling stochastic functions and noisy data
π‘ Lessons Learned
- Building a user base takes active effort - giving talks and engaging online made all the difference
- Good documentation converts interested users into regular contributors
- This was my first open source package and learned all about packaging, testing, and CI/CD
Want smarter parameter space exploration? Check out Adaptive on GitHub or try our interactive tutorial!
#OpenSource #Python #ScientificComputing #ParallelComputing #Programming