Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Design

Imagine you are a detective trying to solve a mystery, but you can't see the crime scene. Instead, you have a magical, super-complex video game simulator that can recreate the crime over and over again. You know the rules of the game (the physics of the universe), but you don't know the specific settings the "criminal" used to create the mess you see. Your goal is to figure out those hidden settings by running the simulator, watching the results, and adjusting your guesses. This is the world of Simulation-Based Inference (SBI). It's how scientists figure out the hidden rules of everything from how drugs interact with cells to how stars explode, using computer models because the real experiments are too expensive or dangerous to run.

But here's the catch: running these simulators is incredibly slow and expensive. Imagine if every time you pressed "play" on your game console, it cost you a dollar and took ten minutes to load. You only have a limited budget of "plays" before you run out of money. So, you need to be smart about which scenarios you simulate. This is where Bayesian Optimal Experimental Design (BOED) comes in. It's like a super-smart strategist that asks, "If I run the game with this specific setting, will it teach me the most about the criminal's hidden settings?" The paper tackles a tricky problem: usually, the best strategy to find the answer (BOED) and the best way to learn the game's rules (SBI) are treated as separate tasks, often requiring the simulator to be "transparent" (mathematically easy to tweak), which real-world scientific simulators often aren't.

The authors, Vincent D. Zaballa and Elliot E. Hui, have built a new method called SBI-BOED that bridges these two worlds. Think of it as teaching a single AI agent to do two things at once: learn the game's rules while figuring out the best moves to make. They discovered that a mathematical tool used to measure how much information one thing gives you about another (called Mutual Information) can actually be used as a training guide. By tweaking a specific "knob" in their math (a parameter they call $\lambda$), they found a way to make the AI learn the rules more accurately without needing to see the simulator's internal code.

In their tests, they treated the simulator like a "black box"—a machine where you put inputs in and get outputs out, but you can't peek inside to see the gears turning. They showed that their method could find the best experiments to run and learn the hidden parameters just as well as, or better than, existing methods, even when those methods had access to the simulator's inner workings. They found that by balancing the desire for "surprising" experiments with the need for "accurate" learning, they could get better results with fewer simulator runs. For example, on a complex biological model, their method used about 50 times fewer simulator calls than some previous top-tier methods to reach similar accuracy. They also showed that if you just chase the "most informative" experiment without being careful, you might end up with a model that looks smart but is actually wrong about the details. Their approach suggests that the best way to learn is to be a curious, balanced detective who values both the thrill of a new clue and the precision of the evidence.