Progressive Context Enrichment for LLM-Based MILP Formulation in Open-Pit Mine Production Scheduling

Imagine you are the captain of a massive, floating city made of rock, and your job is to decide exactly which chunks of stone to dig out, when to dig them, and how to process them to make the most money over the next twenty years. This isn't just a game of "dig here, dig there"; it's a high-stakes puzzle called Open-Pit Mine Production Scheduling. The catch? The rules are incredibly strict. You can't dig a block of rock until the ones sitting on top of it are gone (like a Jenga tower), you can't process more rock than your factory can chew, and the mix of rock you send to the factory has to be just the right quality. Solving this puzzle perfectly requires a super-smart math tool called Mixed Integer Linear Programming (MILP), which is basically a way of writing down every single rule and goal in a language that computers can solve. But here's the problem: writing these math rules is so hard that only a few experts in the world can do it, and they often get stuck when the mine changes or the rules get tricky.

Enter the new kids on the block: Large Language Models (LLMs). These are the AI chatbots you might know, like the ones that write essays or code. They are amazing at understanding language and even writing computer programs. The big question scientists asked was: Could we just tell an AI, "Here is a mine, here are the rules, please write the math code to solve it," and have it do the job of a human expert? It sounds like a dream, but AI has a habit of getting confused when things get complicated. It might write code that looks right and gives a number, but secretly, it's ignoring the rules. This paper dives deep into whether we can teach these AIs to be better mine planners by giving them better hints, and it discovers that the way we ask the question matters more than the AI itself.

The AI Mine Planner's Training Camp

The researchers at Lakehead University and Memorial University of Newfoundland set up a clever experiment to see if they could turn a general-purpose AI into a mine-scheduling wizard without teaching it any new math. They didn't want to retrain the AI from scratch; instead, they wanted to see if they could just "enrich the context"—basically, give the AI better study materials before it tried to write the solution.

They created a four-step training camp for four different AI models (ChatGPT, Gemini, DeepSeek, and Copilot). Think of it like teaching a student to write a complex essay:

1. The Basic Prompt: They started with the bare minimum: a simple text description of the mine and the rules. It was like giving a student a prompt like "Write about mining" and expecting a thesis.
2. The Simulation Log: Next, they added a "diary" from a greedy robot that had already tried to run the mine. This log showed what happened period by period: which rocks were picked, which were skipped, and when the factory got full. It was like showing the student a diary of someone who actually tried to solve the problem, so the AI could see the behavior of the rules.
3. The Simulator Code: Then, they gave the AI the actual source code of that greedy robot. This wasn't just a diary; it was the instruction manual the robot used to make decisions. It showed the logic: "If the factory is full, stop." "If the block above isn't gone, wait."
4. The Annotated Code: Finally, they took that source code and added sticky notes (annotations) everywhere. They labeled every line of code with what it meant in math terms, like "This line is the Mining Capacity Rule" or "This is the Precedence Rule." This was the ultimate cheat sheet, translating the robot's logic directly into the language of the math problem.

The "Silent Failure" Trap

The most surprising discovery in this study wasn't just about how well the AIs did, but about how badly they could hide their mistakes. The researchers found that if you just asked the AI to write the code and run it, many of the models would produce a solution that looked perfect. It would give a number for the profit (called Net Present Value or NPV) and say, "Success! The mine is scheduled!"

But when the researchers looked closer, they found "silent failures." These were solutions that were technically "feasible" (the computer didn't crash) but were actually broken. The AI had forgotten to enforce one or more of the most important rules, like the rule that says you can't dig a block until the one above it is gone. The AI found a loophole, or just ignored the rule, and the computer happily calculated a profit for a mine that couldn't physically exist. It was like a student writing an essay that looks great but completely ignores the prompt; the teacher might give it a passing grade at first glance, but it's actually nonsense.

The paper explicitly rules out the idea that a "feasible" result means the AI got it right. In fact, for three of the four AI models, adding more information (like the logs and code) actually increased the number of these silent failures. The AIs got more confident, but they were confidently wrong. Only one model, Gemini, managed to avoid these silent failures entirely, no matter how much information was given.

The Magic of the "Cheat Sheet"

Once the researchers filtered out the silent failures and looked only at the solutions that were actually valid, the results became very clear.

• Just text wasn't enough: When the AI only had the basic text description, it almost never got it right.
• The Log helped a little: Seeing the robot's diary (the simulation log) helped the AI understand the flow of time and resources, improving the results slightly.
• The Code was the game-changer: When the AI was given the actual source code of the simulator, the quality of the solutions jumped up significantly. The code acted as a bridge, showing the AI exactly how the rules worked in practice. This was the most informative element for getting the overall profit number right.
• Annotations made it precise: The final step, adding the "sticky notes" (annotations) to the code, didn't always make the profit number higher, but it made the structure of the solution much more accurate. It helped the AI understand exactly which part of the code corresponded to which math rule. This was especially helpful for the trickiest rules, like the ones about the order of digging (precedence) and mixing rock grades.

Who Won the Race?

Among the four AI models tested, Gemini was the clear standout. It was the only model that didn't produce any silent failures, meaning every time it said it had a solution, it actually had a valid one. When given the annotated code, Gemini's solutions were incredibly close to the perfect human-made benchmark, with a "penalized relative error" (a score that counts both mistakes and wrong answers) dropping from 100% (total failure) down to just 21.6%.

The other models, like ChatGPT and Copilot, struggled more. They often produced solutions that looked good but were structurally flawed. Interestingly, giving them the annotated code didn't always help them; for some models, the extra notes actually confused them or led to new types of errors. This suggests that different AIs "think" differently, and what works as a helpful hint for one might be noise for another.

The Takeaway

The paper concludes that the main problem with using AI for complex math problems like mine scheduling isn't that the AI isn't smart enough; it's that we weren't giving it enough context. A simple text description is too vague. However, simply throwing more text at it isn't the answer either. The key is to provide executable logic—the actual code that simulates the rules—and then label that code so the AI knows what each part means.

This approach suggests that we don't need to train a new, super-expensive AI model to be a mine planner. Instead, we can take a general AI and give it a "scaffold" of simulation logs and annotated code to help it translate human operational rules into perfect math. It's a bit like how a student might struggle to write a physics equation from a word problem, but if you give them the lab notes and the formulas used in the lab, they can suddenly solve it.

The study also warns us to be careful. Just because an AI says it has a solution doesn't mean it's right. We have to check the details, the "silent failures," to make sure the AI hasn't just found a shortcut that breaks the rules. But with the right kind of context enrichment, specifically using simulator code and annotations, we can get AI to do the heavy lifting of mine planning without needing a PhD in mathematics for every single job.