AI Strategy: How to Choose What AI Product to Implement
This paper introduces the expected ROI (eROI) framework, which helps firms overcome the uncertainty of AI project selection by separately evaluating the potential value, likelihood of success, and implementation cost of each idea to build a robust portfolio of high-potential bets.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are the captain of a massive spaceship, and you have a limited supply of fuel. You need to decide which distant planets to visit. Some planets look shiny and promising on your radar, while others look mysterious and potentially dangerous. In the world of business, this "fuel" is money and time, and the "planets" are new projects. For a long time, companies trying to use Artificial Intelligence (AI) have been like captains guessing which planet to visit based on how cool the planet looks, rather than checking if they can actually land there or if it's even worth the trip. This paper dives into that exact problem: how do you choose which AI project to build when you can't be 100% sure it will work? The authors introduce a new way of thinking called "eROI" (expected Return on Investment). Instead of trying to guess a single magic number for profit, they suggest breaking the decision down into three simple questions: How much money would we make if it works? How likely is it to actually work? And how much will it cost to build? It's like checking a map, a weather report, and a budget before you ever leave the dock.
The authors, who are experts in business and data science, argue that most companies fail at AI not because the technology is too hard, but because they pick the wrong projects to start with. They propose a framework called eROI (expected Return on Investment) to help leaders sort through hundreds of ideas and pick the winners. The core idea is simple but powerful: stop trying to mash everything into one confusing number. Instead, separate your decision into three distinct pillars.
First, ask: "How valuable would it be if it worked?" This isn't just about the immediate cash; it's about whether the project fits the company's big goals, if it creates new data assets for the future, or if it teaches the team something new.
Second, ask: "How likely is it to work?" This is the tricky part. AI is full of scientific uncertainty. Just because you have the data doesn't mean the computer can figure out the pattern. The authors break this down into four checks: Will customers use it? Can our engineers build it? Do we have the right data? And most importantly, is the science even possible? If the answer to any of these is "maybe not," the whole project is risky.
Third, ask: "What will it cost?" This isn't just the price of the software. It includes the time your team spends, the cost of buying new data, and the hard work of fitting the new tool into your existing daily routines.
To show how this works, the authors tell the story of a real estate company called Compass. They looked at three different AI ideas the company was considering:
The "Likely-to-Sell" Tool: This was a system that looked at an agent's contact list and flagged people who were most likely to put their house up for sale soon.
- The Verdict: This was a home run. It was Very High Value because it directly helped agents make more sales. It was High Likelihood because the data was clear and the science was straightforward. It was Medium Cost because the team already had the tools to build it.
- The Result: The company built it, and it ended up generating hundreds of millions of dollars in extra revenue.
The "Time-on-Market" Tool: This was a fancy calculator that tried to predict exactly how long a house would sit on the market at a specific price.
- The Verdict: This was a trap. While it looked High Value on paper, the Likelihood of Success was Low. Why? Because the science behind it was broken. To predict how price causes a house to sell faster, you need to know what would have happened if the price were different. But you can never see that "what if" scenario in real life. The authors call this a "causal" problem, and without special experiments, the AI couldn't solve it.
- The Result: The company wisely shelved this idea. Building it would have been a waste of money because the science simply wasn't there yet.
The "Renovation Visualization" Tool: This was a Generative AI tool that would take a photo of a run-down kitchen and show what it would look like after a renovation.
- The Verdict: This was a "maybe." The Value was Low to Medium because there were already cheaper ways to do this (like virtual staging). The Likelihood was Medium because the AI might make weird mistakes that would make buyers angry. The Cost was Medium.
- The Result: Even though it sounded cool and used the latest "Generative AI" buzzword, the math didn't add up. The company built a version of it, but it didn't take off because the risk of errors was too high compared to the benefit.
The paper suggests that the secret to success isn't just having a great idea or the smartest engineers. It's about having a portfolio of bets. You don't put all your fuel into the one "best" project. Instead, you mix a few sure things (Quick Wins), a few big risky dreams (Moonshots), and some research projects to figure out if the big dreams are even possible.
The authors are careful to say that you don't need perfect math to use this. You don't need to calculate the exact profit down to the penny. You just need to use simple ratings like "Low," "Medium," "High," or "Very High" for each of the three pillars. This helps teams stop arguing about whether a project is "good" or "bad" and start having real conversations about why it might fail or succeed.
In the end, the paper teaches us that choosing an AI project is like planning a road trip. You can't just pick the destination with the prettiest picture. You have to check if the car can handle the road (Technical Certainty), if you have enough gas (Investment), and if the destination is actually worth the drive (Value). By separating these questions, companies can stop wasting money on projects that look good but are impossible to build, and start funding the ones that will actually change the game.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.