2 months ago
The 3 Things You Need To Know About Predictive AI

Originally published in Forbes, June 29, 2024.

Some problems are better solved with predictive AI than with generative AI. To run more effectively, many business operations need prediction more than they need the generation of new content. That’s why predictive AI is the kind of AI companies turn to for improving the effectiveness of large-scale processes. Predictive models decide whom to contact, approve, test, warn, investigate, incarcerate, set up on a date or medicate. They target operational decisions: Market to those likely to buy. Approve for a loan those likely to pay on time. Test those likely to have a disease.

But the world isn’t making it easy for predictive AI to succeed. First, genAI’s current popularity leaves predictive AI a relatively unsung hero. Second, prediction boils down to probabilities and probabilities aren’t sexy. A cultural aversion hinders companies from embracing them. For many business professionals, the topic of probability can seem at best boring and at worst arcane and complex.

But there’s no way around it: Embracing predictive AI means becoming a business that acts probabilistically. We don’t generally have absolute, highly-confident predictions. There’s no magic crystal ball. But we do have the next best thing: A number between 0 and 100 that expresses the expected chance of a certain outcome or behavior. That’s a probability. And that’s what you get from a predictive model generated by machine learning. You could just as well call the model a “probability calculator.”

The culture backlashes strongly against probabilities. In The Empire Strikes Back, our trustworthy android C-3PO nervously exclaims, “The possibility of successfully navigating an asteroid field is approximately 3,720 to 1.” But our beloved hero Han Solo steers his ship right into danger, snapping back, “Never tell me the odds!” The screenwriters may have been reflecting a popular sentiment. I feel uncool admitting that, if I were ship captain, I’d want to know our chances.

In contrast, the beloved movie Moneyball—and the bestselling book on which it’s based—glamorize the successful collaboration between quant and stakeholder with a true success story. Working closely with a data scientist, the general manager of the Oakland Athletics baseball team triumphed with an unexpectedly high performance.

The problem is, for all its crowd-pleasing drama, Moneyball represents the epitome of glossing over the math. It does little to teach leaders how to collaborate with quants.

But if you take a deep breath and a good look, you’ll find that predictive AI isn’t hard to understand. The required upskilling is accessible, not arcane. Business professionals involved with predictive AI must ramp up on a semi-technical understanding that comes down to three things: 1) what’s predicted, 2) how well and 3) what’s done about it.

For the first of these three, you work with data professionals to establish what outcome or behavior to put probabilities on, such as whether a customer will click, buy, lie, die, cancel their subscription or commit an act of fraud.

For the second, you must help establish which metrics to report on for determining whether an ML model is production-ready. This includes straightforward business metrics such as the improvement to profit or savings a deployment is expected to deliver. Spoiler alert: accuracy is usually an impertinent and misleading metric.

For the third, you must help establish how predictions are acted upon. For example, if the model predicts that a customer will buy if contacted, then include that customer in a marketing campaign. If a transaction is predicted to be fraudulent, then block or audit it.

Most predictive AI projects fail to reach deployment. I believe this stems largely from a lack of deep collaboration with business-side stakeholders. To improve this dismal track record, business professionals must involve themselves in the details of these three aspects of a predictive AI project so that they can weigh in from an informed perspective, providing their business-side vantage so that the project stays on track not only technically but pragmatically. After all, if you don’t get your hands dirty, when it comes time to authorize a predictive AI deployment, your feet are liable to get cold.

About the author
Eric Siegel is a leading consultant and former Columbia University professor who helps companies deploy machine learning. He is the founder of the long-running Machine Learning Week conference series and its new sister, Generative AI Applications Summit, the instructor of the acclaimed online course “Machine Learning Leadership and Practice – End-to-End Mastery,” executive editor of The Machine Learning Times and a frequent keynote speaker. He wrote the bestselling Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die, which has been used in courses at hundreds of universities, as well as The AI Playbook: Mastering the Rare Art of Machine Learning Deployment. Eric’s interdisciplinary work bridges the stubborn technology/business gap. At Columbia, he won the Distinguished Faculty award when teaching the graduate computer science courses in ML and AI. Later, he served as a business school professor at UVA Darden. Eric also publishes op-eds on analytics and social justice. You can follow him on LinkedIn.