In anticipation of his upcoming conference presentation, Detecting Incorrect Payments: The Payment Defect Model, at Predictive Analytics World for Financial in Las Vegas, June 3-7, 2018, we asked 
Richard Lee, Manager of Operations Reporting Consistency at John Hancock Financial, a few questions about his work in predictive analytics.
Q: In your work with predictive analytics, what behavior or outcome do your models predict?
A: Most of the work I do is geared towards needle in the haystack type of problems. These are the most interesting cases as overall accuracy is moot point due to the nature of the data. In this case, detecting defects, if I was to simply say that all payments are accurate I would be correct 99% of the times, that’s great however it doesn’t help us much with preventing the unlikely event of a defective payment being made.
Q: How does predictive analytics deliver value at your organization – what is one specific way in which it actively drives decisions or operations?
A: Our business is constantly growing and ten years ago that would have meant that the staff size would have grown as well however that is not efficient and is rather antiquated mode of operation. Through analytics our operations areas can make smarter and more targeted process improvements that result in higher productivity and therefore our organization can handle growing business with virtually unchanged staffing needs.
Q: Can you describe a quantitative result, such as the predictive lift of your model or the ROI of an analytics initiative?
A: The current model results in 60% reduced effort needed to discover the same (slightly greater) volume of defects then our previous process resulting in capacity gains that can be shifted towards other preventative measures. At its peak the model produces 6x lift.
Q: What surprising discovery or insight have you unearthed in your data?
A: It is hardly a surprise, none the less it always amazes me that there are always people in operations that have a very good sense of the problem as well as possible solutions however they lack the skills to quantify the data and produce further analysis. I worked closely with the operations and their ideas, as futuristic to them as it may have seemed, would have resulted in significant improvement. If I had to pick something surprising about data it would have to be the discovery that some data, at times, was overwritten when alterations were made in task processing rendering it almost useless.
Q: Sneak preview: Please tell us a take-away that you will provide during your talk at Predictive Analytics World.
A: Test, Test, Test… the smaller data set you have the more you have to test. CRISP-DM process is invaluable in analyzing data to spans over many years.
———————
Don’t miss Richard’s conference presentation, Detecting Incorrect Payments: The Payment Defect Model, on Wednesday, June 6, 2018 from 4:15 to 5:00 pm at Predictive Analytics World for Financial in Las Vegas. Click here to register to attend.
By: Eric Siegel, Founder, Predictive Analytics World
Eric Siegel is the founder of Predictive Analytics World (www.pawcon.com) — the leading cross-vendor conference series consisting of 10 annual events in New York, Chicago, San Francisco, Washington DC, London, and Berlin — and the author of the award-winning book, Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die – Revised and Updated Edition, (Wiley, 2016).