2 years ago
We Must Perfect Predictive Models for Generative AI to Deliver on the AI Revolution

Originally published in VentureBeat, April 30, 2023.

Throughout 2022, generative AI captured the public’s imagination.

With the release of Stable Diffusion, Dall-E2, and ChatGPT-3, people could engage with AI first-hand, watching with awe as seemingly intelligent systems created art, composed songs, penned poetry and wrote passable college essays.

Only a few months later, some investors have begun narrowing their focus. They’re only interested in companies building generative AI, relegating those working on predictive models to the realm of  “old school” AI.

However, generative AI alone won’t fulfill the promise of the AI revolution. The sci-fi future that many people anticipate accompanying the widespread adoption of AI depends on the success of predictive models. Self-driving cars, robotic attendants, personalized healthcare and many other innovations hinge on perfecting “old school” AI.

Generative AI’s Great Leap Forward?

Predictive and generative AI are designed to perform different tasks.

Predictive models infer information about different data points so that they can make decisions. Is this an image of a dog or a cat? Is this tumor benign or malignant? A human supervises the model’s training, telling it whether its outputs are correct. Based on the training data it encounters, the model learns to respond to different scenarios in different ways.

Generative models produce new data points based on what they learn from their training data. These models typically train in an unsupervised manner, analyzing the data without human input and drawing their own conclusions.

For years, generative models had the more difficult tasks, such as trying to learn to generate photorealistic images or create textual information that answers questions accurately, and progress moved slowly.

Then, an increase in the availability of compute power enabled machine learning (ML) teams to build foundation models: Massive unsupervised models that train vast amounts of data (sometimes all the data available on the internet). Over the past couple of years, ML engineers have calibrated these generative foundation models — feeding them subsets of annotated data to target outputs for specific objectives — so that they can be used for practical applications.

Fine-Tuning AI

ChatGPT-3 is a good example. It’s a version of Chat GPT, a foundation model that’s trained on vast amounts of unlabeled data. To create ChatGPT, OpenAI hired 6,000 annotators to label an appropriate subset of data, and its ML engineers then used that data to fine tune the model to teach it to generate specific information.

With these sorts of fine-tuning methods, generative models have begun to create outputs of which they were previously incapable, and the result has been a swift proliferation of functional generative models. This sudden expansion makes it appear that the generative AI has leapfrogged the performance of existing predictive AI systems.

Appearances, however, can be deceiving.

The Real-World Use Cases for Predictive and Generative AI

When it comes to current real-world use cases for these models, people use generative and predictive AI in very different ways.

Predictive AI has largely been used to free up people’s time by automating human processes to perform at very high levels of accuracy and with minimal human oversight.

In contrast, the current iteration of generative AI is mostly being used to augment rather than replace human workloads. Most of the current use cases for generative AI still require human oversight. For instance, these models have been used to draft documents and co-author code, but humans are still “in the loop,” reviewing and editing the outputs.

At the moment, generative models haven’t yet been applied to high-stakes use cases, so  it doesn’t matter much if they have large error rates. Their current applications, such as creating art or writing essays, don’t carry much risk. If a generative model produces an image of a woman with eyes too blue to be realistic, what harm is really done?

Predictive AI Has Real-World Impact

Many of the use cases for predictive AI, on the other hand, do carry risks that can have very real impact on people’s lives. As a result, these models must achieve high-performance benchmarks before they’re released into the wild. Whereas a marketer might use a generative model to draft a blog post that’s 80% as good as the one they would have written themselves, no hospital would use a medical diagnostic system that predicts with only 80% accuracy.

While on the surface, it may appear that generative models have taken a giant leap forward in terms of performance when compared to their predictive counterparts, all things equal, most predictive models are actually required to perform at a higher level of accuracy because their use cases demand it.

Even lower-stakes predictive AI models, such as email filtering, need to meet high-performance thresholds. If a spam email lands in a user’s inbox, it’s not the end of world, but if an important email gets filtered directly to spam, the results could be severe.

The capacity at which generative AI can currently perform is far from the threshold required to make the leap into production for high-risk applications. Using a generative text-to-image model with likely error rates to make art may have enthralled the general public, but no medical publishing company would use that same model to generate images of benign and malignant tumors to teach medical students. The stakes are simply too high.

The Business Value of AI

While predictive AI may have recently taken a backseat in terms of media coverage, in the near-to medium-term, it’s still these systems that are likely to deliver the greatest value for business and society.

Although generative AI creates new data of the world, it’s less useful for solving problems on existing data. Most of the urgent large-scale problems that humans need to solve require making inferences about, and decisions based on, real world data.

Predictive AI systems can already read documents, control temperature, analyze weather patterns, evaluate medical images, assess property damage and more. They can generate immense business value by automating vast amounts of data and document processing. Financial institutions, for instance, use predictive AI to review and categorize millions of transactions each day, saving employees from this time and labor-intensive tasks.

However, many of the real-world applications for predictive AI that have the potential to transform our day-to-day lives depend on perfecting existing models so that they achieve the performance benchmarks required to enter production. Closing the prototype-production performance gap is the most challenging part of model development, but it’s essential if AI systems are to reach their potential.

The Future of Generative and Predictive AI

So has generative AI been overhyped?

Not exactly. Having generative models capable of delivering value is an exciting development. For the first time, people can interact with AI systems that don’t just automate but create — an activity of which only humans were previously capable.

Nonetheless, the current performance metrics for generative AI aren’t as well defined as those for predictive AI, and measuring the accuracy of a generative model is difficult. If the technology is going to one day be used for practical applications — such as writing a textbook — it will ultimately need to have performance requirements similar to that of generative models. Likewise, predictive and generative AI will merge eventually.

Mimicking human intelligence and performance requires having one system that is both predictive and generative, and that system will need to perform both of these functions at high levels of accuracy.

In the meantime, however, if we really want to accelerate the AI revolution, we shouldn’t abandon “old school AI” for its flashier cousin. Instead, we need to focus on perfecting predictive AI systems and putting resources into closing the prototype-production gap for predictive models.

If we don’t, ten years from now, we might be able to create a symphony from text-to-sound models, but we’ll still be driving ourselves.

Originally published by VentureBeat.