Mike Tung is the founder and CEO of Diffbot.
The black box era of AI development is over. Deep learning had a great run in the 2010s to 2020s. End-to-end deep learning networks, the technology that catapulted image classification and large language models from academic research to mainstream consciousness, have also led to numerous concerns. with AI bias and AI harm. That’s because a neural network, essentially a large array of numerical weights, is inscrutable for explanation when the AI inevitably makes a wrong prediction.
While this phenomenon is nothing new to the AI research community, as AI systems move out of the research lab and more and more into our daily lives, the inscrutability of deep learning systems has moved from a theoretical concern to a concern. practice. Deep learning networks have been very successful in optimizing the goal for which they were designed – namely, the overall accuracy in some artificial training set. But the next generation of AI systems will optimize a different measure – namely, the trust between a user and AI in real-world applications.
Knowledge allows explanations
Consider two different websites that recommend books for you to read. The first simply provides a sorted list of recommendations on what to read next, perhaps based on your previous reading history. The second site includes with each recommendation a reason for the recommendation. For example, he recommends the first book because you’ve just finished your prequel by the same author. The second book is recommended because you’re a fan of alternate history sci-fi plots that feature time travel.
Recommendations from which site would you trust the most?
The first website is how most AI-driven products have worked over the past decade. They are based on statistical patterns extracted from large datasets of private users acquired over long periods of time. The algorithms used are not transparent, nor are the objective functions being optimized. Does the system optimize recommendations based on what other people like or what generates the most revenue from your sales or advertisers? Are the recommendations based on my recent behavior or model of my preferences ten years ago when I first registered on the site? If a recommendation seems wrong, there is no way to understand why she made these predictions.
There have been recent calls to these open source systems, but even access to source code and learned weights of machine learning models would not be sufficient to provide satisfactory explanations for their predictions. Blind optimization of accuracy leads to recommendations without context (e.g. recommending you buy a TV based on your browsing history even if you bought one last week, or a list of recommendations that look like the same product instead of results several).
On the second site, recommendations have explanations because the systems have knowledge about the products they are recommending (i.e., the products are not just data points associated through statistical correlations, but correspond to structured data about the real-world object) . The second system has access to a world knowledge database (also known as a knowledge graph) with book entities and attributes linked to other entities, such as their genre, plot, author, characters, and all other entities with attributes of their own. .
In next-gen AI systems, the intelligence of the system’s recommendations is determined by the quality of the underlying knowledge graph that the system has access to (i.e., how comprehensive, accurate, and deep its facts are).
Just as a teacher asks a student on a test to “show off their work” to prove they understand a concept, next-generation AI systems will also have this capability, explaining their predictions based on the knowledge the system has.
Data provenance allows auditability
Another defining characteristic of the AI systems of the previous decade is their reliance on “dark” pools of behavioral data from private users. Advertising and current consumer search engines are emblematic of this type of AI system. They cannot explain a result because the result is based on a statistical pattern that has been calculated from many data points over a long period of time. The data points that went into the calculation are private information and are usually grouped in the behaviors of many users, so they cannot and should not be shown in the user experience as an explanation.
Next-gen AI systems don’t rely as heavily on private behavior data to make recommendations. As the quality of next-generation systems is largely determined by the quality of the common knowledge graph, it is less dependent on the private date. The facts in the common knowledge graph will have a chain of data provenance (i.e. citations where the fact originally came from, much like a student cites primary sources when writing a research paper). This data provenance will allow the user to “check sources” and audit the chain of reasoning of an AI system. If a user doesn’t like a particular primary source or has a preference for certain sources, this data provenance chain allows the system to only use facts from sources the user trusts. What is factual and accurate information should not depend on historical user behavior.
Towards a more reliable AI
As you undertake the next AI project in your organization, ask yourself not just whether AI optimizes your data, but whether AI has access to all the knowledge needed to deliver a human and credible experience. Map the stakeholders who maintain this knowledge, whether internally or through externally maintained knowledge graphs. Insist on a clear line of data provenance so forecasts can be audited and sources can be cited.
The era of black box AI systems is over. Next-generation systems will optimize the explainability and reliability of the overall human AI system, and knowledge graphs will serve as a key ingredient that makes these systems more explainable, inspectable, auditable, and, ultimately, controllable.
The Forbes Technology Council is an invite-only community for world-class CIOs, CTOs and technology executives. do i qualify?