Industrial AI: Three Metrics and Four Levels to Measure Trust

On October 30, 2026, IIoT World published a write-up of a panel at the Industrial AI Summit 2026 built around a simple question: what must artificial intelligence prove before operators act on its recommendations? The panelists were Tim Burke (Arch Systems), Abby Eon (Velotic), Bryce Barnes (LG CNS) and Sahil Yadav (AOI), moderated by Jeff Winter of IIoT World. A caveat: the article is sponsored by Arch Systems, although it is described as editorially independent, and it reports the opinions of a panel rather than independent research.

High adoption, weak governance

The piece cites several survey figures that describe the gap between experimentation and control:

• 72% of manufacturers have adopted AI in some form, but only 10% have put it into production at scale (Parsec 2026 report);
• 62% focus AI on operations, but only 7% have tested an incident response plan for AI-related events;
• only 12% believe they could pass an independent audit of their AI governance (Grant Thornton 2026 survey).

The article does not say which industries these figures cover or how many manufacturers were surveyed, so they should be read as indications of a trend.

From "AI ready" to "decision ready"

According to the panelists, a model is AI ready when it can produce an answer; it is decision ready when someone, based on that answer, actually changes a production schedule or a quality process. For AI to influence operations, it should be grounded in real data, contextualized, explainable, governed and bounded.

Eon identified the ability to flag uncertainty as the most important capability. A model that cannot tell a sensor reading of zero from a failed sensor will give wrong instructions with complete confidence.

"Operators do not trust systems that are always sure." — Abby Eon

Three metrics for operations leaders

Yadav proposed judging AI with operational indicators rather than model accuracy alone:

• Mean time to explanation: how long it takes a person to understand why the AI made a recommendation;
• Trust-adjusted risk: the cost if the AI gets it wrong;
• Adoption rate: the share of recommendations operators actually act on.

All three measures should be captured before AI is introduced, and the test cases should be chosen by the operations lead.

The autonomy scale

The panelists described four levels, each with its own requirements:

• Observe: the data picture must be accurate;
• Recommend: accuracy must be demonstrated on incidents already classified by people;
• Act with supervision: requires safety kill switches, audit traceability, explainability and a limited blast radius;
• Self-heal: requires a proven track record over time, reversible actions and risk within accepted limits.

Outlook and advice

Within three years, according to the panel, generative AI will change how decisions are selected and presented, but people will still make them. Areas cited as overhyped included humanoid robots, co-pilots on messy data, LLMs on their own and autonomous agents on the shop floor. Burke warned against three-year data integration projects run solely by IT that deliver nothing along the way. The suggestion is to start with a boring, non-critical task, as AOI did with skeptical engineers. The first use cases to scale were varied: quality root-cause analysis, downtime attribution, workflow assistance and predictive maintenance.

What to do with this on the plant floor

For production directors and IT/OT managers, the value lies in the method: set baselines first, choose concrete test cases and move up a level only when the evidence supports it. In MES and shop-floor data collection projects, the realistic levels to start from are observe and recommend: they require reliable, contextualized data, the foundation on which any later autonomy rests.