Your Factory Is Full of AI Nobody Approved, and It Arrived Through Procurement

Ask a chief data officer to describe the company's AI estate and the answer is usually tidy. A predictive maintenance pilot. Vision inspection on two lines. A copilot rollout in engineering. Each has an owner, a business case and a slide in the steering deck.

Then walk the floor.

The machining centre that arrived last quarter adjusts its own tool offsets using an embedded vision model. The compressors are running a vendor's anomaly detection. The new spectrometer classifies defects with a model trained on other customers' data, quite possibly including your competitors.

None of it is in the inventory. None of it went through model validation. It came in the way a pallet jack does, through procurement, evaluated on price, delivery and warranty.

Two estates, and the bigger one is invisible

Farooque Munshi, a partner at EY who leads data and AI for advanced manufacturing across the Americas, describes most manufacturers as running two AI estates simultaneously.

One is deliberate, governed and shown to the board. The other is larger, growing faster, and assembled by category managers who have no idea they are making architecture decisions, because nothing in the purchasing process tells them that is what they are doing.

It is getting difficult to buy equipment that is not intelligent. And every embedded model has precisely the properties that made AI a governance problem in the first place: trained on data you cannot inspect, making decisions that affect your output, and capable of drifting quietly.

For the models your own people built, there are owners, monitoring and rollback plans. For anything bought this year, Munshi says the honest answer is usually that nobody knows, and that nothing was signed which would allow anyone to find out.

We didn't know the machine had a model in it is not a defence

Both systems are working correctly

The failure is not incompetence in either department, which is why it persists.

The request-for-proposal template predates embedded AI. The contract playbook covers intellectual property, liability and spare parts, not retraining cadence or data provenance. Meanwhile the governance function was designed to catch models built in-house and software bought as software, and capital equipment sits in a different budget under a different approval chain.

Both systems are doing what they were designed to do. The gap is between them.

This is also why the obvious fix does not work. Telling procurement to loop in the AI team is the same instruction that failed to make developers consult security teams, and for the same reason: a team working to a deadline will not reliably escalate something voluntarily. The check has to be built into the purchasing process, so that identifying embedded AI is a standard part of evaluating equipment rather than an extra step somebody has to remember.

What goes wrong first

The immediate risk is silent drift. A material supplier changes, or the vendor pushes an update overnight, and an embedded model starts making poor calls carrying the full authority of automation. There is no baseline and no audit trail.

The operational consequence is worse than the error. Operators get burned once and then route around the intelligence permanently, which means the capability you paid for is switched off by people who no longer trust it, and nobody upstream is told.

The second risk is data leakage, and it is structural rather than accidental. Vendors pool operational data across their installed base to improve their models, then sell the improved model to everyone. Years of hard-won process optimisation end up quietly averaged into the industry mean, and the plant across town gets the benefit.

The regulatory clock

The high-risk obligations of the EU AI Act have been postponed to December 2027, which is being read in some quarters as permission to wait.

Munshi's view is that this is a mistake, because some of the responsibilities fall on deployers rather than only on vendors, including human oversight, monitoring and record keeping. Separately, the new Machinery Regulation brings certain AI-enabled safety functions into conformity assessment from 2027.

Neither regime accepts the explanation that is currently the truthful one in most plants. Not knowing the machine had a model in it is not a defence. It is the exact governance gap the rules were written to expose.