Physical AI · Accountability

The robot that should worry you isn’t the humanoid in the demo

It’s the cart that just learned to choose its own route. Our line: if nobody on the floor can explain why an AI-guided machine did something, it doesn’t run again until someone can.

ByGlappy Insights
Published1 October 2026
Reading time5 min
A wheeled cart loaded with storage bins moves down a long, dimly lit warehouse aisle at night while a worker in a hi-vis vest walks in the distance.

Imagine a hypothetical warehouse on a Tuesday night shift. A self-guided cart that has run the same aisle loop for two years gets a software update. It can now pick its own route around congestion. At 2:14 a.m. it takes a shortcut through a loading bay nobody expected it to use and clips a pallet jack. No one is hurt. The supervisor stops the cart and asks the obvious question: why did it go that way?

Nobody on the floor can answer. The vendor’s support line opens at nine. The plant manager wants to know if the other carts are safe to run. The night shift has to decide whether to restart them now or work the rest of the shift by hand.

The humanoid robot gets the keynote and the headlines. The machine that actually changes your risk is quieter: the cart, the arm, the inspection camera that used to follow rules and now makes choices.

01Everyone wants it. Few have it running.

In March and April 2026, TCS surveyed CXOs and vice presidents at 300 manufacturing companies across North America and Europe. The appetite is obvious. 77% expect significant or transformational impact from physical AI in warehouse operations, 75% in assembly and manufacturing, and 72% in logistics and material movement. No organization surveyed plans to reduce investment, and 26% plan to increase it.1

Getting it to work is another matter. 68% were still in experimental or pre-deployment stages.1 In diginomica’s write-up of the same survey, only 9% had moved a pilot into operations, 4% were running deployments across more than five facilities, and a separate 6% were seeing measurable financial returns.2

So most of the industry is about to move from pilots to real floors. That’s exactly when the next number starts to matter.

02Nearly half can’t say who answers for a failure

44% of the manufacturers TCS surveyed reported unclear or no formal accountability structure for physical AI failures. 40% said they are unprepared for emerging regulatory requirements.1

Sit with the first figure. In close to half of these companies, if an AI-guided machine damages stock, stops a line or injures someone, it isn’t clear who is responsible. The operator? The engineer who configured it? The vendor? The manager who signed off the pilot?

That question gets harder as machines make more of their own choices. When a fixed-path robot fails, it’s a maintenance ticket: a sensor, a motor, a bad weld. When a machine chose a new route and failed, the question runs through the model, the training data, the settings and whoever approved all three. Without an owner, every one of those people can reasonably say it wasn’t theirs.

Who gets hurt? The worker standing in the loading bay. The night supervisor who has to make a call with no information. And the company, which finds out during an incident review that nobody was ever on the hook.

03The rules are coming whether you’re ready or not

Regulators are moving in the same direction. In Europe, diginomica notes, the EU Machinery Regulation 2023/1230 comes into full force on 20 January and requires third-party conformity assessment for safety components with fully or partially self-evolving behavior.2 Manufacturers selling into or operating in the EU will have to show roles, operating limits, human oversight and change control, not just say they have them.

The 40% who say they aren’t prepared have a deadline. The rest of us have a preview.

04Explain it, or it doesn’t run

Our line is short: if nobody on the floor can explain why an AI-guided machine did something, it doesn’t run again until someone can.

That sounds strict. It’s actually cheap compared with the alternative, which is restarting a machine you don’t understand because the shift is behind. To make the rule workable, you need four things in place before the pilot leaves the test cell:

  • A named owner for each deployment. One person, by role, on site or on call, who has the authority to pause the system and is expected to use it.
  • Written limits. Which decisions the machine can make on its own, which need a person to approve, and when it must hand control back. One page, posted where operators can see it.
  • Logs a supervisor can read. What the machine sensed, what it decided and what it did next, kept long enough for someone to reconstruct an event the same shift. Not just a vendor-only diagnostic file.
  • A stop rule people actually use. Any unexplained action means a pause and a short review. Not a reboot and a shrug.

05The network is part of the machine

The demo never shows this part. Machines that coordinate with each other, stream sensor data and log their decisions depend on the plant’s infrastructure. That means wireless coverage with no dead spots in the aisles they use, networks segmented so a robot fleet isn’t sharing space with guest Wi-Fi, and storage for the event data your logs depend on.

In diginomica’s reporting, TCS points to fragmented data environments and legacy integration as major barriers to scaling.2 Those are infrastructure problems before they’re AI problems. A cart that loses its connection mid-aisle isn’t an IT inconvenience. It’s a safety question. Plan the network and the logging as part of the deployment from day one.

06What to do Monday

  1. List every machine on your floor that makes its own choices, including the ones that got that ability through a software update.
  2. Write a name next to each one. If you can’t, that’s your first gap.
  3. Run the 2:14 a.m. test. Pick one machine and ask: if it did something unexpected tonight, could the night supervisor find out why before the next shift? If the answer is no, fix the logs before you scale.

The humanoid may show up one day. The cart is already here. Make sure someone can explain what it does.

—

When you want a second pair of eyes. Glappy designs and installs the network and infrastructure that industrial systems run on. If you’re planning a physical AI pilot, we can help you get the foundations and the logging right first. Book a time.

A note on the numbers

The figures come from a vendor survey of large manufacturers in North America and Europe; TCS sells physical AI services. The deployment and returns figures and the regulatory note are as reported by diginomica. The warehouse scenario is hypothetical. The “explain it or it doesn’t run” line is Glappy’s opinion, not a research finding.

Sources

  1. Tata Consultancy Services, Future-Ready Manufacturing: TCS Physical AI Readiness Report 2026, press release 22 Jul 2026. CXOs and VPs at 300 manufacturing companies in North America and Europe; surveyed March–April 2026. www.tcs.com/who-we-are/newsroom/press-release/tcs-study-reveals-physical-ai-is-now-mainstream-as-manufacturers-scale-ai-implementationTCS sells physical AI and manufacturing services.
  2. diginomica, “Why physical AI pilots stall – the TCS worldview as autonomy runs into accountability,” 2026, reporting the same TCS survey. diginomica.com/why-physical-ai-pilots-stall-tcs-worldview-autonomy-runs-accountability
About

Glappy Insights

Field notes from Glappy Inc, a Richardson, Texas company working on AI, networks, security and infrastructure. We write about the consequences of AI for the systems and the people who answer for them. Glappy is an OpenAI Select Partner and a Dell Technologies Authorized Partner.

Follow Glappy on LinkedIn ↗