The Edge-to-Cloud Latency Budget: Real-Time Inference for Safety-Critical Equipment

When AI monitors equipment whose failure can injure people or release hazardous material, how fast the system responds stops being a performance detail. It becomes part of the safety case. A correct fault prediction that arrives after the window to act has closed is not much better than no prediction.

This article explains how to build a latency budget for condition monitoring and inference across edge and cloud, how to tier equipment so you spend edge hardware where it matters, and how to keep AI in its proper place relative to the safety systems that must not depend on it.

First principle: AI is not the safety function

Before any architecture discussion, separate two things. Safety instrumented functions, the trips and interlocks that bring a process to a safe state, belong in a safety instrumented system designed and verified under the process safety standards your site follows, such as IEC 61511 or ISA 84. AI condition monitoring sits in front of that layer. Its job is early warning: spotting degradation early enough that people can plan an intervention before the safety system ever has to act.

Regulation points the same way. OSHA's process safety management standard exists to prevent or minimize the consequences of catastrophic releases of toxic, reactive, flammable, or explosive chemicals; it requires the process hazard analysis to address engineering and administrative controls, including detection methodologies that give early warning of releases, and it requires written management-of-change procedures for changes to technology and equipment in a covered process ^[1]. An AI monitoring layer on a covered process is exactly the kind of change that procedure should capture.

The December 2025 joint CISA and partner guidance on AI in operational technology reaches a similar conclusion from the security side: maintain oversight, ensure transparency, and integrate AI into incident response plans ^[2].

Anatomy of a latency budget

A latency budget is a stack of measured segments, each with its own allocation:

Measure every handoff, not just end to end. End-to-end numbers hide where time is consumed and where margin is thinning. Preprocessing is a frequent surprise: transforms on high-frequency vibration data or normalization of image frames can take a meaningful share of the budget before the model ever runs.

The ceiling for the whole budget comes from the process, not the hardware. Your process hazard analysis and the failure modes you are trying to catch tell you how quickly a developing fault can become dangerous. The detection chain has to fit inside that window with margin, and any part of the window that belongs to a safety function belongs to the safety system, not to the AI.

Tier equipment by consequence

Not every asset needs edge inference. Tier by consequence and by how fast faults develop:

Edge hardware is now capable of serious inference near the machine. NVIDIA, for example, rates its Jetson Orin NX at up to 100 TOPS of AI performance and the AGX Orin at up to 275 TOPS ^[3]. Capacity is rarely the constraint for condition monitoring models; environmental rating, power, maintainability, and security usually are.

Model partitioning: what runs where

Split the work so the time-critical path never crosses the wide-area network:

• At the edge: preprocessing, a compact scoring model, confirmation logic, and local alarming for Tier 1 and Tier 2 assets.
• In the cloud: model training, fleet comparisons, long-horizon trend analysis, and reporting.
• Between them: buffered, store-and-forward synchronization, so edge scoring continues when the link drops and history backfills when it returns.

Compressing models helps them fit edge hardware. ONNX Runtime, for example, supports 8-bit linear quantization, but the documentation warns it is not a lossless transformation and may affect accuracy, and that the speed benefit depends on hardware support for efficient int8 inference ^[4]. Validate a quantized model against the full-precision version on your own fault data before deploying it.

Network failover that works at 3am

Assume the network will fail at the worst time:

• Edge first for anything time critical. If the cloud is unreachable, local scoring and local alarms keep working.
• Buffer locally. Store raw features and scores on the edge device and backfill when connectivity returns.
• Degrade visibly. When a component is down or slow, raise a diagnostic alarm so operators know the monitoring layer is impaired, rather than letting it fail silently.
• Keep OT boundaries. The UK NCSC's OT connectivity guidance recommends brokering data exchange between OT and IT through a DMZ with secure, standardised protocols and ensuring no inbound connectivity from IT to OT ^[5]. Design cloud links outbound from the plant.

NIST SP 800-82r3 provides the broader guidance on securing OT while addressing its performance, reliability, and safety requirements ^[6].

Edge hardware selection

Choose edge hardware the way you choose any plant equipment:

Put each edge node in the CMMS with an owner, a PM task, and a rollback plan. Our article on treating edge AI as plant equipment covers this operating model in detail.

Enforcing the budget

A budget only matters if it is monitored:

1. Instrument every segment with timestamps, as above. 2. Alarm on breaches, not just on outright failures, so slow drift in latency is caught early. 3. Degrade automatically: if inference exceeds its budget, fall back to simpler rules or thresholds and raise a diagnostic alarm. 4. Review monthly: latency percentiles by segment, breaches, and any events where the monitoring layer was impaired. 5. Treat changes as MOC: model updates, threshold changes, and network changes on Tier 1 assets go through management of change ^[1].

For the maintenance side of the loop, our guide to CMMS integration covers turning alerts into planned work.

Frequently asked questions

Can AI inference be used as a safety shutdown?

Safety instrumented functions should stay in a safety instrumented system designed and verified under your site's process safety standards. AI condition monitoring is an early-warning layer in front of it.

Should safety-relevant inference run at the edge or in the cloud?

At the edge for the time-critical path, with local alarming, and in the cloud for training, fleet analysis, and reporting.

Does model quantization reduce accuracy?

It can. ONNX Runtime notes that quantization is not a lossless transformation ^[4]. Validate the quantized model on your own fault data.

How do we handle network outages?

Score and alarm locally, buffer data on the edge device, backfill when the link returns, and raise a diagnostic alarm whenever the monitoring layer is impaired.

Your first latency budget in 30 minutes

Pick one Tier 1 asset. Write down the fault you most need to catch and how quickly it can become dangerous, from your hazard analysis. List the six segments above, put your best measured or estimated time against each, and compare the total with the window. If the margin is thin, the next step is measurement, not more hardware.

References

[1] OSHA, "29 CFR 1910.119, Process safety management of highly hazardous chemicals." https://www.osha.gov/laws-regs/regulations/standardnumber/1910/1910.119

[2] CISA, "CISA, Australia, and Partners Author Joint Guidance on Securely Integrating Artificial Intelligence in Operational Technology," December 3, 2025. https://www.cisa.gov/news-events/alerts/2025/12/03/cisa-australia-and-partners-author-joint-guidance-securely-integrating-artificial-intelligence

[3] NVIDIA, "Jetson Orin." https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/

[4] ONNX Runtime, "Quantize ONNX Models." https://onnxruntime.ai/docs/performance/model-optimizations/quantization.html

[5] UK National Cyber Security Centre, "Secure connectivity principles for OT, Principle 4: Use standardised and secure protocols." https://www.ncsc.gov.uk/collection/operational-technology/secure-connectivity/principle-4

[6] NIST, "SP 800-82 Rev. 3, Guide to Operational Technology (OT) Security," September 2023. https://csrc.nist.gov/pubs/sp/800/82/r3/final