From Pilot Alerts to Planned Work: Why PdM Demos Stall Before Deployment

Day 61 of a 90-day pilot. The model flags rising vibration and temperature on a line-critical gearbox, the reliability engineer confirms the trend looks like a developing bearing fault, and the pilot vendor screenshots the chart for the steering committee. Everyone agrees the catch was real. Eleven days later the gearbox seizes on second shift, the line goes down for most of a shift, and the post-mortem says the same thing every post-mortem says: we knew.

Nobody wrote a work order.

A predictive maintenance pilot is production-ready when every alert reaches a named reviewer, becomes a scheduled work order with parts and labor attached, and closes with verified findings recorded against the asset. Detection is not the deliverable. The decision path is. Most pilots get scoped to prove that a signal exists, which is the cheapest part of the program, and skip the expensive parts: ownership, routing, work order quality, and closed-loop verification.

The Demo That Wins the Room and Dies in the Backlog

Pilot scopes are written by people who need to prove the technology works. That is a reasonable goal, and it produces a reasonable scope: instrument twenty assets, collect baseline data, show alerts that correlate with real degradation. What it does not produce is a maintenance decision.

Condition monitoring products are explicit that the workflow does not end at the alert. Amazon Monitron, for example, documents a path from sensor readings through alert notification, investigation, corrective action, and then technician feedback recorded against the abnormality ^[3]. The alert is step three of six. Pilots routinely end at step three and then ask an executive committee to judge the whole program on it.

The mechanism behind the stall is mundane. During a pilot, the reliability engineer is the entire routing layer. They watch the dashboard, they interpret the trend, they walk over to the planner, and they make the case verbally. That works for twenty assets and one enthusiastic engineer. It does not survive vacation, night shift, a second site, or the two hundred assets that come with plant-wide rollout. When the human router leaves, the alerts keep firing into a dashboard nobody owns.

Detection quality is also not usually the constraint. Modern condition monitoring stacks combine fixed thresholds with machine learning analysis of vibration and temperature data ^[7], and threshold-plus-model analysis is good enough to catch the failure modes that matter on rotating equipment. The constraint is that a correct alert with no owner, no decision window, and no CMMS destination has exactly the same operational value as no alert at all.

The Five Handoffs Where Pilot Alerts Go to Die

Draw the chain end to end before you score a pilot. Signal, reliability review, planner triage, scheduled work order, technician execution, verified findings back to the model. Five handoffs. Each one has a specific way of breaking and a specific observable symptom you can look for this week.

• Signal to review. What breaks: no named reviewer per asset class, and no off-shift escalation. Symptom: alerts land in a shared inbox or a group chat where nobody is accountable for reading them. Owner who fixes it: the reliability lead, by publishing a per-asset-class reviewer list with a backup.
• Review to planner. What breaks: the reviewer's judgment never becomes a formal request. Symptom: the decision exists in a Teams message, not in the CMMS. Owner: reliability lead and planning lead jointly, by agreeing on a single request object.
• Planner to schedule. What breaks: the planner cannot place the asset. Symptom: the equipment cannot be matched to a functional location in the SAP PM hierarchy, or the sensor names a bearing position the equipment master does not recognize. Owner: the master data owner, before rollout, not during.
• Technician to feedback. What breaks: findings go on paper or into a comment field nobody reads. Symptom: work order long text says "checked bearing, ok." Owner: planning lead, by making a structured findings field mandatory on predictive work orders.
• Feedback to model. What breaks: no route for confirmed or rejected outcomes to reach whoever tunes the system. Symptom: the same nuisance alert recurs monthly for a year. Owner: the reliability engineer plus whoever owns the monitoring configuration.

Score the pilot on handoff completion rate. For the last ten alerts, how many made it all the way to a closed work order with recorded findings? That single number tells you more than precision and recall. Most pilot scopes exclude the CMMS write path entirely, which guarantees a stall at handoff three no matter how good the detection is.

Write the Work Order Before You Tune the Model

In week one of a pilot, before threshold tuning, draft the target work order using the fields your plant already uses. Not a mockup. The real fields, approved by the planning lead.

A predictive-triggered work order needs, at minimum:

• Functional location and equipment number as they exist in the CMMS equipment master, not the sensor's nickname
• Maintenance activity type that identifies this as condition-based, so you can report on it separately later
• Failure mode code from your existing FMEA code set, so the work is comparable to historical failures
• Priority and recommended execution window, expressed as a date range tied to the next production opportunity
• Evidence link to the trend, the operating context, and the measurement history
• Required parts with storeroom availability checked, and estimated labor hours agreed with the crew lead

Evidence has to travel with the work order. A technician sent to inspect a gearbox needs the vibration trend, the load and speed context when the change appeared, and the suspected failure mode. A confidence score tells them nothing they can act on. Condition monitoring tools present measurement history and abnormality detail precisely so the person investigating can see what changed and record how it resolved ^[8]. If your integration drops that context on the way into the CMMS, you have shipped a rumor.

Integration reality differs by platform. In SAP PM, decide early whether predictive output creates a notification for planner conversion or an order directly, because that choice determines who holds scheduling authority. IBM Maximo needs the asset hierarchy and location mapping settled before any automated work order lands cleanly. Fiix and UpKeep can accept API-created requests, which makes a planner approval queue easy to implement and easy to skip. Keep a human approval step for at least the first several months of production routing. Planners keep authority over schedule and priority, and you get an audit trail of accepted and rejected recommendations. That trail is the raw material for later automation. Our notes on CMMS integration patterns and work order automation go deeper on which objects to write and when, and our guide to connecting sensor data to a CMMS for automated work order triggers walks through the integration itself.

Asset Criticality Decides Which Alerts Deserve a Planner

Undifferentiated routing is the fastest way to lose planner trust. When a nuisance alert on a redundant transfer pump consumes the same attention as a real signal on the line-stopping extruder, planners learn to treat the whole channel as noise. You cannot recover that trust with better models later.

Tier the routing:

• Tier 1, line-stopping single points of failure (main extruder drive, press hydraulic unit): route to the named reliability reviewer and the planner on the shift that raises the alert, and write your own decision deadline into the routing policy so no Tier 1 signal crosses a shift change unowned. Require the full measurement trend, the operating context at the time of the alert, and a suspected failure mode before the review can be closed, and record what was found against the alert so the next reviewer inherits the evidence.
• Tier 2, high consequence with partial redundancy (one of two chilled water pumps, critical conveyor gearbox): route to the reviewer daily, decision within three days, trend and failure mode required.
• Tier 3, moderate consequence, spared equipment (redundant transfer pumps, non-critical fans): batch into a weekly review, no individual planner notification, trend only.
• Tier 4, run-to-failure by policy (small utility fans, low-cost pumps): monitor for trending and reporting only, no alert routing at all.

Tie these tiers to reliability work you already have rather than inventing a new ranking. Your RCM analysis, your FMEA failure mode codes, and the criticality ranking already driving PM frequency decisions under ISO 55000 asset management practice are the correct inputs. Federal facility O&M program guidance treats maintenance planning as a structured program discipline rather than an ad hoc activity ^[5], and criticality-based routing is that discipline applied to alerts.

Start production routing on tier 1 and tier 2 only. Expand after decision latency stabilizes, not after the sensor budget arrives. And never let sensor availability choose your scope. Plenty of plants instrument what is easy to reach and then wonder why the alert stream does not correlate with their downtime Pareto.

The Production-Readiness Checklist Pilots Usually Skip

Run this before you call anything production. Group it so you can assign each block to one owner.

Data readiness

• Asset IDs reconciled across historian tags, SCADA naming, and the CMMS equipment master
• Naming conventions documented and enforced for new sensor installs
• Operating states labeled so alerts are not raised during planned idle, changeover, or cleaning
• Sensor mount locations documented with photos, including bearing position

Routing readiness

• Named reviewer and backup per asset class
• Escalation path for off-shift and weekend alerts
• Documented decision window per criticality tier
• CMMS destination object identified (notification, request, or order) and tested

Workflow readiness

• Work order template approved in writing by the planning lead
• Parts availability checked against storeroom min/max for the failure modes you expect to detect
• Labor estimate agreed with the crew lead, including access and lockout time

People readiness

• Funded program owner named for after the pilot ends
• Technicians briefed on what a predictive work order means and what evidence they will receive
• Operations aware of how condition-based work enters the schedule

Verification readiness

• Structured findings capture field mandatory on every predictive work order
• Monthly review of confirmed versus unconfirmed alerts
• Written process for retiring or retuning models that keep being wrong

Data readiness is where most programs quietly fail, because a correct alert on a misidentified asset is worse than no alert. If your equipment master and historian disagree about what a tag points to, fix that first; we covered the failure patterns in predictive alerts on the wrong asset.

A Go/No-Go Rubric That Beats Demo Enthusiasm

Score six dimensions before expanding. A pilot that scores well on detection and poorly on ownership is not a partial success. It is a program that will consume budget and produce a dashboard. If you are still choosing a monitoring approach rather than scoring one already running, our buyer's rubric for evaluating condition monitoring options covers the selection criteria that come earlier.

Use conditional go deliberately. Deploy on a narrow asset set, tier 1 only, with a fixed re-review date sixty days out and one or two named owners. That is far better than a plant-wide rollout that spreads an unresolved routing gap across four hundred assets. The same staged logic applies when you move a condition monitoring pilot to scale. When funding the named owner is the thing holding up a decision, the costs, savings, and payback period for predictive maintenance lays out the inputs a sponsor will ask you for.

Technician Feedback Is the Only Model Validation That Sticks

The crew's confirmed-or-not findings are the labeled data your program needs, and most pilots throw them away. Condition monitoring platforms build technician feedback into the loop on purpose, capturing whether an alert reflected a real machine problem and how it was resolved ^[1]. That feedback is what separates a monitoring system that improves from one that slowly loses credibility.

Keep the capture short enough that a technician completes it at the machine:

Alert ID: PDM-4412
Asset: EXT-02 gearbox, drive end bearing
Suspected failure mode: outer race defect
What was found: outer race spalling confirmed on visual
Part replaced: bearing 22216 E, qty 1
Sensor mount checked: yes, torque ok
Would you want this alert again? YES

Consider a plant where two technicians reject three alerts in a row on the same fan. The instinct is to blame the model. The reliability engineer pulls the raw trend instead and finds broadband noise appearing only at certain speeds, with no matching change in temperature. The root cause is a loose accelerometer mount, not a bad model. Without a structured rejection record and a reviewer willing to look at raw measurements, that plant retires a working configuration and keeps a real fault undetected.

Production machine learning guidance is consistent on this point: measure the system in production and monitor the pipeline, not just the model in isolation ^[6]. The NIST AI Risk Management Framework frames the same obligation as governing, measuring, and managing risk across the lifecycle of an AI-enabled system rather than at one validation gate ^[2]. On the plant floor, that governance is a weekly reliability meeting where rejected alerts get reviewed next to confirmed saves, so the crew sees that their input changes behavior. Root cause notes and findings text also become institutional memory, which matters more every year as experienced technicians retire.

One more operational note: sensor and gateway paths cross OT network boundaries, so bring your controls and security owners into the architecture conversation early rather than after a pilot needs a firewall change ^[4].

Questions Maintenance Leaders Ask Before Scaling a Pilot

How long should a predictive maintenance pilot run? Long enough to observe at least one real intervention cycle on a tier 1 asset, including the work order, the execution, and the findings. Duration matters less than whether the full loop closed even once.

Should alerts create work orders automatically? Not at first. Route them to a planner approval queue so planners keep schedule authority and you build an accepted-versus-rejected record. Automate creation only after that record shows consistent acceptance for a specific failure mode and asset class.

Who should own the program after the pilot? Someone inside reliability or maintenance with budget and a reporting line into operations. A vendor cannot own it, and an IT-only owner will not survive the first schedule conflict.

What if the model produces false positives? Investigate mounting, operating state labeling, and asset identification before retuning thresholds. Document the retune or retire decision and who made it.

Should we switch platforms if the pilot stalls at the planner handoff? A stalled handoff is usually a routing and ownership gap rather than a detection gap, and changing vendors restarts your baseline without building the CMMS write path you were missing. Fix reviewer assignment, work order completeness, and findings capture first, then judge the platform on what it did with those in place. If you are genuinely at a selection decision, compare predictive maintenance platforms for manufacturers against the six dimensions above.

Your First Week After the Pilot Review

Do one thing this week. Sit down with a planner, open the last ten pilot alerts, and try to build a complete, schedulable work order from each one using nothing but what the alert gave you. It takes about an hour. Every missing field, unmatched equipment number, and absent parts reference will surface in that hour, and you will leave with a punch list nobody could produce from a dashboard review.

Start tracking one metric: median time from alert to a recorded maintenance decision, with the deciding person's name attached. Not alert volume. Not model accuracy. Decision latency with an accountable name.

Then run a 30/60/90. By day 30, reconcile asset IDs across historian and CMMS and get the work order template approved in writing. By day 60, route tier 1 alerts into a planner approval queue with mandatory findings capture. By day 90, publish the first confirmed-versus-unconfirmed alert review to the same steering committee that approved the pilot.

Go back to that gearbox. The model was right, the engineer was right, and the asset still failed. The difference between a saved asset and a case study about a missed catch was one routed work order. Hold the standard accordingly: no alert goes live in production until you can name who reviews it, what decision it triggers, and where it lands in the CMMS.

References

[1] AWS, "What is Amazon Monitron? - Amazon Monitron", AWS documentation. https://docs.aws.amazon.com/Monitron/latest/user-guide/what-is-monitron.html

[2] NIST, "AI Risk Management Framework | NIST", 2023. https://www.nist.gov/itl/ai-risk-management-framework

[3] AWS, "The Amazon Monitron workflow - Amazon Monitron", AWS documentation. https://docs.aws.amazon.com/Monitron/latest/user-guide/deployed-workflow.html

[4] NIST, "Guide to Operational Technology (OT) Security | CSRC", 2023. https://csrc.nist.gov/pubs/sp/800/82/r3/final

[5] U.S. Department of Energy, "Operations and Maintenance in Federal Facilities | Department of Energy", DOE guidance. https://www.energy.gov/cmei/femp/operations-and-maintenance-federal-facilities

[6] Google, "Rules of Machine Learning: | Google for Developers", Google developer guidance. https://developers.google.com/machine-learning/guides/rules-of-ml?hl=en

[7] AWS, "How Amazon Monitron works - Amazon Monitron", AWS documentation. https://docs.aws.amazon.com/Monitron/latest/user-guide/how-monitron-works.html

[8] AWS, "Understanding sensor measurements and monitoring machine abnormalities - Amazon Monitron", AWS documentation. https://docs.aws.amazon.com/Monitron/latest/user-guide/anom-monitoring-chapter.html