AI to Production · tool 8 of 9
Running AI in Production
Running AI in Production: keeps a live system good, safe and affordable.
Best for: After launch · Time: Ongoing
Domain: Across all domains · Focus area: Monitoring and Controlling
How it works
AI quality can change with no code change, as data, users and vendors’ models shift. Monitor quality, cost, latency and safety, and have a rollback ready.
- Log inputs and outputs within your privacy rules.
- Set cost alerts.
- Pin model versions where you can.
How to apply it
- Choose live metrics.
- Sample outputs for review.
- Set alerts and a rollback.
- Review weekly and feed failures into evaluation.
In practice
Costs double after a popular new feature. An alert fires, and a cheaper model handles simple requests.
Pro tip
Plan for the vendor changing the model underneath you.
Use it
- Playbook: Our AI gave a harmful answer
Quick check
Why can AI quality fall without any code change?
Sources
Last reviewed 2026-11-02 against the PMBOK® Guide, 8th edition · Practitioner review pending · How this guide is made