Imagine this: it is Monday morning, and a developer opens a routine pull request. Tests pass. The team’s AI reviewer does not respond. Someone reruns the workflow. Same result. The build is healthy; the assistant is not.
The on-call engineer checks the integration and finds a model-unavailable error. The model was retired. A notice reached an inbox weeks earlier, but never became an owned engineering task.
This is a hypothetical scenario used to explain the failure mode—not a report of a specific company outage.
1. The warning is not the outage
Back at the hypothetical team’s desk, the first mistake becomes clear: “deprecated” was treated as a distant housekeeping problem. It was actually a countdown.
Anthropic distinguishes a deprecated model, which still works but is scheduled for retirement, from a retired one, whose requests fail. Its documentation also notes that partner-operated platforms can follow different retirement schedules. Check the Claude model-deprecation documentation.
Google publishes its own Gemini API deprecation schedule. AWS documents Amazon Bedrock’s model lifecycle separately. The practical lesson is to track the endpoint and platform you actually use—not just a model’s marketing name.
A pinned version makes model selection more controlled. It does not buy permanent access. A provider can retire the version you pinned. Keep the lifecycle source beside the configuration, and recheck it during each migration.
2. Find the dependency nobody remembered
The team changes the model setting in its reviewer. The visible error disappears. Then a nightly documentation job fails: it had its own model ID. This is where a one-line fix becomes dependency management.
Search repositories, CI definitions, agent configuration, scheduled jobs, environment settings, and provider usage reports. Do not stop at the main application. Include fallback models and developer tools that can select a model independently.
Give each relevant notice an issue with a named owner and migration deadline. Set that deadline before retirement. A two-week buffer is an illustrative team policy, not a provider promise; short-notice changes may require a smaller scope and an explicit decision to pause a feature.
Keep model routing configurable and reviewed. But do not mistake centralized configuration for compatibility: prompts, tool schemas, timeouts, and evaluation results need to travel with the change.
3. A successful response can still be a failed migration
Now the reviewer answers again. Everyone relaxes—until a developer notices that its proposed fix removes an important validation check. The replacement is available. The workflow has regressed.
This is the second failure to design for: not a red error, but a plausible answer that is wrong for your repository. Test representative tasks from your own work, such as fixing a regression, following project conventions, editing several files, and rejecting malicious instructions inside retrieved content.
Run generated code in an isolated environment without production credentials. Use authorized, sanitized examples and repeat representative cases to account for output variability. Decide your acceptance gates before comparing candidates.
| Gate | Evidence—not a feeling |
|---|---|
| Correctness | Requested behavior works; regression tests pass. |
| Tool safety | Arguments validate; approvals remain enforced; no unintended writes. |
| Latency | End-to-end time, including slow-tail requests and retries. |
| Economics | Cost per accepted task, including failures and human rework. |
| Recovery | Interrupted tasks can resume without duplicate side effects. |
Do not assume a cheaper token price produces a cheaper workflow. More retries or repair work can erase the saving. For a related starting point, read From Prompt to Production: Evaluate AI Features.
4. Change a little. Measure. Then expand.
The team’s next move is deliberately smaller: run the replacement on an internal slice of work, inspect the results, and expand only when its gates pass.
Google’s SRE guidance on canary releases explains the value of limited exposure and representative measurements before a broad rollout. Applying that principle to AI means measuring task quality alongside availability—not treating a successful HTTP response as success.
Correctness · tools · latency · cost
No ↓
Keep the current safe path
Yes ↓
Monitor quality and errors
If results hold: expand gradually, then confirm zero calls to the retiring model.
If results regress: stop expansion and use an available, tested fallback.
Shadow comparisons can help, but a shadow agent must not execute live side effects. Replaying a task without isolation can duplicate a write, email, or deployment. Record model identity, prompt version, error class, latency, and task outcome while keeping secrets and unnecessary customer data out of logs.
5. The fallback is a user experience
Finally, the team asks a better question: if every approved model is unavailable, what can the developer still do?
For suggestions, preserve their work and make manual editing easy. For pull-request review, keep deterministic checks running and request a human review. For an autonomous task, pause consequential actions and retain enough state for a controlled restart.
“AI review is temporarily unavailable. Your changes are saved, and automated tests are still running. Please request a human review.”
Example message—use it only when those statements match your product’s actual behavior.
A fallback must be available, evaluated, and approved to receive the data. Do not silently send private code to a new provider. Bound retries for transient errors; repeatedly calling a retired model is not recovery.
Before retirement, the old model may remain a rollback option. Afterwards, it does not. Your recovery path must survive that deadline. Maintain the same permission boundaries throughout the migration; see the production AI agent guardrails.
6. Rehearse a boring ending
The hypothetical team finishes by simulating model failure in staging. This time, the reviewer explains the interruption, tests continue, and nobody loses a patch. The alert reaches an owner. The runbook describes an available recovery path. That is the ending worth designing.
A small team can start today: inventory the models behind its most important feature, assign one owner, and rehearse that feature without AI. No elaborate platform required.
Questions teams usually ask
Does pinning a model ID prevent retirement?
No. It controls the version you request, not how long the provider makes that version available. Track availability independently.
Should every team have a second provider?
Not necessarily. A tested manual path may be safer and simpler. Cross-provider fallback adds evaluation, data-handling, capacity, and integration work.
How many evaluation cases are enough?
There is no universal number. Cover critical journeys and failure modes, repeat variable tasks, and expand the suite with real regressions. Confidence should match the consequence of failure.
Go deeper: the original sources
Official references checked for this revision. Schedules change; verify the current entry for your platform. The scenario and operational checklist above are editorial synthesis, not a vendor guarantee.
- Anthropic — model lifecycle, retirement, and migration guidance
- Google — Gemini API deprecation schedules
- AWS — Amazon Bedrock model lifecycle
- Google SRE — canary releases and rollout evaluation
Join the discussion
Which model dependency would catch your team off guard? Share a lesson or a failure mode you want to test.
d