A silent model swap is when the model serving an API alias changes without notice. You call the same name, but a different build answers, and nothing in the response tells you.
Most production traffic points at a moving alias, not a dated snapshot. That alias is a convenience: it always resolves to “the current version.” The catch is that “current” can change under you, and providers do not always announce when it does. The request looks identical, the model id in the response looks identical, and yet the behavior on the other end has moved.
Why providers swap
The reasons are ordinary operations, not conspiracy: a cheaper build to serve load, a capacity reroute, a safety or formatting patch shipped mid-cycle. From the provider's side the alias is doing its job. From your side, a dependency changed and you were not told.
Why it breaks things
A prompt tuned against the old build can regress overnight. An evaluation you ran last quarter may no longer describe the endpoint you are calling this quarter. Output formats that your parser depends on can shift by a whitespace or a preamble and break a pipeline. And a compliance or safety argument that rests on “we tested this model” quietly stops being true when the model is not the one you tested. The failure is silent precisely because nothing errors; the system just behaves a little differently.
How to catch a swap
The detector is a canary: a small set of fixed prompts with exactly one correct, stable answer, run at temperature zero so the reply should be identical day after day. Add a rotating nonce so the answer cannot be served from a cache. Fingerprint each reply and compare it to the established baseline. When the fixed answer moves and stays moved, the build underneath almost certainly changed. Because the probes are short and greedy, this runs for pennies a day, which is what makes daily monitoring practical at all.
Modelometer runs this canary every day against frontier endpoints and publishes the result, so a serving change is caught and dated the moment it happens rather than discovered weeks later in a broken pipeline.
Common questions
How do I know if my model changed?
Compare a fixed, deterministic probe over time. If a temperature-zero prompt that returned one stable answer starts returning a different stable answer, the build serving your alias likely changed.
Do providers announce model changes?
Dated snapshot versions are stable by contract. Moving aliases (the default for most traffic) can be repointed with little or no notice, which is why independent monitoring exists.
What is a canary probe?
A fixed, deterministic prompt with a known stable answer, run on a schedule and fingerprinted. A change in the fingerprint flags a possible serving swap.