A silent model swap is when the model serving an API alias changes without notice. You call the same name, but a different build answers, and nothing in the response tells you.
Most production traffic points at a moving alias, not a dated snapshot. That alias is a convenience: it always resolves to “the current version.” The catch is that “current” can change under you, and providers do not always announce when it does. The request looks identical, the model id in the response looks identical, and yet the behavior on the other end has moved.
Why providers swap
The reasons are ordinary operations, not conspiracy: a cheaper build to serve load, a capacity reroute, a safety or formatting patch shipped mid-cycle. From the provider's side the alias is doing its job. From your side, a dependency changed and you were not told.
Why it breaks things
A prompt tuned against the old build can regress overnight. An evaluation you ran last quarter may no longer describe the endpoint you are calling this quarter. Output formats that your parser depends on can shift by a whitespace or a preamble and break a pipeline. And a compliance or safety argument that rests on “we tested this model” quietly stops being true when the model is not the one you tested. The failure is silent precisely because nothing errors; the system just behaves a little differently.
How to catch a swap
The detector is a canary: a small set of fixed prompts, sent on a schedule. Fingerprint each reply and compare it to the established baseline. When the fixed answer moves and stays moved, that is a sign that something in how the model is served may have changed, which can also come from ordinary variation on the provider's side. Because the probes are short, this runs for pennies a day, which is what makes daily monitoring practical at all.
Modelometer runs this canary every day against frontier endpoints and publishes the result, so a serving change is recorded with the date of the reading that first showed it rather than discovered weeks later in a broken pipeline.
Common questions
How do I know if my model changed?
Compare a fixed probe over time. If a temperature-zero prompt that returned one answer starts returning a different one, that is a sign that something in how the model is served may have changed, which can also come from ordinary variation on the provider's side.
Do providers announce model changes?
Dated snapshot versions are stable by contract. Moving aliases (the default for most traffic) can be repointed with little or no notice, which is why independent monitoring exists.
What is a canary probe?
A fixed prompt with a known stable answer, run on a schedule and fingerprinted. A change in the fingerprint flags a possible serving swap.