Enterprises using multiple AI models are underestimating failure rates by 2.25x

AI Summary

A new study reveals that enterprises using multiple AI models underestimate failure rates by 2.25 times due to the co-failure ceiling effect, challenging common assumptions about model orchestration. The study advises developers to combine models of matched quality to avoid negative performance impacts and higher operational costs.

A team routing queries across a coding specialist, a logic specialist, and a generalist model assumes each will cover the others' blind spots. A new study evaluating 67 frontier models from 21 providers shows that assumption is mathematically flawed — and the flaw has a name: the co-failure ceiling. The assumption works like this: as long as two models don't usually fail on the exact same prompts, combining them is supposed to create a safety net against failures. The real limit on orchestration is not how often models disagree, but the percentage of prompts where every model in the pool gives the wrong answer at once. By ignoring the co-failure ceiling, enterprises are building complex, expensive routing infrastructure to chase performance gains that do not exist. Fortunately, developers can use this same math to build a cost-free test that determines exactly when multi-model orchestration will actually pay off. The hidden costs of the multi-model strategy To orchestrate multiple language models, developers typically rely on three architectures. Model routers act as traffic cops, sending complex queries to expensive models and simple queries to cheaper ones. Cascades send every prompt to a cheap model first, only escalating to a premium model if the initial system signals low confidence. Finally, approaches like Mixture-of-Agents (MoA) fuse multiple models by asking them the same question and generating a synthesized answer from their combined outputs. These architectures introduce a "shadow price" to inference costs. Every time a development team implements a router or a cascade, they pay a premium in added system latency, complex infrastructure maintenance, and increased governance risks across multiple API providers. To justify these operational costs, engineers rely on “pairwise error correlation” to select their model pool. Imagine a developer has Model A, which writes excellent Python but fails at SQL, and Model B, which writes excellent SQL but fails at Python. Because they fail on different types of prompts, their pairwise error correlation is low. The developer assumes that by placing a routing layer in front of them, they have created a composite system that rarely fails at coding. According to the study, throwing diverse models together based on low correlation can actually hurt performance if the models are not equally capable — when you vote across diverse but unequal models, the weaker ones often gang up and outvote the smartest one. Josef Chen, author of the paper, told VentureBeat that in their experiments, "Naive majority voting across unequal models had negative mean gain (minus 10 points on our hard mix): diverse-but-weaker members outvote the strong one." The actionable advice for developers is to "combine only models within a matched quality band." If you cannot match quality, take the single-model baseline and spend your budget on the best model available. The paper provides one bright spot for this approach regarding MoA architectures. When building ensembles, teams often use "Self-MoA," where they query the same premium model multiple times to generate a synthesized answer. The researchers found that at matched quality, building a diverse ensemble of models with low pairwise correlation beats a high-correlation Self-MoA setup. However, when teams use that same pairwise correlation metric to predict the absolute accuracy of their overall system, the math breaks down. "So teams pay the orchestration overhead up front (latency, complexity, multi-provider operations) on the assumption that a diversity dividend arrives later," Chen said. "Usually it doesn't, because today's best models agree, and, worse, they fail on the same queries … the prompt simply carries little signal about which model will be the one that's right when the frontier disagrees." Why the math fails: the co-failure ceiling The core finding of the study centers on a metric called the "co-failure rate" — the formal name for the all-wrong scenario described above. No router, voting system, or cascade can ever achieve an accuracy higher than the ceiling it imposes. The coding, logic, and generalist pool shows low pairwise correlation on routine prompts — they rarely fail together. But the co-failure ceiling represents the obscure, highly complex edge case that pushes past the limits of current AI architectures. If a prompt is so difficult that all three models hallucinate or fail, it does not matter how intelligently the router distributes the task. The entire pool wipes out at once. The researchers tested their 67-model pool, which included GPT-5.5, Claude Opus 4.8, and Gemini 3.1 Pro, on the open-ended MATH-500 math benchmark. Based on standard pairwise correlation, statistical models predicted that the entire pool would wipe out simultaneously on only 2.3% of the questions. In reality, the co-failure rate was 5.2%. Standard correlation metrics underestimated the failure rate by roughly 2.25 times. The culprit is not ju