Toward Collective Model Intelligence: Characterizing and Coordinating Models

First, Model Eval constructs evidence-based capability profiles by combining query-level performance estimation with cross-domain prediction, reducing the need for exhaustive evaluation. It comes from a paper posted to arXiv on 20 October 2026. Independently developed AI models accumulate diverse and complementary capabilities, but a growing model collection does not automatically realize Collective AI. Addressing this gap requires understanding model capabilities with limited evaluation, organizing complementary models to collaborate on diverse tasks, and executing model calls efficiently under resource constraints. We present a systematic approach to pursuing Collective Model Intelligence through three connected components. Second, Mio (Models in One) organizes complementary model candidates and supports task-conditioned, cost-aware selection and collaboration as task requirements evolve. Third, Mos (Model Serving OS) separates model selection from execution and provides endpoint scheduling and shared model deployment and distribution. Experiments show lower capability-estimation errors under limited evaluation. Across nine public benchmarks, Mio 1.0 Lite achieves the highest average score and lowest average cost among compared methods, with per-request costs 73.7-97.5% lower than those of the best-scoring alternatives. A prewarmed shared cache accelerates model-file retrieval by up to 23.3 times. This work lays a foundation for Collective Model Intelligence by characterizing and coordinating the diverse capabilities of an expanding model universe.

Why it counts

First, Model Eval constructs evidence-based capability profiles by combining query-level performance estimation with cross-domain prediction, reducing the need for exhaustive evaluation.

Sources

Primary source: primary source

What is not known

This brief does not claim independent replication. Claims that appear only on X and not in the primary source stay unknown.