PortBench: A Correlation-Aware, Full-Pipeline Benchmark for LLM-Driven Portfolio Management

Across ten frontier LLMs, strong financial QA performance fails to translate into superior portfolio performance: only 32.5\% of 120 evaluations beat equal weighting on Sharpe across four market periods. It comes from a paper posted to arXiv on 16 September 2026. Large language models (LLMs) have shown strong performance across diverse financial tasks, yet portfolio management (PM) remains poorly benchmarked. Existing benchmarks exhibit two gaps: they are often equity-only and ignore cross-asset correlations; they fail to evaluate the complete PM decision pipeline. We introduce PortBench, a benchmark spanning six heterogeneous asset classes from 2015 to 2025. PortBench comprises a static QA dataset of 6,269 questions across seven task templates and a dynamic five-stage allocation pipeline. To evaluate these layers, we introduce two metrics: a dual-layer correlation score for inter-class hedging and intra-class concentration, and CEPS, which quantifies how reasoning errors compound across pipeline stages. We further evaluate under three stress windows and three risk profiles, and support real-time evaluation to mitigate pretraining contamination on historical markets. Our source code is available at \href{https://github.com/AgenticFinLab/portbench}{this https URL}.

Why it counts

Across ten frontier LLMs, strong financial QA performance fails to translate into superior portfolio performance: only 32.5\% of 120 evaluations beat equal weighting on Sharpe across four market periods.

Sources

Primary source: primary source

What is not known

This brief does not claim independent replication. Claims that appear only on X and not in the primary source stay unknown.