Decision-Induced Ranking Explains Prediction Inflation and Excessive Turnover in SPO-Based Portfolio Optimization

Decision-Induced Ranking, Prediction Inflation, and Turnover in SPO-Based Portfolio Optimization

Introduction

The paper "Decision-Induced Ranking Explains Prediction Inflation and Excessive Turnover in SPO-Based Portfolio Optimization" (2605.01176) examines the behavior of decision-focused learning (DFL), particularly the Smart Predict-then-Optimize (SPO) framework, in portfolio optimization. The authors provide both mechanistic and empirical analyses to understand why SPO-trained models often produce inflated return predictions and excessive portfolio turnover, phenomena that challenge practical implementability. The study leverages Karush–Kuhn–Tucker (KKT) based characterizations, detailed ablation experiments, and practical stabilization techniques to articulate the core mechanism underlying SPO-induced decisions.

Decision-Focused Learning and Portfolio Optimization Mechanisms

Traditional predict-then-optimize (PtO) approaches optimize predictors for accuracy, subsequently using their outputs as inputs for downstream optimization, commonly in portfolio allocation tasks. However, these approaches often fail to guarantee that increased prediction accuracy results in better decision quality, especially given the nonlinear and non-stationary nature of asset returns. DFL—by directly optimizing predictors for downstream decision quality—aims to bridge this gap.

Analytically, the paper highlights that portfolio optimization decisions are fundamentally governed by the ranking over risk- and transaction-cost-adjusted marginal scores. Through KKT conditions, the portfolio allocation is shown to depend less on absolute predicted return levels and more critically on the structure of their cross-sectional ranking, adjusted for risk and transaction costs. This ranking-driven allocation mechanism can result in highly concentrated portfolios and unstable transitions, as the decision-relevant ordering overshadows calibration of predicted return magnitudes.

Figure 1: Monthly predicted and realized mean returns under $\lambda=20$, demonstrating the large fluctuations and inflation in DFL predictions compared to both realized and PTO predictions.

Empirical Evidence: Prediction Inflation and Turnover Instability

Empirical investigation reveals that portfolios managed using SPO+ surrogate loss exhibit both prediction inflation and excessive turnover across multiple datasets (DOW, ETF_A, ETF_B). Predicted returns from SPO-trained models display much wider dispersion than both true and PTO-predicted returns. This is directly linked to the optimization mechanism which tends to amplify asset score gaps to induce sharper allocation decisions. Increasing risk aversion within the mean–variance framework did not sufficiently mitigate turnover, confirming the instability is not solely attributable to downstream risk penalization.

Figure 2: Distribution of predicted and realized returns under $\lambda=20$ for Standard, Clip, and Rescale settings. The effect of clipping and rescaling is evident in terms of prediction concentration and preserved ranking structure.

Practical Interventions: Stabilizing SPO-Based Portfolios

To address instability, the paper evaluates practical interventions that operate at both prediction and portfolio levels:

1. Prediction Scale Control:
   • Clipping restricts predicted returns to realistic bounds, suppressing outlier signals without altering overall ranking structure.
   • Min–max Rescaling normalizes the entire prediction vector within each period, bounding magnitudes while preserving intra-period rank order.
2. Portfolio Adjustment:
   • Partial Portfolio Adjustment reduces turnover by gradually transitioning the current allocation towards the SPO optimizer’s target, controlled by an adjustment speed parameter $\delta$.

Empirical results confirm that partial adjustment (Adj) and its combinations (Clip+Adj, Rescale+Adj) dominate in reducing turnover and volatility. Clipping alone yields improved concentration but limited turnover reduction, while rescaling preserves decision-inducing ranking signals. Notably, Clip+Adj is the most conservative and stable, whereas Rescale+Adj frequently achieves superior returns and Sharpe ratios.

Numerical Results and Interpretations

Ablation studies across varying risk aversion $\lambda$ and asset universes demonstrate that turnover reduction is primarily achieved via partial portfolio adjustment—prediction scale control delivers additional stability but is secondary in effect compared to smoothing realized portfolio changes. Ranking preservation (via rescale) maintains the strength of SPO-induced signals, supporting the ranking-centric interpretation. Both turnover and volatility are minimized with combination strategies that incorporate adjustment, validating the view that practical implementability necessitates portfolio-level control mechanisms in addition to prediction-level constraints.

Distributional Analysis of Predicted Returns

The comparative distributional analysis further substantiates the finding that SPO-trained predictions are not calibrated return forecasts but decision-inducing signals with exaggerated score gaps. Clipping and rescaling alter magnitude but do not fundamentally change the cross-sectional ranking, meaning that allocation instability persists unless portfolio transition is directly constrained.

Implications and Prospects

The work delivers robust evidence that DFL—and SPO in particular—shapes predictors to favor decision-induced ranking signals over calibrated forecasting. This shift leads to inflated predicted returns, aggressive portfolio concentrations, and excessive turnover. For practical portfolio management, this necessitates the introduction of realistic output bounds and turnover control to reconcile decision optimality with trading implementability.

Theoretically, the findings reinforce interpretations of DFL as learning-to-rank, inviting further exploration into ranking-sensitive loss formulations and integrated optimization constraints. Practically, the control mechanisms evaluated here—prediction clipping, rescaling, and partial adjustment—are viable for enhancing stability and realism in algorithmic trading strategies premised on decision-focused learning.

Future Directions

Future research should address the gap between mathematically optimal decisions under DFL objectives and the pragmatic requirements of financial markets, such as diversification, liquidity-aware trading, and adaptive turnover regulation. Integrated learning objectives that directly encode ranking margins or decision smoothness, or that embed financial constraints into the training loop, may yield portfolios that are both theoretically optimal and practically reasonable. Extended empirical assessment across broader asset classes and market regimes would further clarify the robustness and universality of these strategies.

Conclusion

This paper rigorously characterizes the mechanism of SPO-based decision-focused learning in portfolio optimization, substantiating the role of decision-induced ranking in prediction inflation and turnover. Through KKT-based analysis, extensive empirical testing, and practical intervention studies, it identifies ranking-driven score gaps as the primary source of instability and proposes actionable stabilization strategies. The findings emphasize a trade-off between exploiting decision-focused signals and ensuring implementable trading behavior, pointing towards ranking-aware losses and realistic constraint integration as promising future research avenues in DFL-driven portfolio strategies.