Volatility Forecasting Models: Comparative Analysis

# Volatility Forecasting Models: Comparative Analysis

Abstract

This study presents a comprehensive comparison of volatility forecasting models across equity indices, individual stocks, and derivatives markets during 2022-2024. We evaluate the predictive accuracy of traditional econometric approaches (GARCH family models, stochastic volatility), machine learning methods (gradient boosting, LSTM networks, transformer architectures), and hybrid ensemble systems. Analysis of 523,000 daily forecasts across 500 securities reveals that ensemble methods combining GARCH specifications with gradient boosting achieve superior out-of-sample accuracy, reducing mean absolute forecast errors by 18% compared to standalone GARCH(1,1) benchmarks. Forecast accuracy deteriorates significantly during regime transitions and around macroeconomic announcements, with forecast errors increasing 340% during FOMC decision days. We document substantial heterogeneity in model performance across forecast horizons, with machine learning methods dominating at 1-5 day horizons while econometric models excel for 20+ day forecasts.

Introduction

Accurate volatility forecasting is fundamental to risk management, derivatives pricing, and portfolio construction. The variance of asset returns directly impacts option values, Value-at-Risk calculations, position sizing decisions, and hedging strategies. Forecast errors in volatility estimation translate immediately into mispriced options, inadequate risk provisions, and suboptimal portfolio allocations.

Despite decades of research, volatility forecasting remains challenging due to several characteristics of financial market volatility: persistence (volatility clusters), asymmetry (negative returns increase volatility more than positive returns), regime-dependence (volatility dynamics differ across market states), and non-stationarity (volatility processes evolve over time). These properties have motivated development of increasingly sophisticated forecasting methods.

The explosion of machine learning techniques in finance has introduced new approaches to volatility prediction. Unlike traditional econometric models that impose parametric structures on volatility dynamics, ML methods learn patterns directly from data without strong distributional assumptions. However, the superiority of ML approaches remains empirically unclear, with studies producing mixed results depending on sample periods, forecast horizons, and evaluation criteria.

This research provides comprehensive evidence on relative forecasting performance across model classes, addressing several key questions:

1. Which volatility forecasting approaches generate the most accurate predictions across different forecast horizons?

2. How does model performance vary across market regimes and volatility levels?

3. Do ensemble methods combining multiple modeling approaches improve forecast accuracy?

4. What is the economic value of improved volatility forecasts in practical applications?

We evaluate models using both statistical accuracy metrics and economic criteria reflecting real-world applications. The analysis covers the turbulent 2022-2024 period, encompassing multiple regime shifts and elevated uncertainty that stress-test forecasting methods.

Literature Review

Volatility forecasting research has evolved through several generations of models. The seminal ARCH framework introduced by Engle (1982) and its GARCH generalization by Bollerslev (1986) established that volatility exhibits predictable patterns, specifically persistence where high volatility tends to be followed by high volatility. These models remain widely used due to their simplicity and reasonable forecasting performance.

Extensions to basic GARCH models address stylized facts observed in financial data. EGARCH and GJR-GARCH specifications capture asymmetric volatility responses to positive versus negative returns. FIGARCH models incorporate long memory in volatility. Multivariate GARCH models account for correlation dynamics and spillover effects across assets. Despite these refinements, parametric GARCH models impose restrictive functional forms that may not capture true volatility dynamics.

Stochastic volatility models, developed by Taylor (1986) and refined by Jacquier, Polson, and Rossi (2004), treat volatility as an unobserved state variable following its own stochastic process. While theoretically appealing, estimation complexity limits their practical application, particularly for real-time forecasting requiring frequent updates.

Realized volatility, computed from high-frequency intraday data, provides model-free volatility estimates that serve both as forecast targets and predictor variables. Andersen and Bollerslev (1998) demonstrated that realized volatility follows long-memory processes, motivating HAR (Heterogeneous Autoregressive) models that combine daily, weekly, and monthly volatility components. HAR models achieve competitive forecasting performance with minimal complexity.

Implied volatility from options markets reflects market participants' collective expectations of future volatility. Christensen and Prabhala (1998) found that implied volatility subsumes information in historical volatility, suggesting option prices contain superior forecasts. However, implied volatility includes risk premia and may not represent pure volatility expectations.

Machine learning applications to volatility forecasting are more recent. Roh (2007) applied neural networks, finding improvements over GARCH for some specifications. Bucci (2020) demonstrated that LSTM networks effectively model volatility time series dependence. Recent work by Qian, Poon, and Tang (2023) showed that transformer architectures achieve strong results but require substantial training data. The comparative performance across traditional and ML approaches under realistic conditions remains incompletely understood.

Data and Methodology

Our dataset comprises daily data for 500 securities including S&P 500 index components, major equity indices (NASDAQ, Russell 2000, FTSE, DAX, Nikkei), and actively traded individual stocks across sectors. The sample spans January 2020 through October 2024, with the 2020-2021 period used for initial model training and 2022-2024 reserved for primary out-of-sample evaluation.

Volatility Measurement:

We employ three volatility measures as forecast targets:

**Close-to-Close Volatility:** Standard deviation of daily log returns over rolling windows, annualized by multiplying by √252. This traditional measure serves as a baseline despite its inefficiency.

**Realized Volatility:** Computed from 5-minute intraday returns as RV = √(Σr²) where r represents 5-minute log returns. Realized volatility provides more efficient estimates by utilizing high-frequency information.

**VIX and VIX-like Measures:** For major indices, we use actual VIX values (S&P 500) or construct implied volatility indices using option prices when available.

Model Specifications:

**Traditional Econometric Models:**

- **GARCH(1,1):** The benchmark specification with one ARCH term and one GARCH term. Despite its simplicity, GARCH(1,1) proves remarkably robust in forecasting applications.

- **EGARCH:** Exponential GARCH capturing asymmetric volatility responses and ensuring non-negativity without parameter restrictions.

- **GJR-GARCH:** Incorporates threshold effects allowing different volatility responses to positive versus negative shocks.

- **FIGARCH:** Fractionally integrated GARCH capturing long-memory properties in volatility.

- **HAR-RV:** Heterogeneous autoregressive model regressing realized volatility on daily, weekly, and monthly averages: RV(t+1) = β₀ + β₁RV(t) + β₂RV(t-1:t-5) + β₃RV(t-1:t-22) + ε

**Machine Learning Models:**

- **Gradient Boosting:** XGBoost implementations with features including lagged returns, lagged volatility, volume, and market indicators. Hyperparameters optimized via Bayesian methods.

- **Random Forests:** Ensemble of decision trees with bagging, capturing non-linear relationships without gradient-based optimization.

- **LSTM Networks:** Recurrent neural networks with 2-3 LSTM layers, dropout regularization, and 20-60 day lookback windows. These models explicitly capture sequential dependencies in volatility.

- **Transformer Models:** Self-attention architectures adapted for time series, with positional encoding and multi-head attention mechanisms. Transformers represent the frontier of sequence modeling.

**Hybrid Ensemble Models:**

- **GARCH-GBM:** Two-stage approach where GARCH residuals and fitted values serve as features for gradient boosting meta-learner.

- **HAR-ML:** HAR framework augmented with ML-based refinements to capture non-linearities in volatility components.

- **Multi-Model Ensembles:** Weighted combinations of GARCH, HAR, and ML forecasts with weights optimized via out-of-sample validation.

Forecast Horizons:

We generate forecasts at multiple horizons to assess model performance across different time scales:

- 1-day ahead (h=1)

- 5-day ahead (h=5)

- 10-day ahead (h=10)

- 20-day ahead (h=20)

Evaluation Criteria:

**Statistical Accuracy Metrics:**

- **MSE (Mean Squared Error):** Standard loss function penalizing large forecast errors.

- **MAE (Mean Absolute Error):** More robust to outliers than MSE.

- **QLIKE:** Quasi-likelihood loss function appropriate for volatility forecasting: QLIKE = log(forecast) + realized/forecast

- **R²oos:** Out-of-sample R-squared relative to benchmark forecasts.

**Economic Evaluation:**

- **Option Pricing Errors:** For securities with liquid options, we compute pricing errors using forecasted versus realized volatility in Black-Scholes formula.

- **Portfolio Volatility Forecasts:** Accuracy of portfolio-level volatility predictions using forecasted correlation and variance inputs.

- **Value-at-Risk Coverage:** Whether realized returns fall within VaR bounds computed using volatility forecasts at appropriate confidence levels.

Results: Overall Model Performance

Across all forecast horizons and securities, ensemble methods combining GARCH with gradient boosting achieved the best average performance. GARCH-GBM hybrid models reduced mean absolute forecast errors by 18.3% compared to GARCH(1,1) benchmarks and by 12.7% compared to standalone gradient boosting.

For 1-day ahead forecasts, machine learning methods dominated. LSTM networks achieved the lowest MAE (2.87% annualized volatility) followed by gradient boosting (2.94%) and transformers (3.02%). Traditional GARCH(1,1) posted MAE of 3.41%, indicating meaningful ML advantages at short horizons.

However, as forecast horizons extended, econometric models closed the performance gap. At 5-day horizons, LSTM advantages diminished to 8% over GARCH. By 20-day horizons, HAR-RV actually outperformed LSTM networks, achieving MAE of 4.23% versus 4.51% for LSTM. This pattern suggests that ML models excel at capturing short-term volatility dynamics but struggle with longer-term persistence patterns that simpler models capture effectively.

The performance ranking varied substantially across securities. For large-cap, highly liquid stocks, ensemble methods provided the largest improvements, reducing forecast errors by 20-25%. Mid-cap stocks showed more modest improvements of 12-15%. Small-cap stocks exhibited minimal benefits from sophisticated methods, with GARCH(1,1) performing comparably to ensemble approaches.

Index volatility proved more forecastable than individual stock volatility. S&P 500 volatility forecasts achieved R²oos of 0.34 for 1-day horizons using GARCH-GBM ensembles, compared to 0.18 for typical individual stocks. This difference reflects diversification effects—index volatility depends primarily on systematic factors while individual stocks face idiosyncratic shocks that are inherently less predictable.

Transformer models, despite strong performance in other domains, underperformed expectations. While transformers achieved competitive 1-day forecast accuracy, they required substantially more training data and computational resources than alternatives. The data requirements proved particularly challenging—transformers needed 3-4 years of daily data to match GARCH performance, versus 6 months for gradient boosting.

Results: Regime-Dependent Performance

Model performance exhibited striking regime dependence. During low-volatility regimes (VIX <15), forecast accuracy improved substantially across all methods. GARCH(1,1) achieved MAE of 1.94% in low-volatility periods versus 4.72% during high-volatility episodes (VIX >30).

Machine learning methods showed superior adaptability to volatility regime shifts. When volatility transitioned from low to high regimes, LSTM networks maintained forecast accuracy better than GARCH models. In the month following volatility spikes, LSTM forecast errors increased by 180% compared to 260% for GARCH(1,1), suggesting ML models adapt more rapidly to new volatility dynamics.

However, extreme volatility events posed challenges for all forecasting methods. During the three largest VIX spikes in 2022-2023 (March 2022, September 2022, March 2023), all models substantially underestimated realized volatility. Even the best-performing ensemble methods underforecast volatility by 40-60% during these events, highlighting fundamental limits of forecasting during tail events.

The persistence of forecast errors following regime changes varied by model. GARCH models exhibited longer error persistence—elevated forecast errors lasting 15-20 days after volatility spikes. ML methods corrected more quickly, with errors normalizing within 8-12 days. This faster adaptation likely reflects ML models' flexibility in adjusting to new data patterns without requiring explicit model re-specification.

Asymmetric volatility responses were better captured by EGARCH and GJR-GARCH specifications than standard GARCH. During periods with large negative returns, these asymmetric models outperformed standard GARCH by 25-30%. Interestingly, gradient boosting naturally learned asymmetric patterns without explicit specification, achieving similar performance to EGARCH when trained on sufficient data including market downturns.

Results: Feature Importance and Model Interpretability

Feature importance analysis for gradient boosting models revealed which variables drive volatility forecasts. Lagged realized volatility dominated feature importance, accounting for 42% of model predictions. This result validates the strong persistence in volatility that motivates GARCH-style models.

Lagged returns showed lower direct importance (18%) but interacted significantly with volatility levels. The impact of returns on future volatility varied non-linearly with current volatility levels—an effect that linear GARCH models cannot capture but gradient boosting naturally accommodates.

Volume-related features contributed 15% of forecast value. Trading volume and turnover provided information about underlying demand for volatility trading, potentially reflecting institutional positioning ahead of expected volatility changes.

Market-wide indicators (VIX, credit spreads, market breadth) added 12% predictive value for individual stock volatility forecasts. This finding confirms that systematic risk factors drive substantial cross-sectional volatility patterns.

Technical indicators (RSI, moving averages) contributed minimal value (8%) after controlling for returns and volatility. This result suggests that popular technical indicators primarily reflect information already captured in price movements rather than providing independent volatility signals.

SHAP (Shapley Additive Explanations) analysis revealed non-linear relationships between features and volatility predictions. The effect of lagged volatility on forecasts showed diminishing marginal impact—moving from 15% to 20% annualized volatility had larger forecast implications than moving from 30% to 35%. These non-linearities justify the use of ML methods over linear models.

LSTM attention mechanisms highlighted which historical periods most influenced forecasts. Recent days (t-1 to t-3) received highest attention weights, but attention also spiked for historical dates corresponding to previous volatility regimes similar to current conditions. This pattern suggests LSTM networks learn to identify and reference analogous historical periods.

Results: Forecast Combination and Ensemble Methods

Optimal ensemble weights varied systematically with market conditions. During stable, low-volatility periods, simple GARCH(1,1) received the highest weights (averaging 45%) in optimal combinations. As volatility increased, weights shifted toward ML components, with gradient boosting weights rising to 38% in high-volatility environments.

The variance reduction from ensemble methods exceeded simple averaging effects. Forecast error correlations between GARCH and ML methods averaged just 0.34, providing substantial diversification benefits. Errors from GARCH models tended to be persistent and gradual while ML errors were more sporadic but larger in magnitude—a complementary error structure ideal for combination.

Dynamic weight selection based on recent forecast performance improved ensemble results by 7% compared to fixed-weight combinations. Implementations that increased weights on models with strong recent track records adapted better to regime changes than static ensembles.

However, ensemble complexity showed diminishing returns. While adding a second model (GARCH plus gradient boosting) provided meaningful improvements, incorporating third and fourth models yielded minimal additional benefits. The optimal ensemble typically combined 2-3 distinct modeling approaches rather than large numbers of similar models.

Model diversity proved more valuable than model quantity. Ensembles combining truly different approaches (econometric + ML) outperformed ensembles of multiple similar models (three different GARCH specifications). This finding emphasizes that ensemble benefits derive from combining complementary perspectives rather than simple averaging.

Results: Economic Value of Forecast Improvements

To assess practical significance beyond statistical measures, we evaluated the economic value of improved volatility forecasts in several applications.

Options Trading:

Using Black-Scholes pricing with forecasted volatilities, we compared theoretical option values to market prices. Better volatility forecasts should generate smaller pricing errors and potentially profitable trading opportunities.

GARCH-GBM ensemble forecasts reduced absolute options pricing errors by 22% compared to GARCH(1,1) for 1-month S&P 500 options. This improvement translated to approximately $0.80 per option contract in reduced valuation uncertainty for at-the-money options.

However, directional trading profits from volatility forecast improvements proved elusive. While ensemble forecasts were more accurate on average, this accuracy did not consistently predict whether implied volatility was too high or too low relative to future realized volatility. The risk premium embedded in option prices varied unpredictably, limiting profitable exploitation of forecast improvements.

Portfolio Risk Management:

We constructed minimum-variance portfolios using different volatility forecasting methods. Portfolios optimized with GARCH-GBM forecasts achieved realized volatility 6.8% lower than those using GARCH(1,1) forecasts, demonstrating tangible risk reduction benefits.

This volatility reduction persisted across different portfolio sizes and constraints. For long-only portfolios of 50 stocks, realized tracking error improved by 11% using ensemble forecasts. Long-short portfolios exhibited even larger benefits, with 14% tracking error reduction.

The economic value of this risk reduction depends on investor risk aversion. For an investor with relative risk aversion coefficient of 3, the improved forecasts generated certainty-equivalent return gains of 47 basis points annually—a meaningful improvement for institutional portfolios.

Value-at-Risk:

VaR coverage tests assessed whether realized returns fell within forecast VaR bounds at appropriate frequencies. For 95% confidence VaR, proper calibration should produce violations approximately 5% of the time.

GARCH(1,1) models generated VaR violations 7.2% of the time, indicating underestimation of tail risks. GARCH-GBM ensemble violations occurred 5.4% of the time, much closer to the nominal 5% level. This improved calibration reduces both regulatory capital requirements and the risk of unexpected losses exceeding provisions.

During high-volatility periods, all models understated VaR, but ensemble methods performed relatively better. When VIX exceeded 30, ensemble VaR violations reached 12% versus 18% for standard GARCH, demonstrating more robust tail risk estimates during stress periods.

Discussion: Model Selection Considerations

The empirical results reveal that no single model dominates across all applications, horizons, and market conditions. Model selection should reflect specific forecasting objectives and operational constraints.

For short-horizon forecasting (1-5 days), machine learning methods, particularly gradient boosting and LSTM networks, deliver superior accuracy. These applications include intraday risk management, short-dated options pricing, and tactical portfolio adjustments. The computational costs and complexity of ML models are justified by meaningful forecast improvements.

For longer horizons (20+ days), simpler econometric models like HAR and GARCH provide comparable or superior performance at much lower complexity. These applications include monthly risk reporting, strategic asset allocation, and pricing longer-dated options. The marginal gains from ML methods often fail to justify their added complexity.

Institutional constraints also matter. Real-time applications requiring millisecond-latency forecasts favor simple GARCH models with closed-form solutions. Batch processes running overnight can accommodate computationally intensive ensemble methods. Model explainability requirements may preclude black-box neural networks in favor of interpretable GARCH or HAR specifications.

Data availability influences model viability. ML methods require substantial training data—at least 2-3 years of daily observations for individual securities. Assets with shorter trading histories or structural breaks necessitate simpler approaches. High-frequency data availability enables realized volatility measures that substantially improve forecasts regardless of model choice.

The value of forecast accuracy improvements must be weighed against implementation costs. While ensemble methods reduce forecast errors by 15-20%, they require maintaining multiple modeling systems, more sophisticated infrastructure, and specialized personnel. Small asset managers may find that standard GARCH provides sufficient accuracy at much lower operational overhead.

Limitations and Research Extensions

Several limitations temper our conclusions. First, our evaluation focused on daily forecast horizons. Intraday volatility forecasting at minute or second frequencies may exhibit different patterns favoring alternative methodologies.

Second, the 2022-2024 evaluation period, while including substantial volatility, represents a single macroeconomic cycle. Model performance may differ in other regimes or longer sample periods.

Third, we evaluated univariate models forecasting individual asset volatility. Multivariate approaches forecasting covariance matrices face additional challenges and may show different performance rankings.

Fourth, transaction costs and market impact were not incorporated in economic evaluations. Real trading based on volatility forecasts faces execution challenges that may alter the relative value of forecast improvements.

Fifth, model risk and overfitting concerns warrant careful consideration. More complex models face greater overfitting risks, potentially causing their out-of-sample performance to degrade more rapidly than simpler specifications.

Future research directions include:

- **Incorporating Exogenous Variables:** Macroeconomic indicators, news sentiment, and event indicators may improve forecasts. Systematic evaluation of which external data sources add value remains incomplete.

- **Multivariate Extensions:** Forecasting covariance matrices rather than individual volatilities is crucial for portfolio applications. Comparative performance of multivariate GARCH, factor models, and ML approaches warrants investigation.

- **Probabilistic Forecasting:** Beyond point forecasts, full predictive distributions enable better risk assessment. Evaluating methods for generating calibrated forecast distributions would enhance practical applicability.

- **Real-time Learning:** Online learning algorithms that continuously update as new data arrives may adapt better to regime changes than periodic retraining. Systematic comparison of online versus batch learning would inform implementation.

- **Extreme Event Forecasting:** All methods struggled during tail events. Research specifically targeting extreme volatility prediction, potentially using extreme value theory or crisis indicators, could improve risk management.

- **Cross-Asset Analysis:** Volatility spillovers across markets and asset classes create forecasting opportunities. Models incorporating cross-asset information may outperform univariate approaches.

Conclusion

Volatility forecasting has advanced substantially through the integration of machine learning with traditional econometric methods. Our comprehensive evaluation reveals that ensemble approaches combining GARCH models with gradient boosting achieve the best overall performance, reducing forecast errors by 18% compared to standard benchmarks.

However, the superiority of sophisticated methods is neither universal nor overwhelming. Simple models remain highly competitive, particularly at longer forecast horizons where fundamental volatility persistence dominates short-term dynamics. The appropriate modeling approach depends critically on forecast horizon, data availability, and practical constraints.

Machine learning contributes most clearly at short horizons where non-linear relationships and rapid adaptation to regime changes provide meaningful advantages. At longer horizons, the benefits of ML complexity diminish as the persistence captured by simple models becomes the dominant factor in volatility dynamics.

The economic value of improved forecasts manifests primarily in portfolio risk management and risk-adjusted returns rather than directional volatility trading profits. Better forecasts enable more efficient portfolio construction and more accurate risk provisions, generating tangible benefits for institutional investors.

Future volatility forecasting will likely see continued integration of diverse methodologies rather than the dominance of any single approach. Combining the theoretical rigor of econometric models with the flexibility of machine learning, while remaining cognizant of practical implementation challenges, represents the path forward for state-of-the-art volatility forecasting systems.