Do Deep Learning Methods Improve Financial Forecasts?

Views and opinions expressed are those of the authors and do not necessarily represent official positions or policy of the Office of Financial Research or the U.S. Department of the Treasury.

Forecasting financial markets, from stock returns to bond prices to early signs of funding stress, informs decisions across monetary policy, bank supervision, and risk management. New methods are being developed at a remarkable pace as a result of progress in artificial intelligence, with each new method promising sharper predictions than the last. Researchers often disagree about which method works best, but settling that question requires comparing methods using the same data under the same rules. That is the role of a benchmark.

There are few openly available benchmarks for financial markets, and until now, not many have focused specifically on finance. The gap matters because of a well-established result in machine learning and in particular, the no free lunch theorem that states that no single forecasting method is best for every kind of data. A method that excels at predicting electricity demand or retail sales may not accurately predict bond yields or exchange rates. Knowing which methods work for financial markets requires testing on financial data prepared in the way market specialists actually use it.

Building such a benchmark is not just a matter of gathering data, and this is where subject matter expertise becomes essential. Cleaning financial data requires consequential choices for a specific market like which securities are tradable, how to treat unreliable or stale prices, how to handle quotes that violate basic no-arbitrage relationships, and the time period. The same raw source can yield meaningfully different datasets depending on how it is handled, and those differences ultimately alter a study’s conclusions. Constructing each dataset correctly requires a familiarity with a particular market’s conventions. A fair comparison also requires holding the data fixed so that differences in measured performance reflect the methods themselves rather than the data preparation behind them.

In the working paper, “An Open Benchmark for Evaluating Time Series Forecasting Methods across Financial Markets,” OFR researcher Jeremiah Bejarano and coauthors Viren Desai, Kausthub Keshava, Arsh Kumar, Zixiao Wang, Vincent Hanyang Xu, and Yangge Xu construct the first comprehensive open benchmark focused specifically on financial markets. They assemble a standardized collection of datasets spanning seven asset classes—equities, corporate bonds, Treasuries, foreign exchange, commodities, credit default swaps, and options—along with measures of funding stress and bank balance sheet health. Each dataset is cleaned using methods matching those developed by a seminal academic paper for that market so the data match how specialists use them. The authors then evaluate about a dozen forecasting methods on identical data, ranging from classical statistical models to the latest deep learning architectures.

Grouping the methods into families and comparing them as classes makes the central pattern clear (Figure 1). No family of methods is best everywhere, and the value of the newer deep learning methods depends on the specific forecast.

Pooled across all datasets (Panel A), every family of methods is at least better than the naïve benchmark (a simple historical average). However, the newer hybrid and deep learning methods post the best scores. The families finish fairly close in the aggregate, and the differences that matter for practice emerge once the datasets are separated by type (Panel B).

For funding stress basis spreads, the newer hybrid and deep learning methods deliver large and genuine gains that doubling their predictive accuracy. These spreads contain recurring structure, such as predictable funding pressures around the end of each quarter, that the more flexible methods are well suited to capture. This is exactly the kind of signal financial stability monitors watch, since basis spreads tend to widen when funding becomes scarce and markets come under strain.

For bank and intermediary indicators, the newer methods remain competitive with the best classical models, with deep learning methods posting the highest median accuracy on the typical series, though the simpler classical methods are steadier on average. Asset returns, such as stocks, bonds, and currencies, are the hardest case because no family improves on a simple historical average. This result is consistent with decades of asset pricing research and one that holds for all the methods tested here, which forecast each series from its own past alone.

These results make the no free lunch theorem concrete within a single financial system. The newest deep learning methods work best when the data contain rich, repeating structure, such as funding stress basis spreads, and offer less when a simple benchmark is hard to beat, such as asset returns. Because the best method depends on the task, a domain-specific benchmark built on fixed, carefully prepared data is what makes a credible choice possible. By holding the data fixed and releasing the full data cleaning pipeline, the authors make these comparisons reproducible, and they give researchers a common yardstick for judging whether each new forecasting method represents genuine progress for financial markets.