AI/ML Platform & Applications

Scaling time-series forecasting models to cope with the multi-verse with Ray

September 19, 2:30 PM - 3:00 PM
View Slides

At JPMorgan Chase we need robust forecasts of market prices and other time-series to provide a high-quality financial service to our clients. However, time-series forecasts in financial markets are difficult due to the richness of the systems involved and low signal-to-noise ratio. Furthermore, often the underlying drivers (e.g. Inflation) are non-stationary. In this work, we present the concept of probabilistic forecasting at scale powered by Ray, and we apply this technique to improve time-series models for multiple use cases relevant to the finance industry.

When forecasting time-series, one must consider the question: "Is the future likely to represent this exact version of history?" Or can we take random subsets of the past and develop a probabilistic model to forecast the future based on multiple slices of past data (i.e. a multi-verse approach)? This kind of sampling (called back-testing) helps remove the influence of outlier data points, while being representative of history, but requires much, much more processing. This is why we scale our project with Ray. To be more specific, probabilistic forecasting and non-stationarity necessitate large scale compute and distributed ML model development:

  • Probabilistic forecasting requires a distribution of outcomes
  • Non-stationarity is addressed through a back-testing framework where instead of training a single global ML model, a sequence of models is trained periodically over time accounting for the changing macro and micro-economic dynamics

Potential use cases for this work are forecasting stock, commodity & energy prices, interest rates, exchange rates and a large variety of tradable assets. The forecasts could help traders and investors make better informed decisions about buying and selling assets, modeling other assets that depend on these time-series forecasts and managing risk. These use-cases can be expanded to cover different forecast horizons. A short-term model might be appropriate for day trading, while a medium-term model might inform asset selection in a portfolio, and finally a long-term forecast might help with stock valuation or asset construction decisions (e.g. wind-farms needed by 2050).

Our research team has created a platform for large scale ML based time-series forecasting for the above use cases. The backbone of the platform is Ray and its capabilities to efficiently distribute large scale computations on cloud leveraging Kubernetes. This JPMorgan Chase platform is used internally to conduct probabilistic regression, feature engineering, feature selection, hyper-parameter optimization, and probabilistic analysis metrics, with scaling powered by Ray.

Distributed time-series forecasting has the potential to drive significant improvements in efficiency, profitability, and sustainability, thanks to the power of Ray.


Peyman Tavallali and Savinay Narendra are Vice Presidents and Applied AI ML Leads at JPMorgan Chase's Machine Learning Center of Excellence.

Berowne D Hlavaty is an Executive Director in JPMorgan Chase's Big Data & AI Strategies Research team.

About Peyman

Peyman Tavallali joined JPMC in May-2022 focusing on probabilistic regression/forecasting, TS analysis, ML uncertainty quantification, explainable ML, and distributed computation since then. In general, Peyman uses applied ML and applied math to assimilate in and provide value to JPMC.

Before joining JPMC, Peyman was working at NASA's JPL leading and participating in many of JPL's and NASA's cutting-edge technology projects using tools in ML and applied math. He also led numerous uncertainty quantification ML efforts for different use cases at JPL.

Peyman Tavallali

Vice President Applied AI ML Lead, JP Morgan Chase
Photo of Ray Summit pillows
Ray Summit 23 logo

Ready to Register?

Come connect with the global community of thinkers and disruptors who are building and deploying the next generation of AI and ML applications.

Photo of Ray pillows and Raydiate sign
Photo of Raydiate sign

Join the Conversation

Ready to get involved in the Ray community before the conference? Ask a question in the forums. Open a pull request. Or share why you’re excited with the hashtag #RaySummit on Twitter.