TimeGPT vs Statistical Models for Forecasting SPY

In the previous post of this TimeGPT series, we used TimeGPT to forecast the price of SPY in the next 7 days. In this post, we compare TimeGPT's performance against traditional statistical forecasting methods, namely the AutoARIMA Model and Optimized Theta Model, which are two well-known and versatile time series models.

We will be using the same dataset - last year's SPY adjusted closing prices, and apply each algorithm to forecast the next 7 days' prices. Here's a sample of the dataset:

Setup Models

AutoARIMA and Optimized Theta Model are available from StatsForecast package. First install the package:

$ pip install statsforecast

Then set up the models:

from statsforecast.models import AutoARIMA, OptimizedTheta

season_length = 7 # Daily price data
horizon = 7 # Prediction horizon

models = [AutoARIMA(season_length=season_length), OptimizedTheta(season_length=season_length)]
sf = StatsForecast(df=df, models=models, freq='C')  # 'C' for custom business days

Cross Validation

For each model, we will perform a 5-window cross validation, in each window we split the data for training and predict the remaining 7-day prices.

cv_sf = sf.cross_validation(df=df, h=horizon, step_size=horizon, n_windows=5, level=[90])
cv_timegpt = nixtla_client.cross_validation(
    df, 
    h=horizon, 
    n_windows=5, 
    freq='C',
    level=[90]
)

Evaluation Metrics and Result

For each model and their predictions, we compute Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Symmetric Mean Absolute Percentage Error (SMAPE) as evaluation metrics. The lower the metric, the better the prediction performance.

Optimized Theta Model is the clear winner across all evaluation metrics. AutoARIMA beats TimeGPT in 4 out of 5 metrics.

Summary

In this post, we compared TimeGPT, AutoARIMA and Optimized Theta on forecasting SPY's closing prices in the next 7 days. I encourage you to try different algorithms on your time series data and share your thoughts!