\documentclass[11pt,a4paper]{article}

% Packages
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{graphicx}
\usepackage{booktabs}
\usepackage{hyperref}
\usepackage{geometry}
\usepackage{float}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{natbib}
\usepackage{setspace}
\usepackage{enumitem}
\usepackage{longtable}
\usepackage{multirow}
\usepackage{xcolor}
\usepackage{listings}

\geometry{margin=1in}
\onehalfspacing

% Title
\title{Historic Options Dataset: Comprehensive Description and Summary Statistics\\
\large SPY, IWM, and QQQ Options Data for Machine Learning-Based Volatility Modeling}
\author{Philipp Dubach\\
\texttt{github.com/philippdubach}}
\date{\today}

\begin{document}

\maketitle

\begin{abstract}
This document provides a comprehensive description of a large-scale dataset comprising options data for three major U.S. equity ETFs: SPY (SPDR S\&P 500 ETF Trust), IWM (iShares Russell 2000 ETF), and QQQ (Invesco QQQ Trust, Nasdaq-100). The combined dataset contains over 53 million option contracts spanning from January 2008 to December 2025, with associated Greeks, implied volatilities, and market microstructure variables. We present detailed summary statistics, distributional analyses, and temporal patterns relevant for researchers investigating implied volatility surface modeling, option pricing, and volatility forecasting using machine learning methods. The dataset covers multiple market regimes including the 2008 financial crisis, the COVID-19 pandemic volatility spike, and subsequent market recoveries, providing rich variation for model training and evaluation.
\end{abstract}

\tableofcontents
\newpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}
\label{sec:introduction}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The implied volatility surface represents one of the most important structures in quantitative finance, encoding market expectations about future asset price distributions across different strike prices and maturities. Understanding and modeling this surface has profound implications for option pricing, risk management, and trading strategies.

This document describes a comprehensive dataset of options on three major U.S. equity ETFs designed to support research in:
\begin{itemize}[noitemsep]
    \item Implied volatility surface modeling and interpolation
    \item Machine learning approaches to option pricing
    \item Volatility forecasting and term structure analysis
    \item Market microstructure analysis of options markets
    \item Arbitrage-free volatility surface construction
    \item Cross-asset volatility correlation studies
\end{itemize}

The dataset includes:
\begin{itemize}[noitemsep]
    \item \textbf{SPY}: S\&P 500 ETF Trust --- Large-cap U.S. equities
    \item \textbf{IWM}: iShares Russell 2000 ETF --- Small-cap U.S. equities
    \item \textbf{QQQ}: Invesco QQQ Trust --- Nasdaq-100 technology-focused equities
\end{itemize}

These three ETFs together provide exposure to different segments of the U.S. equity market, enabling comparative analysis of volatility behavior across market capitalizations and sector exposures.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Data Source and Collection}
\label{sec:data_source}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Data Provider}

The options data was collected from market data providers offering historical option chain snapshots. Each record represents a single option contract observed on a specific trading date, capturing the full option chain available for each underlying on that day.

\subsection{Collection Methodology}

Data collection follows a systematic backfill procedure:
\begin{enumerate}[noitemsep]
    \item Daily option chain snapshots captured at market close
    \item All available strikes and expirations included
    \item Greeks computed using standard Black-Scholes-Merton framework
    \item Implied volatility extracted via Newton-Raphson iteration
\end{enumerate}

\subsection{Data Quality Procedures}

Quality control measures implemented:
\begin{itemize}[noitemsep]
    \item Duplicate contract filtering using unique contract identifiers
    \item Validation of arbitrage-free constraints
    \item Detection and logging of failed data retrievals
    \item Cross-validation of Greeks against theoretical bounds
\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Database Schema}
\label{sec:schema}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The data is stored in SQLite databases (one per underlying) with identical schemas.

\subsection{Options Data Table}

\begin{table}[H]
\centering
\caption{Schema for \texttt{options\_data} table}
\label{tab:schema}
\begin{tabular}{llp{8cm}}
\toprule
\textbf{Column} & \textbf{Type} & \textbf{Description} \\
\midrule
contract\_id & TEXT & Unique option contract identifier \\
symbol & TEXT & Underlying symbol (SPY, IWM, or QQQ) \\
expiration & TEXT & Contract expiration date (YYYY-MM-DD) \\
strike & REAL & Strike price in USD \\
type & TEXT & Option type ('call' or 'put') \\
last & REAL & Last traded price \\
mark & REAL & Mid-market price \\
bid & REAL & Best bid price \\
bid\_size & INTEGER & Size at best bid \\
ask & REAL & Best ask price \\
ask\_size & INTEGER & Size at best ask \\
volume & INTEGER & Daily trading volume \\
open\_interest & INTEGER & Open interest \\
date & TEXT & Observation date (YYYY-MM-DD) \\
implied\_volatility & REAL & Black-Scholes implied volatility \\
delta & REAL & First derivative w.r.t. underlying \\
gamma & REAL & Second derivative w.r.t. underlying \\
theta & REAL & Time decay per day \\
vega & REAL & Sensitivity to volatility \\
rho & REAL & Sensitivity to interest rate \\
in\_the\_money & INTEGER & ITM indicator (0 or 1) \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Underlying Prices Table}

The \texttt{underlying\_prices} table stores daily OHLCV data for each underlying, enabling computation of moneyness and realized volatility measures.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Dataset Overview}
\label{sec:overview}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Available Datasets}

\begin{table}[H]
\centering
\caption{Dataset Summary}
\label{tab:datasets}
\begin{tabular}{llrrr}
\toprule
\textbf{Symbol} & \textbf{Description} & \textbf{Records} & \textbf{Trading Days} & \textbf{Period} \\
\midrule
SPY & SPDR S\&P 500 ETF Trust & 24,681,665 & 4,514 & 2008--2025 \\
IWM & iShares Russell 2000 ETF & 13,379,573 & 4,509 & 2008--2025 \\
QQQ & Invesco QQQ Trust (Nasdaq-100) & 15,345,882 & 3,700 & 2011--2025 \\
\midrule
\textbf{Total} & & \textbf{53,407,120} & & \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Market Regimes Covered}

The dataset spans multiple distinct market regimes:

\begin{itemize}[noitemsep]
    \item \textbf{2008-2009}: Global Financial Crisis, peak VIX $>$ 80
    \item \textbf{2010-2019}: Extended bull market with occasional volatility spikes
    \item \textbf{2020}: COVID-19 pandemic, VIX spike to 82
    \item \textbf{2021-2025}: Post-pandemic normalization and rate hiking cycle
\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{SPY Dataset}
\label{sec:spy}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Overview}

SPY (SPDR S\&P 500 ETF Trust) tracks the S\&P 500 Index and is one of the most liquid ETFs globally. SPY options are among the most actively traded equity options.

\begin{table}[H]
\centering
\caption{SPY Dataset Summary}
\label{tab:spy_summary}
\begin{tabular}{lr}
\toprule
\textbf{Metric} & \textbf{Value} \\
\midrule
Total Records & 24,681,665 \\
Start Date & 2008-01-02 \\
End Date & 2025-12-12 \\
Trading Days & 4,514 \\
Years Covered & 17.9 \\
Calls & 12,340,771 \\
Puts & 12,340,894 \\
Avg. Records per Day & 5,467 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Implied Volatility Statistics}

\begin{table}[H]
\centering
\caption{SPY Implied Volatility Summary Statistics}
\label{tab:spy_iv_stats}
\begin{tabular}{lrrrrrrrr}
\toprule
\textbf{Type} & \textbf{N} & \textbf{Mean} & \textbf{Std} & \textbf{Min} & \textbf{Q1} & \textbf{Median} & \textbf{Q3} & \textbf{Max} \\
\midrule
Call & 12.3M & 0.220 & 0.320 & 0.015 & 0.103 & 0.161 & 0.239 & 9.995 \\
Put & 12.3M & 0.321 & 0.319 & 0.015 & 0.181 & 0.249 & 0.366 & 9.995 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Volume and Open Interest}

\begin{table}[H]
\centering
\caption{SPY Volume and Open Interest Statistics}
\label{tab:spy_volume}
\begin{tabular}{lrrrr}
\toprule
\textbf{Type} & \textbf{Total Volume} & \textbf{Avg Volume} & \textbf{Total OI} & \textbf{Avg OI} \\
\midrule
Call & 7.27B & 589 & 28.8B & 2,333 \\
Put & 9.99B & 810 & 55.8B & 4,518 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Greeks Statistics}

\begin{table}[H]
\centering
\caption{SPY Option Greeks Summary}
\label{tab:spy_greeks}
\begin{tabular}{lrrrrr}
\toprule
\textbf{Greek} & \textbf{Mean} & \textbf{Std} & \textbf{Min} & \textbf{Median} & \textbf{Max} \\
\midrule
Delta & 0.121 & 0.620 & -1.000 & 0.000 & 1.000 \\
Gamma & 0.008 & 0.024 & 0.000 & 0.003 & 5.251 \\
Theta & -0.054 & 0.134 & -24.719 & -0.025 & 3.393 \\
Vega & 0.376 & 0.532 & 0.000 & 0.155 & 4.360 \\
Rho & -0.028 & 1.568 & -166.894 & 0.000 & 11.352 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{SPY Figures}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/spy_iv_time_series.pdf}
\caption{SPY Implied Volatility Time Series (2008-2025)}
\label{fig:spy_iv_ts}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/spy_iv_distribution.pdf}
\caption{SPY Implied Volatility Distribution by Option Type}
\label{fig:spy_iv_dist}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/spy_volume_analysis.pdf}
\caption{SPY Volume and Open Interest Analysis}
\label{fig:spy_volume}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/spy_term_structure.pdf}
\caption{SPY Implied Volatility Term Structure}
\label{fig:spy_term}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/spy_volatility_smile.pdf}
\caption{SPY Volatility Smile (Recent Trading Day)}
\label{fig:spy_smile}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/spy_greeks_distribution.pdf}
\caption{SPY Option Greeks Distribution}
\label{fig:spy_greeks}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{IWM Dataset}
\label{sec:iwm}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Overview}

IWM (iShares Russell 2000 ETF) tracks the Russell 2000 Index, providing exposure to small-cap U.S. equities. Small-cap stocks typically exhibit higher volatility than large-caps.

\begin{table}[H]
\centering
\caption{IWM Dataset Summary}
\label{tab:iwm_summary}
\begin{tabular}{lr}
\toprule
\textbf{Metric} & \textbf{Value} \\
\midrule
Total Records & 13,379,573 \\
Start Date & 2008-01-02 \\
End Date & 2025-12-15 \\
Trading Days & 4,509 \\
Years Covered & 17.9 \\
Calls & 6,689,698 \\
Puts & 6,689,875 \\
Avg. Records per Day & 2,967 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Implied Volatility Statistics}

\begin{table}[H]
\centering
\caption{IWM Implied Volatility Summary Statistics}
\label{tab:iwm_iv_stats}
\begin{tabular}{lrrrrrrrr}
\toprule
\textbf{Type} & \textbf{N} & \textbf{Mean} & \textbf{Std} & \textbf{Min} & \textbf{Q1} & \textbf{Median} & \textbf{Q3} & \textbf{Max} \\
\midrule
Call & 6.7M & 0.255 & 0.331 & 0.015 & 0.130 & 0.195 & 0.283 & 9.995 \\
Put & 6.7M & 0.345 & 0.320 & 0.015 & 0.206 & 0.279 & 0.395 & 9.995 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Volume and Open Interest}

\begin{table}[H]
\centering
\caption{IWM Volume and Open Interest Statistics}
\label{tab:iwm_volume}
\begin{tabular}{lrrrr}
\toprule
\textbf{Type} & \textbf{Total Volume} & \textbf{Avg Volume} & \textbf{Total OI} & \textbf{Avg OI} \\
\midrule
Call & 1.08B & 162 & 9.57B & 1,430 \\
Put & 2.02B & 301 & 21.28B & 3,181 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Greeks Statistics}

\begin{table}[H]
\centering
\caption{IWM Option Greeks Summary}
\label{tab:iwm_greeks}
\begin{tabular}{lrrrrr}
\toprule
\textbf{Greek} & \textbf{Mean} & \textbf{Std} & \textbf{Min} & \textbf{Median} & \textbf{Max} \\
\midrule
Delta & 0.122 & 0.619 & -1.000 & 0.000 & 1.000 \\
Gamma & 0.020 & 0.052 & 0.000 & 0.009 & 5.251 \\
Theta & -0.022 & 0.054 & -5.820 & -0.011 & 3.393 \\
Vega & 0.145 & 0.199 & 0.000 & 0.063 & 2.076 \\
Rho & -0.003 & 0.283 & -13.152 & 0.000 & 2.076 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{IWM Figures}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/iwm_iv_time_series.pdf}
\caption{IWM Implied Volatility Time Series (2008-2025)}
\label{fig:iwm_iv_ts}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/iwm_iv_distribution.pdf}
\caption{IWM Implied Volatility Distribution by Option Type}
\label{fig:iwm_iv_dist}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/iwm_volume_analysis.pdf}
\caption{IWM Volume and Open Interest Analysis}
\label{fig:iwm_volume}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/iwm_term_structure.pdf}
\caption{IWM Implied Volatility Term Structure}
\label{fig:iwm_term}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/iwm_volatility_smile.pdf}
\caption{IWM Volatility Smile (Recent Trading Day)}
\label{fig:iwm_smile}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/iwm_greeks_distribution.pdf}
\caption{IWM Option Greeks Distribution}
\label{fig:iwm_greeks}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{QQQ Dataset}
\label{sec:qqq}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Overview}

QQQ (Invesco QQQ Trust) tracks the Nasdaq-100 Index, providing exposure to the largest non-financial companies listed on the Nasdaq, with heavy weighting toward technology stocks.

\begin{table}[H]
\centering
\caption{QQQ Dataset Summary}
\label{tab:qqq_summary}
\begin{tabular}{lr}
\toprule
\textbf{Metric} & \textbf{Value} \\
\midrule
Total Records & 15,345,882 \\
Start Date & 2011-03-23 \\
End Date & 2025-12-15 \\
Trading Days & 3,700 \\
Years Covered & 14.7 \\
Calls & 7,672,920 \\
Puts & 7,672,962 \\
Avg. Records per Day & 4,148 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Implied Volatility Statistics}

\begin{table}[H]
\centering
\caption{QQQ Implied Volatility Summary Statistics}
\label{tab:qqq_iv_stats}
\begin{tabular}{lrrrrrrrr}
\toprule
\textbf{Type} & \textbf{N} & \textbf{Mean} & \textbf{Std} & \textbf{Min} & \textbf{Q1} & \textbf{Median} & \textbf{Q3} & \textbf{Max} \\
\midrule
Call & 7.7M & 0.267 & 0.345 & 0.015 & 0.121 & 0.188 & 0.299 & 9.995 \\
Put & 7.7M & 0.342 & 0.317 & 0.015 & 0.187 & 0.271 & 0.403 & 9.995 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Volume and Open Interest}

\begin{table}[H]
\centering
\caption{QQQ Volume and Open Interest Statistics}
\label{tab:qqq_volume}
\begin{tabular}{lrrrr}
\toprule
\textbf{Type} & \textbf{Total Volume} & \textbf{Avg Volume} & \textbf{Total OI} & \textbf{Avg OI} \\
\midrule
Call & 3.59B & 468 & 13.93B & 1,816 \\
Put & 4.09B & 533 & 22.70B & 2,960 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Greeks Statistics}

\begin{table}[H]
\centering
\caption{QQQ Option Greeks Summary}
\label{tab:qqq_greeks}
\begin{tabular}{lrrrrr}
\toprule
\textbf{Greek} & \textbf{Mean} & \textbf{Std} & \textbf{Min} & \textbf{Median} & \textbf{Max} \\
\midrule
Delta & 0.119 & 0.619 & -1.000 & 0.000 & 1.000 \\
Gamma & 0.006 & 0.020 & 0.000 & 0.002 & 5.251 \\
Theta & -0.080 & 0.185 & -27.181 & -0.035 & 3.393 \\
Vega & 0.477 & 0.675 & 0.000 & 0.182 & 5.102 \\
Rho & -0.064 & 2.236 & -179.893 & 0.000 & 18.125 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{QQQ Figures}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/qqq_iv_time_series.pdf}
\caption{QQQ Implied Volatility Time Series (2011-2025)}
\label{fig:qqq_iv_ts}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/qqq_iv_distribution.pdf}
\caption{QQQ Implied Volatility Distribution by Option Type}
\label{fig:qqq_iv_dist}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/qqq_volume_analysis.pdf}
\caption{QQQ Volume and Open Interest Analysis}
\label{fig:qqq_volume}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/qqq_term_structure.pdf}
\caption{QQQ Implied Volatility Term Structure}
\label{fig:qqq_term}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/qqq_volatility_smile.pdf}
\caption{QQQ Volatility Smile (Recent Trading Day)}
\label{fig:qqq_smile}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{figures/qqq_greeks_distribution.pdf}
\caption{QQQ Option Greeks Distribution}
\label{fig:qqq_greeks}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Cross-Asset Comparison}
\label{sec:comparison}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Implied Volatility Comparison}

\begin{table}[H]
\centering
\caption{Cross-Asset Implied Volatility Comparison}
\label{tab:iv_comparison}
\begin{tabular}{lrrrrr}
\toprule
\textbf{Symbol} & \textbf{Mean IV (Call)} & \textbf{Mean IV (Put)} & \textbf{IV Skew} & \textbf{Median IV} \\
\midrule
SPY & 0.220 & 0.321 & 0.101 & 0.210 \\
IWM & 0.255 & 0.345 & 0.090 & 0.234 \\
QQQ & 0.267 & 0.342 & 0.075 & 0.227 \\
\bottomrule
\end{tabular}
\end{table}

Key observations:
\begin{itemize}[noitemsep]
    \item IWM (small-cap) exhibits the highest overall implied volatility, consistent with greater uncertainty in small-cap stocks
    \item SPY shows the largest put-call IV skew (0.101), reflecting strong demand for downside protection on the broad market index
    \item QQQ has the smallest IV skew (0.075), potentially due to technology sector characteristics
    \item All three ETFs show the characteristic negative skew (puts more expensive than calls)
\end{itemize}

\subsection{Liquidity Comparison}

\begin{table}[H]
\centering
\caption{Cross-Asset Liquidity Comparison}
\label{tab:liquidity_comparison}
\begin{tabular}{lrrrr}
\toprule
\textbf{Symbol} & \textbf{Total Volume (B)} & \textbf{Avg Daily Vol} & \textbf{Total OI (B)} & \textbf{Records/Day} \\
\midrule
SPY & 17.26 & 700 & 84.6 & 5,467 \\
IWM & 3.10 & 232 & 30.9 & 2,967 \\
QQQ & 7.68 & 501 & 36.6 & 4,148 \\
\bottomrule
\end{tabular}
\end{table}

SPY is by far the most liquid options market, with more than double the volume of QQQ and five times that of IWM.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Data Quality Assessment}
\label{sec:quality}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Data Completeness}

All datasets maintain high data quality with minimal missing values:
\begin{itemize}[noitemsep]
    \item Implied volatility: $>$99\% available across all datasets
    \item Greeks (delta, gamma, theta, vega): $>$98\% available
    \item Volume and open interest: 100\% available
    \item Bid-ask quotes: $>$97\% available
\end{itemize}

\subsection{Data Anomalies}

Potential data quality issues identified (consistent across datasets):
\begin{itemize}[noitemsep]
    \item IV $>$ 5.0: $<$0.1\% of records (extreme values during illiquid periods)
    \item Zero bid-ask spread: $<$0.1\% (stale quotes)
    \item Negative open interest: 0\% (validated)
\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Implications for Machine Learning}
\label{sec:ml_implications}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Feature Engineering Considerations}

The dataset supports various feature engineering approaches:

\begin{enumerate}[noitemsep]
    \item \textbf{Moneyness measures}: $K/S$, $\ln(K/S)$, standardized moneyness
    \item \textbf{Time features}: $\sqrt{\tau}$, $\ln(\tau)$, day-of-week effects
    \item \textbf{Cross-sectional features}: Smile slope, curvature, term slope
    \item \textbf{Historical features}: Rolling IV, realized vol ratios
    \item \textbf{Market microstructure}: Bid-ask spread, volume ratios
    \item \textbf{Cross-asset features}: IV spreads between SPY/IWM/QQQ
\end{enumerate}

\subsection{Sample Size Considerations}

With over 53 million records across three underlyings:
\begin{itemize}[noitemsep]
    \item Sufficient for deep learning approaches
    \item Supports temporal train/validation/test splits
    \item Enables regime-specific model training
    \item Allows for extensive cross-validation
    \item Supports transfer learning across underlyings
\end{itemize}

\subsection{Potential Modeling Targets}

\begin{enumerate}[noitemsep]
    \item \textbf{IV prediction}: Forecast next-day implied volatility
    \item \textbf{Surface interpolation}: Fill missing strikes/maturities
    \item \textbf{Smile dynamics}: Model evolution of smile parameters
    \item \textbf{Option pricing}: Direct price prediction
    \item \textbf{Cross-asset modeling}: Predict IV relationships between underlyings
    \item \textbf{Regime detection}: Identify market stress periods
\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Conclusion}
\label{sec:conclusion}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

This dataset provides a comprehensive foundation for research in volatility modeling and machine learning applications to option pricing. The 17-year span captures multiple market regimes, while the granular contract-level data enables sophisticated feature engineering and model development.

Key dataset strengths:
\begin{itemize}[noitemsep]
    \item Large combined sample size (53M+ records) suitable for deep learning
    \item Three complementary ETFs covering different market segments
    \item Complete option Greeks for feature engineering
    \item High temporal resolution enabling time series analysis
    \item Coverage of multiple volatility regimes
    \item High data quality with minimal missing values
    \item Cross-asset analysis capabilities for correlation studies
\end{itemize}

The dataset is suitable for developing and benchmarking machine learning models for implied volatility surface modeling, contributing to the growing literature on data-driven approaches to derivative pricing.

\end{document}
