OptionMetrics has major data revisions 2024 08 relative to 2023 08

[chenandrewy]

After running the whole repo, we found large t-stat revisions in option-based predictors:

The revisions for share issuance and credit ratings can be traced to bug fixes. But the options code has not changed.

We happened to have some OptionMetrics vol surface data from 2023 08. If we compare last year's vintage to this year, we get revisions to implied vol of more than 5 p.p. for roughly 10% of stock-months:

We find evidence of similar revisions in the price dataset. There seem to have been revisions in previous years, but not like the magnitude we see this year. For example, here are revisions to the SmileSlope predictor:

And here are some correlations for signals

As seen above, we don't find major revisions in the option volumes.

We've heard a bit from other researchers about revisions to OptionMetrics too.

[chenandrewy]

For the 2024 08 release, we opted to use the 2023 08 options signals. For clear record keeping, I used the following script to splice the new and old data: https://github.com/OpenSourceAP/CrossSection/blob/master/Signals/Code/04_OM_Splicer_2024.R. This script was run after running the signals code but before running the portfolios code. It just removes an option predictors from the Signals/Data/ subfolders and replaces them with the 2023 08 versions.

[chenandrewy]

Seems like OptionMetrics updated their methodology in March 2024: https://optionmetrics.com/news/optionmetrics-releases-ivydb-us-6-0-and-ivydb-etf-4-0-with-proprietary-methodologies-for-faster-more-precise-options-implied-volatilities/.

This "New Proprietary" methodology was probably hard to use for generating trading signals in the past ;) But also, it seems options researchers need to decide what to do with the new observations with > 300% implied vol.

For now, using the pre-2024 OptionMetrics data is the most transparent method.

[chenandrewy]

It seems that we (@tomz23 , me, +OptionMetrics) inadvertently extended Muravyev, Pearson, and Pollet's JFE R&R ("Why Does Option Market Information Predict Stock Returns?").

MPP found that CPVolSpread and skew1 predictability is to a significant extent eaten up by stock borrowing fees (see their Tables 4 and 5). They also show that CPVolSpread signal is linked to the borrow fee theoretically (Equation 5).

A natural implication of their paper is, then, that constructing CPVolSpread and skew1 based on borrow-fee-adjusted implied vols will reduce predictability. That's exactly what we (with the help of OptionMetric's "New Proprietary Implied Lending Fees Methodology") find.

H/T Dima Muravyev for pointing us to their paper.

[chenandrewy]

MPP point out the importance of accounting for borrow fees on page 2

Perhaps OptionMetrics should cite their paper? That would certainly be a useful reference for users of the data.

[chenandrewy]

In the 2025 release, we're still kicking this can down the road. Hopefully in the future we'll compute our own implied vols the old fashioned way and replicate Bali-Hovakimian and others. But now for I asked @claude to patch as follows:

# Plan for patching Option Metrics Implied Vol signals

In 2024-08, we found that Option Metrics had significantly revised their implied vol calculations, leading to a sharp drop in predictor performance.
See https://github.com/OpenSourceAP/CrossSection/issues/156.

Eventually, we hope to re-do the implied vol calculations ourselves to replicate the original papers. But for now, we'll continue to patch by using the 2023-08 vintage of the data.

## Task 

Work on only one script at a time.

For the script you are working on:

- At the beginning of add a check for whether `pyCode/config.py` `PATCH_OPTIONM_IV == True`. 
- If true, use the `openassetpricing` package to download the 2023 vintage and stop the script there.
```python
print("WARNING: PATCH_OPTIONM_IV is True, using 2023 vintage from openassetpricing")
print("See https://github.com/OpenSourceAP/CrossSection/issues/156")
from openassetpricing import OpenAP
openap = OpenAP(2023)
df = openap.dl_signal('polars', ['dVolCall'])
df = df.rename({'yyyymm': 'time_avail_m'})
save_predictor(df, 'dVolCall')
sys.exit()