Modeling Cascaded Delay Feedback for Online Net Conversion Rate Prediction: Benchmark, Insights and Solutions

Here is the official implenmentation of our WWW 2026 paper Modeling Cascaded Delay Feedback for Online Net Conversion Rate Prediction: Benchmark, Insights and Solutions. This repository provides a comprehensive benchmark and open-source toolkit for Modeling Cascaded Delay Feedback in online Net Conversion Rate (NetCVR) prediction.

In this work, we present systematic insights into the cascading nature of delayed feedback signals and propose effective modeling solutions. This codebase includes datasets, models, training pipelines, and evaluation tools to support future research in delay feedback modeling.

---

📦 Dataset

The experiments are based on a large-scale industrial dataset from Alibaba, capturing multi-stage user behaviors including click, add-to-cart, payment, and refund, with precise timestamps for modeling delay dynamics.

👉 Dataset Information:
CASCADE dataset on HuggingFace

📁 Data structure includes:

• User/item/Related Features
• Timestamps for each conversion stage (click_time, pay_time, refund_time)

data source should be placed under data/CASCADE/.

---

🧪 Baseline Models

Below are the baseline models included in this benchmark, along with their original paper references and corresponding implementation scripts.

Model Name	Model Reference Script
ESDFM	ali_reesdfm_stream_pretrain.py
MISS	ali_remiss_stream_train.py
DFSN	ali_redfsn_stream_train.py
Oracle	ali_reoracle_stream_train.py
FNW	ali_refnw_stream_train.py
FNC	ali_refnc_stream_train.py
Defuse	ali_redefuse_stream_train.py
Defer	ali_redefer_stream_train.py
DDFM	ali_reddfm_stream_train.py
TESLA (Ours)	ali_TESLA_stream_train.py

📁 Project Structure

AirBench4OpenSource/
├── data/ # Raw and metadata files
├── dataloader/ # Custom data loading modules
├── datasets/ # Dataset classes and preprocessing scripts
├── log/ # Training logs and evaluation outputs
├── models/ # Model architectures (e.g., CascadeNet, ESDFM)
├── mx_utils/ # Utility functions: metrics, config, logging, etc.
├── trainers/ # Training and evaluation logic
├── examples/ # Example scripts for quick start
├── requirements.txt # Required Python packages
├── README.md # This file
└── LICENSE # MIT License

---

🚀 Quick Start

1. Clone the Repository

git clone git@gitlab.alibaba-inc.com:CASCADE/AirBench4OpenSource.git
cd AirBench4OpenSource
python -m venv venv
source venv/bin/activate # Linux/Mac# venv\Scripts\activate # Windows# Install dependencies
pip install -r requirements.txt

2. processing data

Download the CASCADE dataset on HuggingFace and process it by using scripts below and place it under data/CASCADE/.

# to process data
python process_CASCADE_with_MappingDict.py

3. run an example script

To run the main training script for our model, use:

# to direct run our model 
python AirBench4OpenSource/ali_TESLA_stream_train.py

Specifically, you need to run the following pre-training scripts in advance:

# Step 1: Pre-train the base model
python AirBench4OpenSource/ali_esdfmRf_PLE_pretrain.py
# Step 2: Pre-train the inw-tn-pay delay feedback model
python AirBench4OpenSource/ali_esdfmRF_inw_tn_pay_pretrain.py
# Step 3: Pre-train the inw-tn-refund delay feedback model
python AirBench4OpenSource/ali_esdfmRF_inw_tn_refund_pretrain.py

These scripts will generate the necessary checkpoint files (model weights), which are then loaded by ali_TESLA_stream_train.py during training. More usage examples can be found in the scripts under the examples/ directory.