Add Bagz format support by Marlon666 · Pull Request #4496 · AI-Hypercomputer/maxtext

Marlon666 · 2026-07-15T23:16:20Z

Description

This Pull Request introduces end-to-end support for the Gemini-backed Bagz (.bagz) dataset format within MaxText's PyGrain input pipeline.

Context & Problem Solved:
Previously, MaxText users relying on the ultra-efficient Bagz format for large-scale pre-training faced challenges when bridging cloud storage with PyGrain workers. This PR enables a highly optimized, POSIX-compliant file ingestion path designed explicitly for local filesystem and naturally GCS Fuse mounts. By avoiding complex REST/RPC shims, this approach achieves robust, multiprocess-safe read scalability out-of-the-box.

Scope of Changes:

Input Pipeline: Added the BagzDataSource wrapper in input_pipeline_utils.py for parallel and resilient Bagz shard reader instantiation.
Grain Integration: Registered bagz as a first-class supported file type in grain_data_processing.py, enabling native ParseFeatures and NormalizeFeatures handling.
Configurations: Expanded base.yml and types.py to seamlessly accept "bagz" as a valid grain_file_type.
Tooling: Created download_hf_dataset_as_bagz.py with multi-worker parallelism, HuggingFace streaming, and robust checkpoint recovery.

FIXES: (If there is a Github Issue #, paste it here, otherwise leave blank)

Tests

The implementation was successfully validated end-to-end on both Data Generation and Model Training levels:

Parallel Shard Generation: Successfully streamed and converted the Salesforce/wikitext dataset (wikitext-103-raw-v1) into 38 contiguous .bagz file shards using multi-worker execution.
End-to-End Model Training: Executed a 3-step offline training loop on a CPU-only environment using the generated local Bagz dataset via DECOUPLE_GCLOUD=TRUE (Log URL attached below).

Commands used for local testing:

# Data Generation
python tools/data_generation/download_hf_dataset_as_bagz.py \
--dataset Salesforce/wikitext \
--config '{"name": "wikitext-103-raw-v1"}' \
--output ~/datasets/wikitext_bagz \
--file-size-mb 10 \
--workers 2
# MaxText CPU Smoke Testexport DECOUPLE_GCLOUD=TRUE
python -m maxtext.trainers.pre_train.train \
src/maxtext/configs/base.yml \
run_name="bagz_cpu_smoke_test" \
steps=3 \
dataset_type="grain" \
grain_file_type="bagz" \
grain_train_files="$HOME/datasets/wikitext_bagz/*.bagz" \
grain_eval_files="$HOME/datasets/wikitext_bagz/*.bagz" \
tokenize_eval_data=True \
per_device_batch_size=1 \
max_target_length=64 \
enable_checkpointing=false \
override_model_config=true \
base_num_decoder_layers=1 \
base_emb_dim=32 \
base_mlp_dim=32 \
head_dim=8 \
base_num_kv_heads=1 \
base_num_query_heads=1
Test Logs URL: https://paste.googleplex.com/6387838210932736

lepan-google · 2026-07-17T20:46:26Z

+ """A fully picklable and fork-safe RandomAccessDataSource for Bagz files."""
+ def __init__(self, paths: list[str] | str):
+ if isinstance(paths, (list, tuple)):
+ self._path = ",".join(sorted(paths))


Why we have to sort the paths here?

aireenmei

Thanks for adding the feature! Could you also add unit tests in grain_data_processing_test.py? You will need a class setup class similar to _GrainArrayRecordSetup and a test class similar to GrainArrayRecordProcessingTest

aireenmei · 2026-07-30T00:07:26Z

+# limitations under the License.
+
+"""
+Download a HuggingFace dataset via streaming and save as Bagz files.


Could you add some example commands in the docstring?

Done, I've added detailed example commands in the docstring covering basic local conversion, GCSFUSE-mounted bucket output with custom prefix/file size, and authenticated access for gated datasets

Implements end-to-end support for the Bagz (.bagz) dataset format inside the MaxText training and data processing pipeline. Enables highly efficient, POSIX-compliant file loading optimized specifically for GCS FUSE and local filesystem environments. Scope of Changes: - Input Pipeline: Integrated upstream `grain.BagzDataSource` for direct, multiprocess-safe reading of Bagz file shards via PyGrain. - Data Processing: Registered `bagz` as a first-class supported file type across PyGrain data loaders, feature normalizers, and tokenizers. - Tooling: Created `download_hf_dataset_as_bagz.py` with multi-worker support, HuggingFace streaming, and resilient checkpointing for GCSFUSE mounts. Testing: - Unit Tests: Added comprehensive unit test suite in `tests/unit/bagz_data_processing_test.py` covering RandomAccessDataSource compatibility, MapDataset, ElasticIterator (single and multi-worker), and end-to-end `get_datasets` integration. - End-to-End Local Smoke Test: Executed a 3-step offline training loop on a local CPU environment using the newly generated Bagz dataset via `DECOUPLE_GCLOUD=TRUE`. - Parallel Data Generation: Successfully converted and streamed HF datasets into multi-shard Bagz files (38 shards, Salesforce/wikitext).

…ring Add practical example commands to the docstring of `download_hf_dataset_as_bagz.py`: - Basic local directory conversion (Salesforce/wikitext) - Write-optimized GCSFUSE mounted bucket output with custom shard prefix and file size - Private/gated HuggingFace dataset download using auth token

Marlon666 · 2026-07-15T23:16:20Z

Description

This Pull Request introduces end-to-end support for the Gemini-backed Bagz (.bagz) dataset format within MaxText's PyGrain input pipeline.

Context & Problem Solved:
Previously, MaxText users relying on the ultra-efficient Bagz format for large-scale pre-training faced challenges when bridging cloud storage with PyGrain workers. This PR enables a highly optimized, POSIX-compliant file ingestion path designed explicitly for local filesystem and naturally GCS Fuse mounts. By avoiding complex REST/RPC shims, this approach achieves robust, multiprocess-safe read scalability out-of-the-box.

Scope of Changes:

Input Pipeline: Added the BagzDataSource wrapper in input_pipeline_utils.py for parallel and resilient Bagz shard reader instantiation.
Grain Integration: Registered bagz as a first-class supported file type in grain_data_processing.py, enabling native ParseFeatures and NormalizeFeatures handling.
Configurations: Expanded base.yml and types.py to seamlessly accept "bagz" as a valid grain_file_type.
Tooling: Created download_hf_dataset_as_bagz.py with multi-worker parallelism, HuggingFace streaming, and robust checkpoint recovery.

FIXES: (If there is a Github Issue #, paste it here, otherwise leave blank)

Tests

The implementation was successfully validated end-to-end on both Data Generation and Model Training levels:

Parallel Shard Generation: Successfully streamed and converted the Salesforce/wikitext dataset (wikitext-103-raw-v1) into 38 contiguous .bagz file shards using multi-worker execution.
End-to-End Model Training: Executed a 3-step offline training loop on a CPU-only environment using the generated local Bagz dataset via DECOUPLE_GCLOUD=TRUE (Log URL attached below).

Commands used for local testing:

# Data Generation
python tools/data_generation/download_hf_dataset_as_bagz.py \
--dataset Salesforce/wikitext \
--config '{"name": "wikitext-103-raw-v1"}' \
--output ~/datasets/wikitext_bagz \
--file-size-mb 10 \
--workers 2
# MaxText CPU Smoke Testexport DECOUPLE_GCLOUD=TRUE
python -m maxtext.trainers.pre_train.train \
src/maxtext/configs/base.yml \
run_name="bagz_cpu_smoke_test" \
steps=3 \
dataset_type="grain" \
grain_file_type="bagz" \
grain_train_files="$HOME/datasets/wikitext_bagz/*.bagz" \
grain_eval_files="$HOME/datasets/wikitext_bagz/*.bagz" \
tokenize_eval_data=True \
per_device_batch_size=1 \
max_target_length=64 \
enable_checkpointing=false \
override_model_config=true \
base_num_decoder_layers=1 \
base_emb_dim=32 \
base_mlp_dim=32 \
head_dim=8 \
base_num_kv_heads=1 \
base_num_query_heads=1
Test Logs URL: https://paste.googleplex.com/6387838210932736

lepan-google · 2026-07-17T20:46:26Z

+ """A fully picklable and fork-safe RandomAccessDataSource for Bagz files."""
+ def __init__(self, paths: list[str] | str):
+ if isinstance(paths, (list, tuple)):
+ self._path = ",".join(sorted(paths))


Why we have to sort the paths here?

aireenmei

Thanks for adding the feature! Could you also add unit tests in grain_data_processing_test.py? You will need a class setup class similar to _GrainArrayRecordSetup and a test class similar to GrainArrayRecordProcessingTest

aireenmei · 2026-07-30T00:07:26Z

+# limitations under the License.
+
+"""
+Download a HuggingFace dataset via streaming and save as Bagz files.


Could you add some example commands in the docstring?

Done, I've added detailed example commands in the docstring covering basic local conversion, GCSFUSE-mounted bucket output with custom prefix/file size, and authenticated access for gated datasets

Implements end-to-end support for the Bagz (.bagz) dataset format inside the MaxText training and data processing pipeline. Enables highly efficient, POSIX-compliant file loading optimized specifically for GCS FUSE and local filesystem environments. Scope of Changes: - Input Pipeline: Integrated upstream `grain.BagzDataSource` for direct, multiprocess-safe reading of Bagz file shards via PyGrain. - Data Processing: Registered `bagz` as a first-class supported file type across PyGrain data loaders, feature normalizers, and tokenizers. - Tooling: Created `download_hf_dataset_as_bagz.py` with multi-worker support, HuggingFace streaming, and resilient checkpointing for GCSFUSE mounts. Testing: - Unit Tests: Added comprehensive unit test suite in `tests/unit/bagz_data_processing_test.py` covering RandomAccessDataSource compatibility, MapDataset, ElasticIterator (single and multi-worker), and end-to-end `get_datasets` integration. - End-to-End Local Smoke Test: Executed a 3-step offline training loop on a local CPU environment using the newly generated Bagz dataset via `DECOUPLE_GCLOUD=TRUE`. - Parallel Data Generation: Successfully converted and streamed HF datasets into multi-shard Bagz files (38 shards, Salesforce/wikitext).

…ring Add practical example commands to the docstring of `download_hf_dataset_as_bagz.py`: - Basic local directory conversion (Salesforce/wikitext) - Write-optimized GCSFUSE mounted bucket output with custom shard prefix and file size - Private/gated HuggingFace dataset download using auth token

Marlon666 · 2026-07-15T23:16:20Z

Description

This Pull Request introduces end-to-end support for the Gemini-backed Bagz (.bagz) dataset format within MaxText's PyGrain input pipeline.

Context & Problem Solved:
Previously, MaxText users relying on the ultra-efficient Bagz format for large-scale pre-training faced challenges when bridging cloud storage with PyGrain workers. This PR enables a highly optimized, POSIX-compliant file ingestion path designed explicitly for local filesystem and naturally GCS Fuse mounts. By avoiding complex REST/RPC shims, this approach achieves robust, multiprocess-safe read scalability out-of-the-box.

Scope of Changes:

Input Pipeline: Added the BagzDataSource wrapper in input_pipeline_utils.py for parallel and resilient Bagz shard reader instantiation.
Grain Integration: Registered bagz as a first-class supported file type in grain_data_processing.py, enabling native ParseFeatures and NormalizeFeatures handling.
Configurations: Expanded base.yml and types.py to seamlessly accept "bagz" as a valid grain_file_type.
Tooling: Created download_hf_dataset_as_bagz.py with multi-worker parallelism, HuggingFace streaming, and robust checkpoint recovery.

FIXES: (If there is a Github Issue #, paste it here, otherwise leave blank)

Tests

The implementation was successfully validated end-to-end on both Data Generation and Model Training levels:

Parallel Shard Generation: Successfully streamed and converted the Salesforce/wikitext dataset (wikitext-103-raw-v1) into 38 contiguous .bagz file shards using multi-worker execution.
End-to-End Model Training: Executed a 3-step offline training loop on a CPU-only environment using the generated local Bagz dataset via DECOUPLE_GCLOUD=TRUE (Log URL attached below).

Commands used for local testing:

# Data Generation
python tools/data_generation/download_hf_dataset_as_bagz.py \
--dataset Salesforce/wikitext \
--config '{"name": "wikitext-103-raw-v1"}' \
--output ~/datasets/wikitext_bagz \
--file-size-mb 10 \
--workers 2
# MaxText CPU Smoke Testexport DECOUPLE_GCLOUD=TRUE
python -m maxtext.trainers.pre_train.train \
src/maxtext/configs/base.yml \
run_name="bagz_cpu_smoke_test" \
steps=3 \
dataset_type="grain" \
grain_file_type="bagz" \
grain_train_files="$HOME/datasets/wikitext_bagz/*.bagz" \
grain_eval_files="$HOME/datasets/wikitext_bagz/*.bagz" \
tokenize_eval_data=True \
per_device_batch_size=1 \
max_target_length=64 \
enable_checkpointing=false \
override_model_config=true \
base_num_decoder_layers=1 \
base_emb_dim=32 \
base_mlp_dim=32 \
head_dim=8 \
base_num_kv_heads=1 \
base_num_query_heads=1
Test Logs URL: https://paste.googleplex.com/6387838210932736

lepan-google · 2026-07-17T20:46:26Z

+ """A fully picklable and fork-safe RandomAccessDataSource for Bagz files."""
+ def __init__(self, paths: list[str] | str):
+ if isinstance(paths, (list, tuple)):
+ self._path = ",".join(sorted(paths))


Why we have to sort the paths here?

aireenmei

Thanks for adding the feature! Could you also add unit tests in grain_data_processing_test.py? You will need a class setup class similar to _GrainArrayRecordSetup and a test class similar to GrainArrayRecordProcessingTest

aireenmei · 2026-07-30T00:07:26Z

+# limitations under the License.
+
+"""
+Download a HuggingFace dataset via streaming and save as Bagz files.


Could you add some example commands in the docstring?

Done, I've added detailed example commands in the docstring covering basic local conversion, GCSFUSE-mounted bucket output with custom prefix/file size, and authenticated access for gated datasets

Implements end-to-end support for the Bagz (.bagz) dataset format inside the MaxText training and data processing pipeline. Enables highly efficient, POSIX-compliant file loading optimized specifically for GCS FUSE and local filesystem environments. Scope of Changes: - Input Pipeline: Integrated upstream `grain.BagzDataSource` for direct, multiprocess-safe reading of Bagz file shards via PyGrain. - Data Processing: Registered `bagz` as a first-class supported file type across PyGrain data loaders, feature normalizers, and tokenizers. - Tooling: Created `download_hf_dataset_as_bagz.py` with multi-worker support, HuggingFace streaming, and resilient checkpointing for GCSFUSE mounts. Testing: - Unit Tests: Added comprehensive unit test suite in `tests/unit/bagz_data_processing_test.py` covering RandomAccessDataSource compatibility, MapDataset, ElasticIterator (single and multi-worker), and end-to-end `get_datasets` integration. - End-to-End Local Smoke Test: Executed a 3-step offline training loop on a local CPU environment using the newly generated Bagz dataset via `DECOUPLE_GCLOUD=TRUE`. - Parallel Data Generation: Successfully converted and streamed HF datasets into multi-shard Bagz files (38 shards, Salesforce/wikitext).

…ring Add practical example commands to the docstring of `download_hf_dataset_as_bagz.py`: - Basic local directory conversion (Salesforce/wikitext) - Write-optimized GCSFUSE mounted bucket output with custom shard prefix and file size - Private/gated HuggingFace dataset download using auth token

Marlon666 · 2026-07-15T23:16:20Z

Description

This Pull Request introduces end-to-end support for the Gemini-backed Bagz (.bagz) dataset format within MaxText's PyGrain input pipeline.

Context & Problem Solved:
Previously, MaxText users relying on the ultra-efficient Bagz format for large-scale pre-training faced challenges when bridging cloud storage with PyGrain workers. This PR enables a highly optimized, POSIX-compliant file ingestion path designed explicitly for local filesystem and naturally GCS Fuse mounts. By avoiding complex REST/RPC shims, this approach achieves robust, multiprocess-safe read scalability out-of-the-box.

Scope of Changes:

Input Pipeline: Added the BagzDataSource wrapper in input_pipeline_utils.py for parallel and resilient Bagz shard reader instantiation.
Grain Integration: Registered bagz as a first-class supported file type in grain_data_processing.py, enabling native ParseFeatures and NormalizeFeatures handling.
Configurations: Expanded base.yml and types.py to seamlessly accept "bagz" as a valid grain_file_type.
Tooling: Created download_hf_dataset_as_bagz.py with multi-worker parallelism, HuggingFace streaming, and robust checkpoint recovery.

FIXES: (If there is a Github Issue #, paste it here, otherwise leave blank)

Tests

The implementation was successfully validated end-to-end on both Data Generation and Model Training levels:

Parallel Shard Generation: Successfully streamed and converted the Salesforce/wikitext dataset (wikitext-103-raw-v1) into 38 contiguous .bagz file shards using multi-worker execution.
End-to-End Model Training: Executed a 3-step offline training loop on a CPU-only environment using the generated local Bagz dataset via DECOUPLE_GCLOUD=TRUE (Log URL attached below).

Commands used for local testing:

# Data Generation
python tools/data_generation/download_hf_dataset_as_bagz.py \
--dataset Salesforce/wikitext \
--config '{"name": "wikitext-103-raw-v1"}' \
--output ~/datasets/wikitext_bagz \
--file-size-mb 10 \
--workers 2
# MaxText CPU Smoke Testexport DECOUPLE_GCLOUD=TRUE
python -m maxtext.trainers.pre_train.train \
src/maxtext/configs/base.yml \
run_name="bagz_cpu_smoke_test" \
steps=3 \
dataset_type="grain" \
grain_file_type="bagz" \
grain_train_files="$HOME/datasets/wikitext_bagz/*.bagz" \
grain_eval_files="$HOME/datasets/wikitext_bagz/*.bagz" \
tokenize_eval_data=True \
per_device_batch_size=1 \
max_target_length=64 \
enable_checkpointing=false \
override_model_config=true \
base_num_decoder_layers=1 \
base_emb_dim=32 \
base_mlp_dim=32 \
head_dim=8 \
base_num_kv_heads=1 \
base_num_query_heads=1
Test Logs URL: https://paste.googleplex.com/6387838210932736

lepan-google · 2026-07-17T20:46:26Z

+ """A fully picklable and fork-safe RandomAccessDataSource for Bagz files."""
+ def __init__(self, paths: list[str] | str):
+ if isinstance(paths, (list, tuple)):
+ self._path = ",".join(sorted(paths))


Why we have to sort the paths here?

aireenmei

Thanks for adding the feature! Could you also add unit tests in grain_data_processing_test.py? You will need a class setup class similar to _GrainArrayRecordSetup and a test class similar to GrainArrayRecordProcessingTest

aireenmei · 2026-07-30T00:07:26Z

+# limitations under the License.
+
+"""
+Download a HuggingFace dataset via streaming and save as Bagz files.


Could you add some example commands in the docstring?

Done, I've added detailed example commands in the docstring covering basic local conversion, GCSFUSE-mounted bucket output with custom prefix/file size, and authenticated access for gated datasets

Implements end-to-end support for the Bagz (.bagz) dataset format inside the MaxText training and data processing pipeline. Enables highly efficient, POSIX-compliant file loading optimized specifically for GCS FUSE and local filesystem environments. Scope of Changes: - Input Pipeline: Integrated upstream `grain.BagzDataSource` for direct, multiprocess-safe reading of Bagz file shards via PyGrain. - Data Processing: Registered `bagz` as a first-class supported file type across PyGrain data loaders, feature normalizers, and tokenizers. - Tooling: Created `download_hf_dataset_as_bagz.py` with multi-worker support, HuggingFace streaming, and resilient checkpointing for GCSFUSE mounts. Testing: - Unit Tests: Added comprehensive unit test suite in `tests/unit/bagz_data_processing_test.py` covering RandomAccessDataSource compatibility, MapDataset, ElasticIterator (single and multi-worker), and end-to-end `get_datasets` integration. - End-to-End Local Smoke Test: Executed a 3-step offline training loop on a local CPU environment using the newly generated Bagz dataset via `DECOUPLE_GCLOUD=TRUE`. - Parallel Data Generation: Successfully converted and streamed HF datasets into multi-shard Bagz files (38 shards, Salesforce/wikitext).

…ring Add practical example commands to the docstring of `download_hf_dataset_as_bagz.py`: - Basic local directory conversion (Salesforce/wikitext) - Write-optimized GCSFUSE mounted bucket output with custom shard prefix and file size - Private/gated HuggingFace dataset download using auth token

Marlon666 · 2026-07-15T23:16:20Z

Description

This Pull Request introduces end-to-end support for the Gemini-backed Bagz (.bagz) dataset format within MaxText's PyGrain input pipeline.

Context & Problem Solved:
Previously, MaxText users relying on the ultra-efficient Bagz format for large-scale pre-training faced challenges when bridging cloud storage with PyGrain workers. This PR enables a highly optimized, POSIX-compliant file ingestion path designed explicitly for local filesystem and naturally GCS Fuse mounts. By avoiding complex REST/RPC shims, this approach achieves robust, multiprocess-safe read scalability out-of-the-box.

Scope of Changes:

Input Pipeline: Added the BagzDataSource wrapper in input_pipeline_utils.py for parallel and resilient Bagz shard reader instantiation.
Grain Integration: Registered bagz as a first-class supported file type in grain_data_processing.py, enabling native ParseFeatures and NormalizeFeatures handling.
Configurations: Expanded base.yml and types.py to seamlessly accept "bagz" as a valid grain_file_type.
Tooling: Created download_hf_dataset_as_bagz.py with multi-worker parallelism, HuggingFace streaming, and robust checkpoint recovery.

FIXES: (If there is a Github Issue #, paste it here, otherwise leave blank)

Tests

The implementation was successfully validated end-to-end on both Data Generation and Model Training levels:

Parallel Shard Generation: Successfully streamed and converted the Salesforce/wikitext dataset (wikitext-103-raw-v1) into 38 contiguous .bagz file shards using multi-worker execution.
End-to-End Model Training: Executed a 3-step offline training loop on a CPU-only environment using the generated local Bagz dataset via DECOUPLE_GCLOUD=TRUE (Log URL attached below).

Commands used for local testing:

# Data Generation
python tools/data_generation/download_hf_dataset_as_bagz.py \
--dataset Salesforce/wikitext \
--config '{"name": "wikitext-103-raw-v1"}' \
--output ~/datasets/wikitext_bagz \
--file-size-mb 10 \
--workers 2
# MaxText CPU Smoke Testexport DECOUPLE_GCLOUD=TRUE
python -m maxtext.trainers.pre_train.train \
src/maxtext/configs/base.yml \
run_name="bagz_cpu_smoke_test" \
steps=3 \
dataset_type="grain" \
grain_file_type="bagz" \
grain_train_files="$HOME/datasets/wikitext_bagz/*.bagz" \
grain_eval_files="$HOME/datasets/wikitext_bagz/*.bagz" \
tokenize_eval_data=True \
per_device_batch_size=1 \
max_target_length=64 \
enable_checkpointing=false \
override_model_config=true \
base_num_decoder_layers=1 \
base_emb_dim=32 \
base_mlp_dim=32 \
head_dim=8 \
base_num_kv_heads=1 \
base_num_query_heads=1
Test Logs URL: https://paste.googleplex.com/6387838210932736

lepan-google · 2026-07-17T20:46:26Z

+ """A fully picklable and fork-safe RandomAccessDataSource for Bagz files."""
+ def __init__(self, paths: list[str] | str):
+ if isinstance(paths, (list, tuple)):
+ self._path = ",".join(sorted(paths))


Why we have to sort the paths here?

aireenmei

Thanks for adding the feature! Could you also add unit tests in grain_data_processing_test.py? You will need a class setup class similar to _GrainArrayRecordSetup and a test class similar to GrainArrayRecordProcessingTest

aireenmei · 2026-07-30T00:07:26Z

+# limitations under the License.
+
+"""
+Download a HuggingFace dataset via streaming and save as Bagz files.


Could you add some example commands in the docstring?

Done, I've added detailed example commands in the docstring covering basic local conversion, GCSFUSE-mounted bucket output with custom prefix/file size, and authenticated access for gated datasets

Implements end-to-end support for the Bagz (.bagz) dataset format inside the MaxText training and data processing pipeline. Enables highly efficient, POSIX-compliant file loading optimized specifically for GCS FUSE and local filesystem environments. Scope of Changes: - Input Pipeline: Integrated upstream `grain.BagzDataSource` for direct, multiprocess-safe reading of Bagz file shards via PyGrain. - Data Processing: Registered `bagz` as a first-class supported file type across PyGrain data loaders, feature normalizers, and tokenizers. - Tooling: Created `download_hf_dataset_as_bagz.py` with multi-worker support, HuggingFace streaming, and resilient checkpointing for GCSFUSE mounts. Testing: - Unit Tests: Added comprehensive unit test suite in `tests/unit/bagz_data_processing_test.py` covering RandomAccessDataSource compatibility, MapDataset, ElasticIterator (single and multi-worker), and end-to-end `get_datasets` integration. - End-to-End Local Smoke Test: Executed a 3-step offline training loop on a local CPU environment using the newly generated Bagz dataset via `DECOUPLE_GCLOUD=TRUE`. - Parallel Data Generation: Successfully converted and streamed HF datasets into multi-shard Bagz files (38 shards, Salesforce/wikitext).

…ring Add practical example commands to the docstring of `download_hf_dataset_as_bagz.py`: - Basic local directory conversion (Salesforce/wikitext) - Write-optimized GCSFUSE mounted bucket output with custom shard prefix and file size - Private/gated HuggingFace dataset download using auth token

Marlon666 · 2026-07-15T23:16:20Z

Description

This Pull Request introduces end-to-end support for the Gemini-backed Bagz (.bagz) dataset format within MaxText's PyGrain input pipeline.

Context & Problem Solved:
Previously, MaxText users relying on the ultra-efficient Bagz format for large-scale pre-training faced challenges when bridging cloud storage with PyGrain workers. This PR enables a highly optimized, POSIX-compliant file ingestion path designed explicitly for local filesystem and naturally GCS Fuse mounts. By avoiding complex REST/RPC shims, this approach achieves robust, multiprocess-safe read scalability out-of-the-box.

Scope of Changes:

Input Pipeline: Added the BagzDataSource wrapper in input_pipeline_utils.py for parallel and resilient Bagz shard reader instantiation.
Grain Integration: Registered bagz as a first-class supported file type in grain_data_processing.py, enabling native ParseFeatures and NormalizeFeatures handling.
Configurations: Expanded base.yml and types.py to seamlessly accept "bagz" as a valid grain_file_type.
Tooling: Created download_hf_dataset_as_bagz.py with multi-worker parallelism, HuggingFace streaming, and robust checkpoint recovery.

FIXES: (If there is a Github Issue #, paste it here, otherwise leave blank)

Tests

The implementation was successfully validated end-to-end on both Data Generation and Model Training levels:

Parallel Shard Generation: Successfully streamed and converted the Salesforce/wikitext dataset (wikitext-103-raw-v1) into 38 contiguous .bagz file shards using multi-worker execution.
End-to-End Model Training: Executed a 3-step offline training loop on a CPU-only environment using the generated local Bagz dataset via DECOUPLE_GCLOUD=TRUE (Log URL attached below).

Commands used for local testing:

# Data Generation
python tools/data_generation/download_hf_dataset_as_bagz.py \
--dataset Salesforce/wikitext \
--config '{"name": "wikitext-103-raw-v1"}' \
--output ~/datasets/wikitext_bagz \
--file-size-mb 10 \
--workers 2
# MaxText CPU Smoke Testexport DECOUPLE_GCLOUD=TRUE
python -m maxtext.trainers.pre_train.train \
src/maxtext/configs/base.yml \
run_name="bagz_cpu_smoke_test" \
steps=3 \
dataset_type="grain" \
grain_file_type="bagz" \
grain_train_files="$HOME/datasets/wikitext_bagz/*.bagz" \
grain_eval_files="$HOME/datasets/wikitext_bagz/*.bagz" \
tokenize_eval_data=True \
per_device_batch_size=1 \
max_target_length=64 \
enable_checkpointing=false \
override_model_config=true \
base_num_decoder_layers=1 \
base_emb_dim=32 \
base_mlp_dim=32 \
head_dim=8 \
base_num_kv_heads=1 \
base_num_query_heads=1
Test Logs URL: https://paste.googleplex.com/6387838210932736

lepan-google · 2026-07-17T20:46:26Z

+ """A fully picklable and fork-safe RandomAccessDataSource for Bagz files."""
+ def __init__(self, paths: list[str] | str):
+ if isinstance(paths, (list, tuple)):
+ self._path = ",".join(sorted(paths))


Why we have to sort the paths here?

aireenmei

Thanks for adding the feature! Could you also add unit tests in grain_data_processing_test.py? You will need a class setup class similar to _GrainArrayRecordSetup and a test class similar to GrainArrayRecordProcessingTest

aireenmei · 2026-07-30T00:07:26Z

+# limitations under the License.
+
+"""
+Download a HuggingFace dataset via streaming and save as Bagz files.


Could you add some example commands in the docstring?

Done, I've added detailed example commands in the docstring covering basic local conversion, GCSFUSE-mounted bucket output with custom prefix/file size, and authenticated access for gated datasets

Implements end-to-end support for the Bagz (.bagz) dataset format inside the MaxText training and data processing pipeline. Enables highly efficient, POSIX-compliant file loading optimized specifically for GCS FUSE and local filesystem environments. Scope of Changes: - Input Pipeline: Integrated upstream `grain.BagzDataSource` for direct, multiprocess-safe reading of Bagz file shards via PyGrain. - Data Processing: Registered `bagz` as a first-class supported file type across PyGrain data loaders, feature normalizers, and tokenizers. - Tooling: Created `download_hf_dataset_as_bagz.py` with multi-worker support, HuggingFace streaming, and resilient checkpointing for GCSFUSE mounts. Testing: - Unit Tests: Added comprehensive unit test suite in `tests/unit/bagz_data_processing_test.py` covering RandomAccessDataSource compatibility, MapDataset, ElasticIterator (single and multi-worker), and end-to-end `get_datasets` integration. - End-to-End Local Smoke Test: Executed a 3-step offline training loop on a local CPU environment using the newly generated Bagz dataset via `DECOUPLE_GCLOUD=TRUE`. - Parallel Data Generation: Successfully converted and streamed HF datasets into multi-shard Bagz files (38 shards, Salesforce/wikitext).

…ring Add practical example commands to the docstring of `download_hf_dataset_as_bagz.py`: - Basic local directory conversion (Salesforce/wikitext) - Write-optimized GCSFUSE mounted bucket output with custom shard prefix and file size - Private/gated HuggingFace dataset download using auth token

Marlon666 · 2026-07-15T23:16:20Z

Description

This Pull Request introduces end-to-end support for the Gemini-backed Bagz (.bagz) dataset format within MaxText's PyGrain input pipeline.

Context & Problem Solved:
Previously, MaxText users relying on the ultra-efficient Bagz format for large-scale pre-training faced challenges when bridging cloud storage with PyGrain workers. This PR enables a highly optimized, POSIX-compliant file ingestion path designed explicitly for local filesystem and naturally GCS Fuse mounts. By avoiding complex REST/RPC shims, this approach achieves robust, multiprocess-safe read scalability out-of-the-box.

Scope of Changes:

Input Pipeline: Added the BagzDataSource wrapper in input_pipeline_utils.py for parallel and resilient Bagz shard reader instantiation.
Grain Integration: Registered bagz as a first-class supported file type in grain_data_processing.py, enabling native ParseFeatures and NormalizeFeatures handling.
Configurations: Expanded base.yml and types.py to seamlessly accept "bagz" as a valid grain_file_type.
Tooling: Created download_hf_dataset_as_bagz.py with multi-worker parallelism, HuggingFace streaming, and robust checkpoint recovery.

FIXES: (If there is a Github Issue #, paste it here, otherwise leave blank)

Tests

The implementation was successfully validated end-to-end on both Data Generation and Model Training levels:

Parallel Shard Generation: Successfully streamed and converted the Salesforce/wikitext dataset (wikitext-103-raw-v1) into 38 contiguous .bagz file shards using multi-worker execution.
End-to-End Model Training: Executed a 3-step offline training loop on a CPU-only environment using the generated local Bagz dataset via DECOUPLE_GCLOUD=TRUE (Log URL attached below).

Commands used for local testing:

# Data Generation
python tools/data_generation/download_hf_dataset_as_bagz.py \
--dataset Salesforce/wikitext \
--config '{"name": "wikitext-103-raw-v1"}' \
--output ~/datasets/wikitext_bagz \
--file-size-mb 10 \
--workers 2
# MaxText CPU Smoke Testexport DECOUPLE_GCLOUD=TRUE
python -m maxtext.trainers.pre_train.train \
src/maxtext/configs/base.yml \
run_name="bagz_cpu_smoke_test" \
steps=3 \
dataset_type="grain" \
grain_file_type="bagz" \
grain_train_files="$HOME/datasets/wikitext_bagz/*.bagz" \
grain_eval_files="$HOME/datasets/wikitext_bagz/*.bagz" \
tokenize_eval_data=True \
per_device_batch_size=1 \
max_target_length=64 \
enable_checkpointing=false \
override_model_config=true \
base_num_decoder_layers=1 \
base_emb_dim=32 \
base_mlp_dim=32 \
head_dim=8 \
base_num_kv_heads=1 \
base_num_query_heads=1
Test Logs URL: https://paste.googleplex.com/6387838210932736

lepan-google · 2026-07-17T20:46:26Z

+ """A fully picklable and fork-safe RandomAccessDataSource for Bagz files."""
+ def __init__(self, paths: list[str] | str):
+ if isinstance(paths, (list, tuple)):
+ self._path = ",".join(sorted(paths))


Why we have to sort the paths here?

aireenmei

Thanks for adding the feature! Could you also add unit tests in grain_data_processing_test.py? You will need a class setup class similar to _GrainArrayRecordSetup and a test class similar to GrainArrayRecordProcessingTest

aireenmei · 2026-07-30T00:07:26Z

+# limitations under the License.
+
+"""
+Download a HuggingFace dataset via streaming and save as Bagz files.


Could you add some example commands in the docstring?

Done, I've added detailed example commands in the docstring covering basic local conversion, GCSFUSE-mounted bucket output with custom prefix/file size, and authenticated access for gated datasets

Implements end-to-end support for the Bagz (.bagz) dataset format inside the MaxText training and data processing pipeline. Enables highly efficient, POSIX-compliant file loading optimized specifically for GCS FUSE and local filesystem environments. Scope of Changes: - Input Pipeline: Integrated upstream `grain.BagzDataSource` for direct, multiprocess-safe reading of Bagz file shards via PyGrain. - Data Processing: Registered `bagz` as a first-class supported file type across PyGrain data loaders, feature normalizers, and tokenizers. - Tooling: Created `download_hf_dataset_as_bagz.py` with multi-worker support, HuggingFace streaming, and resilient checkpointing for GCSFUSE mounts. Testing: - Unit Tests: Added comprehensive unit test suite in `tests/unit/bagz_data_processing_test.py` covering RandomAccessDataSource compatibility, MapDataset, ElasticIterator (single and multi-worker), and end-to-end `get_datasets` integration. - End-to-End Local Smoke Test: Executed a 3-step offline training loop on a local CPU environment using the newly generated Bagz dataset via `DECOUPLE_GCLOUD=TRUE`. - Parallel Data Generation: Successfully converted and streamed HF datasets into multi-shard Bagz files (38 shards, Salesforce/wikitext).

…ring Add practical example commands to the docstring of `download_hf_dataset_as_bagz.py`: - Basic local directory conversion (Salesforce/wikitext) - Write-optimized GCSFUSE mounted bucket output with custom shard prefix and file size - Private/gated HuggingFace dataset download using auth token

Marlon666 · 2026-07-15T23:16:20Z

Description

This Pull Request introduces end-to-end support for the Gemini-backed Bagz (.bagz) dataset format within MaxText's PyGrain input pipeline.

Context & Problem Solved:
Previously, MaxText users relying on the ultra-efficient Bagz format for large-scale pre-training faced challenges when bridging cloud storage with PyGrain workers. This PR enables a highly optimized, POSIX-compliant file ingestion path designed explicitly for local filesystem and naturally GCS Fuse mounts. By avoiding complex REST/RPC shims, this approach achieves robust, multiprocess-safe read scalability out-of-the-box.

Scope of Changes:

Input Pipeline: Added the BagzDataSource wrapper in input_pipeline_utils.py for parallel and resilient Bagz shard reader instantiation.
Grain Integration: Registered bagz as a first-class supported file type in grain_data_processing.py, enabling native ParseFeatures and NormalizeFeatures handling.
Configurations: Expanded base.yml and types.py to seamlessly accept "bagz" as a valid grain_file_type.
Tooling: Created download_hf_dataset_as_bagz.py with multi-worker parallelism, HuggingFace streaming, and robust checkpoint recovery.

FIXES: (If there is a Github Issue #, paste it here, otherwise leave blank)

Tests

The implementation was successfully validated end-to-end on both Data Generation and Model Training levels:

Parallel Shard Generation: Successfully streamed and converted the Salesforce/wikitext dataset (wikitext-103-raw-v1) into 38 contiguous .bagz file shards using multi-worker execution.
End-to-End Model Training: Executed a 3-step offline training loop on a CPU-only environment using the generated local Bagz dataset via DECOUPLE_GCLOUD=TRUE (Log URL attached below).

Commands used for local testing:

# Data Generation
python tools/data_generation/download_hf_dataset_as_bagz.py \
--dataset Salesforce/wikitext \
--config '{"name": "wikitext-103-raw-v1"}' \
--output ~/datasets/wikitext_bagz \
--file-size-mb 10 \
--workers 2
# MaxText CPU Smoke Testexport DECOUPLE_GCLOUD=TRUE
python -m maxtext.trainers.pre_train.train \
src/maxtext/configs/base.yml \
run_name="bagz_cpu_smoke_test" \
steps=3 \
dataset_type="grain" \
grain_file_type="bagz" \
grain_train_files="$HOME/datasets/wikitext_bagz/*.bagz" \
grain_eval_files="$HOME/datasets/wikitext_bagz/*.bagz" \
tokenize_eval_data=True \
per_device_batch_size=1 \
max_target_length=64 \
enable_checkpointing=false \
override_model_config=true \
base_num_decoder_layers=1 \
base_emb_dim=32 \
base_mlp_dim=32 \
head_dim=8 \
base_num_kv_heads=1 \
base_num_query_heads=1
Test Logs URL: https://paste.googleplex.com/6387838210932736

lepan-google · 2026-07-17T20:46:26Z

+ """A fully picklable and fork-safe RandomAccessDataSource for Bagz files."""
+ def __init__(self, paths: list[str] | str):
+ if isinstance(paths, (list, tuple)):
+ self._path = ",".join(sorted(paths))


Why we have to sort the paths here?

aireenmei

Thanks for adding the feature! Could you also add unit tests in grain_data_processing_test.py? You will need a class setup class similar to _GrainArrayRecordSetup and a test class similar to GrainArrayRecordProcessingTest

aireenmei · 2026-07-30T00:07:26Z

+# limitations under the License.
+
+"""
+Download a HuggingFace dataset via streaming and save as Bagz files.


Could you add some example commands in the docstring?

Done, I've added detailed example commands in the docstring covering basic local conversion, GCSFUSE-mounted bucket output with custom prefix/file size, and authenticated access for gated datasets

Implements end-to-end support for the Bagz (.bagz) dataset format inside the MaxText training and data processing pipeline. Enables highly efficient, POSIX-compliant file loading optimized specifically for GCS FUSE and local filesystem environments. Scope of Changes: - Input Pipeline: Integrated upstream `grain.BagzDataSource` for direct, multiprocess-safe reading of Bagz file shards via PyGrain. - Data Processing: Registered `bagz` as a first-class supported file type across PyGrain data loaders, feature normalizers, and tokenizers. - Tooling: Created `download_hf_dataset_as_bagz.py` with multi-worker support, HuggingFace streaming, and resilient checkpointing for GCSFUSE mounts. Testing: - Unit Tests: Added comprehensive unit test suite in `tests/unit/bagz_data_processing_test.py` covering RandomAccessDataSource compatibility, MapDataset, ElasticIterator (single and multi-worker), and end-to-end `get_datasets` integration. - End-to-End Local Smoke Test: Executed a 3-step offline training loop on a local CPU environment using the newly generated Bagz dataset via `DECOUPLE_GCLOUD=TRUE`. - Parallel Data Generation: Successfully converted and streamed HF datasets into multi-shard Bagz files (38 shards, Salesforce/wikitext).

…ring Add practical example commands to the docstring of `download_hf_dataset_as_bagz.py`: - Basic local directory conversion (Salesforce/wikitext) - Write-optimized GCSFUSE mounted bucket output with custom shard prefix and file size - Private/gated HuggingFace dataset download using auth token

Marlon666 changed the title ~~Add Bagz format support with GCS FUSE compatibility~~Add Bagz format supportJul 15, 2026

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/configs/types.py

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

aireenmei self-assigned this Jul 25, 2026

aireenmei reviewed Jul 30, 2026
View reviewed changes

Marlon666 force-pushed the feat/bagz-gcsfuse-support branch from 85206e8 to 7abdb4dCompare August 4, 2026 21:56

Marlon666 requested a review from shuningjin as a code owner August 4, 2026 21:56

Marlon666 requested review from aireenmei and lepan-google August 4, 2026 22:25

Marlon666 changed the title ~~Add Bagz format support with GCS FUSE compatibility~~Add Bagz format supportJul 15, 2026

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/configs/types.py

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

aireenmei self-assigned this Jul 25, 2026

aireenmei reviewed Jul 30, 2026
View reviewed changes

Marlon666 force-pushed the feat/bagz-gcsfuse-support branch from 85206e8 to 7abdb4dCompare August 4, 2026 21:56

Marlon666 requested a review from shuningjin as a code owner August 4, 2026 21:56

Marlon666 requested review from aireenmei and lepan-google August 4, 2026 22:25

Marlon666 changed the title ~~Add Bagz format support with GCS FUSE compatibility~~Add Bagz format supportJul 15, 2026

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/configs/types.py

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

aireenmei self-assigned this Jul 25, 2026

aireenmei reviewed Jul 30, 2026
View reviewed changes

Marlon666 force-pushed the feat/bagz-gcsfuse-support branch from 85206e8 to 7abdb4dCompare August 4, 2026 21:56

Marlon666 requested a review from shuningjin as a code owner August 4, 2026 21:56

Marlon666 requested review from aireenmei and lepan-google August 4, 2026 22:25

Marlon666 changed the title ~~Add Bagz format support with GCS FUSE compatibility~~Add Bagz format supportJul 15, 2026

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/configs/types.py

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

aireenmei self-assigned this Jul 25, 2026

aireenmei reviewed Jul 30, 2026
View reviewed changes

Marlon666 force-pushed the feat/bagz-gcsfuse-support branch from 85206e8 to 7abdb4dCompare August 4, 2026 21:56

Marlon666 requested a review from shuningjin as a code owner August 4, 2026 21:56

Marlon666 requested review from aireenmei and lepan-google August 4, 2026 22:25

Marlon666 changed the title ~~Add Bagz format support with GCS FUSE compatibility~~Add Bagz format supportJul 15, 2026

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/configs/types.py

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

aireenmei self-assigned this Jul 25, 2026

aireenmei reviewed Jul 30, 2026
View reviewed changes

Marlon666 force-pushed the feat/bagz-gcsfuse-support branch from 85206e8 to 7abdb4dCompare August 4, 2026 21:56

Marlon666 requested a review from shuningjin as a code owner August 4, 2026 21:56

Marlon666 requested review from aireenmei and lepan-google August 4, 2026 22:25

Marlon666 changed the title ~~Add Bagz format support with GCS FUSE compatibility~~Add Bagz format supportJul 15, 2026

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/configs/types.py

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

aireenmei self-assigned this Jul 25, 2026

aireenmei reviewed Jul 30, 2026
View reviewed changes

Marlon666 force-pushed the feat/bagz-gcsfuse-support branch from 85206e8 to 7abdb4dCompare August 4, 2026 21:56

Marlon666 requested a review from shuningjin as a code owner August 4, 2026 21:56

Marlon666 requested review from aireenmei and lepan-google August 4, 2026 22:25

Marlon666 changed the title ~~Add Bagz format support with GCS FUSE compatibility~~Add Bagz format supportJul 15, 2026

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/configs/types.py

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

aireenmei self-assigned this Jul 25, 2026

aireenmei reviewed Jul 30, 2026
View reviewed changes

Marlon666 force-pushed the feat/bagz-gcsfuse-support branch from 85206e8 to 7abdb4dCompare August 4, 2026 21:56

Marlon666 requested a review from shuningjin as a code owner August 4, 2026 21:56

Marlon666 requested review from aireenmei and lepan-google August 4, 2026 22:25

Marlon666 changed the title ~~Add Bagz format support with GCS FUSE compatibility~~Add Bagz format supportJul 15, 2026

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/configs/types.py

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

Comment threadsrc/maxtext/input_pipeline/grain_data_processing.py Outdated

lepan-google reviewed Jul 17, 2026
View reviewed changes

aireenmei self-assigned this Jul 25, 2026

aireenmei reviewed Jul 30, 2026
View reviewed changes

Marlon666 force-pushed the feat/bagz-gcsfuse-support branch from 85206e8 to 7abdb4dCompare August 4, 2026 21:56

Marlon666 requested a review from shuningjin as a code owner August 4, 2026 21:56

Marlon666 requested review from aireenmei and lepan-google August 4, 2026 22:25

Uh oh!

Conversation

Marlon666 commented Jul 15, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Tests

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aireenmei left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Marlon666 commented Jul 15, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Tests

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aireenmei left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Marlon666 commented Jul 15, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Tests

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aireenmei left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Marlon666 commented Jul 15, 2026•
edited
Loading

Marlon666 commented Jul 15, 2026•
edited
Loading

Marlon666 commented Jul 15, 2026•
edited
Loading

Marlon666 commented Jul 15, 2026•
edited
Loading

Marlon666 commented Jul 15, 2026•
edited
Loading