GELATTO: Geospatial Relation-Aware Soft Target Learning for Multimodal Region Representation

1. Environment Setup

conda create -n gelatto python=3.10 -y
conda activate gelatto
pip install torch torchvision transformers scikit-learn tqdm pillow numpy pandas matplotlib
pip install geopandas shapely

Set paths before running:

export INPUT_DATA=/path/to/CityLens-data
export HF_HOME=/path/to/.cache/huggingface
export TORCH_HOME=/path/to/.cache/torch

---

2. Data Folder Structure

Expected dataset layout:

CityLens-data/
├── satellite_image/
│ ├── Tokyo/
│ │ ├── 12345_67890.png
│ │ └── ...
│ └── ...
├── POI_sentence_vocab/
│ ├── Tokyo_poi_processed.csv
│ └── ...
├── validated/
│ ├── Dataset/
│ │ ├── all_global_pop_task_all.json
│ │ ├── all_global_gdp_task_all.json
│ │ └── ...
│ └── Benchmark/
│ ├── all_global_pop_task.json
│ └── ...
└── patchify_smart/
├── satellite/
└── poi_sentence/

The patchify_smart/ directory is generated by the patch generation step below.

---

3. Patch Generation

Prepare OSM divider GeoJSON files:

export OSM_CACHE=/path/to/osm_divider_geojson_cache

Run patch generation:

bash run_patch.sh

This creates:

$INPUT_DATA/patchify_smart/
├── satellite/
│ └── Tokyo/
│ └── <tile>_p0.png
└── poi_sentence/
└── Tokyo/
└── Tokyo_poi_sentence.csv

---

4. Pretraining + Probing

Run pretraining and downstream probing:

bash run.sh

Outputs are saved under:

results/GELATTO/
├── seed0.pt
├── seed0_loss.csv
├── loss_plots/
└── probe_results.txt

---

5. Probe Only

Evaluate a pretrained checkpoint:

bash probe.sh /path/to/checkpoint.pt

Example:

bash probe.sh results/GELATTO/seed0.pt gelatto_probe results/gelatto_probe.txt

This evaluates the checkpoint on all downstream probing tasks.

---

Dataset

Download the dataset from:

• Pretrain & downstream label
• OSM divider