Training-Free VFM-guided Dynamic Refinement for Domain Generalized Semantic Segmentation

Accepted by IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)

Visualization

.

Abstract

Domain Generalized Semantic Segmentation (DGSS) has recently attracted lots of research attention aiming to achieve robust segmentation performance on unseen domains, which aligns with diverse real-world applications. Visual Foundation Model (VFM), depending on large-scale pre-trained data and exquisite training strategies with good generalizability, has been explored and mined in some downstream tasks like DGSS. However, existing VFM-based DGSS methods predominantly focus on fine-tuning the VFM to adapt them to the semantic segmentation task. While this improves task alignment, it inevitably introduces additional training overhead and still results in static prediction behavior at inference time, making the model unable to adapt to varying target-domain distributions during deployment.

To address these issues, we propose a Training-free VFM-guided Dynamic Refinement (TVDR) framework for the DGSS task, operating at the inference stage. First, a maximum category voting module is proposed to smooth the segmentation result of large masks generated by Segment Anything Model (SAM). Second, a small-object under-segmentation optimization strategy is proposed to improve the generalization of the small objects. Finally, a fusion refinement module is designed to refine the segmentation results of the above strategies. Compared with existing methods, our strategy does not require any parameter updates for VFM and has the advantages of plug-and-play and flexible deployment. It provides an efficient and practical new paradigm for cross-domain semantic segmentation tasks. Extensive experiments on widely-used benchmarks verified the effectiveness of the proposed approaches.

Citation

@article{zhang2026tvdr,
  title={Training-Free VFM-guided Dynamic Refinement for Domain Generalized Semantic Segmentation},
  author={Yuhang Zhang, Binbin Wei, Wei Li, Xin Yang, Tiantian Zeng, Wenbin Zou},
  journal={IEEE Transactions on Circuits and Systems for Video Technology},
  year={2026}
}