DepthClassNet

Official implementation ofDepthClassNet: A Multitask Framework for Monocular Depth Estimation and Texture Classification in Endoscopic Imaging (Abdallah & Raza, MIUA 2025).
Springer • DOI: 10.1007/978-3-031-98691-8_17

DepthClassNet is a novel multitask framework for monocular depth estimation and texture classification in endoscopic (colonoscopy) imaging. It predicts per-pixel depth from a single RGB frame while classifying tissue texture, improving spatial understanding and scene interpretation for downstream clinical research.

DepthClassNet predictions on the UCL dataset

DepthClassNet predictions on the UCL dataset

Keywords: monocular depth estimation, depth prediction, endoscopy, colonoscopy, medical imaging, PyTorch, multitask learning, texture classification, Swin Transformer, CLIP

DepthClassNet Environment

• Python 3.11.10 (recommended)
• PyTorch ≥ 2.2
• See requirements.txt for full dependencies

Create & activate virtual environment

python3 -m venv myenv source myenv/bin/activate

DepthClassNet Installation dependencies

pip install -r requirements.txt

Datasets

• UCL colonoscopy dataset
• C3VD

Dataset tree

• Once downloaded, organise the datasets exactly as shown below; the dataloader relies on this layout.

Data/
├── c3vd/
│ ├── cecum_t1_a/
│ │ ├── 0000_color.png
│ │ ├── 0000_depth.tiff
│ │ ├── …
│ │ ├── 0275_color.png
│ │ └── 0275_depth.tiff
│ ├── cecum_t1_b/
│ │ ├── 0000_color.png
│ │ ├── 0000_depth.tiff
│ │ └── …
│ ├── cecum_t2_b/
│ └── trans_t4_a/
├── ucl/
│ ├── C_T3_L2_3_resized_FrameBuffer_0315.png
│ ├── C_T3_L2_3_resized_Depth_0315.png
│ ├── …
│ ├── C_T3_L2_3_resized_FrameBuffer_4515.png
│ └── C_T3_L2_3_resized_Depth_4515.png
└── splits/
├── ucl_train.txt
├── ucl_val.txt
└── ucl_test.txt

DepthClassNet Checkpoints:

Official pretrained weights can be downloaded here: DepthClassNet Checkpoints (OneDrive)

License

This repository is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License.

✔ Free for research and educational use.
❌ Commercial use is not permitted.

Citation

If you use this code for your research, please cite our paper:

@InProceedings{10.1007/978-3-031-98691-8_17,
author="Abdallah, Bashayerand Raza, Shan E. Ahmed",
editor="Ali, Shariband Hogg, David C.and Peckham, Michelle",
title="DepthClassNet: A Multitask Framework for Monocular Depth Estimation and Texture Classification in Endoscopic Imaging",
booktitle="Medical Image Understanding and Analysis",
year="2026",
publisher="Springer Nature Switzerland",
address="Cham",
pages="230--246",
abstract="Monocular depth estimation can play a critical role in medical imaging, providing spatial information that enhances diagnostic accuracy and supports precise surgical interventions. The texture classification in the endoscopic images significantly contributes to the differentiation of tissue types and the identification of pathological changes. Building on this knowledge, we introduce DepthClassNet, an innovative multitask framework designed to simultaneously perform monocular depth estimation and texture classification in endoscopic imaging. Our approach employs a tri-encoder model to integrate RGB images, edge maps, and textual descriptions. The architecture comprises a SWIN transformer as an image encoder, a convolutional neural network (CNN) as an edge encoder, and a modified CLIP text encoder for embedding class textual descriptions. Features from the image and edge encoders are effectively combined via a Feature Fusion Module (FFM), and high-resolution depth outputs are reconstructed through a decoder and depth projection block. We introduce an image embedding block that converts visual data from the SWIN encoder into embeddings that align with CLIP text embeddings. The classification head then computes similarity scores, scales them by a learnable temperature {\$}{\$}t{\$}{\$}t, and converts them into probabilities. By designing a loss function that combines depth, edge and classification losses with specific weights, our multitask architecture achieves state-of-the-art results on the Colonoscopy Depth - UCL dataset for depth estimation and texture classification.",
isbn="978-3-031-98691-8"
}