Skip to content
BranchesTags

Repository files navigation

Auto-Compressing Subset Pruning for Semantic Image Segmentation

Pytorch Implementation of Auto-Compressing Subset Pruning for Semantic Image Segmentation.

Introduction

ACoSP is an online pruning algorithm that compresses convolutional neural networks during training. It learns to select a subset of channels from convolutional layers through a sigmoid function, as shown in the figure. For each channel a w_i is used to scale activations.

ACoSP selection scheme.

The segmentation maps display compressed PSPNet-50 models trained on Cityscapes. The models are up to 16 times smaller.

Repository

This repository is a PyTorch implementation of ACoSP based on hszhao/semseg. It was used to run all experiments used for the publication and is meant to guarantee reproducibility and audibility of our results.

The training, test and configuration infrastructure is kept close to semseg, with only some minor modifications to enable more reproducibility and integrate our pruning code. The model/ package contains the PSPNet50 and SegNet model definitions. In acosp/ all code required to prune during training is defined.

The current configs expect a special folder structure (but can be easily adapted):

  • /data: Datasets, Pretrained-weights
  • /logs/exp: Folder to store experiments

Installation

  1. Clone the repository:

    git clone git@github.com:merantix/acosp.git

  2. Install ACoSP including requirements:

    pip install .

Using ACoSP

The implementation of ACoSP is encapsulated in /acosp. It can be applied to CNN models, e.g.

importtorchvisionmodel=torchvision.models.resnet18()

, with just a few steps:

  1. Create a pruner and adapt the model:
fromacosp.prunerimportSoftTopKPruner# Create pruner objectpruner=SoftTopKPruner(
starting_epoch=0,
ending_epoch=100, # Pruning durationfinal_sparsity=0.5, # Final sparsity
)
  1. Add masking layers to your model.
# Sometimes you want to keep all channels in some of the convolution layers # model.my_conv.unprunable = True# Add sigmoid soft k masks to modelpruner.configure_model(model)

These layers mask out a fraction (pruner.final_sparsity) of the channels during training.

  1. In your training loop update the temperature of all masking layers:
# Update the temperature in all masking layerspruner.update_mask_layers(model, epoch)
  1. Convert the soft pruning to hard pruning when ending_epoch is reached:
importacosp.injectifepoch==pruner.ending_epoch:
# Convert to binary channel maskacosp.inject.soft_to_hard_k(model)

Experiments

  1. Highlight:

    • All initialization models, trained models are available. The structure is: 
      | init/ # initial models
| exp/
|-- ade20k/ # ade20k/camvid/cityscapes/voc2012/cifar10
| |-- pspnet50_{SPARSITY}/ # the sparsity refers to the relative amount of weights that are removed. I.e. sparsity=0.75 <==> compression_ratio=4 | |-- model # model files
| |-- ... # config/train/test files
|-- evals/ # all result with class wise IoU/Acc

  2. Hardware Requirements: At least 60GB (PSPNet50) / 16GB (SegNet) of GPU RAM. Can be distributed to multiple GPUs.

  3. Train:

    • Download related datasets and symlink the paths to them as follows (you can alternatively modify the relevant paths specified in folder config):

      mkdir -p /
ln -s /path_to_ade20k_dataset /data/ade20k

    • Download ImageNet pre-trained models and put them under folder /data for weight initialization. Remember to use the right dataset format detailed in FAQ.md.

    • Specify the gpu used in config then do training. (Training using acosp have only been carried out on a single GPU. And not been tested with DDP). The general structure to access individual configs is as follows:

      sh tool/train.sh ${DATASET}${CONFIG_NAME_WITHOUT_DATASET}

      E.g. to train a PSPNet50 on the ade20k dataset and use the config `config/ade20k/ade20k_pspnet50.yaml', execute:

      sh tool/train.sh ade20k pspnet50

  4. Test:

    • Download trained segmentation models and put them under folder specified in config or modify the specified paths.

    • For full testing (get listed performance):

      sh tool/test.sh ade20k pspnet50

  5. Visualization: tensorboardX incorporated for better visualization.

    tensorboard --logdir=/logs/exp/ade20k

  6. Other:

    • Resources: GoogleDrive LINK contains shared models, visual predictions and data lists.
    • Models: ImageNet pre-trained models and trained segmentation models can be accessed. Note that our ImageNet pretrained models are slightly different from original ResNet implementation in the beginning part.
    • Predictions: Visual predictions of several models can be accessed.
    • Datasets: attributes (names and colors) are in folder dataset and some sample lists can be accessed.
    • Some FAQs: FAQ.md.

Performance

Description: mIoU/mAcc stands for mean IoU, mean accuracy of each class and all pixel accuracy respectively. General parameters cross different datasets are listed below:

  • Network: {NETWORK} @ ACoSP-{COMPRESSION_RATIO}
  • Train Parameters: sync_bn(True), scale_min(0.5), scale_max(2.0), rotate_min(-10), rotate_max(10), zoom_factor(8), aux_weight(0.4), base_lr(1e-2), power(0.9), momentum(0.9), weight_decay(1e-4).
  • Test Parameters: ignore_label(255).

  1. ADE20K: Train Parameters: classes(150), train_h(473), train_w(473), epochs(100). Test Parameters: classes(150), test_h(473), test_w(473), base_size(512).

    • Setting: train on train (20210 images) set and test on val (2000 images) set.
    NetworkmIoU/mAcc
    PSPNet5041.42/51.48
    PSPNet50 @ ACoSP-238.97/49.56
    PSPNet50 @ ACoSP-433.67/43.17
    PSPNet50 @ ACoSP-828.04/35.60
    PSPNet50 @ ACoSP-1619.39/25.52

  2. PASCAL VOC 2012: Train Parameters: classes(21), train_h(473), train_w(473), epochs(50). Test Parameters: classes(21), test_h(473), test_w(473), base_size(512).

    • Setting: train on train_aug (10582 images) set and test on val (1449 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.30/85.27
    PSPNet50 @ ACoSP-272.71/81.87
    PSPNet50 @ ACoSP-465.84/77.12
    PSPNet50 @ ACoSP-858.26/69.65
    PSPNet50 @ ACoSP-1648.06/58.83

  3. Cityscapes: Train Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), epochs(200). Test Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), base_size(2048).

    • Setting: train on fine_train (2975 images) set and test on fine_val (500 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.35/84.27
    PSPNet50 @ ACoSP-274.11/81.73
    PSPNet50 @ ACoSP-471.50/79.40
    PSPNet50 @ ACoSP-866.06/74.33
    PSPNet50 @ ACoSP-1659.49/67.74
    SegNet65.12/73.85
    SegNet @ ACoSP-264.62/73.19
    SegNet @ ACoSP-460.77/69.57
    SegNet @ ACoSP-854.34/62.48
    SegNet @ ACoSP-1644.12/50.87

  4. CamVid: Train Parameters: classes(11), train_h(360), train_w(720), epochs(450). Test Parameters: classes(11), test_h(360), test_w(720), base_size(360).

    • Setting: train on train (367 images) set and test on test (233 images) set.
    NetworkmIoU/mAcc
    SegNet55.49+-0.85/65.44+-1.01
    SegNet @ ACoSP-251.85+-0.83/61.86+-0.85
    SegNet @ ACoSP-450.10+-1.11/59.79+-1.49
    SegNet @ ACoSP-847.25+-1.18/56.87+-1.10
    SegNet @ ACoSP-1642.27+-1.95/51.25+-2.02

  5. Cifar10: Train Parameters: classes(10), train_h(32), train_w(32), epochs(50). Test Parameters: classes(10), test_h(32), test_w(32), base_size(32).

    • Setting: train on train (50000 images) set and test on test (10000 images) set.
    NetworkmAcc
    ResNet1889.68
    ResNet18 @ ACoSP-288.50
    ResNet18 @ ACoSP-486.21
    ResNet18 @ ACoSP-881.06
    ResNet18 @ ACoSP-1676.81

Citation

If you find the paper, acosp/ code or trained models useful, please consider citing:

@article{Ditschuneit2022AutoCompressingSP,
title={Auto-Compressing Subset Pruning for Semantic Image Segmentation},
author={Konstantin Ditschuneit and J. Otterbach},
journal={ArXiv},
year={2022},
volume={abs/2201.11103}
}

For the general training code, please also consider referencing hszhao/semseg.

Question

Some FAQ.md collected. You are welcome to send pull requests or give some advices. Contact information: at.

About

Semantic Segmentation in Pytorch

Resources

Readme
License
Activity
Custom properties

Stars

10 stars

Watchers

0 watching

Forks

3 forks
Report repository

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
GitHub - merantix/acosp: Semantic Segmentation in Pytorch · GitHub
Skip to content
BranchesTags
Repository files navigation
Auto-Compressing Subset Pruning for Semantic Image Segmentation
Pytorch Implementation of Auto-Compressing Subset Pruning for Semantic Image Segmentation.
Introduction
ACoSP is an online pruning algorithm that compresses convolutional neural networks during training.
It learns to select a subset of channels from convolutional layers through a sigmoid function, as shown in the figure.
For each channel a w_i is used to scale activations.
ACoSP selection scheme.
The segmentation maps display compressed PSPNet-50 models trained on Cityscapes. The models are up to 16 times smaller.
Repository
This repository is a PyTorch implementation of ACoSP based on hszhao/semseg. It was
used to run all experiments used for the publication and is meant to guarantee reproducibility and audibility of our
results.
The training, test and configuration infrastructure is kept close to semseg, with only some minor modifications to
enable more reproducibility and integrate our pruning code. The model/ package contains the PSPNet50 and SegNet model definitions. In acosp/ all code required to prune during training is defined.
The current configs expect a special folder structure (but can be easily adapted):
  • /data: Datasets, Pretrained-weights
  • /logs/exp: Folder to store experiments
Installation
  1. Clone the repository:
    git clone git@github.com:merantix/acosp.git
  2. Install ACoSP including requirements:
    pip install .
Using ACoSP
The implementation of ACoSP is encapsulated in /acosp. It can be applied to CNN models, e.g.
importtorchvisionmodel=torchvision.models.resnet18()
, with just a few steps:
  1. Create a pruner and adapt the model:
fromacosp.prunerimportSoftTopKPruner# Create pruner objectpruner=SoftTopKPruner(
starting_epoch=0,
ending_epoch=100, # Pruning durationfinal_sparsity=0.5, # Final sparsity
)
  1. Add masking layers to your model.
# Sometimes you want to keep all channels in some of the convolution layers # model.my_conv.unprunable = True# Add sigmoid soft k masks to modelpruner.configure_model(model)
These layers mask out a fraction (pruner.final_sparsity) of the channels during training.
  1. In your training loop update the temperature of all masking layers:
# Update the temperature in all masking layerspruner.update_mask_layers(model, epoch)
  1. Convert the soft pruning to hard pruning when ending_epoch is reached:
importacosp.injectifepoch==pruner.ending_epoch:
# Convert to binary channel maskacosp.inject.soft_to_hard_k(model)
Experiments
  1. Highlight:
    • All initialization models, trained models are available.
The structure is:

      | init/ # initial models
| exp/
|-- ade20k/ # ade20k/camvid/cityscapes/voc2012/cifar10
| |-- pspnet50_{SPARSITY}/ # the sparsity refers to the relative amount of weights that are removed. I.e. sparsity=0.75 <==> compression_ratio=4 | |-- model # model files
| |-- ... # config/train/test files
|-- evals/ # all result with class wise IoU/Acc

  2. Hardware Requirements: At least 60GB (PSPNet50) / 16GB (SegNet) of GPU RAM. Can be distributed to multiple GPUs.
  3. Train:
    • Download related datasets and symlink the paths to them as follows (you can alternatively modify the relevant
paths specified in folder config):
      mkdir -p /
ln -s /path_to_ade20k_dataset /data/ade20k

    • Download ImageNet pre-trained models and put them under folder /data for weight
initialization. Remember to use the right dataset format detailed in FAQ.md.
    • Specify the gpu used in config then do training. (Training using acosp have only been carried out on a single
GPU. And not been tested with DDP). The general structure to access individual configs is as follows:
      sh tool/train.sh ${DATASET}${CONFIG_NAME_WITHOUT_DATASET}
      E.g. to train a PSPNet50 on the ade20k dataset and use the config `config/ade20k/ade20k_pspnet50.yaml', execute:
      sh tool/train.sh ade20k pspnet50
  4. Test:
    • Download trained segmentation models and put them under folder specified in config or modify the specified paths.
    • For full testing (get listed performance):
      sh tool/test.sh ade20k pspnet50
  5. Visualization: tensorboardX incorporated for better visualization.
    tensorboard --logdir=/logs/exp/ade20k
  6. Other:
    • Resources: GoogleDrive LINK contains shared models, visual predictions and data lists.
    • Models: ImageNet pre-trained models and trained segmentation models can be accessed. Note that our ImageNet
pretrained models are slightly different from
original ResNet implementation in
the beginning part.
    • Predictions: Visual predictions of several models can be accessed.
    • Datasets: attributes (names and colors) are in folder dataset and some sample lists can be accessed.
    • Some FAQs: FAQ.md.
Performance
Description: mIoU/mAcc stands for mean IoU, mean accuracy of each class and all pixel accuracy respectively.
General parameters cross different datasets are listed below:
  • Network: {NETWORK} @ ACoSP-{COMPRESSION_RATIO}
  • Train Parameters: sync_bn(True), scale_min(0.5), scale_max(2.0), rotate_min(-10), rotate_max(10), zoom_factor(8),
aux_weight(0.4), base_lr(1e-2), power(0.9), momentum(0.9), weight_decay(1e-4).
  • Test Parameters: ignore_label(255).
  1. ADE20K:
Train Parameters: classes(150), train_h(473), train_w(473), epochs(100). Test Parameters: classes(150), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train (20210 images) set and test on val (2000 images) set.
    NetworkmIoU/mAcc
    PSPNet5041.42/51.48
    PSPNet50 @ ACoSP-238.97/49.56
    PSPNet50 @ ACoSP-433.67/43.17
    PSPNet50 @ ACoSP-828.04/35.60
    PSPNet50 @ ACoSP-1619.39/25.52
  2. PASCAL VOC 2012:
Train Parameters: classes(21), train_h(473), train_w(473), epochs(50). Test Parameters: classes(21), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train_aug (10582 images) set and test on val (1449 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.30/85.27
    PSPNet50 @ ACoSP-272.71/81.87
    PSPNet50 @ ACoSP-465.84/77.12
    PSPNet50 @ ACoSP-858.26/69.65
    PSPNet50 @ ACoSP-1648.06/58.83
  3. Cityscapes:
Train Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), epochs(200). Test
Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), base_size(2048).
    • Setting: train on fine_train (2975 images) set and test on fine_val (500 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.35/84.27
    PSPNet50 @ ACoSP-274.11/81.73
    PSPNet50 @ ACoSP-471.50/79.40
    PSPNet50 @ ACoSP-866.06/74.33
    PSPNet50 @ ACoSP-1659.49/67.74
    SegNet65.12/73.85
    SegNet @ ACoSP-264.62/73.19
    SegNet @ ACoSP-460.77/69.57
    SegNet @ ACoSP-854.34/62.48
    SegNet @ ACoSP-1644.12/50.87
  4. CamVid:
Train Parameters: classes(11), train_h(360), train_w(720), epochs(450). Test Parameters: classes(11), test_h(360),
test_w(720), base_size(360).
    • Setting: train on train (367 images) set and test on test (233 images) set.
    NetworkmIoU/mAcc
    SegNet55.49+-0.85/65.44+-1.01
    SegNet @ ACoSP-251.85+-0.83/61.86+-0.85
    SegNet @ ACoSP-450.10+-1.11/59.79+-1.49
    SegNet @ ACoSP-847.25+-1.18/56.87+-1.10
    SegNet @ ACoSP-1642.27+-1.95/51.25+-2.02
  5. Cifar10:
Train Parameters: classes(10), train_h(32), train_w(32), epochs(50). Test Parameters: classes(10), test_h(32),
test_w(32), base_size(32).
    • Setting: train on train (50000 images) set and test on test (10000 images) set.
    NetworkmAcc
    ResNet1889.68
    ResNet18 @ ACoSP-288.50
    ResNet18 @ ACoSP-486.21
    ResNet18 @ ACoSP-881.06
    ResNet18 @ ACoSP-1676.81
Citation
If you find the paper, acosp/ code or trained models useful, please consider citing:
@article{Ditschuneit2022AutoCompressingSP,
title={Auto-Compressing Subset Pruning for Semantic Image Segmentation},
author={Konstantin Ditschuneit and J. Otterbach},
journal={ArXiv},
year={2022},
volume={abs/2201.11103}
}

For the general training code, please also consider referencing hszhao/semseg.
Question
Some FAQ.md collected. You are welcome to send pull requests or give some advices. Contact
information: at.
About
Semantic Segmentation in Pytorch
Resources
Readme
License
Activity
Custom properties
Stars
10 stars
Watchers
0 watching
Forks
3 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - merantix/acosp: Semantic Segmentation in Pytorch · GitHub
Skip to content
BranchesTags
Repository files navigation
Auto-Compressing Subset Pruning for Semantic Image Segmentation
Pytorch Implementation of Auto-Compressing Subset Pruning for Semantic Image Segmentation.
Introduction
ACoSP is an online pruning algorithm that compresses convolutional neural networks during training.
It learns to select a subset of channels from convolutional layers through a sigmoid function, as shown in the figure.
For each channel a w_i is used to scale activations.
ACoSP selection scheme.
The segmentation maps display compressed PSPNet-50 models trained on Cityscapes. The models are up to 16 times smaller.
Repository
This repository is a PyTorch implementation of ACoSP based on hszhao/semseg. It was
used to run all experiments used for the publication and is meant to guarantee reproducibility and audibility of our
results.
The training, test and configuration infrastructure is kept close to semseg, with only some minor modifications to
enable more reproducibility and integrate our pruning code. The model/ package contains the PSPNet50 and SegNet model definitions. In acosp/ all code required to prune during training is defined.
The current configs expect a special folder structure (but can be easily adapted):
  • /data: Datasets, Pretrained-weights
  • /logs/exp: Folder to store experiments
Installation
  1. Clone the repository:
    git clone git@github.com:merantix/acosp.git
  2. Install ACoSP including requirements:
    pip install .
Using ACoSP
The implementation of ACoSP is encapsulated in /acosp. It can be applied to CNN models, e.g.
importtorchvisionmodel=torchvision.models.resnet18()
, with just a few steps:
  1. Create a pruner and adapt the model:
fromacosp.prunerimportSoftTopKPruner# Create pruner objectpruner=SoftTopKPruner(
starting_epoch=0,
ending_epoch=100, # Pruning durationfinal_sparsity=0.5, # Final sparsity
)
  1. Add masking layers to your model.
# Sometimes you want to keep all channels in some of the convolution layers # model.my_conv.unprunable = True# Add sigmoid soft k masks to modelpruner.configure_model(model)
These layers mask out a fraction (pruner.final_sparsity) of the channels during training.
  1. In your training loop update the temperature of all masking layers:
# Update the temperature in all masking layerspruner.update_mask_layers(model, epoch)
  1. Convert the soft pruning to hard pruning when ending_epoch is reached:
importacosp.injectifepoch==pruner.ending_epoch:
# Convert to binary channel maskacosp.inject.soft_to_hard_k(model)
Experiments
  1. Highlight:
    • All initialization models, trained models are available.
The structure is:

      | init/ # initial models
| exp/
|-- ade20k/ # ade20k/camvid/cityscapes/voc2012/cifar10
| |-- pspnet50_{SPARSITY}/ # the sparsity refers to the relative amount of weights that are removed. I.e. sparsity=0.75 <==> compression_ratio=4 | |-- model # model files
| |-- ... # config/train/test files
|-- evals/ # all result with class wise IoU/Acc

  2. Hardware Requirements: At least 60GB (PSPNet50) / 16GB (SegNet) of GPU RAM. Can be distributed to multiple GPUs.
  3. Train:
    • Download related datasets and symlink the paths to them as follows (you can alternatively modify the relevant
paths specified in folder config):
      mkdir -p /
ln -s /path_to_ade20k_dataset /data/ade20k

    • Download ImageNet pre-trained models and put them under folder /data for weight
initialization. Remember to use the right dataset format detailed in FAQ.md.
    • Specify the gpu used in config then do training. (Training using acosp have only been carried out on a single
GPU. And not been tested with DDP). The general structure to access individual configs is as follows:
      sh tool/train.sh ${DATASET}${CONFIG_NAME_WITHOUT_DATASET}
      E.g. to train a PSPNet50 on the ade20k dataset and use the config `config/ade20k/ade20k_pspnet50.yaml', execute:
      sh tool/train.sh ade20k pspnet50
  4. Test:
    • Download trained segmentation models and put them under folder specified in config or modify the specified paths.
    • For full testing (get listed performance):
      sh tool/test.sh ade20k pspnet50
  5. Visualization: tensorboardX incorporated for better visualization.
    tensorboard --logdir=/logs/exp/ade20k
  6. Other:
    • Resources: GoogleDrive LINK contains shared models, visual predictions and data lists.
    • Models: ImageNet pre-trained models and trained segmentation models can be accessed. Note that our ImageNet
pretrained models are slightly different from
original ResNet implementation in
the beginning part.
    • Predictions: Visual predictions of several models can be accessed.
    • Datasets: attributes (names and colors) are in folder dataset and some sample lists can be accessed.
    • Some FAQs: FAQ.md.
Performance
Description: mIoU/mAcc stands for mean IoU, mean accuracy of each class and all pixel accuracy respectively.
General parameters cross different datasets are listed below:
  • Network: {NETWORK} @ ACoSP-{COMPRESSION_RATIO}
  • Train Parameters: sync_bn(True), scale_min(0.5), scale_max(2.0), rotate_min(-10), rotate_max(10), zoom_factor(8),
aux_weight(0.4), base_lr(1e-2), power(0.9), momentum(0.9), weight_decay(1e-4).
  • Test Parameters: ignore_label(255).
  1. ADE20K:
Train Parameters: classes(150), train_h(473), train_w(473), epochs(100). Test Parameters: classes(150), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train (20210 images) set and test on val (2000 images) set.
    NetworkmIoU/mAcc
    PSPNet5041.42/51.48
    PSPNet50 @ ACoSP-238.97/49.56
    PSPNet50 @ ACoSP-433.67/43.17
    PSPNet50 @ ACoSP-828.04/35.60
    PSPNet50 @ ACoSP-1619.39/25.52
  2. PASCAL VOC 2012:
Train Parameters: classes(21), train_h(473), train_w(473), epochs(50). Test Parameters: classes(21), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train_aug (10582 images) set and test on val (1449 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.30/85.27
    PSPNet50 @ ACoSP-272.71/81.87
    PSPNet50 @ ACoSP-465.84/77.12
    PSPNet50 @ ACoSP-858.26/69.65
    PSPNet50 @ ACoSP-1648.06/58.83
  3. Cityscapes:
Train Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), epochs(200). Test
Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), base_size(2048).
    • Setting: train on fine_train (2975 images) set and test on fine_val (500 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.35/84.27
    PSPNet50 @ ACoSP-274.11/81.73
    PSPNet50 @ ACoSP-471.50/79.40
    PSPNet50 @ ACoSP-866.06/74.33
    PSPNet50 @ ACoSP-1659.49/67.74
    SegNet65.12/73.85
    SegNet @ ACoSP-264.62/73.19
    SegNet @ ACoSP-460.77/69.57
    SegNet @ ACoSP-854.34/62.48
    SegNet @ ACoSP-1644.12/50.87
  4. CamVid:
Train Parameters: classes(11), train_h(360), train_w(720), epochs(450). Test Parameters: classes(11), test_h(360),
test_w(720), base_size(360).
    • Setting: train on train (367 images) set and test on test (233 images) set.
    NetworkmIoU/mAcc
    SegNet55.49+-0.85/65.44+-1.01
    SegNet @ ACoSP-251.85+-0.83/61.86+-0.85
    SegNet @ ACoSP-450.10+-1.11/59.79+-1.49
    SegNet @ ACoSP-847.25+-1.18/56.87+-1.10
    SegNet @ ACoSP-1642.27+-1.95/51.25+-2.02
  5. Cifar10:
Train Parameters: classes(10), train_h(32), train_w(32), epochs(50). Test Parameters: classes(10), test_h(32),
test_w(32), base_size(32).
    • Setting: train on train (50000 images) set and test on test (10000 images) set.
    NetworkmAcc
    ResNet1889.68
    ResNet18 @ ACoSP-288.50
    ResNet18 @ ACoSP-486.21
    ResNet18 @ ACoSP-881.06
    ResNet18 @ ACoSP-1676.81
Citation
If you find the paper, acosp/ code or trained models useful, please consider citing:
@article{Ditschuneit2022AutoCompressingSP,
title={Auto-Compressing Subset Pruning for Semantic Image Segmentation},
author={Konstantin Ditschuneit and J. Otterbach},
journal={ArXiv},
year={2022},
volume={abs/2201.11103}
}

For the general training code, please also consider referencing hszhao/semseg.
Question
Some FAQ.md collected. You are welcome to send pull requests or give some advices. Contact
information: at.
About
Semantic Segmentation in Pytorch
Resources
Readme
License
Activity
Custom properties
Stars
10 stars
Watchers
0 watching
Forks
3 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - merantix/acosp: Semantic Segmentation in Pytorch · GitHub
Skip to content
BranchesTags
Repository files navigation
Auto-Compressing Subset Pruning for Semantic Image Segmentation
Pytorch Implementation of Auto-Compressing Subset Pruning for Semantic Image Segmentation.
Introduction
ACoSP is an online pruning algorithm that compresses convolutional neural networks during training.
It learns to select a subset of channels from convolutional layers through a sigmoid function, as shown in the figure.
For each channel a w_i is used to scale activations.
ACoSP selection scheme.
The segmentation maps display compressed PSPNet-50 models trained on Cityscapes. The models are up to 16 times smaller.
Repository
This repository is a PyTorch implementation of ACoSP based on hszhao/semseg. It was
used to run all experiments used for the publication and is meant to guarantee reproducibility and audibility of our
results.
The training, test and configuration infrastructure is kept close to semseg, with only some minor modifications to
enable more reproducibility and integrate our pruning code. The model/ package contains the PSPNet50 and SegNet model definitions. In acosp/ all code required to prune during training is defined.
The current configs expect a special folder structure (but can be easily adapted):
  • /data: Datasets, Pretrained-weights
  • /logs/exp: Folder to store experiments
Installation
  1. Clone the repository:
    git clone git@github.com:merantix/acosp.git
  2. Install ACoSP including requirements:
    pip install .
Using ACoSP
The implementation of ACoSP is encapsulated in /acosp. It can be applied to CNN models, e.g.
importtorchvisionmodel=torchvision.models.resnet18()
, with just a few steps:
  1. Create a pruner and adapt the model:
fromacosp.prunerimportSoftTopKPruner# Create pruner objectpruner=SoftTopKPruner(
starting_epoch=0,
ending_epoch=100, # Pruning durationfinal_sparsity=0.5, # Final sparsity
)
  1. Add masking layers to your model.
# Sometimes you want to keep all channels in some of the convolution layers # model.my_conv.unprunable = True# Add sigmoid soft k masks to modelpruner.configure_model(model)
These layers mask out a fraction (pruner.final_sparsity) of the channels during training.
  1. In your training loop update the temperature of all masking layers:
# Update the temperature in all masking layerspruner.update_mask_layers(model, epoch)
  1. Convert the soft pruning to hard pruning when ending_epoch is reached:
importacosp.injectifepoch==pruner.ending_epoch:
# Convert to binary channel maskacosp.inject.soft_to_hard_k(model)
Experiments
  1. Highlight:
    • All initialization models, trained models are available.
The structure is:

      | init/ # initial models
| exp/
|-- ade20k/ # ade20k/camvid/cityscapes/voc2012/cifar10
| |-- pspnet50_{SPARSITY}/ # the sparsity refers to the relative amount of weights that are removed. I.e. sparsity=0.75 <==> compression_ratio=4 | |-- model # model files
| |-- ... # config/train/test files
|-- evals/ # all result with class wise IoU/Acc

  2. Hardware Requirements: At least 60GB (PSPNet50) / 16GB (SegNet) of GPU RAM. Can be distributed to multiple GPUs.
  3. Train:
    • Download related datasets and symlink the paths to them as follows (you can alternatively modify the relevant
paths specified in folder config):
      mkdir -p /
ln -s /path_to_ade20k_dataset /data/ade20k

    • Download ImageNet pre-trained models and put them under folder /data for weight
initialization. Remember to use the right dataset format detailed in FAQ.md.
    • Specify the gpu used in config then do training. (Training using acosp have only been carried out on a single
GPU. And not been tested with DDP). The general structure to access individual configs is as follows:
      sh tool/train.sh ${DATASET}${CONFIG_NAME_WITHOUT_DATASET}
      E.g. to train a PSPNet50 on the ade20k dataset and use the config `config/ade20k/ade20k_pspnet50.yaml', execute:
      sh tool/train.sh ade20k pspnet50
  4. Test:
    • Download trained segmentation models and put them under folder specified in config or modify the specified paths.
    • For full testing (get listed performance):
      sh tool/test.sh ade20k pspnet50
  5. Visualization: tensorboardX incorporated for better visualization.
    tensorboard --logdir=/logs/exp/ade20k
  6. Other:
    • Resources: GoogleDrive LINK contains shared models, visual predictions and data lists.
    • Models: ImageNet pre-trained models and trained segmentation models can be accessed. Note that our ImageNet
pretrained models are slightly different from
original ResNet implementation in
the beginning part.
    • Predictions: Visual predictions of several models can be accessed.
    • Datasets: attributes (names and colors) are in folder dataset and some sample lists can be accessed.
    • Some FAQs: FAQ.md.
Performance
Description: mIoU/mAcc stands for mean IoU, mean accuracy of each class and all pixel accuracy respectively.
General parameters cross different datasets are listed below:
  • Network: {NETWORK} @ ACoSP-{COMPRESSION_RATIO}
  • Train Parameters: sync_bn(True), scale_min(0.5), scale_max(2.0), rotate_min(-10), rotate_max(10), zoom_factor(8),
aux_weight(0.4), base_lr(1e-2), power(0.9), momentum(0.9), weight_decay(1e-4).
  • Test Parameters: ignore_label(255).
  1. ADE20K:
Train Parameters: classes(150), train_h(473), train_w(473), epochs(100). Test Parameters: classes(150), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train (20210 images) set and test on val (2000 images) set.
    NetworkmIoU/mAcc
    PSPNet5041.42/51.48
    PSPNet50 @ ACoSP-238.97/49.56
    PSPNet50 @ ACoSP-433.67/43.17
    PSPNet50 @ ACoSP-828.04/35.60
    PSPNet50 @ ACoSP-1619.39/25.52
  2. PASCAL VOC 2012:
Train Parameters: classes(21), train_h(473), train_w(473), epochs(50). Test Parameters: classes(21), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train_aug (10582 images) set and test on val (1449 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.30/85.27
    PSPNet50 @ ACoSP-272.71/81.87
    PSPNet50 @ ACoSP-465.84/77.12
    PSPNet50 @ ACoSP-858.26/69.65
    PSPNet50 @ ACoSP-1648.06/58.83
  3. Cityscapes:
Train Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), epochs(200). Test
Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), base_size(2048).
    • Setting: train on fine_train (2975 images) set and test on fine_val (500 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.35/84.27
    PSPNet50 @ ACoSP-274.11/81.73
    PSPNet50 @ ACoSP-471.50/79.40
    PSPNet50 @ ACoSP-866.06/74.33
    PSPNet50 @ ACoSP-1659.49/67.74
    SegNet65.12/73.85
    SegNet @ ACoSP-264.62/73.19
    SegNet @ ACoSP-460.77/69.57
    SegNet @ ACoSP-854.34/62.48
    SegNet @ ACoSP-1644.12/50.87
  4. CamVid:
Train Parameters: classes(11), train_h(360), train_w(720), epochs(450). Test Parameters: classes(11), test_h(360),
test_w(720), base_size(360).
    • Setting: train on train (367 images) set and test on test (233 images) set.
    NetworkmIoU/mAcc
    SegNet55.49+-0.85/65.44+-1.01
    SegNet @ ACoSP-251.85+-0.83/61.86+-0.85
    SegNet @ ACoSP-450.10+-1.11/59.79+-1.49
    SegNet @ ACoSP-847.25+-1.18/56.87+-1.10
    SegNet @ ACoSP-1642.27+-1.95/51.25+-2.02
  5. Cifar10:
Train Parameters: classes(10), train_h(32), train_w(32), epochs(50). Test Parameters: classes(10), test_h(32),
test_w(32), base_size(32).
    • Setting: train on train (50000 images) set and test on test (10000 images) set.
    NetworkmAcc
    ResNet1889.68
    ResNet18 @ ACoSP-288.50
    ResNet18 @ ACoSP-486.21
    ResNet18 @ ACoSP-881.06
    ResNet18 @ ACoSP-1676.81
Citation
If you find the paper, acosp/ code or trained models useful, please consider citing:
@article{Ditschuneit2022AutoCompressingSP,
title={Auto-Compressing Subset Pruning for Semantic Image Segmentation},
author={Konstantin Ditschuneit and J. Otterbach},
journal={ArXiv},
year={2022},
volume={abs/2201.11103}
}

For the general training code, please also consider referencing hszhao/semseg.
Question
Some FAQ.md collected. You are welcome to send pull requests or give some advices. Contact
information: at.
About
Semantic Segmentation in Pytorch
Resources
Readme
License
Activity
Custom properties
Stars
10 stars
Watchers
0 watching
Forks
3 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
GitHub - merantix/acosp: Semantic Segmentation in Pytorch · GitHub
Skip to content
BranchesTags
Repository files navigation
Auto-Compressing Subset Pruning for Semantic Image Segmentation
Pytorch Implementation of Auto-Compressing Subset Pruning for Semantic Image Segmentation.
Introduction
ACoSP is an online pruning algorithm that compresses convolutional neural networks during training.
It learns to select a subset of channels from convolutional layers through a sigmoid function, as shown in the figure.
For each channel a w_i is used to scale activations.
ACoSP selection scheme.
The segmentation maps display compressed PSPNet-50 models trained on Cityscapes. The models are up to 16 times smaller.
Repository
This repository is a PyTorch implementation of ACoSP based on hszhao/semseg. It was
used to run all experiments used for the publication and is meant to guarantee reproducibility and audibility of our
results.
The training, test and configuration infrastructure is kept close to semseg, with only some minor modifications to
enable more reproducibility and integrate our pruning code. The model/ package contains the PSPNet50 and SegNet model definitions. In acosp/ all code required to prune during training is defined.
The current configs expect a special folder structure (but can be easily adapted):
  • /data: Datasets, Pretrained-weights
  • /logs/exp: Folder to store experiments
Installation
  1. Clone the repository:
    git clone git@github.com:merantix/acosp.git
  2. Install ACoSP including requirements:
    pip install .
Using ACoSP
The implementation of ACoSP is encapsulated in /acosp. It can be applied to CNN models, e.g.
importtorchvisionmodel=torchvision.models.resnet18()
, with just a few steps:
  1. Create a pruner and adapt the model:
fromacosp.prunerimportSoftTopKPruner# Create pruner objectpruner=SoftTopKPruner(
starting_epoch=0,
ending_epoch=100, # Pruning durationfinal_sparsity=0.5, # Final sparsity
)
  1. Add masking layers to your model.
# Sometimes you want to keep all channels in some of the convolution layers # model.my_conv.unprunable = True# Add sigmoid soft k masks to modelpruner.configure_model(model)
These layers mask out a fraction (pruner.final_sparsity) of the channels during training.
  1. In your training loop update the temperature of all masking layers:
# Update the temperature in all masking layerspruner.update_mask_layers(model, epoch)
  1. Convert the soft pruning to hard pruning when ending_epoch is reached:
importacosp.injectifepoch==pruner.ending_epoch:
# Convert to binary channel maskacosp.inject.soft_to_hard_k(model)
Experiments
  1. Highlight:
    • All initialization models, trained models are available.
The structure is:

      | init/ # initial models
| exp/
|-- ade20k/ # ade20k/camvid/cityscapes/voc2012/cifar10
| |-- pspnet50_{SPARSITY}/ # the sparsity refers to the relative amount of weights that are removed. I.e. sparsity=0.75 <==> compression_ratio=4 | |-- model # model files
| |-- ... # config/train/test files
|-- evals/ # all result with class wise IoU/Acc

  2. Hardware Requirements: At least 60GB (PSPNet50) / 16GB (SegNet) of GPU RAM. Can be distributed to multiple GPUs.
  3. Train:
    • Download related datasets and symlink the paths to them as follows (you can alternatively modify the relevant
paths specified in folder config):
      mkdir -p /
ln -s /path_to_ade20k_dataset /data/ade20k

    • Download ImageNet pre-trained models and put them under folder /data for weight
initialization. Remember to use the right dataset format detailed in FAQ.md.
    • Specify the gpu used in config then do training. (Training using acosp have only been carried out on a single
GPU. And not been tested with DDP). The general structure to access individual configs is as follows:
      sh tool/train.sh ${DATASET}${CONFIG_NAME_WITHOUT_DATASET}
      E.g. to train a PSPNet50 on the ade20k dataset and use the config `config/ade20k/ade20k_pspnet50.yaml', execute:
      sh tool/train.sh ade20k pspnet50
  4. Test:
    • Download trained segmentation models and put them under folder specified in config or modify the specified paths.
    • For full testing (get listed performance):
      sh tool/test.sh ade20k pspnet50
  5. Visualization: tensorboardX incorporated for better visualization.
    tensorboard --logdir=/logs/exp/ade20k
  6. Other:
    • Resources: GoogleDrive LINK contains shared models, visual predictions and data lists.
    • Models: ImageNet pre-trained models and trained segmentation models can be accessed. Note that our ImageNet
pretrained models are slightly different from
original ResNet implementation in
the beginning part.
    • Predictions: Visual predictions of several models can be accessed.
    • Datasets: attributes (names and colors) are in folder dataset and some sample lists can be accessed.
    • Some FAQs: FAQ.md.
Performance
Description: mIoU/mAcc stands for mean IoU, mean accuracy of each class and all pixel accuracy respectively.
General parameters cross different datasets are listed below:
  • Network: {NETWORK} @ ACoSP-{COMPRESSION_RATIO}
  • Train Parameters: sync_bn(True), scale_min(0.5), scale_max(2.0), rotate_min(-10), rotate_max(10), zoom_factor(8),
aux_weight(0.4), base_lr(1e-2), power(0.9), momentum(0.9), weight_decay(1e-4).
  • Test Parameters: ignore_label(255).
  1. ADE20K:
Train Parameters: classes(150), train_h(473), train_w(473), epochs(100). Test Parameters: classes(150), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train (20210 images) set and test on val (2000 images) set.
    NetworkmIoU/mAcc
    PSPNet5041.42/51.48
    PSPNet50 @ ACoSP-238.97/49.56
    PSPNet50 @ ACoSP-433.67/43.17
    PSPNet50 @ ACoSP-828.04/35.60
    PSPNet50 @ ACoSP-1619.39/25.52
  2. PASCAL VOC 2012:
Train Parameters: classes(21), train_h(473), train_w(473), epochs(50). Test Parameters: classes(21), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train_aug (10582 images) set and test on val (1449 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.30/85.27
    PSPNet50 @ ACoSP-272.71/81.87
    PSPNet50 @ ACoSP-465.84/77.12
    PSPNet50 @ ACoSP-858.26/69.65
    PSPNet50 @ ACoSP-1648.06/58.83
  3. Cityscapes:
Train Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), epochs(200). Test
Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), base_size(2048).
    • Setting: train on fine_train (2975 images) set and test on fine_val (500 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.35/84.27
    PSPNet50 @ ACoSP-274.11/81.73
    PSPNet50 @ ACoSP-471.50/79.40
    PSPNet50 @ ACoSP-866.06/74.33
    PSPNet50 @ ACoSP-1659.49/67.74
    SegNet65.12/73.85
    SegNet @ ACoSP-264.62/73.19
    SegNet @ ACoSP-460.77/69.57
    SegNet @ ACoSP-854.34/62.48
    SegNet @ ACoSP-1644.12/50.87
  4. CamVid:
Train Parameters: classes(11), train_h(360), train_w(720), epochs(450). Test Parameters: classes(11), test_h(360),
test_w(720), base_size(360).
    • Setting: train on train (367 images) set and test on test (233 images) set.
    NetworkmIoU/mAcc
    SegNet55.49+-0.85/65.44+-1.01
    SegNet @ ACoSP-251.85+-0.83/61.86+-0.85
    SegNet @ ACoSP-450.10+-1.11/59.79+-1.49
    SegNet @ ACoSP-847.25+-1.18/56.87+-1.10
    SegNet @ ACoSP-1642.27+-1.95/51.25+-2.02
  5. Cifar10:
Train Parameters: classes(10), train_h(32), train_w(32), epochs(50). Test Parameters: classes(10), test_h(32),
test_w(32), base_size(32).
    • Setting: train on train (50000 images) set and test on test (10000 images) set.
    NetworkmAcc
    ResNet1889.68
    ResNet18 @ ACoSP-288.50
    ResNet18 @ ACoSP-486.21
    ResNet18 @ ACoSP-881.06
    ResNet18 @ ACoSP-1676.81
Citation
If you find the paper, acosp/ code or trained models useful, please consider citing:
@article{Ditschuneit2022AutoCompressingSP,
title={Auto-Compressing Subset Pruning for Semantic Image Segmentation},
author={Konstantin Ditschuneit and J. Otterbach},
journal={ArXiv},
year={2022},
volume={abs/2201.11103}
}

For the general training code, please also consider referencing hszhao/semseg.
Question
Some FAQ.md collected. You are welcome to send pull requests or give some advices. Contact
information: at.
About
Semantic Segmentation in Pytorch
Resources
Readme
License
Activity
Custom properties
Stars
10 stars
Watchers
0 watching
Forks
3 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - merantix/acosp: Semantic Segmentation in Pytorch · GitHub
Skip to content
BranchesTags
Repository files navigation
Auto-Compressing Subset Pruning for Semantic Image Segmentation
Pytorch Implementation of Auto-Compressing Subset Pruning for Semantic Image Segmentation.
Introduction
ACoSP is an online pruning algorithm that compresses convolutional neural networks during training.
It learns to select a subset of channels from convolutional layers through a sigmoid function, as shown in the figure.
For each channel a w_i is used to scale activations.
ACoSP selection scheme.
The segmentation maps display compressed PSPNet-50 models trained on Cityscapes. The models are up to 16 times smaller.
Repository
This repository is a PyTorch implementation of ACoSP based on hszhao/semseg. It was
used to run all experiments used for the publication and is meant to guarantee reproducibility and audibility of our
results.
The training, test and configuration infrastructure is kept close to semseg, with only some minor modifications to
enable more reproducibility and integrate our pruning code. The model/ package contains the PSPNet50 and SegNet model definitions. In acosp/ all code required to prune during training is defined.
The current configs expect a special folder structure (but can be easily adapted):
  • /data: Datasets, Pretrained-weights
  • /logs/exp: Folder to store experiments
Installation
  1. Clone the repository:
    git clone git@github.com:merantix/acosp.git
  2. Install ACoSP including requirements:
    pip install .
Using ACoSP
The implementation of ACoSP is encapsulated in /acosp. It can be applied to CNN models, e.g.
importtorchvisionmodel=torchvision.models.resnet18()
, with just a few steps:
  1. Create a pruner and adapt the model:
fromacosp.prunerimportSoftTopKPruner# Create pruner objectpruner=SoftTopKPruner(
starting_epoch=0,
ending_epoch=100, # Pruning durationfinal_sparsity=0.5, # Final sparsity
)
  1. Add masking layers to your model.
# Sometimes you want to keep all channels in some of the convolution layers # model.my_conv.unprunable = True# Add sigmoid soft k masks to modelpruner.configure_model(model)
These layers mask out a fraction (pruner.final_sparsity) of the channels during training.
  1. In your training loop update the temperature of all masking layers:
# Update the temperature in all masking layerspruner.update_mask_layers(model, epoch)
  1. Convert the soft pruning to hard pruning when ending_epoch is reached:
importacosp.injectifepoch==pruner.ending_epoch:
# Convert to binary channel maskacosp.inject.soft_to_hard_k(model)
Experiments
  1. Highlight:
    • All initialization models, trained models are available.
The structure is:

      | init/ # initial models
| exp/
|-- ade20k/ # ade20k/camvid/cityscapes/voc2012/cifar10
| |-- pspnet50_{SPARSITY}/ # the sparsity refers to the relative amount of weights that are removed. I.e. sparsity=0.75 <==> compression_ratio=4 | |-- model # model files
| |-- ... # config/train/test files
|-- evals/ # all result with class wise IoU/Acc

  2. Hardware Requirements: At least 60GB (PSPNet50) / 16GB (SegNet) of GPU RAM. Can be distributed to multiple GPUs.
  3. Train:
    • Download related datasets and symlink the paths to them as follows (you can alternatively modify the relevant
paths specified in folder config):
      mkdir -p /
ln -s /path_to_ade20k_dataset /data/ade20k

    • Download ImageNet pre-trained models and put them under folder /data for weight
initialization. Remember to use the right dataset format detailed in FAQ.md.
    • Specify the gpu used in config then do training. (Training using acosp have only been carried out on a single
GPU. And not been tested with DDP). The general structure to access individual configs is as follows:
      sh tool/train.sh ${DATASET}${CONFIG_NAME_WITHOUT_DATASET}
      E.g. to train a PSPNet50 on the ade20k dataset and use the config `config/ade20k/ade20k_pspnet50.yaml', execute:
      sh tool/train.sh ade20k pspnet50
  4. Test:
    • Download trained segmentation models and put them under folder specified in config or modify the specified paths.
    • For full testing (get listed performance):
      sh tool/test.sh ade20k pspnet50
  5. Visualization: tensorboardX incorporated for better visualization.
    tensorboard --logdir=/logs/exp/ade20k
  6. Other:
    • Resources: GoogleDrive LINK contains shared models, visual predictions and data lists.
    • Models: ImageNet pre-trained models and trained segmentation models can be accessed. Note that our ImageNet
pretrained models are slightly different from
original ResNet implementation in
the beginning part.
    • Predictions: Visual predictions of several models can be accessed.
    • Datasets: attributes (names and colors) are in folder dataset and some sample lists can be accessed.
    • Some FAQs: FAQ.md.
Performance
Description: mIoU/mAcc stands for mean IoU, mean accuracy of each class and all pixel accuracy respectively.
General parameters cross different datasets are listed below:
  • Network: {NETWORK} @ ACoSP-{COMPRESSION_RATIO}
  • Train Parameters: sync_bn(True), scale_min(0.5), scale_max(2.0), rotate_min(-10), rotate_max(10), zoom_factor(8),
aux_weight(0.4), base_lr(1e-2), power(0.9), momentum(0.9), weight_decay(1e-4).
  • Test Parameters: ignore_label(255).
  1. ADE20K:
Train Parameters: classes(150), train_h(473), train_w(473), epochs(100). Test Parameters: classes(150), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train (20210 images) set and test on val (2000 images) set.
    NetworkmIoU/mAcc
    PSPNet5041.42/51.48
    PSPNet50 @ ACoSP-238.97/49.56
    PSPNet50 @ ACoSP-433.67/43.17
    PSPNet50 @ ACoSP-828.04/35.60
    PSPNet50 @ ACoSP-1619.39/25.52
  2. PASCAL VOC 2012:
Train Parameters: classes(21), train_h(473), train_w(473), epochs(50). Test Parameters: classes(21), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train_aug (10582 images) set and test on val (1449 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.30/85.27
    PSPNet50 @ ACoSP-272.71/81.87
    PSPNet50 @ ACoSP-465.84/77.12
    PSPNet50 @ ACoSP-858.26/69.65
    PSPNet50 @ ACoSP-1648.06/58.83
  3. Cityscapes:
Train Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), epochs(200). Test
Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), base_size(2048).
    • Setting: train on fine_train (2975 images) set and test on fine_val (500 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.35/84.27
    PSPNet50 @ ACoSP-274.11/81.73
    PSPNet50 @ ACoSP-471.50/79.40
    PSPNet50 @ ACoSP-866.06/74.33
    PSPNet50 @ ACoSP-1659.49/67.74
    SegNet65.12/73.85
    SegNet @ ACoSP-264.62/73.19
    SegNet @ ACoSP-460.77/69.57
    SegNet @ ACoSP-854.34/62.48
    SegNet @ ACoSP-1644.12/50.87
  4. CamVid:
Train Parameters: classes(11), train_h(360), train_w(720), epochs(450). Test Parameters: classes(11), test_h(360),
test_w(720), base_size(360).
    • Setting: train on train (367 images) set and test on test (233 images) set.
    NetworkmIoU/mAcc
    SegNet55.49+-0.85/65.44+-1.01
    SegNet @ ACoSP-251.85+-0.83/61.86+-0.85
    SegNet @ ACoSP-450.10+-1.11/59.79+-1.49
    SegNet @ ACoSP-847.25+-1.18/56.87+-1.10
    SegNet @ ACoSP-1642.27+-1.95/51.25+-2.02
  5. Cifar10:
Train Parameters: classes(10), train_h(32), train_w(32), epochs(50). Test Parameters: classes(10), test_h(32),
test_w(32), base_size(32).
    • Setting: train on train (50000 images) set and test on test (10000 images) set.
    NetworkmAcc
    ResNet1889.68
    ResNet18 @ ACoSP-288.50
    ResNet18 @ ACoSP-486.21
    ResNet18 @ ACoSP-881.06
    ResNet18 @ ACoSP-1676.81
Citation
If you find the paper, acosp/ code or trained models useful, please consider citing:
@article{Ditschuneit2022AutoCompressingSP,
title={Auto-Compressing Subset Pruning for Semantic Image Segmentation},
author={Konstantin Ditschuneit and J. Otterbach},
journal={ArXiv},
year={2022},
volume={abs/2201.11103}
}

For the general training code, please also consider referencing hszhao/semseg.
Question
Some FAQ.md collected. You are welcome to send pull requests or give some advices. Contact
information: at.
About
Semantic Segmentation in Pytorch
Resources
Readme
License
Activity
Custom properties
Stars
10 stars
Watchers
0 watching
Forks
3 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - merantix/acosp: Semantic Segmentation in Pytorch · GitHub
Skip to content
BranchesTags
Repository files navigation
Auto-Compressing Subset Pruning for Semantic Image Segmentation
Pytorch Implementation of Auto-Compressing Subset Pruning for Semantic Image Segmentation.
Introduction
ACoSP is an online pruning algorithm that compresses convolutional neural networks during training.
It learns to select a subset of channels from convolutional layers through a sigmoid function, as shown in the figure.
For each channel a w_i is used to scale activations.
ACoSP selection scheme.
The segmentation maps display compressed PSPNet-50 models trained on Cityscapes. The models are up to 16 times smaller.
Repository
This repository is a PyTorch implementation of ACoSP based on hszhao/semseg. It was
used to run all experiments used for the publication and is meant to guarantee reproducibility and audibility of our
results.
The training, test and configuration infrastructure is kept close to semseg, with only some minor modifications to
enable more reproducibility and integrate our pruning code. The model/ package contains the PSPNet50 and SegNet model definitions. In acosp/ all code required to prune during training is defined.
The current configs expect a special folder structure (but can be easily adapted):
  • /data: Datasets, Pretrained-weights
  • /logs/exp: Folder to store experiments
Installation
  1. Clone the repository:
    git clone git@github.com:merantix/acosp.git
  2. Install ACoSP including requirements:
    pip install .
Using ACoSP
The implementation of ACoSP is encapsulated in /acosp. It can be applied to CNN models, e.g.
importtorchvisionmodel=torchvision.models.resnet18()
, with just a few steps:
  1. Create a pruner and adapt the model:
fromacosp.prunerimportSoftTopKPruner# Create pruner objectpruner=SoftTopKPruner(
starting_epoch=0,
ending_epoch=100, # Pruning durationfinal_sparsity=0.5, # Final sparsity
)
  1. Add masking layers to your model.
# Sometimes you want to keep all channels in some of the convolution layers # model.my_conv.unprunable = True# Add sigmoid soft k masks to modelpruner.configure_model(model)
These layers mask out a fraction (pruner.final_sparsity) of the channels during training.
  1. In your training loop update the temperature of all masking layers:
# Update the temperature in all masking layerspruner.update_mask_layers(model, epoch)
  1. Convert the soft pruning to hard pruning when ending_epoch is reached:
importacosp.injectifepoch==pruner.ending_epoch:
# Convert to binary channel maskacosp.inject.soft_to_hard_k(model)
Experiments
  1. Highlight:
    • All initialization models, trained models are available.
The structure is:

      | init/ # initial models
| exp/
|-- ade20k/ # ade20k/camvid/cityscapes/voc2012/cifar10
| |-- pspnet50_{SPARSITY}/ # the sparsity refers to the relative amount of weights that are removed. I.e. sparsity=0.75 <==> compression_ratio=4 | |-- model # model files
| |-- ... # config/train/test files
|-- evals/ # all result with class wise IoU/Acc

  2. Hardware Requirements: At least 60GB (PSPNet50) / 16GB (SegNet) of GPU RAM. Can be distributed to multiple GPUs.
  3. Train:
    • Download related datasets and symlink the paths to them as follows (you can alternatively modify the relevant
paths specified in folder config):
      mkdir -p /
ln -s /path_to_ade20k_dataset /data/ade20k

    • Download ImageNet pre-trained models and put them under folder /data for weight
initialization. Remember to use the right dataset format detailed in FAQ.md.
    • Specify the gpu used in config then do training. (Training using acosp have only been carried out on a single
GPU. And not been tested with DDP). The general structure to access individual configs is as follows:
      sh tool/train.sh ${DATASET}${CONFIG_NAME_WITHOUT_DATASET}
      E.g. to train a PSPNet50 on the ade20k dataset and use the config `config/ade20k/ade20k_pspnet50.yaml', execute:
      sh tool/train.sh ade20k pspnet50
  4. Test:
    • Download trained segmentation models and put them under folder specified in config or modify the specified paths.
    • For full testing (get listed performance):
      sh tool/test.sh ade20k pspnet50
  5. Visualization: tensorboardX incorporated for better visualization.
    tensorboard --logdir=/logs/exp/ade20k
  6. Other:
    • Resources: GoogleDrive LINK contains shared models, visual predictions and data lists.
    • Models: ImageNet pre-trained models and trained segmentation models can be accessed. Note that our ImageNet
pretrained models are slightly different from
original ResNet implementation in
the beginning part.
    • Predictions: Visual predictions of several models can be accessed.
    • Datasets: attributes (names and colors) are in folder dataset and some sample lists can be accessed.
    • Some FAQs: FAQ.md.
Performance
Description: mIoU/mAcc stands for mean IoU, mean accuracy of each class and all pixel accuracy respectively.
General parameters cross different datasets are listed below:
  • Network: {NETWORK} @ ACoSP-{COMPRESSION_RATIO}
  • Train Parameters: sync_bn(True), scale_min(0.5), scale_max(2.0), rotate_min(-10), rotate_max(10), zoom_factor(8),
aux_weight(0.4), base_lr(1e-2), power(0.9), momentum(0.9), weight_decay(1e-4).
  • Test Parameters: ignore_label(255).
  1. ADE20K:
Train Parameters: classes(150), train_h(473), train_w(473), epochs(100). Test Parameters: classes(150), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train (20210 images) set and test on val (2000 images) set.
    NetworkmIoU/mAcc
    PSPNet5041.42/51.48
    PSPNet50 @ ACoSP-238.97/49.56
    PSPNet50 @ ACoSP-433.67/43.17
    PSPNet50 @ ACoSP-828.04/35.60
    PSPNet50 @ ACoSP-1619.39/25.52
  2. PASCAL VOC 2012:
Train Parameters: classes(21), train_h(473), train_w(473), epochs(50). Test Parameters: classes(21), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train_aug (10582 images) set and test on val (1449 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.30/85.27
    PSPNet50 @ ACoSP-272.71/81.87
    PSPNet50 @ ACoSP-465.84/77.12
    PSPNet50 @ ACoSP-858.26/69.65
    PSPNet50 @ ACoSP-1648.06/58.83
  3. Cityscapes:
Train Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), epochs(200). Test
Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), base_size(2048).
    • Setting: train on fine_train (2975 images) set and test on fine_val (500 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.35/84.27
    PSPNet50 @ ACoSP-274.11/81.73
    PSPNet50 @ ACoSP-471.50/79.40
    PSPNet50 @ ACoSP-866.06/74.33
    PSPNet50 @ ACoSP-1659.49/67.74
    SegNet65.12/73.85
    SegNet @ ACoSP-264.62/73.19
    SegNet @ ACoSP-460.77/69.57
    SegNet @ ACoSP-854.34/62.48
    SegNet @ ACoSP-1644.12/50.87
  4. CamVid:
Train Parameters: classes(11), train_h(360), train_w(720), epochs(450). Test Parameters: classes(11), test_h(360),
test_w(720), base_size(360).
    • Setting: train on train (367 images) set and test on test (233 images) set.
    NetworkmIoU/mAcc
    SegNet55.49+-0.85/65.44+-1.01
    SegNet @ ACoSP-251.85+-0.83/61.86+-0.85
    SegNet @ ACoSP-450.10+-1.11/59.79+-1.49
    SegNet @ ACoSP-847.25+-1.18/56.87+-1.10
    SegNet @ ACoSP-1642.27+-1.95/51.25+-2.02
  5. Cifar10:
Train Parameters: classes(10), train_h(32), train_w(32), epochs(50). Test Parameters: classes(10), test_h(32),
test_w(32), base_size(32).
    • Setting: train on train (50000 images) set and test on test (10000 images) set.
    NetworkmAcc
    ResNet1889.68
    ResNet18 @ ACoSP-288.50
    ResNet18 @ ACoSP-486.21
    ResNet18 @ ACoSP-881.06
    ResNet18 @ ACoSP-1676.81
Citation
If you find the paper, acosp/ code or trained models useful, please consider citing:
@article{Ditschuneit2022AutoCompressingSP,
title={Auto-Compressing Subset Pruning for Semantic Image Segmentation},
author={Konstantin Ditschuneit and J. Otterbach},
journal={ArXiv},
year={2022},
volume={abs/2201.11103}
}

For the general training code, please also consider referencing hszhao/semseg.
Question
Some FAQ.md collected. You are welcome to send pull requests or give some advices. Contact
information: at.
About
Semantic Segmentation in Pytorch
Resources
Readme
License
Activity
Custom properties
Stars
10 stars
Watchers
0 watching
Forks
3 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
GitHub - merantix/acosp: Semantic Segmentation in Pytorch · GitHub
Skip to content
BranchesTags
Repository files navigation
Auto-Compressing Subset Pruning for Semantic Image Segmentation
Pytorch Implementation of Auto-Compressing Subset Pruning for Semantic Image Segmentation.
Introduction
ACoSP is an online pruning algorithm that compresses convolutional neural networks during training.
It learns to select a subset of channels from convolutional layers through a sigmoid function, as shown in the figure.
For each channel a w_i is used to scale activations.
ACoSP selection scheme.
The segmentation maps display compressed PSPNet-50 models trained on Cityscapes. The models are up to 16 times smaller.
Repository
This repository is a PyTorch implementation of ACoSP based on hszhao/semseg. It was
used to run all experiments used for the publication and is meant to guarantee reproducibility and audibility of our
results.
The training, test and configuration infrastructure is kept close to semseg, with only some minor modifications to
enable more reproducibility and integrate our pruning code. The model/ package contains the PSPNet50 and SegNet model definitions. In acosp/ all code required to prune during training is defined.
The current configs expect a special folder structure (but can be easily adapted):
  • /data: Datasets, Pretrained-weights
  • /logs/exp: Folder to store experiments
Installation
  1. Clone the repository:
    git clone git@github.com:merantix/acosp.git
  2. Install ACoSP including requirements:
    pip install .
Using ACoSP
The implementation of ACoSP is encapsulated in /acosp. It can be applied to CNN models, e.g.
importtorchvisionmodel=torchvision.models.resnet18()
, with just a few steps:
  1. Create a pruner and adapt the model:
fromacosp.prunerimportSoftTopKPruner# Create pruner objectpruner=SoftTopKPruner(
starting_epoch=0,
ending_epoch=100, # Pruning durationfinal_sparsity=0.5, # Final sparsity
)
  1. Add masking layers to your model.
# Sometimes you want to keep all channels in some of the convolution layers # model.my_conv.unprunable = True# Add sigmoid soft k masks to modelpruner.configure_model(model)
These layers mask out a fraction (pruner.final_sparsity) of the channels during training.
  1. In your training loop update the temperature of all masking layers:
# Update the temperature in all masking layerspruner.update_mask_layers(model, epoch)
  1. Convert the soft pruning to hard pruning when ending_epoch is reached:
importacosp.injectifepoch==pruner.ending_epoch:
# Convert to binary channel maskacosp.inject.soft_to_hard_k(model)
Experiments
  1. Highlight:
    • All initialization models, trained models are available.
The structure is:

      | init/ # initial models
| exp/
|-- ade20k/ # ade20k/camvid/cityscapes/voc2012/cifar10
| |-- pspnet50_{SPARSITY}/ # the sparsity refers to the relative amount of weights that are removed. I.e. sparsity=0.75 <==> compression_ratio=4 | |-- model # model files
| |-- ... # config/train/test files
|-- evals/ # all result with class wise IoU/Acc

  2. Hardware Requirements: At least 60GB (PSPNet50) / 16GB (SegNet) of GPU RAM. Can be distributed to multiple GPUs.
  3. Train:
    • Download related datasets and symlink the paths to them as follows (you can alternatively modify the relevant
paths specified in folder config):
      mkdir -p /
ln -s /path_to_ade20k_dataset /data/ade20k

    • Download ImageNet pre-trained models and put them under folder /data for weight
initialization. Remember to use the right dataset format detailed in FAQ.md.
    • Specify the gpu used in config then do training. (Training using acosp have only been carried out on a single
GPU. And not been tested with DDP). The general structure to access individual configs is as follows:
      sh tool/train.sh ${DATASET}${CONFIG_NAME_WITHOUT_DATASET}
      E.g. to train a PSPNet50 on the ade20k dataset and use the config `config/ade20k/ade20k_pspnet50.yaml', execute:
      sh tool/train.sh ade20k pspnet50
  4. Test:
    • Download trained segmentation models and put them under folder specified in config or modify the specified paths.
    • For full testing (get listed performance):
      sh tool/test.sh ade20k pspnet50
  5. Visualization: tensorboardX incorporated for better visualization.
    tensorboard --logdir=/logs/exp/ade20k
  6. Other:
    • Resources: GoogleDrive LINK contains shared models, visual predictions and data lists.
    • Models: ImageNet pre-trained models and trained segmentation models can be accessed. Note that our ImageNet
pretrained models are slightly different from
original ResNet implementation in
the beginning part.
    • Predictions: Visual predictions of several models can be accessed.
    • Datasets: attributes (names and colors) are in folder dataset and some sample lists can be accessed.
    • Some FAQs: FAQ.md.
Performance
Description: mIoU/mAcc stands for mean IoU, mean accuracy of each class and all pixel accuracy respectively.
General parameters cross different datasets are listed below:
  • Network: {NETWORK} @ ACoSP-{COMPRESSION_RATIO}
  • Train Parameters: sync_bn(True), scale_min(0.5), scale_max(2.0), rotate_min(-10), rotate_max(10), zoom_factor(8),
aux_weight(0.4), base_lr(1e-2), power(0.9), momentum(0.9), weight_decay(1e-4).
  • Test Parameters: ignore_label(255).
  1. ADE20K:
Train Parameters: classes(150), train_h(473), train_w(473), epochs(100). Test Parameters: classes(150), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train (20210 images) set and test on val (2000 images) set.
    NetworkmIoU/mAcc
    PSPNet5041.42/51.48
    PSPNet50 @ ACoSP-238.97/49.56
    PSPNet50 @ ACoSP-433.67/43.17
    PSPNet50 @ ACoSP-828.04/35.60
    PSPNet50 @ ACoSP-1619.39/25.52
  2. PASCAL VOC 2012:
Train Parameters: classes(21), train_h(473), train_w(473), epochs(50). Test Parameters: classes(21), test_h(473),
test_w(473), base_size(512).
    • Setting: train on train_aug (10582 images) set and test on val (1449 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.30/85.27
    PSPNet50 @ ACoSP-272.71/81.87
    PSPNet50 @ ACoSP-465.84/77.12
    PSPNet50 @ ACoSP-858.26/69.65
    PSPNet50 @ ACoSP-1648.06/58.83
  3. Cityscapes:
Train Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), epochs(200). Test
Parameters: classes(19), train_h(713/512 -PSP/SegNet), train_h(713/1024 -PSP/SegNet), base_size(2048).
    • Setting: train on fine_train (2975 images) set and test on fine_val (500 images) set.
    NetworkmIoU/mAcc
    PSPNet5077.35/84.27
    PSPNet50 @ ACoSP-274.11/81.73
    PSPNet50 @ ACoSP-471.50/79.40
    PSPNet50 @ ACoSP-866.06/74.33
    PSPNet50 @ ACoSP-1659.49/67.74
    SegNet65.12/73.85
    SegNet @ ACoSP-264.62/73.19
    SegNet @ ACoSP-460.77/69.57
    SegNet @ ACoSP-854.34/62.48
    SegNet @ ACoSP-1644.12/50.87
  4. CamVid:
Train Parameters: classes(11), train_h(360), train_w(720), epochs(450). Test Parameters: classes(11), test_h(360),
test_w(720), base_size(360).
    • Setting: train on train (367 images) set and test on test (233 images) set.
    NetworkmIoU/mAcc
    SegNet55.49+-0.85/65.44+-1.01
    SegNet @ ACoSP-251.85+-0.83/61.86+-0.85
    SegNet @ ACoSP-450.10+-1.11/59.79+-1.49
    SegNet @ ACoSP-847.25+-1.18/56.87+-1.10
    SegNet @ ACoSP-1642.27+-1.95/51.25+-2.02
  5. Cifar10:
Train Parameters: classes(10), train_h(32), train_w(32), epochs(50). Test Parameters: classes(10), test_h(32),
test_w(32), base_size(32).
    • Setting: train on train (50000 images) set and test on test (10000 images) set.
    NetworkmAcc
    ResNet1889.68
    ResNet18 @ ACoSP-288.50
    ResNet18 @ ACoSP-486.21
    ResNet18 @ ACoSP-881.06
    ResNet18 @ ACoSP-1676.81
Citation
If you find the paper, acosp/ code or trained models useful, please consider citing:
@article{Ditschuneit2022AutoCompressingSP,
title={Auto-Compressing Subset Pruning for Semantic Image Segmentation},
author={Konstantin Ditschuneit and J. Otterbach},
journal={ArXiv},
year={2022},
volume={abs/2201.11103}
}

For the general training code, please also consider referencing hszhao/semseg.
Question
Some FAQ.md collected. You are welcome to send pull requests or give some advices. Contact
information: at.
About
Semantic Segmentation in Pytorch
Resources
Readme
License
Activity
Custom properties
Stars
10 stars
Watchers
0 watching
Forks
3 forks
Report repository
Releases
Packages
Used by
Contributors
Languages