This is the code repository for the paper A hierarchical semantic segmentation framework for computer-vision-based bridge column damage detection

Overview

Bridge-damage-segmentation

This is the code repository for the paper A hierarchical semantic segmentation framework for computer-vision-based bridge column damage detection submitted to the IC-SHM Challenge 2021. The semantic segmentation framework used in this paper leverages importance sampling, semantic mask, and multi-scale test time augmentation to achieve a 0.836 IoU for scene component segmentation and a 0.467 IoU for concrete damage segmentation on the Tokaido Dataset. The framework was implemented on MMSegmentation using Python.

Highlights

Models used in the framework

Backbones

  • HRNet
  • Swin
  • ResNest

Decoder Heads

  • PSPNet
  • UperNet
  • OCRNet

Performance

The following table reports IoUs for structural component segmentation.

Architecture Slab Beam Column Non-structural Rail Sleeper Average
Ensemble 0.891 0.880 0.859 0.660 0.623 0.701 0.785
Ensemble + Importance sampling 0.915 0.912 0.958 0.669 0.618 0.892 0.827
Ensemble + Importance sampling + Multi-scale TTA 0.924 0.929 0.965 0.681 0.621 0.894 0.836

The following table reports IoUs for damage segmentation of pure texture images.

Architecture Concrete damage Exposed rebar Average
Ensemble 0.356 0.536 0.446
Ensemble + Importance sampling 0.708 0.714 0.711
Ensemble + Importance sampling + Multi-scale TTA 0.698 0.727 0.712

The following table reports IoUs for damage segmentation of real scene images.

Architecture Concrete damage Exposed rebar Average
Ensemble 0.235 0.365 0.300
Ensemble + Importance sampling 0.340 0.557 0.448
Ensemble + Importance sampling + Multi-scale TTA 0.350 0.583 0.467
Ensemble + Importance sampling + Multi-scale TTA + Mask 0.379 0.587 0.483

Environment

The code is developed under the following configurations.

  • Hardware: >= 2 GPUs for training, >= 1 GPU for testing. The script supports sbatch training and testing on computer clusters.
  • Software:
    • System: Ubuntu 16.04.3 LTS
    • CUDA >= 10.1
  • Dependencies:

Usage

Environment

  1. Install conda and create a conda environment

    $ conda create -n open-mmlab
    $ source activate open-mmlab
    $ conda install pytorch=1.6.0 torchvision cudatoolkit=10.1 -c pytorch
  2. Install mmcv-full

    $ pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.6.0/index.html
  3. Install mmsegmentation

    $ pip install git+https://github.com/open-mmlab/mmsegmentation.git
  4. Install other dependencies

    $ pip install opencv, tqdm, numpy, scipy
  5. Download the Tokaido dataset from IC-SHM Challenge 2021.

Training

  1. Example single model training using multiple GPUs
    $ python3 -m torch.distributed.launch --nproc_per_node=2 --nnodes=2 --master_port=$RANDOM ./apis/train_damage_real.py \
      --nw hrnet \
      --cp $CHECKPOINT_DIR \
      --dr $DATA_ROOT \
      --conf $MODEL_CONFIG \
      --bs 16 \
      --train_split $TRAIN_SPLIT_PATH \
      --val_split $VAL_SPLIT_PATH \
      --width 1920 \
      --height 1080 \
      --distributed \
      --iter 100000 \
      --log_iter 10000 \
      --eval_iter 10000 \
      --checkpoint_iter 10000 \
      --multi_loss \
      --ohem \
      --job_name dmg
  2. Example shell script for preparing the whole dataset and train all models for the whole pipeline.
    $ ./scripts/main_training_script.sh

Evlauation

  1. Eval one model

    $ python3 ./test/test.py \
      --nw hrnet \
      --task single \
      --cp $CONFIG_PATH \
      --dr $DATA_ROOT \
      --split_csv $RAW_CSV_PATH \
      --save_path $OUTPOUT_DIR \
      --img_dir $INPUT_IMG_DIR \
      --ann_dir $INPUT_GT_DIR \
      --split $TEST_SPLIT_PATH \
      --type cmp \
      --width 640 \
      --height 360
  2. Example shell script for testing the whole pipeline and generate the output using the IC-SHM Challenge format.

    $ ./scripts/main_testing_script.sh
  3. Visualization (Add the --cmp flag when visualizing components.)

    $ ./modules/viz_label.py \
      --input $SEG_DIR
      --output $OUTPUT_DIR
      --raw_input $IMG_DIR
      --cmp 

Reference

If you find the code useful, please cite the following paper.

Owner
Jingxiao Liu
PhD Candidate at Stanford University
Jingxiao Liu
Official code repository for Continual Learning In Environments With Polynomial Mixing Times

Official code for Continual Learning In Environments With Polynomial Mixing Times Continual Learning in Environments with Polynomial Mixing Times This

Sharath Raparthy 1 Dec 19, 2021
Automated image registration. Registrationimation was too much of a mouthful.

alignimation Automated image registration. Registrationimation was too much of a mouthful. This repo contains the code used for my blog post Alignimat

Ethan Rosenthal 9 Oct 13, 2022
Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.

WECHSEL Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models. arXiv: https://arx

Institute of Computational Perception 45 Dec 29, 2022
[NeurIPS 2020] Official Implementation: "SMYRF: Efficient Attention using Asymmetric Clustering".

SMYRF: Efficient attention using asymmetric clustering Get started: Abstract We propose a novel type of balanced clustering algorithm to approximate a

Giannis Daras 46 Dec 22, 2022
Bridging Composite and Real: Towards End-to-end Deep Image Matting

Bridging Composite and Real: Towards End-to-end Deep Image Matting Please note that the official repository of the paper Bridging Composite and Real:

Jizhizi_Li 30 Oct 31, 2022
Masked regression code - Masked Regression

Masked Regression MR - Python Implementation This repositery provides a python implementation of MR (Masked Regression). MR can efficiently synthesize

Arbish Akram 1 Dec 23, 2021
Mmrotate - OpenMMLab Rotated Object Detection Benchmark

OpenMMLab website HOT OpenMMLab platform TRY IT OUT πŸ“˜ Documentation | πŸ› οΈ Insta

OpenMMLab 1.2k Jan 04, 2023
PyG (PyTorch Geometric) - A library built upon PyTorch to easily write and train Graph Neural Networks (GNNs)

PyG (PyTorch Geometric) is a library built upon PyTorch to easily write and train Graph Neural Networks (GNNs) for a wide range of applications related to structured data.

PyG 16.5k Jan 08, 2023
A high-level Python library for Quantum Natural Language Processing

lambeq About lambeq is a toolkit for quantum natural language processing (QNLP). Documentation: https://cqcl.github.io/lambeq/ User support: lambeq-su

Cambridge Quantum 315 Jan 01, 2023
Real-CUGAN - Real Cascade U-Nets for Anime Image Super Resolution

Real Cascade U-Nets for Anime Image Super Resolution δΈ­ζ–‡ | English πŸ”₯ Real-CUGAN

tarsin 111 Dec 28, 2022
Faster RCNN pytorch windows

Faster-RCNN-pytorch-windows Faster RCNN implementation with pytorch for windows Open cmd, compile this comands: cd lib python setup.py build develop T

Hwa-Rang Kim 1 Nov 11, 2022
Differentiable Optimizers with Perturbations in Pytorch

Differentiable Optimizers with Perturbations in PyTorch This contains a PyTorch implementation of Differentiable Optimizers with Perturbations in Tens

Jake Tuero 54 Jun 22, 2022
Diffusion Normalizing Flow (DiffFlow) Neurips2021

Diffusion Normalizing Flow (DiffFlow) Reproduce setup environment The repo heavily depends on jam, a personal toolbox developed by Qsh.zh. The API may

76 Jan 01, 2023
Biomarker identification for COVID-19 Severity in BALF cells Single-cell RNA-seq data

scBALF Covid-19 dataset Analysis Here is the Github page that has the codes for the bioinformatics pipeline described in the paper COVID-Datathon: Bio

Nami Niyakan 2 May 21, 2022
You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors

You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors In this paper, we propose a novel local descriptor-based fra

Haiping Wang 80 Dec 15, 2022
Kaggle G2Net Gravitational Wave Detection : 2nd place solution

Kaggle G2Net Gravitational Wave Detection : 2nd place solution

Hiroshechka Y 33 Dec 26, 2022
Federated Deep Reinforcement Learning for the Distributed Control of NextG Wireless Networks.

FDRL-PC-Dyspan Federated Deep Reinforcement Learning for the Distributed Control of NextG Wireless Networks. This repository contains the entire code

Peyman Tehrani 17 Nov 18, 2022
Train a deep learning net with OpenStreetMap features and satellite imagery.

DeepOSM Classify roads and features in satellite imagery, by training neural networks with OpenStreetMap (OSM) data. DeepOSM can: Download a chunk of

TrailBehind, Inc. 1.3k Nov 24, 2022
Аналитика доходности инвСстиционного портфСля Π² Π’ΠΈΠ½ΡŒΠΊΠΎΡ„Ρ„ Π±Ρ€ΠΎΠΊΠ΅Ρ€Π΅

Аналитика доходности инвСстиционного портфСля Виньков Π’ΠΈΠ΄Π΅ΠΎ Π½Π° YouTube Для Ρ€Π°Π±ΠΎΡ‚Ρ‹ скрипта Π½ΡƒΠΆΠ½ΠΎ ΡƒΡΡ‚Π°Π½ΠΎΠ²ΠΈΡ‚ΡŒ Ρ‚Ρ€ΠΈ ΠΏΠ΅Ρ€Π΅ΠΌΠ΅Π½Π½Ρ‹Ρ… окруТСния: export TINKOFF_TO

Alexey Goloburdin 64 Dec 17, 2022
Implementation of the Point Transformer layer, in Pytorch

Point Transformer - Pytorch Implementation of the Point Transformer self-attention layer, in Pytorch. The simple circuit above seemed to have allowed

Phil Wang 501 Jan 03, 2023