This is the code repository for the paper A hierarchical semantic segmentation framework for computer-vision-based bridge column damage detection

Overview

Bridge-damage-segmentation

This is the code repository for the paper A hierarchical semantic segmentation framework for computer-vision-based bridge column damage detection submitted to the IC-SHM Challenge 2021. The semantic segmentation framework used in this paper leverages importance sampling, semantic mask, and multi-scale test time augmentation to achieve a 0.836 IoU for scene component segmentation and a 0.467 IoU for concrete damage segmentation on the Tokaido Dataset. The framework was implemented on MMSegmentation using Python.

Highlights

Models used in the framework

Backbones

  • HRNet
  • Swin
  • ResNest

Decoder Heads

  • PSPNet
  • UperNet
  • OCRNet

Performance

The following table reports IoUs for structural component segmentation.

Architecture Slab Beam Column Non-structural Rail Sleeper Average
Ensemble 0.891 0.880 0.859 0.660 0.623 0.701 0.785
Ensemble + Importance sampling 0.915 0.912 0.958 0.669 0.618 0.892 0.827
Ensemble + Importance sampling + Multi-scale TTA 0.924 0.929 0.965 0.681 0.621 0.894 0.836

The following table reports IoUs for damage segmentation of pure texture images.

Architecture Concrete damage Exposed rebar Average
Ensemble 0.356 0.536 0.446
Ensemble + Importance sampling 0.708 0.714 0.711
Ensemble + Importance sampling + Multi-scale TTA 0.698 0.727 0.712

The following table reports IoUs for damage segmentation of real scene images.

Architecture Concrete damage Exposed rebar Average
Ensemble 0.235 0.365 0.300
Ensemble + Importance sampling 0.340 0.557 0.448
Ensemble + Importance sampling + Multi-scale TTA 0.350 0.583 0.467
Ensemble + Importance sampling + Multi-scale TTA + Mask 0.379 0.587 0.483

Environment

The code is developed under the following configurations.

  • Hardware: >= 2 GPUs for training, >= 1 GPU for testing. The script supports sbatch training and testing on computer clusters.
  • Software:
    • System: Ubuntu 16.04.3 LTS
    • CUDA >= 10.1
  • Dependencies:

Usage

Environment

  1. Install conda and create a conda environment

    $ conda create -n open-mmlab
    $ source activate open-mmlab
    $ conda install pytorch=1.6.0 torchvision cudatoolkit=10.1 -c pytorch
  2. Install mmcv-full

    $ pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.6.0/index.html
  3. Install mmsegmentation

    $ pip install git+https://github.com/open-mmlab/mmsegmentation.git
  4. Install other dependencies

    $ pip install opencv, tqdm, numpy, scipy
  5. Download the Tokaido dataset from IC-SHM Challenge 2021.

Training

  1. Example single model training using multiple GPUs
    $ python3 -m torch.distributed.launch --nproc_per_node=2 --nnodes=2 --master_port=$RANDOM ./apis/train_damage_real.py \
      --nw hrnet \
      --cp $CHECKPOINT_DIR \
      --dr $DATA_ROOT \
      --conf $MODEL_CONFIG \
      --bs 16 \
      --train_split $TRAIN_SPLIT_PATH \
      --val_split $VAL_SPLIT_PATH \
      --width 1920 \
      --height 1080 \
      --distributed \
      --iter 100000 \
      --log_iter 10000 \
      --eval_iter 10000 \
      --checkpoint_iter 10000 \
      --multi_loss \
      --ohem \
      --job_name dmg
  2. Example shell script for preparing the whole dataset and train all models for the whole pipeline.
    $ ./scripts/main_training_script.sh

Evlauation

  1. Eval one model

    $ python3 ./test/test.py \
      --nw hrnet \
      --task single \
      --cp $CONFIG_PATH \
      --dr $DATA_ROOT \
      --split_csv $RAW_CSV_PATH \
      --save_path $OUTPOUT_DIR \
      --img_dir $INPUT_IMG_DIR \
      --ann_dir $INPUT_GT_DIR \
      --split $TEST_SPLIT_PATH \
      --type cmp \
      --width 640 \
      --height 360
  2. Example shell script for testing the whole pipeline and generate the output using the IC-SHM Challenge format.

    $ ./scripts/main_testing_script.sh
  3. Visualization (Add the --cmp flag when visualizing components.)

    $ ./modules/viz_label.py \
      --input $SEG_DIR
      --output $OUTPUT_DIR
      --raw_input $IMG_DIR
      --cmp 

Reference

If you find the code useful, please cite the following paper.

Owner
Jingxiao Liu
PhD Candidate at Stanford University
Jingxiao Liu
WaveFake: A Data Set to Facilitate Audio DeepFake Detection

WaveFake: A Data Set to Facilitate Audio DeepFake Detection This is the code repository for our NeurIPS 2021 (Track on Datasets and Benchmarks) paper

Chair for Sys­tems Se­cu­ri­ty 27 Dec 22, 2022
Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology (LMRL Workshop, NeurIPS 2021)

Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology Self-Supervised Vision Transformers Learn Visual Concepts in Histopatholog

Richard Chen 95 Dec 24, 2022
Using the provided dataset which includes various book features, in order to predict the price of books, using various proposed methods and models.

Using the provided dataset which includes various book features, in order to predict the price of books, using various proposed methods and models.

Nikolas Petrou 1 Jan 13, 2022
The official implementation of A Unified Game-Theoretic Interpretation of Adversarial Robustness.

This repository is the official implementation of A Unified Game-Theoretic Interpretation of Adversarial Robustness. Requirements pip install -r requi

Jie Ren 17 Dec 12, 2022
Implementation of "With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition, BMVC, 2021" in PyTorch

Multimodal Temporal Context Network (MTCN) This repository implements the model proposed in the paper: Evangelos Kazakos, Jaesung Huh, Arsha Nagrani,

Evangelos Kazakos 13 Nov 24, 2022
Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval (NeurIPS'21)

Baleen Baleen is a state-of-the-art model for multi-hop reasoning, enabling scalable multi-hop search over massive collections for knowledge-intensive

Stanford Future Data Systems 22 Dec 05, 2022
This is the first released system towards complex meters` detection and recognition, which is implemented by computer vision techniques.

A three-stage detection and recognition pipeline of complex meters in wild This is the first released system towards detection and recognition of comp

Yan Shu 19 Nov 28, 2022
PyTorch implementation of the paper:A Convolutional Approach to Melody Line Identification in Symbolic Scores.

Symbolic Melody Identification This repository is an unofficial PyTorch implementation of the paper:A Convolutional Approach to Melody Line Identifica

Sophia Y. Chou 3 Feb 21, 2022
A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

Note: This is an alpha (preview) version which is still under refining. nn-Meter is a novel and efficient system to accurately predict the inference l

Microsoft 244 Jan 06, 2023
Unofficial implementation of the ImageNet, CIFAR 10 and SVHN Augmentation Policies learned by AutoAugment using pillow

AutoAugment - Learning Augmentation Policies from Data Unofficial implementation of the ImageNet, CIFAR10 and SVHN Augmentation Policies learned by Au

Philip Popien 1.3k Jan 02, 2023
Hooks for VCOCO

Verbs in COCO (V-COCO) Dataset This repository hosts the Verbs in COCO (V-COCO) dataset and associated code to evaluate models for the Visual Semantic

Saurabh Gupta 131 Nov 24, 2022
Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models

Molecular Sets (MOSES): A benchmarking platform for molecular generation models Deep generative models are rapidly becoming popular for the discovery

MOSES 656 Dec 29, 2022
Codes for "CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation"

CSDI This is the github repository for the NeurIPS 2021 paper "CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation

106 Jan 04, 2023
Anderson Acceleration for Deep Learning

Anderson Accelerated Deep Learning (AADL) AADL is a Python package that implements the Anderson acceleration to speed-up the training of deep learning

Oak Ridge National Laboratory 7 Nov 24, 2022
SafePicking: Learning Safe Object Extraction via Object-Level Mapping, ICRA 2022

SafePicking Learning Safe Object Extraction via Object-Level Mapping Kentaro Wad

Kentaro Wada 49 Oct 24, 2022
Time-Optimal Planning for Quadrotor Waypoint Flight

Time-Optimal Planning for Quadrotor Waypoint Flight This is an example implementation of the paper "Time-Optimal Planning for Quadrotor Waypoint Fligh

Robotics and Perception Group 38 Dec 02, 2022
Nvdiffrast - Modular Primitives for High-Performance Differentiable Rendering

Nvdiffrast – Modular Primitives for High-Performance Differentiable Rendering Modular Primitives for High-Performance Differentiable Rendering Samuli

NVIDIA Research Projects 675 Jan 06, 2023
unet-family: Ultimate version

unet-family: Ultimate version 基于之前my-unet代码,我整理出来了这一份终极版本unet-family,方便其他人阅读。 相比于之前的my-unet代码,代码分类更加规范,有条理 对于clone下来的代码不需要修改各种复杂繁琐的路径问题,直接就可以运行。 并且代码有

2 Sep 19, 2022
Implementation of Shape Generation and Completion Through Point-Voxel Diffusion

Shape Generation and Completion Through Point-Voxel Diffusion Project | Paper Implementation of Shape Generation and Completion Through Point-Voxel Di

Linqi Zhou 103 Dec 29, 2022
CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks

CALVIN CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks Oier Mees, Lukas Hermann, Erick Rosete,

Oier Mees 107 Dec 26, 2022