The Body Part Regression (BPR) model translates the anatomy in a radiologic volume into a machine-interpretable form.

Overview

Copyright © German Cancer Research Center (DKFZ), Division of Medical Image Computing (MIC). Please make sure that your usage of this code is in compliance with the code license: License


Body Part Regression

The Body Part Regression (BPR) model translates the anatomy in a radiologic volume into a machine-interpretable form. Each axial slice maps to a slice score. The slice scores monotonously increase with patient height. In the following figure, you can find example slices for the predicted slice scores: 0, 25, 50, 75, and 100. In each row independent random CT slices are visible with nearly the same target. It can be seen, that the start of the pelvis maps to 0, the upper pelvis region maps to 25, the start of the lungs to 50, the shoulder region to 75, and the head to 100:

decision tree

With the help of a slice-score look-up table, the mapping between certain landmarks to slice scores can be checked. The BPR model learns in a completely self-supervised fashion. There is no need for annotated data for training the model, besides of evaluation purposes.

The BPR model can be used for sorting and labeling radiologic images by body parts. Moreover, it is useful for cropping specific body parts as a pre-processing or post-processing step of medical algorithms. If a body part is invalid for a certain medical algorithm, it can be cropped out before applying the algorithm to the volume.

The Body Part Regression model in this repository is based on the SSBR model from Yan et al. with a few modifications explained in the master thesis "Body Part Regression for CT Volumes".

For CT volumes, a pretrained model for inference exists already. With a simple command from the terminal, the body part information can be calculated for nifti-files.


1. Install package

You can either use conda or just pip to install the bpreg package.

1.1 Install package without conda

  1. Create a new python environment and activate it through:
python -m venv venv_name
source venv_name/bin/activate
  1. Install the package through:
pip install bpreg

1.2 Install package with conda

  1. Create new conda environment and activate environment with:
conda create -n venv_name
conda activate venv_name
  1. Install pip into the environment
conda install pip
  1. Install the package with pip through the command (with your personal anaconda path):
/home/anaconda3/envs/venv_name/bin/pip install bpreg

You can find your personal anaconda path through the command:

which anaconda

Analyze examined body parts

The scope of the pretrained BPR model for CT volumes are body parts from adults from the beginning of the pelvis to the end of the head. Note that due to missing training data, children, pregnant women or legs are not in the scope of the algorithm. To obtain the body part information for nifti-files you need to provide the nifti-files with the file ending *.nii or *.nii.gz in one directory and run the following command:

bpreg_predict -i 
   
     -o 
    

    
   

Tags for the bpreg_predict command:

  • -i (str): input path, origin of nifti-files
  • -o (str): save path for created meta-data json-files
  • --skip (bool): skip already created .json metadata files (default: 1)
  • --model (str): specify model (default: public model from zenodo for CT volumes)
  • --plot (png): create and save plot for each volume with calculated slice score curve.

Through the bpreg_predict command for each nifti-file in the directory input_path a corresponding json-file gets created and saved in the output_path. Moreover, a README file will be saved in the output path, where the information inside the JSON files is explained.

If your input data is not in the nifti-format you can still apply the BPR model by converting the data to a numpy matrix. A tutorial for using the package for CT images in the numpy format can be found in the notebook: docs/notebooks/inference-example-with-npy-arrays.

If you use this model for your work, please make sure to cite the model and the training data as explained at zenodo.

The meta-data files can be used for three main use cases.

  1. Predicting the examined body part
  2. Filter corrupted CT images
  3. Cropping required region from CT images

1. Predicting the examined body part

The label for the predicted examined body part can be found under body part examined tag in the meta-data file. In the following figure, you can find a comparison between the BodyPartExamined tag from the DICOM meta-data header and the predicted body part examined tag from this method. The predicted body part examined tag is more fine-grained and contains less misleading and missing values than the BodyPartExamined tag from the DICOM header:

Pie charts of comparisson between DICOM BodyPartExamined tag and predicted body part examined tag

2. Filter corrupted CT images

Some of the predicted body part examined tags are NONE, which means that the predicted slice score curve for this CT volume looks unexpected (then thevalid z-spacing tag from the meta-data is equal to 0). Based on the NONE tag corrupted CT volumes can be automatically found. In the following, you find in the left a typical CT volume with a corresponding typical slice score curve. Next to the typical CT volume several corrupted CT volumes are shown with the corresponding slice score curves. It can be seen that the slice score curves from the corrupted CT volumes are clearly different from the expected slice score curve. If the slice score curve is looking is monotonously increasing as in the left figure but the predicted body part examined tag is still NONE then this happens because the z-spacing of the CT volume seems to be wrong.

Example figures of slice score curves from corrupted CT images

3. Cropping required region from CT images

The meta-data can be used as well to crop appropriate regions from a CT volume. This can be helpful for medical computer vision algorithms. It can be implemented as a pre-processing or post-processing step and leads to less false-positive predictions in regions which the model has not seen during training: Figure of known region cropping process as pre-processing step or post-processing step for a lung segmentation method


Structure of metadata file

The json-file contains all the metadata regarding the examined body part of the nifti-file. It includes the following tags:

  • cleaned slice-scores: Cleanup of the outcome from the BPR model (smoothing, filtering out outliers).
  • unprocessed slice-scores: Plain outcome of the BPR model.
  • body part examined: Dictionary with the tags: "legs", "pelvis", "abdomen", "chest", "shoulder-neck" and "head". For each body-part, the slice indices are listed, where the body part is visible.
  • body part examined tag: updated tag for BodyPartExamined. Possible values: PELVIS, ABDOMEN, CHEST, NECK, HEAD, HEAD-NECK-CHEST-ABDOMEN-PELVIS, HEAD-NECK-CHEST-ABDOMEN, ...
  • look-up table: reference table to be able to map slice scores to landmarks and vise versa.
  • reverse z-ordering: (0/1) equal to one if patient height decreases with slice index.
  • valid z-spacing: (0/1) equal to one if z-spacing seems to be plausible. The data sanity check is based on the slope of the curve from the cleaned slice-scores.

The information from the meta-data file can be traced back to the unprocessed slice-scores and the look-up table.


Documentation for Body Part Regression

In the docs/notebooks folder, you can find a tutorial on how to use the body part regression model for inference. An example will be presented, were the lungs are detected and cropped automatically from CT volumes. Moreover, a tutorial for training and evaluating a Body Part Regression model can be found.

For a more detailed explanation to the theory behind Body Part Regression and the application use cases have a look into the master thesis "Body Part Regression for CT Images" from Sarah Schuhegger.


Cite Software

Sarah Schuhegger. (2021). MIC-DKFZ/BodyPartRegression: (v1.0). Zenodo. https://doi.org/10.5281/zenodo.5195341

Owner
MIC-DKFZ
Division of Medical Image Computing, German Cancer Research Center (DKFZ)
MIC-DKFZ
Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)

Skyformer This repository is the official implementation of Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr"om Method (NeurIPS 2021).

Qi Zeng 46 Sep 20, 2022
Estimating Example Difficulty using Variance of Gradients

Estimating Example Difficulty using Variance of Gradients This repository contains source code necessary to reproduce some of the main results in the

Chirag Agarwal 48 Dec 26, 2022
Implementation of Research Paper "Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation"

Zero-DCE and Zero-DCE++(Lite architechture for Mobile and edge Devices) Papers Abstract The paper presents a novel method, Zero-Reference Deep Curve E

Tauhid Khan 15 Dec 10, 2022
Just Go with the Flow: Self-Supervised Scene Flow Estimation

Just Go with the Flow: Self-Supervised Scene Flow Estimation Code release for the paper Just Go with the Flow: Self-Supervised Scene Flow Estimation,

Himangi Mittal 50 Nov 22, 2022
NeurIPS'21 Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows

NeurIPS'21 Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows This repo contains the code for the paper Tractable Densit

Layer6 Labs 4 Dec 12, 2022
A unified 3D Transformer Pipeline for visual synthesis

Overview This is the official repo for the paper: NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion. NÜWA is a unified multimodal p

Microsoft 2.6k Jan 06, 2023
4K videos with annotated masks in our ICCV2021 paper 'Internal Video Inpainting by Implicit Long-range Propagation'.

Annotated 4K Videos paper | project website | code | demo video 4K videos with annotated object masks in our ICCV2021 paper: Internal Video Inpainting

Tengfei Wang 21 Nov 05, 2022
PyTorch Implementation of PortaSpeech: Portable and High-Quality Generative Text-to-Speech

PortaSpeech - PyTorch Implementation PyTorch Implementation of PortaSpeech: Portable and High-Quality Generative Text-to-Speech. Model Size Module Nor

Keon Lee 279 Jan 04, 2023
Norm-based Analysis of Transformer

Norm-based Analysis of Transformer Implementations for 2 papers introducing to analyze Transformers using vector norms: Kobayashi+'20 Attention is Not

Goro Kobayashi 52 Dec 05, 2022
PyTorch implementation of the implicit Q-learning algorithm (IQL)

Implicit-Q-Learning (IQL) PyTorch implementation of the implicit Q-learning algorithm IQL (Paper) Currently only implemented for online learning. Offl

Sebastian Dittert 27 Dec 30, 2022
An Efficient Implementation of Analytic Mesh Algorithm for 3D Iso-surface Extraction from Neural Networks

AnalyticMesh Analytic Marching is an exact meshing solution from neural networks. Compared to standard methods, it completely avoids geometric and top

Karbo 45 Dec 21, 2022
Sibur challange 2021 competition - 6 place

sibur challange 2021 Решение на 6 место: https://sibur.ai-community.com/competitions/5/tasks/13 Скор 1.4066/1.4159 public/private. Архитектура - однос

Ivan 5 Jan 11, 2022
Code for A Volumetric Transformer for Accurate 3D Tumor Segmentation

VT-UNet This repo contains the supported pytorch code and configuration files to reproduce 3D medical image segmentaion results of VT-UNet. Environmen

Himashi Amanda Peiris 114 Dec 20, 2022
Implementation of "Semi-supervised Domain Adaptive Structure Learning"

Semi-supervised Domain Adaptive Structure Learning - ASDA This repo contains the source code and dataset for our ASDA paper. Illustration of the propo

3 Dec 13, 2021
Modeling Temporal Concept Receptive Field Dynamically for Untrimmed Video Analysis

Modeling Temporal Concept Receptive Field Dynamically for Untrimmed Video Analysis This is a PyTorch implementation of the model described in our pape

qzhb 6 Jul 08, 2021
Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis (CVPR2022)

Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis Multi-View Consistent Generative Adversarial Networks for 3D-aware

Xuanmeng Zhang 78 Dec 10, 2022
Lazy, a tool for running things in idle time

Lazy, a tool for running things in idle time Mostly used to stop CUDA ML model training from making my desktop unusable. Simply monitors keyboard/mous

N Shepperd 46 Nov 06, 2022
code for Multi-scale Matching Networks for Semantic Correspondence, ICCV

MMNet This repo is the official implementation of ICCV 2021 paper "Multi-scale Matching Networks for Semantic Correspondence.". Pre-requisite conda cr

joey zhao 25 Dec 12, 2022
gACSON software for visualization, processing and analysis of three-dimensional electron microscopy images

gACSON gACSON software is to visualize, segment, and analyze the morphology of neurons in three-dimensional electron microscopy images. If you use any

Andrea Behanova 2 May 31, 2022
Wenzhou-Kean University AI-LAB

AI-LAB This is Wenzhou-Kean University AI-LAB. Our research interests are in Computer Vision and Natural Language Processing. Computer Vision Please g

WKU AI-LAB 10 May 05, 2022