A pytorch-based deep learning framework for multi-modal 2D/3D medical image segmentation

Overview

A 3D multi-modal medical image segmentation library in PyTorch

Contributors Forks Stargazers Issues Open In Colab

We strongly believe in open and reproducible deep learning research. Our goal is to implement an open-source medical image segmentation library of state of the art 3D deep neural networks in PyTorch. We also implemented a bunch of data loaders of the most common medical image datasets. This project started as an MSc Thesis and is currently under further development. Although this work was initially focused on 3D multi-modal brain MRI segmentation we are slowly adding more architectures and data-loaders.

Top priorities 21-07

[Update] 21-07 We have just received a brand new GPU. The project developedment was postponed due to lack of computational resources. We will be back with more updates. Please Watch our Github repository for releases to be notified. We are always looking for passionate open-source contributos. Full credits will be given.

  • Project restructure, API/CLI design ++
  • Minimal test prediction example with pre-trained models
  • Overlapping and non-overlapping inference
  • Finalize preprocessing on Brats datasets
  • Save produced 3d-total-segmentation as nifty files
  • Medical image decathlon dataloaders
  • StructSeg 2019 challenge dataloaders
  • More options for 2D architectures
  • Rewrite manual
  • New notebooks with google colab support

Quick Start

  • If you want to quickly understand the fundamental concepts for deep learning in medical imaging, we strongly advice to check our blog post. We provide a general high-level overview of all the aspects of medical image segmentation and deep learning. For a broader overview on MRI applications find my latest review article.

  • To grasp more fundamental medical imaging concepts, check out our post on coordinate systems and DICOM images.

  • For a more holistic approach on Deep Learning in MRI you may advice my thesis this.

  • Alternatively, you can create a virtual environment and install the requirements. Check the installation folder for more instructions.

  • You can also take a quick glance at the manual.

  • If you do not have a capable environment or device to run this projects then you could give Google Colab a try. It allows you to run the project using a GPU device, free of charge. You may try our Colab demo using this notebook:Open In Colab

Implemented architectures

  • U-Net3D Learning Dense Volumetric Segmentation from Sparse Annotation Learning Dense Volumetric Segmentation from Sparse Annotation

  • V-net Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

  • ResNet3D-VAE 3D MRI brain tumor segmentation using auto-encoder regularization

  • U-Net Convolutional Networks for Biomedical Image Segmentation

  • SkipDesneNet3D 3D Densely Convolutional Networks for Volumetric Segmentation

  • HyperDense-Net A hyper-densely connected CNN for multi-modal image segmentation

  • multi-stream Densenet3D A hyper-densely connected CNN for multi-modal image segmentation

  • DenseVoxelNet Automatic 3D Cardiovascular MR Segmentation with Densely-Connected Volumetric ConvNets

  • MED3D Transfer learning for 3D medical image analysis

  • HighResNet3D On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task

Implemented medical imaging data-loaders

Task Data Info/ Modalities Train/Test Volume size Classes Dataset size (GB)
Iseg 2017 T1, T2 10 / 10 144x192x256 4 0.72
Iseg 2019 T1, T2 10 / 13 144x192x256 4 0.75
MICCAI BraTs2018 FLAIR, T1w, T1gd,T2w 285 / - 240x240x155 9 or 4 2.4
MICCAI BraTs2019 FLAIR, T1w, T1gd,T2w 335 / 125 240x240x155 9 or 4 4
Mrbrains 2018 FLAIR, T1w, T1gd,T2w 8 240x240x48 9 or 4 0.5
IXI brain development Dataset T1,T2 no labels 581 (110~150)x256x256 - 8.7
MICCAI Gleason 2019 Challenge 2D pathology images ~250 5K x 5K - 2.5

Preliminary results

Visual results on Iseg-2017

Iseg and Mr-brains

Model # Params (M) MACS(G) Iseg 2017 DSC (%) Mr-brains 4 classes DSC (%)
Unet3D 17 M 0.9 93.84 88.61
Vnet 45 M 12 87.21 84.09
DenseNet3D 3 M 5.1 81.65 79.85
SkipDenseNet3D 1.5 M 31 - -
DenseVoxelNet 1.8 M 8 - -
HyperDenseNet 10.4 M 5.8 - -

Usage

How to train your model

  • For Iseg-2017 :
python ./examples/train_iseg2017_new.py --args
  • For MR brains 2018 (4 classes)
python ./examples/train_mrbrains_4_classes.py --args
  • For MR brains 2018 (8 classes)
python ./examples/train_mrbrains_9_classes.py --args
  • For MICCAI 2019 Gleason Challenge
python ./examples/test_miccai_2019.py --args
  • The arguments that you can modify are extensively listed in the manual.

Inference

How to test your trained model in a medical image

python ./tests/inference.py --args

Covid-19 segmentation and classification

We provide some implementations around Covid-19 for humanitarian purposes. In detail:

Classification model

  • COVID-Net A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest Radiography Images

Datasets

Classification from 2D images:

3D COVID-19 segmentation dataset

Latest features (06/2020)

  • On the fly 3D total volume visualization
  • Tensorboard and PyTorch 1.4+ support to track training progress
  • Code cleanup and packages creation
  • Offline sub-volume generation
  • Add Hyperdensenet, 3DResnet-VAE, DenseVoxelNet
  • Fix mrbrains,Brats2018,Brats2019, Iseg2019, IXI,MICCAI 2019 gleason challenge dataloaders
  • Add confusion matrix support for understanding training dynamics
  • Some Visualizations

Support

If you really like this repository and find it useful, please consider (★) starring it, so that it can reach a broader audience of like-minded people. It would be highly appreciated :) !

Contributing to Medical ZOO

If you find a bug, create a GitHub issue, or even better, submit a pull request. Similarly, if you have questions, simply post them as GitHub issues. More info on the contribute directory.

Current team

Ilias Papastatis, Sergios Karagianakos and Nikolas Adaloglou

License , citation and acknowledgements

Please advice the LICENSE.md file. For usage of third party libraries and repositories please advise the respective distributed terms. It would be nice to cite the original models and datasets. If you want, you can also cite this work as:

@MastersThesis{adaloglou2019MRIsegmentation,
author = {Adaloglou Nikolaos},
title={Deep learning in medical image analysis: a comparative analysis of
multi-modal brain-MRI segmentation with 3D deep neural networks},
school = {University of Patras},
note="\url{https://github.com/black0017/MedicalZooPytorch}",
year = {2019},
organization={Nemertes}}

Acknowledgements

In general, in the open source community recognizing third party utilities increases the credibility of your software. In deep learning, academics tend to skip acknowledging third party repos for some reason. In essence, we used whatever resource we needed to make this project self-complete, that was nicely written. However, modifications were performed to match the project structure and requirements. Here is the list of the top-based works: HyperDenseNet model. Most of the segmentation losses from here. 3D-SkipDenseNet model from here. 3D-ResNet base model from here. Abstract model class from MimiCry project. Trainer and Writer class from PyTorch template. Covid-19 implementation based on our previous work from here. MICCAI 2019 Gleason challenge data-loaders based on our previous work from here. Basic 2D Unet implementation from here.Vnet model from here

Owner
Adaloglou Nikolas
Human-Centered AI PhD Researcher.
Adaloglou Nikolas
Implementation of BI-RADS-BERT & The Advantages of Section Tokenization.

BI-RADS BERT Implementation of BI-RADS-BERT & The Advantages of Section Tokenization. This implementation could be used on other radiology in house co

1 May 17, 2022
PyTorch implementation of the REMIND method from our ECCV-2020 paper "REMIND Your Neural Network to Prevent Catastrophic Forgetting"

REMIND Your Neural Network to Prevent Catastrophic Forgetting This is a PyTorch implementation of the REMIND algorithm from our ECCV-2020 paper. An ar

Tyler Hayes 72 Nov 27, 2022
Source code for From Stars to Subgraphs

GNNAsKernel Official code for From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness Visualizations GNN-AK(+) GNN-AK(+) with Subgra

44 Dec 19, 2022
Modular Gaussian Processes

Modular Gaussian Processes for Transfer Learning 🧩 Introduction This repository contains the implementation of our paper Modular Gaussian Processes f

Pablo Moreno-Muñoz 10 Mar 15, 2022
Codes to calculate solar-sensor zenith and azimuth angles directly from hyperspectral images collected by UAV. Works only for UAVs that have high resolution GNSS/IMU unit.

UAV Solar-Sensor Angle Calculation Table of Contents About The Project Built With Getting Started Prerequisites Installation Datasets Contributing Lic

Sourav Bhadra 1 Jan 15, 2022
MANO hand model porting for the GraspIt simulator

Learning Joint Reconstruction of Hands and Manipulated Objects - ManoGrasp Porting the MANO hand model to GraspIt! simulator Yana Hasson, Gül Varol, D

Lucas Wohlhart 10 Feb 08, 2022
Pytorch Lightning 1.2k Jan 06, 2023
Only works with the dashboard version / branch of jesse

Jesse optuna Only works with the dashboard version / branch of jesse. The config.yml should be self-explainatory. Installation # install from git pip

Markus K. 8 Dec 04, 2022
Prototypical Networks for Few shot Learning in PyTorch

Prototypical Networks for Few shot Learning in PyTorch Simple alternative Implementation of Prototypical Networks for Few Shot Learning (paper, code)

Orobix 835 Jan 08, 2023
EZ graph is an easy to use AI solution that allows you to make and train your neural networks without a single line of code.

EZ-Graph EZ Graph is a GUI that allows users to make and train neural networks without writing a single line of code. Requirements python 3 pandas num

1 Jul 03, 2022
Keras-tensorflow implementation of Fully Convolutional Networks for Semantic Segmentation(Unfinished)

Keras-FCN Fully convolutional networks and semantic segmentation with Keras. Models Models are found in models.py, and include ResNet and DenseNet bas

645 Dec 29, 2022
Woosung Choi 63 Nov 14, 2022
DeepLab2: A TensorFlow Library for Deep Labeling

DeepLab2 is a TensorFlow library for deep labeling, aiming to provide a unified and state-of-the-art TensorFlow codebase for dense pixel labeling tasks.

Google Research 845 Jan 04, 2023
An index of algorithms for learning causality with data

awesome-causality-algorithms An index of algorithms for learning causality with data. Please cite our survey paper if this index is helpful. @article{

Ruocheng Guo 2.3k Jan 08, 2023
UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss

UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss This repository contains the TensorFlow implementation of the paper UnF

Simon Meister 270 Nov 06, 2022
git《Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A Color Image Denoiser》(2021) GitHub: [fig5]

Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A Color Image Denoiser Abstract The success of deep denoisers on real-world colo

Yue Cao 51 Nov 22, 2022
ULMFiT for Genomic Sequence Data

Genomic ULMFiT This is an implementation of ULMFiT for genomics classification using Pytorch and Fastai. The model architecture used is based on the A

Karl 276 Dec 12, 2022
ECLARE: Extreme Classification with Label Graph Correlations

ECLARE ECLARE: Extreme Classification with Label Graph Correlations @InProceedings{Mittal21b, author = "Mittal, A. and Sachdeva, N. and Agrawal

Extreme Classification 35 Nov 06, 2022
Database Reasoning Over Text project for ACL paper

Database Reasoning over Text This repository contains the code for the Database Reasoning Over Text paper, to appear at ACL2021. Work is performed in

Facebook Research 320 Dec 12, 2022
Research Artifact of USENIX Security 2022 Paper: Automated Side Channel Analysis of Media Software with Manifold Learning

Automated Side Channel Analysis of Media Software with Manifold Learning Official implementation of USENIX Security 2022 paper: Automated Side Channel

Yuanyuan Yuan 175 Jan 07, 2023