Set of models for classifcation of 3D volumes

Overview

Classification models 3D Zoo - Keras and TF.Keras

This repository contains 3D variants of popular CNN models for classification like ResNets, DenseNets, VGG, etc. It also contains weights obtained by converting ImageNet weights from the same 2D models.

This repository is based on great classification_models repo by @qubvel

Architectures:

Installation

pip install classification-models-3D

Examples

Loading model with imagenet weights:
# for keras
from classification_models_3D.keras import Classifiers

# for tensorflow.keras
# from classification_models_3D.tfkeras import Classifiers

ResNet18, preprocess_input = Classifiers.get('resnet18')
model = ResNet18(input_shape=(128, 128, 128, 3), weights='imagenet')

All possible nets for Classifiers.get() method: 'resnet18, 'resnet34', 'resnet50', 'resnet101', 'resnet152', 'seresnet18', 'seresnet34', 'seresnet50', 'seresnet101', 'seresnet152', 'seresnext50', 'seresnext101', 'senet154', 'resnext50', 'resnext101', 'vgg16', 'vgg19', 'densenet121', 'densenet169', 'densenet201', 'inceptionresnetv2', 'inceptionv3', 'mobilenet', 'mobilenetv2'

Convert imagenet weights (2D -> 3D)

Code to convert 2D imagenet weights to 3D variant is available here: convert_imagenet_weights_to_3D_models.py. Weights were obtained with TF2, but works OK with Keras + TF1 as well.

How to choose input shape

If initial 2D model had shape (512, 512, 3) then you can use shape (D, H, W, 3) where D * H * W ~= 512*512, so something like (64, 64, 64, 3) will be ok.

Training with single NVIDIA 1080Ti (11 GB) worked with:

  • DenseNet121, DenseNet169 and ResNet50 with shape (96, 128, 128, 3) and batch size 6
  • DenseNet201 with shape (96, 128, 128, 3) and batch size 5
  • ResNet18 with shape (128, 160, 160, 3) and batch size 6

Related repositories

Unresolved problems

  • There is no DepthwiseConv3D layer in keras, so repo used custom layer from this repo by @alexandrosstergiou which can be slower than native implementation.
  • There is no imagenet weights for 'inceptionresnetv2' and 'inceptionv3'.

Description

This code was used to get 1st place in DrivenData: Advance Alzheimer’s Research with Stall Catchers competition.

More details on ArXiv: https://arxiv.org/abs/2104.01687

Citation

If you find this code useful, please cite it as:

@InProceedings{RSolovyev_2021_stalled,
  author = {Solovyev, Roman and Kalinin, Alexandr A. and Gabruseva, Tatiana},
  title = {3D Convolutional Neural Networks for Stalled Brain Capillary Detection},
  booktitle = {Arxiv: 2104.01687},
  month = {April},
  year = {2021}
}
Comments
  • Update __init__.py

    Update __init__.py

    Using keras 2.9.0, import keras_applications as ka gives the following error:- ModuleNotFoundError: No module named 'keras_applications'

    Instead using from keras import applications as ka works!

    opened by msmuskan 0
  • Pushing current version to PyPI

    Pushing current version to PyPI

    Hello @ZFTurbo,

    if you have time, please push the current updated status (with ConvNeXt) of this repo to PyPI. :)

    Thanks again for the great work and your time!

    Cheers, Dominik

    opened by muellerdo 0
  • Grad cam issue

    Grad cam issue

    Hello ,

    base_model, preprocess_input = Classifiers.get('seresnext50') model = base_model(input_shape=(512, 512, 20, 1 ), weights=None , include_top = False ) x = Flatten()(model.output) x = Dense(1024, activation= 'sigmoid')(x) x = Dense(2, activation= 'sigmoid')(x)

    Trying to train a model , the accuracy is everything resides upto expectation, but the gradcam are quite off from the region of the focus - how the accuracy is good but the grad cam is off the focus of targeted area .

    Using the layer - 'activation-161' as output ref - https://github.com/fitushar/3D-Grad-CAM/blob/master/3DGrad-CAM.ipynb for the gradcam generation code , the results are always at the border of the image.

    opened by ntirupathirao18 0
  • ImportError: cannot import name 'VersionAwareLayers' from 'keras.layers'

    ImportError: cannot import name 'VersionAwareLayers' from 'keras.layers'

    Thank you for the great work.

    I am experiencing the following error over and over, even though I created a brand new tensorflow environment and installed all the necessary libraries in it. Could you please have a look on it and guide me how do I solve this problem? Thank you.

    ImportError: Unable to import 'VersionAwareLayers' from 'keras.layers' (/home/ubuntu/anaconda3/envs/cm_3d/lib/python3.7/site-packages/keras/layers/init.py)

    opened by nasir3843 2
  • 3D DenseNet

    3D DenseNet

    Hello and sorry to bother you beforehand,

    I am currently conducting my master thesis project and I am trying to implement a 3D DenseNet-121 with knee MRIs as input data. While I was searching on how to implement a 3D version of the DenseNet I came across your repository and tried to change it for my application.

    I have some issues regarding my try and I didn't know where else to ask about it and again I am sorry if I am completely of topic asking them here.

    Firstly, my input shapes are (250,320,18,1) and when I give them as input to the 3D DenseNet I developed with stride_size=1 for my Conv_block and pooling_size=(2,2,2) and strides=(2,2,1) for my AveragePooling3D layer in the transition block, the model is constructed properly with the specific input_size, while when I am trying to load a DenseNet121 from classification_models_3d.tfkeras classifiers I am unable to construct it with input_shape(250,320,18,1), stride_size=1 and kernel_size=2. It gives as an error "Negative dimension size... for node pool4_pool/AvgPool3D". Is there a way to specifically define the strides for AvgPool3D layer in the transition block?

    And secondly, I was thinking to load the 3D weights to my 3D DenseNet 121, is there a folder in your repository where I can find your pre-trained weights on imagenet??

    Again thank you for having this repository publicly available and sorry if I am completely of topic asking such things here.

    I look forward for you answer, Kind regards, Anastasis

    opened by alexopoulosanastasis 4
  • What are the limitations on Inceptionv3 input shape?

    What are the limitations on Inceptionv3 input shape?

    I seem to always get this error when I try to create InceptionV3 model no matter what input_shape. What are the limitations on input shape there?

    InvalidArgumentError: Negative dimension size caused by subtracting 3 from 2 for '{{node conv3d_314/Conv3D}} = 
    Conv3D[T=DT_FLOAT, data_format="NDHWC", dilations=[1, 1, 1, 1, 1], padding="VALID", strides=[1, 2, 2, 2, 1]](Placeholder, 
    conv3d_314/Conv3D/ReadVariableOp)' with input shapes: [?,2,17,17,192], [3,3,3,192,320].
    
    opened by mazatov 0
Releases(v1.0.4)
BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

Holy Wu 35 Jan 01, 2023
Interactive web apps created using geemap and streamlit

geemap-apps Introduction This repo demostrates how to build a multi-page Earth Engine App using streamlit and geemap. You can deploy the app on variou

Qiusheng Wu 27 Dec 23, 2022
Neural networks applied in recognizing guitar chords using python, AutoML.NET with C# and .NET Core

Chord Recognition Demo application The demo application is written in C# with .NETCore. As of July 9, 2020, the only version available is for windows

Andres Mauricio Rondon Patiño 24 Oct 22, 2022
Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding by Qiaole Dong*, Chenjie Cao*, Yanwei Fu Paper and Supple

Qiaole Dong 190 Dec 27, 2022
NovelD: A Simple yet Effective Exploration Criterion

NovelD: A Simple yet Effective Exploration Criterion Intro This is an implementation of the method proposed in NovelD: A Simple yet Effective Explorat

29 Dec 05, 2022
Real-time object detection on Android using the YOLO network with TensorFlow

TensorFlow YOLO object detection on Android Source project android-yolo is the first implementation of YOLO for TensorFlow on an Android device. It is

Nataniel Ruiz 624 Jan 03, 2023
BT-Unet: A-Self-supervised-learning-framework-for-biomedical-image-segmentation-using-Barlow-Twins

BT-Unet: A-Self-supervised-learning-framework-for-biomedical-image-segmentation-using-Barlow-Twins Deep learning has brought most profound contributio

Narinder Singh Punn 12 Dec 04, 2022
Python implementation of Lightning-rod Agent, the Stack4Things board-side probe

Iotronic Lightning-rod Agent Python implementation of Lightning-rod Agent, the Stack4Things board-side probe. Free software: Apache 2.0 license Websit

2 May 19, 2022
Deep learning models for change detection of remote sensing images

Change Detection Models (Remote Sensing) Python library with Neural Networks for Change Detection based on PyTorch. ⚡ ⚡ ⚡ I am trying to build this pr

Kaiyu Li 176 Dec 24, 2022
Official pytorch implementation of "DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion"

DSPoint Official implementation of "DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion". Paper link: https://arxiv.org/abs/2111.10

Ziyao Zeng 14 Feb 26, 2022
Differential Privacy for Heterogeneous Federated Learning : Utility & Privacy tradeoffs

Differential Privacy for Heterogeneous Federated Learning : Utility & Privacy tradeoffs In this work, we propose an algorithm DP-SCAFFOLD(-warm), whic

19 Nov 10, 2022
Action Segmentation Evaluation

Reference Action Segmentation Evaluation Code This repository contains the reference code for action segmentation evaluation. If you have a bug-fix/im

5 May 22, 2022
Official implementation for paper: A Latent Transformer for Disentangled Face Editing in Images and Videos.

A Latent Transformer for Disentangled Face Editing in Images and Videos Official implementation for paper: A Latent Transformer for Disentangled Face

InterDigital 108 Dec 09, 2022
[CVPR 2022] CoTTA Code for our CVPR 2022 paper Continual Test-Time Domain Adaptation

CoTTA Code for our CVPR 2022 paper Continual Test-Time Domain Adaptation Prerequisite Please create and activate the following conda envrionment. To r

Qin Wang 87 Jan 08, 2023
Nested cross-validation is necessary to avoid biased model performance in embedded feature selection in high-dimensional data with tiny sample sizes

Pruner for nested cross-validation - Sphinx-Doc Nested cross-validation is necessary to avoid biased model performance in embedded feature selection i

1 Dec 15, 2021
Implementation of CVPR'21: RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction

RfD-Net [Project Page] [Paper] [Video] RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction Yinyu Nie, Ji Hou, Xiaoguang Han, Matthi

Yinyu Nie 162 Jan 06, 2023
An AutoML Library made with Optuna and PyTorch Lightning

An AutoML Library made with Optuna and PyTorch Lightning Installation Recommended pip install -U gradsflow From source pip install git+https://github.

GradsFlow 294 Dec 17, 2022
Code implementation for the paper 'Conditional Gaussian PAC-Bayes'.

CondGauss This repository contains PyTorch code for the paper Stochastic Gaussian PAC-Bayes. A novel PAC-Bayesian training method is implemented. Ther

0 Nov 01, 2021
[ICCV'2021] "SSH: A Self-Supervised Framework for Image Harmonization", Yifan Jiang, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin, Kalyan Sunkavalli, Simon Chen, Sohrab Amirghodsi, Sarah Kong, Zhangyang Wang

SSH: A Self-Supervised Framework for Image Harmonization (ICCV 2021) code for SSH Representative Examples Main Pipeline RealHM DataSet Google Drive Pr

VITA 86 Dec 02, 2022
Consensus score for tripadvisor

ContripScore ContripScore is essentially a score that combines an Internet platform rating and a consensus rating from sentiment analysis (For instanc

Pepe 1 Jan 13, 2022