scAR (single-cell Ambient Remover) is a package for data denoising in single-cell omics.

Overview

scAR

scAR single-cell omics machine learning variational autoencoders denoising

scAR (single cell Ambient Remover) is a package for denoising multiple single cell omics data. It can be used for multiple tasks, such as, sgRNA assignment for scCRISPRseq, identity barcode assignment for cell indexing, protein denoising for CITE-seq, mRNA denoising for scRNAseq, and etc... It is built using probabilistic deep learning, illustrated as follows:

Table of Contents

Installation

Clone this repository,

$ git clone https://github.com/Novartis/scAR.git

Enter the cloned directory:

$ cd scAR

To install the dependencies, create a conda environment:

Please use scAR-gpu if you have an nvidia graphis card and the corresponging driver installed.

$ conda env create -f scAR-gpu.yml

or

Please use scAR-cpu if you don't have a graphis card availalble.

$ conda env create -f scAR-cpu.yml

To activate the scAR conda environment run:

$ conda activate scAR

Usage

There are two ways to run scAR.

  1. Use scAR API if you are Python users
>>> from scAR import model
>>> scarObj = model(adata.X.to_df(), empty_profile)
>>> scarObj.train()
>>> scarObj.inference()
>>> adata.layers["X_scAR_denoised"] = scarObj.native_counts
>>> adata.obsm["X_scAR_assignment"] = scarObj.feature_assignment  # feature assignment, e.g., sgRNAs, tags, and etc.. Only available in 'cropseq' mode

See the tutorials

  1. Run scAR from the command line
$ scar raw_count_matrix.pickle -t technology -e empty_profile.pickle -o output

raw_count_matrix.pickle, a pickle-formatted raw count matrix (MxN) with cells in rows and features in columns
empty_profile.pickle, a pickle-formatted feature frequencies (Nx1) in empty droplets
technology, a string, either 'scRNAseq' or 'CROPseq' or 'CITEseq'

Use scar --help command to see other optional arguments and parameters.

The output folder contains four (or five) files:

output
├── denoised_counts.pickle		# denoised count matrix
├── expected_noise_ratio.pickle	# estimated noise ratio
├── BayesFactor.pickle			# bayesian factor of ambient contamination
├── expected_native_freq.pickle	# estimated native frequencies
└── assignment.pickle			# feature assignment, e.g., sgRNAs, tags, and etc.. Gernerated under 'cropseq' mode

Dependencies

PyTorch 1.8 Python 3.8.6 torchvision 0.9.0 tqdm 4.62.3 scikit-learn 1.0.1

Resources

License

This project is licensed under the terms of License.
Copyright 2022 Novartis International AG.

Reference

If you use scAR in your research, please consider citing our manuscript,

@article {Sheng2022.01.14.476312,
	author = {Sheng, Caibin and Lopes, Rui and Li, Gang and Schuierer, Sven and Waldt, Annick and Cuttat, Rachel and Dimitrieva, Slavica and Kauffmann, Audrey and Durand, Eric and Galli, Giorgio G and Roma, Guglielmo and de Weck, Antoine},
	title = {Probabilistic modeling of ambient noise in single-cell omics data},
	elocation-id = {2022.01.14.476312},
	year = {2022},
	doi = {10.1101/2022.01.14.476312},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2022/01/14/2022.01.14.476312},
	eprint = {https://www.biorxiv.org/content/early/2022/01/14/2022.01.14.476312.full.pdf},
	journal = {bioRxiv}
}
Comments
  • Stochastic rounding to integers for downstream use in TotalVI/SCVI

    Stochastic rounding to integers for downstream use in TotalVI/SCVI

    Hi Caibin,

    I tried using scar's output as input for TotalVI/SCVI. As expected, those gave an error because the input is not integer anymore. I would suggest implementing stochastic rounding to integers as done in SoupX.

    Let me know if you're interested and I can find the time to implement it.

    Regards, Mikhael

    enhancement 
    opened by mdmanurung 9
  • BiocondaBot not triggered

    BiocondaBot not triggered

    Hi @fgypas , I made a new release v0.4.1 but bioconda somehow is not triggered upon the new release.

    In the new release, some codes related to building process have been refactored.

    • All information in setup.py (deleted) is integrated into setup.cfg.
    • An extra pyproject.toml file is added.

    I am wondering whether these affect the bioconda-recipes.

    Many thanks, Caibin

    opened by CaibinSh 7
  • New release

    New release

    Hi @fgypas ,

    I am making a new release. There are mainly three changes: 1) addition of a readthedocs; 2) code reformatting via black and pylint (pylint now can score >7, so I have increase the standard in the Action test from 0.5 to 6); 3) renaming 'scAR' to 'scar'.

    I have a couple of questions regarding whether these changes influence the bioconda recipe.

    • Will renaming package name (scAR) require modification in bioconda PR? All uppercase ('scAR') is changed to lowercase ('scar') in everywhere possible (inc. folder, environment, and etc.) But the repo name may stay as 'scAR' for a while, as renaming repo name requires permission from Nick.

    • Should we exclude the folder of datasets in the conda recipe? In addition, a folder, named 'datasets' contains >100 MBs data is added for the tutorial. Should we exclude it?

    question 
    opened by CaibinSh 3
  • Implementation in scvi-tools

    Implementation in scvi-tools

    Hi scAR team,

    I'm reaching out to gauge interest in having a mirror implementation in scvi-tools for scAR. Given the existing infrastructure in the scvi-tools repository, I was able to create a port of scAR quite easily as an external module. Of course, the implementation will link to this repository as the original and cites the paper in the docs. On top of that, the port would allow users of scvi-tools to use the pretrained scAR encoder for doublet detection using the solo model.

    Here's the pending pull request so you can check out what it would look like in the final implementation: https://github.com/scverse/scvi-tools/pull/1683

    Please let me know what you think!

    opened by ricomnl 2
  • Positive-valued denoising results for ADTs with raw 0 counts

    Positive-valued denoising results for ADTs with raw 0 counts

    Hi scar team!

    Thank you for developing this interesting package. I had a question about the resulting denoised values for CITE-seq experiments.

    I've noticed that some cells that originally have a 0 value for an ADT (as a raw count) will have a positive value (>0) for that ADT after the denoising procedure. Below, I show this case for the CD25 ADT in the 10xPBMC5k CITE-seq dataset (from the tutorial at https://scar-tutorials.readthedocs.io/en/latest/tutorials/scAR_tutorial_denoising_CITEseq.html).

    I'm a bit confused about how to best interpret these values and how they are occurring. Should these be set to 0 after the denoising procedure?

    Screen Shot 2022-05-25 at 1 16 37 AM question 
    opened by diegoalexespi 2
  • Sparsity values for mRNA decontamination?

    Sparsity values for mRNA decontamination?

    Hello,

    I was wondering what the recommendations for the sparsity value would be in denoising mRNA? Specifically if we don't know too much of the data besides UMI/nGenes in the cells etc.? I noticed its generally set at 1 for sgRNA decontamination, but what would the general recommended value be for mRNA?

    Thanks, Chang

    question 
    opened by cnk113 1
  • Number of training epochs + batch size

    Number of training epochs + batch size

    Dear scAR-Team,

    thank you for developing this package. I am currently exploring it and I would like to ask you

    1. how do you determine the number of epochs the user should use for feature_type = "mRNA"? In your tutorials you used 400 epochs and in your paper you mentioned that you fixed the epochs to 800. I applied it for various batch sizes (up to 1000) and noticed that the model is sensitive to it.

    2. I noticed that you use rather small batch-size - is scAR sensitive to the batch-size, it is just due to computational limitations or due to better perfromance?

    Thank you in advance!

    Best,

    question 
    opened by KalinNonchev 1
  • bump to version 0.3.2

    bump to version 0.3.2

    fix(*): changelog docs: adding docstring in documentation docs: adding Release notes in documentation docs: adding docstring in documentation test: adding semantic release refactor: further refactoring codes fix semantic release

    opened by CaibinSh 1
  • ask for permission of Webhooks

    ask for permission of Webhooks

    Hi @kliatsko ,

    We are currently refactoring and adding functionalities to scAR.

    Could you please grant the Webhooks permission for us to automate the documentation?

    Many thanks in advance. Best regards, Caibin on behalf of the scar team @fgypas @Tobias-Ternent @mr-nvs @AlexMTYZ.

    help wanted 
    opened by CaibinSh 1
  • New release

    New release

    • Additions of readthedocs
    • Code refactoring
    1. Renaming module names, e.g. changing "scAR" -> "scar"
    2. Renaming parameter names, e.g.

    changing "scRNAseq_tech" -> "feature_type" changing "model" -> "count_model" changing "scRNAseq_tech" -> "feature_type"

    • Black and Pylint re-formatting the code
    enhancement 
    opened by CaibinSh 1
  • Black github action

    Black github action

    Addition of black github action that runs on every push and every pull request. It shows in the stdout all the changes that need to be made (--diff), but returns exit code 0, even if errors are observed.

    opened by fgypas 1
Releases(v0.4.4)
  • v0.4.4(Aug 9, 2022)

    Documentation

    • Update dependency (03cf19e)
    • Update dependencies (9bd7f1c)
    • Update documentations (418996c)
    • Update dependencies (1bde351)
    • main: Add link to anndata and scanpy (8436e05)
    • main: Update dependencies (984df35)
    • main: Update documentation for .h5 file (2a309e0)
    • Add a link of binary installers (2faed3e)
    • Update documentations (e26a6e9)
    • Add competing methods (8564b2b)
    • scar: Add versionadded directives for parameter sparsity and round_to_int (33e35ca)
    • Update docs (a4da539)
    • Update introduction (a036b24)
    • Change readthedocs template (421e52f)
    • data_generator: Update docs (1f8f668)
    • data_generator: Re-style docs (afef9fb)
    • *: Re-style docs (2d550fa)

    Performance

    • main: Command line tool supports a new input: filtered_feature_bc_matrix.h5 (73bc13e)
    • setup: Add an error raise statement (f4fb1a8)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.3(Jun 15, 2022)

    Fix

    • setup: Fix a bug to allow sample reasonable numbers of droplets (ef6f7e4)
    • main: Fix a bug in main to set default NN number (794ff17)

    Documentation

    • main: Add scanpy as dependency (252a492)

    Performance

    • main: Set a separate batchsize_infer parameter for inference (8727f04)
    • setup: Add an option of random sampling droplets to speed up calculation (ce042dd)
    • setup: Enable manupulate large-scale emptydroplets (15f1840)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.2(Jun 7, 2022)

  • v0.4.1(May 19, 2022)

    What's Changed

    Feature

    • inference: add a round_to_int parameter to round the counts (float) for easy interpretation and better integration into other methods (#47) (902a2b9) (8694239)

    Build

    • setup: replace setup.py with setup.cfg and pyproject.toml (#51) (3dc999a)

    Chore

    Documentation

    • readthedocs: add scAR_logo image (#51) (c34f362)
    • tutorials: add ci=None to speed up plotting (#51) (902a2b9)

    Contributor

    @CaibinSh and @mdmanurung

    Full Changelog: https://github.com/Novartis/scar/compare/v0.4.0...v0.4.1

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(May 5, 2022)

  • v0.3.5(May 3, 2022)

  • v0.3.4(May 1, 2022)

  • v0.3.3(May 1, 2022)

  • v0.3.1(Apr 29, 2022)

  • v0.3.0(Apr 27, 2022)

    What's Changed

    Renaming module names, e.g. changing "scAR" -> "scar" Renaming parameter names, e.g.

    "scRNAseq_tech" -> "feature_type" "model" -> "count_model" "empty_profile" -> "ambient_profile" ...

    • Black and Pylint re-formatting the code
    • New release by @CaibinSh in https://github.com/Novartis/scAR/pull/26

    Contributor

    @CaibinSh @fgypas @mr-nvs @Tobias-Ternent

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.3...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.3(Apr 20, 2022)

    • Add integration test
    • Black formating
    • Bump version to 0.2.3

    Contributors: @fgypas , @mr-nvs and @CaibinSh

    What's Changed

    • Develop by @CaibinSh in https://github.com/Novartis/scAR/pull/19

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.2...v0.2.3

    Source code(tar.gz)
    Source code(zip)
  • v0.2.2(Apr 4, 2022)

    v0.2.2

    • Remove torchaudio
    • Add test data for integration tests
    • Bump version to 0.2.2

    Contributors: @CaibinSh @fgypas

    What's Changed

    • Remove torchaudio, add test data and bump version to 0.2.2 by @fgypas in https://github.com/Novartis/scAR/pull/15

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.1-beta...v0.2.2

    Source code(tar.gz)
    Source code(zip)
  • v0.2.1-beta(Apr 1, 2022)

    • fix a typo in scAR-gpu.yml
    • reorganise init.py files

    Contributor: @CaibinSh

    What's Changed

    • Develop by @CaibinSh in https://github.com/Novartis/scAR/pull/12

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.0-beta...v0.2.1-beta

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0-beta(Apr 1, 2022)

    • Support for training of the model with CPUs
    • Addition of two yaml files for CPU/GPU installation
    • Refactor of setup.py and structure of the package
    • Addition of tests with pytest
    • Addition of lint checks
    • Automate build with github actions (install package and run lint checks and pytest)
    • Update documentation
    • Version 0.2.0

    Co-authored-by: @CaibinSh @mr-nvs @Tobias-Ternent @fgypas

    What's Changed

    • 0.2.0-release by @fgypas in https://github.com/Novartis/scAR/pull/11

    Full Changelog: https://github.com/Novartis/scAR/commits/v0.2.0-beta

    Source code(tar.gz)
    Source code(zip)
Keyword-BERT: Keyword-Attentive Deep Semantic Matching

project discription An implementation of the Keyword-BERT model mentioned in my paper Keyword-Attentive Deep Semantic Matching (Plz cite this github r

1 Nov 14, 2021
Torch implementation of various types of GAN (e.g. DCGAN, ALI, Context-encoder, DiscoGAN, CycleGAN, EBGAN, LSGAN)

gans-collection.torch Torch implementation of various types of GANs (e.g. DCGAN, ALI, Context-encoder, DiscoGAN, CycleGAN, EBGAN). Note that EBGAN and

Minchul Shin 53 Jan 22, 2022
The `rtdl` library + The official implementation of the paper

The `rtdl` library + The official implementation of the paper "Revisiting Deep Learning Models for Tabular Data"

Yandex Research 510 Dec 30, 2022
Towards uncontrained hand-object reconstruction from RGB videos

Towards uncontrained hand-object reconstruction from RGB videos Yana Hasson, Gül Varol, Ivan Laptev and Cordelia Schmid Project page Paper Table of Co

Yana 69 Dec 27, 2022
Data Preparation, Processing, and Visualization for MoVi Data

MoVi-Toolbox Data Preparation, Processing, and Visualization for MoVi Data, https://www.biomotionlab.ca/movi/ MoVi is a large multipurpose dataset of

Saeed Ghorbani 51 Nov 27, 2022
Multimodal Temporal Context Network (MTCN)

Multimodal Temporal Context Network (MTCN) This repository implements the model proposed in the paper: Evangelos Kazakos, Jaesung Huh, Arsha Nagrani,

Evangelos Kazakos 13 Nov 24, 2022
Codes for CVPR2021 paper "PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization"

PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization (CVPR 2021) This is the official implementation of PW

Intelligent Robotics and Machine Vision Lab 42 Dec 18, 2022
A playable implementation of Fully Convolutional Networks with Keras.

keras-fcn A re-implementation of Fully Convolutional Networks with Keras Installation Dependencies keras tensorflow Install with pip $ pip install git

JihongJu 202 Sep 07, 2022
Lightweight Python library for adding real-time object tracking to any detector.

Norfair is a customizable lightweight Python library for real-time 2D object tracking. Using Norfair, you can add tracking capabilities to any detecto

Tryolabs 1.7k Jan 05, 2023
A Simple Long-Tailed Rocognition Baseline via Vision-Language Model

BALLAD This is the official code repository for A Simple Long-Tailed Rocognition Baseline via Vision-Language Model. Requirements Python3 Pytorch(1.7.

Teli Ma 4 Jan 20, 2022
Repository for RNNs using TensorFlow and Keras - LSTM and GRU Implementation from Scratch - Simple Classification and Regression Problem using RNNs

RNN 01- RNN_Classification Simple RNN training for classification task of 3 signal: Sine, Square, Triangle. 02- RNN_Regression Simple RNN training for

Nahid Ebrahimian 13 Dec 13, 2022
simple demo codes for Learning to Teach with Dynamic Loss Functions

Learning to Teach with Dynamic Loss Functions This repo contains the simple demo for the NeurIPS-18 paper: Learning to Teach with Dynamic Loss Functio

Lijun Wu 15 Dec 30, 2021
Implementation of "Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency"

Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency (ICCV2021) Paper Link: https://arxiv.org/abs/2107.11355 This implementation bui

32 Nov 17, 2022
[NeurIPS 2021] A weak-shot object detection approach by transferring semantic similarity and mask prior.

[NeurIPS 2021] A weak-shot object detection approach by transferring semantic similarity and mask prior.

BCMI 49 Jul 27, 2022
Churn-Prediction-Project - In this project, a churn prediction model is developed for a private bank as a term project for Data Mining class.

Churn-Prediction-Project In this project, a churn prediction model is developed for a private bank as a term project for Data Mining class. Project in

1 Jan 03, 2022
Building Ellee — A GPT-3 and Computer Vision Powered Talking Robotic Teddy Bear With Human Level Conversation Intelligence

Using an object detection and facial recognition system built on MobileNetSSDV2 and Dlib and running on an NVIDIA Jetson Nano, a GPT-3 model, Google Speech Recognition, Amazon Polly and servo motors,

24 Oct 26, 2022
Repository for "Toward Practical Monocular Indoor Depth Estimation" (CVPR 2022)

Toward Practical Monocular Indoor Depth Estimation Cho-Ying Wu, Jialiang Wang, Michael Hall, Ulrich Neumann, Shuochen Su [arXiv] [project site] DistDe

Meta Research 122 Dec 13, 2022
Clustergram - Visualization and diagnostics for cluster analysis in Python

Clustergram Visualization and diagnostics for cluster analysis Clustergram is a diagram proposed by Matthias Schonlau in his paper The clustergram: A

Martin Fleischmann 96 Dec 26, 2022
UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning

UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning This is the official PyTorch implementation for UniMoCo pape

dddzg 49 Jan 02, 2023
MT3: Multi-Task Multitrack Music Transcription

MT3: Multi-Task Multitrack Music Transcription MT3 is a multi-instrument automatic music transcription model that uses the T5X framework. This is not

Magenta 867 Dec 29, 2022