The starter repository for submissions to the GeneDisco challenge for optimized experimental design in genetic perturbation experiments

Overview

GeneDisco ICLR-22 Challenge Starter Repository

Python version Library version

The starter repository for submissions to the GeneDisco challenge for optimized experimental design in genetic perturbation experiments.

GeneDisco (to be published at ICLR-22) is a benchmark suite for evaluating active learning algorithms for experimental design in drug discovery. GeneDisco contains a curated set of multiple publicly available experimental data sets as well as open-source implementations of state-of-the-art active learning policies for experimental design and exploration.

Install

pip install -r requirements.txt

Use

Setup

  • Create a cache directory. This will hold any preprocessed and downloaded datasets for faster future invocation.
    • $ mkdir /path/to/genedisco_cache
    • Replace the above with your desired cache directory location.
  • Create an output directory. This will hold all program outputs and results.
    • $ mkdir /path/to/genedisco_output
    • Replace the above with your desired output directory location.

How to Run the Full Benchmark Suite?

Experiments (all baselines, acquisition functions, input and target datasets, multiple seeds) included in GeneDisco can be executed sequentially for e.g. acquired batch size 64, 8 cycles and a bayesian_mlp model using:

run_experiments \
  --cache_directory=/path/to/genedisco_cache  \
  --output_directory=/path/to/genedisco_output  \
  --acquisition_batch_size=64  \
  --num_active_learning_cycles=8  \
  --max_num_jobs=1

Results are written to the folder at /path/to/genedisco_cache, and processed datasets will be cached at /path/to/genedisco_cache (please replace both with your desired paths) for faster startup in future invocations.

Note that due to the number of experiments being run by the above command, we recommend execution on a compute cluster.
The GeneDisco codebase also supports execution on slurm compute clusters (the slurm command must be available on the executing node) using the following command and using dependencies in a Python virtualenv available at /path/to/your/virtualenv (please replace with your own virtualenv path):

run_experiments \
  --cache_directory=/path/to/genedisco_cache  \
  --output_directory=/path/to/genedisco_output  \
  --acquisition_batch_size=64  \
  --num_active_learning_cycles=8  \
  --schedule_on_slurm \
  --schedule_children_on_slurm \
  --remote_execution_virtualenv_path=/path/to/your/virtualenv

Other scheduling systems are currently not supported by default.

How to Run A Single Isolated Experiment (One Learning Cycle)?

To run one active learning loop cycle, for example, with the "topuncertain" acquisition function, the "achilles" feature set and the "schmidt_2021_ifng" task, execute the following command:

active_learning_loop  \
    --cache_directory=/path/to/genedisco/genedisco_cache \
    --output_directory=/path/to/genedisco/genedisco_output \
    --model_name="bayesian_mlp" \
    --acquisition_function_name="topuncertain" \
    --acquisition_batch_size=64 \
    --num_active_learning_cycles=8 \
    --feature_set_name="achilles" \
    --dataset_name="schmidt_2021_ifng" 

How to Evaluate a Custom Acquisition Function?

To run a custom acquisition function, set --acquisition_function_name="custom" and --acquisition_function_path to the file path that contains your custom acquisition function (e.g. main.py in this repo).

active_learning_loop  \
    --cache_directory=/path/to/genedisco/genedisco_cache \
    --output_directory=/path/to/genedisco/genedisco_output \
    --model_name="bayesian_mlp" \
    --acquisition_function_name="custom" \
    --acquisition_function_path=/path/to/src/main.py \
    --acquisition_batch_size=64 \
    --num_active_learning_cycles=8 \
    --feature_set_name="achilles" \
    --dataset_name="schmidt_2021_ifng" 

...where "/path/to/custom_acquisition_function.py" contains code for your custom acquisition function corresponding to the BaseBatchAcquisitionFunction interface, e.g.:

import numpy as np
from typing import AnyStr, List
from slingpy import AbstractDataSource
from slingpy.models.abstract_base_model import AbstractBaseModel
from genedisco.active_learning_methods.acquisition_functions.base_acquisition_function import \
    BaseBatchAcquisitionFunction

class RandomBatchAcquisitionFunction(BaseBatchAcquisitionFunction):
    def __call__(self,
                 dataset_x: AbstractDataSource,
                 batch_size: int,
                 available_indices: List[AnyStr], 
                 last_selected_indices: List[AnyStr] = None, 
                 model: AbstractBaseModel = None,
                 temperature: float = 0.9,
                 ) -> List:
        selected = np.random.choice(available_indices, size=batch_size, replace=False)
        return selected

Note that the last class implementing BaseBatchAcquisitionFunction is loaded by GeneDisco if there are multiple valid acquisition functions present in the loaded file.

Submission instructions

For submission, you will need two things:

Please note that all your submitted code must either be loaded via a dependency in requirements.txt or be present in the src/ directory in this starter repository for the submission to succeed.

Once you have set up your submission environment, you will need to create a lightweight container image that contains your acquisition function.

Submission steps

  • Navigate to the directory to which you have cloned this repo to.
    • $ cd /path/to/genedisco-starter
  • Ensure you have ONE acquisition function (inheriting from BaseBatchAcquisitionFunction) in main.py
    • This is your pre-defined program entry point.
  • Build your container image
    • $ docker build -t submission:latest .
  • Save your image name to a shell variable
    • $ IMAGE="submission:latest"
  • Use the EvalAI-CLI command to submit your image
    • Run the following command to submit your container image:
      • $ evalai push $IMAGE --phase gsk-genedisco-challenge-1528
      • Please note that you have a maximum number of submissions that any submission will be counted against.

That’s it! Our pipeline will take your image and test your function.

If you have any questions or concerns, please reach out to us at [email protected]

Citation

Please consider citing, if you reference or use our methodology, code or results in your work:

@inproceedings{mehrjou2022genedisco,
    title={{GeneDisco: A Benchmark for Experimental Design in Drug Discovery}},
    author={Mehrjou, Arash and Soleymani, Ashkan and Jesson, Andrew and Notin, Pascal and Gal, Yarin and Bauer, Stefan and Schwab, Patrick},
    booktitle={{International Conference on Learning Representations (ICLR)}},
    year={2022}
}

License

License

Authors

Arash Mehrjou, GlaxoSmithKline plc
Jacob A. Sackett-Sanders, GlaxoSmithKline plc
Patrick Schwab, GlaxoSmithKline plc

Acknowledgements

PS, JSS and AM are employees and shareholders of GlaxoSmithKline plc.

Template for creating PyPI project

template-for-creating-pypi-project Template for creating PyPI project Hello there! This is a template for creating a PyPI project. Fork or clone this

4 Apr 25, 2022
Template to quickly start your playwright-python project

Playwright-python template πŸͺ Template to quickly start your playwright-python project Getting started β€’ Demo β€’ Configuration Getting started Clone th

Constantin 1 Dec 13, 2021
A template repository implementing HTML5 Boilerplate 8.0 in Sanic using the Domonic framework.

sanic-domonic-h5bp A template repository implementing HTML5 Boilerplate 8.0 in Sanic using the Domonic framework. If you need frontend interactivity,

PyXY 3 Dec 12, 2022
A simple cookiecutter to create Python Telegram bots, wrapped with Django.

PTB Django cookiecutter A simple cookiecutter to create Python Telegram bots, wrapped with Django. Based on this cool projects python-telegram-bot (PT

Carlos Lugones 20 Nov 12, 2022
Open-source full-stack seed project that uses a React UI powered by a simple Flask API Server

React Flask Authentication Open-source full-stack seed project that uses a React UI powered by a simple Flask API Server.

App Generator 37 Dec 24, 2022
A cookiecutter template for python scripts

cookiecutter-py-script A cookiecutter template for python scripts Prerequisites Git Usage pip install cookiecutter

Zeheng Li 1 Dec 14, 2022
Cookiecutter-allpurpose-minimal-python - A simple cookiecutter template for general-purpose python projects.

cookiecutter-allpurpose-minimal-python A simple cookiecutter template for general-purpose python projects. To use, run pip install cookiecutter cookie

E. Tolga Ayan 2 Jan 24, 2022
A platform for developers πŸ‘©β€πŸ’» who wants to share their programs and projects.

Hacktoberfest-2021 A platform for developers πŸ‘©β€πŸ’» who wants to share their projects and programs. Hacktoberfest has updated their rules and now this

Mayank Choudhary 40 Nov 07, 2022
Simple boilerplate ready for development

StartApp Simple boilerplate ready for development Notes Currently supported frameworks are: FastApi, Flask πŸ”¨ Installation $ sudo pip3 install starta

Sabuhi 16 Oct 16, 2022
Cookiecutter Flask OpenAPI is a template for jumpstarting production-ready Flask projects quickly.

Cookiecutter Flask OpenAPI is a template for jumpstarting production-ready Flask projects quickly. It has a well organized and scalable structure. It uses API design first

Roger Camargo 3 Nov 17, 2022
A boilerplate for Django web applications

Django Hackathon Starter A boilerplate application for Django web applications. If you've attented hackathons, you already know how much time can be w

David Leonard 1.6k Dec 31, 2022
Boilerplate for starting a python project

Python Project Boilerplate Simple boilerplate for starting a python proect. Using the repo Follow following steps to install client on server Create a

Prajwal Dahal 1 Nov 19, 2021
Setup a flask project using a single command, right from creating virtual environment to creating Procfile for deployment.

AutoFlask-Setup About AutoFlask-Setup can help you set up a new Flask Project, right from creating virtual environment to creating Procfile for deploy

Ashutosh Krishna 1 Oct 21, 2021
Um template para quem quiser usar o Docker + PGSQL + Django.

Um template para quem quiser usar o Docker + PGSQL + Django.

Drack 2 Mar 11, 2022
Basic Docker Compose template application with Flask, Celery, Redis, MySQL, SocketIO, Nginx and Gunicorn.

Nginx / Gunicorn / Flask 🐍 / Celery / SocketIO / MySQL / Redis / Docker 🐳 sample application Basic Docker Compose template application for orchestat

Alex Oarga 8 Aug 06, 2022
This is a boilerplate for a basic backend app using Python, Django and SQLite, as developed after tutorials with Programming with Mosh

This is a boilerplate for a basic backend app using Python, Django and SQLite, as developed after tutorials with Programming with Mosh

Gustavo Catala Sverdrup 1 Jan 07, 2022
A project to get you started with Docker and Django.

Docker Django tl;dr $ git clone [email protected]:erroneousboat/docker-django.git $ d

JP Bruins Slot 176 Dec 29, 2022
The starter for the Flask React project

Flask React Project This is the starter for the Flask React project. Getting started Clone this repository (only this branch) git clone https://github

Parker Bolick 2 May 14, 2022
A python starter package to be used as a template for creating your own python packages.

Python Starter Package This is a basic python starter package to be used as a template for creating your own python packages. Github repo: https://git

Mystic 1 Apr 04, 2022
Forkable, Minimal Template for Starknet Projects.

Forkable, Minimal Template for Starknet Projects.

andreas 44 Oct 09, 2022