competitions-v2

Last update: Dec 02, 2022

Related tags

Overview

Codabench

(formerly Codalab Competitions v2)

Installation

$ cp .env_sample .env
$ docker-compose up -d
$ docker-compose exec django ./manage.py migrate
$ docker-compose exec django ./manage.py generate_data
$ docker-compose exec django ./manage.py collectstatic --noinput

You can now login as username "admin" with password "admin" at http://localhost:8000

If you ever need to reset the database, use the script ./reset_db.sh

Running tests

# Non "end to end tests"
$ docker-compose exec django py.test -m "not e2e"

# "End to end tests" (a shell script to launch a selenium docker container)
$ ./run_selenium_tests.sh

# If you are on Mac OSX it is easy to watch these tests, no need to install
# anything just do:
$ open vnc://0.0.0.0:5900

# And login with password "secret"

Example competitions

The repo comes with a couple examples that are used during tests:

v2 test data

src/tests/functional/test_files/submission.zip
src/tests/functional/test_files/competition.zip

v1.5 legacy test data

src/tests/functional/test_files/submission15.zip
src/tests/functional/test_files/competition15.zip

Other Codalab Competition examples

https://github.com/codalab/competition-examples/tree/master/v2/

Building compute worker

To build the normal image:

docker build -t codalab/competitions-v2-compute-worker:latest -f Dockerfile.compute_worker .

To build the GPU version:

docker build -t codalab/competitions-v2-compute-worker:nvidia -f Dockerfile.compute_worker_gpu .

Updating the image

docker push codalab/competitions-v2-compute-worker

Worker setup

# install docker
$ curl https://get.docker.com | sudo sh
$ sudo usermod -aG docker $USER

# >>> reconnect <<<

Start CPU worker

Make a file .env and put this in it:

# Queue URL
BROKER_URL=
   
    

# Location to store submissions/cache -- absolute path!
HOST_DIRECTORY=/your/path/to/codabench/storage

# If SSL is enabled, then uncomment the following line
#BROKER_USE_SSL=True

NOTE /your/path/to/codabench -- this path needs to be volumed into /codabench on the worker, as you can see below.

$ docker run \
    -v /your/path/to/codabench/storage:/codabench \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -d \
    --env-file .env \
    --restart unless-stopped \
    --log-opt max-size=50m \
    --log-opt max-file=3 \
    codalab/competitions-v2-compute-worker:latest

Start GPU worker

nvidia installation instructions

$ nvidia-docker run \
    -v /your/path/to/codabench/storage:/codabench \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /var/lib/nvidia-docker/nvidia-docker.sock:/var/lib/nvidia-docker/nvidia-docker.sock \
    -d \
    --env-file .env \
    --restart unless-stopped \
    --log-opt max-size=50m \
    --log-opt max-file=3 \
    codalab/competitions-v2-compute-worker:nvidia

Worker management

Outside of docker containers install Fabric like so:

pip install fab-classic==1.17.0

Create a server_config.yaml in the root of this repository using:

cp server_config_sample.yaml server_config.yaml

Below is an example server_config.yaml that defines 2 roles comp-gpu and comp-cpu, one with gpu style workers (is_gpu and the nvidia docker_image) and one with cpu style workers

comp-gpu:
  hosts:
    - [email protected]
    - [email protected]
  broker_url: pyamqp://user:[email protected]:port/vhost-gpu
  is_gpu: true
  docker_image: codalab/competitions-v2-compute-worker:nvidia

comp-cpu:
  hosts:
    - [email protected]
  broker_url: pyamqp://user:[email protected]:port/vhost-cpu
  is_gpu: false
  docker_image: codalab/competitions-v2-compute-worker:latest

You can of course create your own docker_image and specify it here.

You can execute commands against a role:

❯ fab -R comp-gpu status
..
[[email protected]] out: CONTAINER ID        IMAGE                                           COMMAND                  CREATED             STATUS              PORTS               NAMES
[[email protected]] out: 1d318268bee1        codalab/competitions-v2-compute-worker:nvidia   "/bin/sh -c 'celery …"   2 hours ago         Up 2 hours                              hardcore_greider
..

❯ fab -R comp-gpu update
..
(updates workers)

See available commands with fab -l

competitions-v2

Related tags

Overview

Codabench

(formerly Codalab Competitions v2)

Installation

Running tests

Example competitions

v2 test data

v1.5 legacy test data

Other Codalab Competition examples

Building compute worker

Worker setup

Start CPU worker

Start GPU worker

Worker management

Owner

CodaLab

Pytorch implementation for the Temporal and Object Quantification Networks (TOQ-Nets).

[CVPR 2021] "The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models" Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Michael Carbin, Zhangyang Wang

🧑‍🔬 verify your TEAL program by experiment and observation

Open source implementation of AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing

The code for replicating the experiments from the LFI in SSMs with Unknown Dynamics paper.

Our implementation used for the MICCAI 2021 FLARE Challenge titled 'Efficient Multi-Organ Segmentation Using SpatialConfiguartion-Net with Low GPU Memory Requirements'.

Dataset Condensation with Contrastive Signals

[SIGGRAPH 2022 Journal Track] AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

Training, generation, and analysis code for Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics

Face Mask Detection on Image and Video using tensorflow and keras

System Design course at HSE (2021)

Drslmarkov - Distributionally Robust Structure Learning for Discrete Pairwise Markov Networks

Asterisk is a framework to generate high-quality training datasets at scale

Contextual Attention Localization for Offline Handwritten Text Recognition

A tool to visualise the results of AlphaFold2 and inspect the quality of structural predictions

Editing a Conditional Radiance Field

PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+

Back to Basics: Efficient Network Compression via IMP

A full pipeline AutoML tool for tabular data

这是一个利用facenet和retinaface实现人脸识别的库，可以进行在线的人脸识别。