Deep Probabilistic Programming Course @ DIKU

Overview

Syllabus

Part I - Introduction to Deep Probabilistic Programming

Week Topic Exercise Links
1 Introduction to Bayesian Inference Read Pattern Recognition and Machine Learning (PRML), Sections 1.1-1.3, 1.5-1.6 & 2-2.3.4 (inclusive ranges), Intro to Bayesian updating paper, and Pyro paper.

Form up groups and ask a question for each chapter/paper you have read.
Pattern Recognition and Machine Learning

Bayesian Updating Paper

Pyro Paper
2 Variational Inference Read the Variational Inference paper and Pyro tutorials on Stochastic Variational Inference (SVI). Ask three questions about them.

Use Pyro’s Variational Inference support to implement the kidney cancer model. See worksheet and Bayesian Data Analysis 3rd Edition (BDA3) Section 2.7.
Variational Inference Paper

Worksheet

Bayesian Data Analysis

Pyro SVI tutorial: Part I and Part II

Pyro Website
3 Hamiltonian Monte Carlo Read paper on Hamiltonian Monte Carlo and blog post on gradient-based Markov Chain Monte Carlo (MCMC). Look at the source code for Mini-MC.

Ask a question each for HMC, the Mini-MC implementation and discrete variable marginalization.

Implement Bayesian Change-point model in Pyro using NUTS.
Hamiltonian Monte Carlo Paper

Gradient-based MCMC

Mini-MC implementation

Change-point model

Pyro NUTS Example
4 Hidden Markov Models and Discrete Variables. Read Paper on Hidden Markov Models and ask three questions about it.

Read Pyro tutorials on Discrete Variables and Gaussian Mixture Models.

Read Pyro Hidden Markov Model code example and describe in your own words what the different models do.

Add amino acid prediction output to the TorusDBN HMM and show that the posterior predictive distribution of the amino acids matches the one found in data.
Hidden Markov Models

Pyro Discrete Variables Tutorial

Pyro Gaussian Mixture Model Tutorial

Pyro Hidden Markov Model Example

TorusDBN

Optional: Epidemological Inference via HMC
5 Bayesian Regression Models Read PRML Chapter 3 on Linear Models.

Ask 3 questions about the chapter.

Read the Pyro tutorials on Bayesian Regression.

Solve the weather check exercise in the worksheet.
Pyro Bayesian Regression: Part I, Part II

Worksheet
6 Variational Auto-Encoders Read Variational Auto Encoders (VAE) foundations Chapters 1 & 2, and Pyro tutorial on VAE. Ask three questions about the paper and tutorial.

Implement Frey Faces model from VAE paper in Pyro. Rely on the existing VAE implementation (see tutorial link).
Variational Auto Encoders Foundations

Pyro Tutorial on VAE
7 Deep Generative Models Read one of these papers: Interpretable Representation VAE, Causal Effect VAE, Deep Markov Model or DRAW (one paper per group).

Try out the relevant Pyro or PyTorch implementation on your choice of relevant dataset which was not used in the paper.

Make a small (10 minute) presentation about the paper and your experiences with the implementation.
Deep Markov Model

Interpretable Representation VAE

Causal Effect VAE

DRAW

Part II - Deep Probabilistic Programming Project

The second part of the course concerns applying the techniques learned in the first part, as a project solving a practical problem. We have several types of projects depending on the interests of the student.

For those interested in boosting their CV and potentially getting a student job, we warmly recommend working with one of our industrial partners on a suitable probabilistic programming project. For those interested in working with a more academic-oriented project, we have different interesting problems in Computer Science and Biology.

Industrial Projects

Company Area Ideas
 Relion Shift-planning AI Shift planning based on worker availability, historical sales data, weather and holidays.

Employee satisfaction quantification based on previously assigned shifts.

Employee shift assignment based on wishes and need
Paperflow Invoice Recognition AI Talk to us
Hypefactors Media and Reputation Tracking AI Talk to us
‹Your Company› ‹Your Area› Interested in collaboration with our group? contact Ahmad Salim to hear more!

Academic Projects

Type Description Notes/Links
Computer Science Implement a predictive scoring model for your favourite sports game, inspired by FiveThirtyEight. FiveThirtyEight Methodology and Models
Computer Science  Implement a ranking system for your favourite video or board game, inspired by Microsoft TrueSkill. Microsoft TrueSkill Model

Can be implemented in Infer.NET using Expectation Propagation
Computer Science Use Inference Compilation in PyProb to implement a CAPTCHA breaker or a Spaceship Generator Inference Compilation and PyProb. You can use the experimental PyProb bindings for Java.

CAPTCHA breaking with Oxford CAPTCHA Generator.

Spaceship Generator
Computer Science Implement asterisk corrector suggested by XKCD XKCD #2337: Asterisk Correction
Computer Science Implement an inference compilation based program-testing tool like QuickCheck or SmallCheck Inference Compilation

QuickCheck

SmallCheck
Computer Science Magic: The Gathering, Automated Deck Construction. Design a model that finds a good deck automatically based on correlations in existing deck design. Ideas like card substitution models could be integrated too. Magic: The Gathering - Meta Site
Computer Science Use probabilistic programming to explore ideas for solving Eternity II (No $2 million prize anymore, but still interesting from a math point of view) Eternity II
Biology Auto-Encoders or Deep Markov Models for Protein Folding Deep Markov Model

Pyro Deep Markov Model
Biology Inference Compilation for Ancestral Reconstruction Inference Compilation and PyProb. Talk to us for details.
Biology Recurrent Causal Effect VAE for modelling mutations in proteins Talk to us for details.

Recommendations

  • Sometimes sampling can be slow on the CPU for complex models, so try using Google Colab and GPUs and see if that provides an improvement.

Acknowledgements

This course has been developed by Thomas Hamelryck and Ahmad Salim Al-Sibahi. Thanks to Ola Rønning for suggesting the Variational Auto Encoders Foundations paper instead of Auto-Encoding Variational Bayes which we originally proposed to read on week 3. Thanks to Richard Michael for testing out the course and provide us with valuable feedback on the content!

A style-based Quantum Generative Adversarial Network

Style-qGAN A style based Quantum Generative Adversarial Network (style-qGAN) model for Monte Carlo event generation. Tutorial We have prepared a noteb

9 Nov 24, 2022
UpChecker is a simple opensource project to host it fast on your server and check is server up, view statistic, get messages if it is down. UpChecker - just run file and use project easy

UpChecker UpChecker is a simple opensource project to host it fast on your server and check is server up, view statistic, get messages if it is down.

Yan 4 Apr 07, 2022
Tutorial on active learning with the Nvidia Transfer Learning Toolkit (TLT).

Active Learning with the Nvidia TLT Tutorial on active learning with the Nvidia Transfer Learning Toolkit (TLT). In this tutorial, we will show you ho

Lightly 25 Dec 03, 2022
Data Consistency for Magnetic Resonance Imaging

Data Consistency for Magnetic Resonance Imaging Data Consistency (DC) is crucial for generalization in multi-modal MRI data and robustness in detectin

Dimitris Karkalousos 19 Dec 12, 2022
CAPRI: Context-Aware Interpretable Point-of-Interest Recommendation Framework

CAPRI: Context-Aware Interpretable Point-of-Interest Recommendation Framework This repository contains a framework for Recommender Systems (RecSys), a

RecSys Lab 8 Jul 03, 2022
Implementations of paper Controlling Directions Orthogonal to a Classifier

Classifier Orthogonalization Implementations of paper Controlling Directions Orthogonal to a Classifier , ICLR 2022, Yilun Xu, Hao He, Tianxiao Shen,

Yilun Xu 33 Dec 01, 2022
OCR-D wrapper for detectron2 based segmentation models

ocrd_detectron2 OCR-D wrapper for detectron2 based segmentation models Introduction Installation Usage OCR-D processor interface ocrd-detectron2-segm

Robert Sachunsky 13 Dec 06, 2022
Datasets, Transforms and Models specific to Computer Vision

torchvision The torchvision package consists of popular datasets, model architectures, and common image transformations for computer vision. Installat

13.1k Jan 02, 2023
Multi-objective gym environments for reinforcement learning.

MO-Gym: Multi-Objective Reinforcement Learning Environments Gym environments for multi-objective reinforcement learning (MORL). The environments follo

Lucas Alegre 74 Jan 03, 2023
Incomplete easy-to-use math solver and PDF generator.

Math Expert Let me do your work Preview preview.mp4 Introduction Math Expert is our (@salastro, @younis-tarek, @marawn-mogeb) math high school graduat

SalahDin Ahmed 22 Jul 11, 2022
wgan, wgan2(improved, gp), infogan, and dcgan implementation in lasagne, keras, pytorch

Generative Adversarial Notebooks Collection of my Generative Adversarial Network implementations Most codes are for python3, most notebooks works on C

tjwei 1.5k Dec 16, 2022
Educational API for 3D Vision using pose to control carton.

Educational API for 3D Vision using pose to control carton.

41 Jul 10, 2022
Code for GNMR in ICDE 2021

GNMR Code for GNMR in ICDE 2021 Please unzip data files in Datasets/MultiInt-ML10M first. Run labcode_preSamp.py (with graph sampling) for ECommerce-c

7 Oct 27, 2022
Dense matching library based on PyTorch

Dense Matching A general dense matching library based on PyTorch. For any questions, issues or recommendations, please contact Prune at

Prune Truong 399 Dec 28, 2022
Adversarial Reweighting for Partial Domain Adaptation

Adversarial Reweighting for Partial Domain Adaptation Code for paper "Xiang Gu, Xi Yu, Yan Yang, Jian Sun, Zongben Xu, Adversarial Reweighting for Par

12 Dec 01, 2022
Applying PVT to Semantic Segmentation

Applying PVT to Semantic Segmentation Here, we take MMSegmentation v0.13.0 as an example, applying PVTv2 to SemanticFPN. For details see Pyramid Visio

35 Nov 30, 2022
code for Fast Point Cloud Registration with Optimal Transport

robot This is the repository for the paper "Accurate Point Cloud Registration with Robust Optimal Transport". We are in the process of refactoring the

28 Jan 04, 2023
An example to implement a new backbone with OpenMMLab framework.

Backbone example on OpenMMLab framework English | 简体中文 Introduction This is an template repo about how to use OpenMMLab framework to develop a new bac

Ma Zerun 22 Dec 29, 2022
A Python Package for Convex Regression and Frontier Estimation

pyStoNED pyStoNED is a Python package that provides functions for estimating multivariate convex regression, convex quantile regression, convex expect

Sheng Dai 17 Jan 08, 2023