Deep Probabilistic Programming Course @ DIKU

Overview

Syllabus

Part I - Introduction to Deep Probabilistic Programming

Week Topic Exercise Links
1 Introduction to Bayesian Inference Read Pattern Recognition and Machine Learning (PRML), Sections 1.1-1.3, 1.5-1.6 & 2-2.3.4 (inclusive ranges), Intro to Bayesian updating paper, and Pyro paper.

Form up groups and ask a question for each chapter/paper you have read.
Pattern Recognition and Machine Learning

Bayesian Updating Paper

Pyro Paper
2 Variational Inference Read the Variational Inference paper and Pyro tutorials on Stochastic Variational Inference (SVI). Ask three questions about them.

Use Pyro’s Variational Inference support to implement the kidney cancer model. See worksheet and Bayesian Data Analysis 3rd Edition (BDA3) Section 2.7.
Variational Inference Paper

Worksheet

Bayesian Data Analysis

Pyro SVI tutorial: Part I and Part II

Pyro Website
3 Hamiltonian Monte Carlo Read paper on Hamiltonian Monte Carlo and blog post on gradient-based Markov Chain Monte Carlo (MCMC). Look at the source code for Mini-MC.

Ask a question each for HMC, the Mini-MC implementation and discrete variable marginalization.

Implement Bayesian Change-point model in Pyro using NUTS.
Hamiltonian Monte Carlo Paper

Gradient-based MCMC

Mini-MC implementation

Change-point model

Pyro NUTS Example
4 Hidden Markov Models and Discrete Variables. Read Paper on Hidden Markov Models and ask three questions about it.

Read Pyro tutorials on Discrete Variables and Gaussian Mixture Models.

Read Pyro Hidden Markov Model code example and describe in your own words what the different models do.

Add amino acid prediction output to the TorusDBN HMM and show that the posterior predictive distribution of the amino acids matches the one found in data.
Hidden Markov Models

Pyro Discrete Variables Tutorial

Pyro Gaussian Mixture Model Tutorial

Pyro Hidden Markov Model Example

TorusDBN

Optional: Epidemological Inference via HMC
5 Bayesian Regression Models Read PRML Chapter 3 on Linear Models.

Ask 3 questions about the chapter.

Read the Pyro tutorials on Bayesian Regression.

Solve the weather check exercise in the worksheet.
Pyro Bayesian Regression: Part I, Part II

Worksheet
6 Variational Auto-Encoders Read Variational Auto Encoders (VAE) foundations Chapters 1 & 2, and Pyro tutorial on VAE. Ask three questions about the paper and tutorial.

Implement Frey Faces model from VAE paper in Pyro. Rely on the existing VAE implementation (see tutorial link).
Variational Auto Encoders Foundations

Pyro Tutorial on VAE
7 Deep Generative Models Read one of these papers: Interpretable Representation VAE, Causal Effect VAE, Deep Markov Model or DRAW (one paper per group).

Try out the relevant Pyro or PyTorch implementation on your choice of relevant dataset which was not used in the paper.

Make a small (10 minute) presentation about the paper and your experiences with the implementation.
Deep Markov Model

Interpretable Representation VAE

Causal Effect VAE

DRAW

Part II - Deep Probabilistic Programming Project

The second part of the course concerns applying the techniques learned in the first part, as a project solving a practical problem. We have several types of projects depending on the interests of the student.

For those interested in boosting their CV and potentially getting a student job, we warmly recommend working with one of our industrial partners on a suitable probabilistic programming project. For those interested in working with a more academic-oriented project, we have different interesting problems in Computer Science and Biology.

Industrial Projects

Company Area Ideas
 Relion Shift-planning AI Shift planning based on worker availability, historical sales data, weather and holidays.

Employee satisfaction quantification based on previously assigned shifts.

Employee shift assignment based on wishes and need
Paperflow Invoice Recognition AI Talk to us
Hypefactors Media and Reputation Tracking AI Talk to us
‹Your Company› ‹Your Area› Interested in collaboration with our group? contact Ahmad Salim to hear more!

Academic Projects

Type Description Notes/Links
Computer Science Implement a predictive scoring model for your favourite sports game, inspired by FiveThirtyEight. FiveThirtyEight Methodology and Models
Computer Science  Implement a ranking system for your favourite video or board game, inspired by Microsoft TrueSkill. Microsoft TrueSkill Model

Can be implemented in Infer.NET using Expectation Propagation
Computer Science Use Inference Compilation in PyProb to implement a CAPTCHA breaker or a Spaceship Generator Inference Compilation and PyProb. You can use the experimental PyProb bindings for Java.

CAPTCHA breaking with Oxford CAPTCHA Generator.

Spaceship Generator
Computer Science Implement asterisk corrector suggested by XKCD XKCD #2337: Asterisk Correction
Computer Science Implement an inference compilation based program-testing tool like QuickCheck or SmallCheck Inference Compilation

QuickCheck

SmallCheck
Computer Science Magic: The Gathering, Automated Deck Construction. Design a model that finds a good deck automatically based on correlations in existing deck design. Ideas like card substitution models could be integrated too. Magic: The Gathering - Meta Site
Computer Science Use probabilistic programming to explore ideas for solving Eternity II (No $2 million prize anymore, but still interesting from a math point of view) Eternity II
Biology Auto-Encoders or Deep Markov Models for Protein Folding Deep Markov Model

Pyro Deep Markov Model
Biology Inference Compilation for Ancestral Reconstruction Inference Compilation and PyProb. Talk to us for details.
Biology Recurrent Causal Effect VAE for modelling mutations in proteins Talk to us for details.

Recommendations

  • Sometimes sampling can be slow on the CPU for complex models, so try using Google Colab and GPUs and see if that provides an improvement.

Acknowledgements

This course has been developed by Thomas Hamelryck and Ahmad Salim Al-Sibahi. Thanks to Ola Rønning for suggesting the Variational Auto Encoders Foundations paper instead of Auto-Encoding Variational Bayes which we originally proposed to read on week 3. Thanks to Richard Michael for testing out the course and provide us with valuable feedback on the content!

Time-series-deep-learning - Developing Deep learning LSTM, BiLSTM models, and NeuralProphet for multi-step time-series forecasting of stock price.

Stock Price Prediction Using Deep Learning Univariate Time Series Predicting stock price using historical data of a company using Neural networks for

Abdultawwab Safarji 7 Nov 27, 2022
A distributed deep learning framework that supports flexible parallelization strategies.

FlexFlow FlexFlow is a deep learning framework that accelerates distributed DNN training by automatically searching for efficient parallelization stra

528 Dec 25, 2022
This is a computer vision based implementation of the popular childhood game 'Hand Cricket/Odd or Even' in python

Hand Cricket Table of Content Overview Installation Game rules Project Details Future scope Overview This is a computer vision based implementation of

Abhinav R Nayak 6 Jan 12, 2022
Barbershop: GAN-based Image Compositing using Segmentation Masks (SIGGRAPH Asia 2021)

Barbershop: GAN-based Image Compositing using Segmentation Masks Barbershop: GAN-based Image Compositing using Segmentation Masks Peihao Zhu, Rameen A

Peihao Zhu 928 Dec 30, 2022
🚗 INGI Dakar 2K21 - Be the first one on the finish line ! 🚗

🚗 INGI Dakar 2K21 - Be the first one on the finish line ! 🚗 This year's first semester Club Info challenge will put you at the head of a car racing

ClubINFO INGI (UCLouvain) 6 Dec 10, 2021
An automated facial recognition based attendance system (desktop application)

Facial_Recognition_based_Attendance_System An automated facial recognition based attendance system (desktop application) Made using Python, Tkinter an

1 Jun 21, 2022
Asymmetric Bilateral Motion Estimation for Video Frame Interpolation, ICCV2021

ABME (ICCV2021) Junheum Park, Chul Lee, and Chang-Su Kim Official PyTorch Code for "Asymmetric Bilateral Motion Estimation for Video Frame Interpolati

Junheum Park 86 Dec 28, 2022
Unofficial PyTorch reimplementation of the paper Swin Transformer V2: Scaling Up Capacity and Resolution

PyTorch reimplementation of the paper Swin Transformer V2: Scaling Up Capacity and Resolution [arXiv 2021].

Christoph Reich 122 Dec 12, 2022
MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation

MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation This repo is the official implementation of "MHFormer: Multi-Hypothesis Transforme

Vegetabird 281 Jan 07, 2023
TeachMyAgent is a testbed platform for Automatic Curriculum Learning methods in Deep RL.

TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL Paper Website Documentation TeachMyAgent is a testbed platform for Automatic Cu

Flowers Team 51 Dec 25, 2022
Trafffic prediction analysis using hybrid models - Machine Learning

Hybrid Machine learning Model Clone the Repository Create a new Directory as assests and download the model from the below link Model Link To Start th

1 Feb 08, 2022
A tensorflow model that predicts if the image is of a cat or of a dog.

Quick intro Hello and thank you for your interest in my project! This is the backend part of a two-repo application. The other part can be found here

Tudor Matei 0 Mar 08, 2022
A fast python implementation of Ray Tracing in One Weekend using python and Taichi

ray-tracing-one-weekend-taichi A fast python implementation of Ray Tracing in One Weekend using python and Taichi. Taichi is a simple "Domain specific

157 Dec 26, 2022
CharacterGAN: Few-Shot Keypoint Character Animation and Reposing

CharacterGAN Implementation of the paper "CharacterGAN: Few-Shot Keypoint Character Animation and Reposing" by Tobias Hinz, Matthew Fisher, Oliver Wan

Tobias Hinz 181 Dec 27, 2022
Freecodecamp Scientific Computing with Python Certification; Solution for Challenge 2: Time Calculator

Assignment Write a function named add_time that takes in two required parameters and one optional parameter: a start time in the 12-hour clock format

Hellen Namulinda 0 Feb 26, 2022
ICSS - Interactive Continual Semantic Segmentation

Presentation This repository contains the code of our paper: Weakly-supervised c

Alteia 9 Jul 23, 2022
A library for preparing, training, and evaluating scalable deep learning hybrid recommender systems using PyTorch.

collie Collie is a library for preparing, training, and evaluating implicit deep learning hybrid recommender systems, named after the Border Collie do

ShopRunner 96 Dec 29, 2022
A proof of concept ai-powered Recaptcha v2 solver

Recaptcha Fullauto I've decided to open source my old Recaptcha v2 solver. My latest version will be opened sourced this summer. I am hoping this proj

Nate 60 Dec 20, 2022
Credit fraud detection in Python using a Jupyter Notebook

Credit-Fraud-Detection - Credit fraud detection in Python using a Jupyter Notebook , using three classification models (Random Forest, Gaussian Naive Bayes, Logistic Regression) from the sklearn libr

Ali Akram 4 Dec 28, 2021
Custom implementation of Corrleation Module

Pytorch Correlation module this is a custom C++/Cuda implementation of Correlation module, used e.g. in FlowNetC This tutorial was used as a basis for

Clément Pinard 361 Dec 12, 2022