[Update] A Pytorch version together with tutorials are added to our sister repository: https://github.com/mlech26l/ncps
This is the official repository for LTC networks described in the paper: https://arxiv.org/abs/2006.04439 This repository allows you to train continuous-time models with backpropagation through-time (BPTT). Available Continuous-time models are:
| Models | References |
|---|---|
| Liquid time-constant Networks | https://arxiv.org/abs/2006.04439 |
| Neural ODEs | https://papers.nips.cc/paper/7892-neural-ordinary-differential-equations.pdf |
| Continuous-time RNNs | https://www.sciencedirect.com/science/article/abs/pii/S089360800580125X |
| Continuous-time Gated Recurrent Units (GRU) | https://arxiv.org/abs/1710.04110 |
All models were implemented and tested with TensorFlow 1.14.0 and python3 on Ubuntu 16.04 and 18.04 machines. All the following steps assume that they are executed under these conditions.
First, we have to download all datasets by running
source download_datasets.shThis script creates a folder data, where all downloaded datasets are stored.
There is exactly one Python module per dataset:
- Hand gesture segmentation:
gesture.py - Room occupancy detection:
occupancy.py - Human activity recognition:
har.py - Traffic volume prediction:
traffic.py - Ozone level forecasting:
ozone.py
Each script accepts the following four arguments:
--model: lstm | ctrnn | ltc | ltc_rk | ltc_ex--epochs: number of training epochs (default 200)--size: number of hidden RNN units (default 32)--log: interval of how often to evaluate validation metric (default 1)
Each script trains the specified model for the given number of epochs and evaluates the
validation performance after every log steps.
At the end of the training, the best-performing checkpoint is restored and the model is evaluated on the test set.
All results are stored in the results folder by appending the result to CSV file.
For example, we can train and evaluate the CT-RNN by executing
python3 har.py --model ctrnnAfter the script is finished there should be a file results/har/ctrnn_32.csv created, containing the following columns:
best epoch: Epoch number that achieved the best validation metrictrain loss: Training loss achieved at the best epochtrain accuracy: Training metric achieved at the best epochvalid loss: Validation loss achieved at the best epochvalid accuracy: Best validation metric achieved during trainingtest loss: Loss on the test settest accuracy: Metric on the test set
| Parameter | Value | Description |
|---|---|---|
| Minibatch size | 16 | Number of training samples over which the gradient descent update is computed |
| Learning rate | 0.001/0.02 | 0.01-0.02 for LTC, 0.001 for all other models. |
| Hidden units | 32 | Number of hidden units of each model |
| Optimizer | Adam | See (Kingma and Ba, 2014) |
| beta_1 | 0.9 | Parameter of the Adam method |
| beta_2 | 0.999 | Parameter of the Adam method |
| epsilon | 1e-08 | Epsilon-hat parameter of the Adam method |
| Number of epochs | 200 | Maximum number of training epochs |
| BPTT length | 32 | Backpropagation through time length in time-steps |
| ODE solver sreps | 1/6 | relative to input sampling period |
| Validation evaluation interval | 1 | Interval of training epochs when the metrics on the validation are evaluated |
Run the main.m file to get trajectory length results for the desired setting tuneable in the code.
