Deep learning PyTorch library for time series forecasting, classification, and anomaly detection

Overview

Deep learning for time series forecasting

Example image Flow forecast is an open-source deep learning for time series forecasting framework. It provides all the latest state of the art models (transformers, attention models, GRUs) and cutting edge concepts with easy to understand interpretability metrics, cloud provider integration, and model serving capabilities. Flow Forecast was the first time series framework to feature support for transformer based models and remains the only true end-to-end deep learnig for time series forecasting framework. Currently Task-TS from CoronaWhy primarily maintains this repository. Pull requests are welcome. Historically, this repository provided open source benchmark and codes for flash flood and river flow forecasting.

For additional tutorials (on Colab) and examples please see our tutorials repository.

branch status
master CircleCI
Build PY Upload Python Package
Documentation Documentation Status
CodeCov codecov
CodeFactor CodeFactor

Getting Started

Using the library

  1. Run pip install flood-forecast
  2. Detailed info on training models can be found on the Wiki.
  3. Check out our Confluence Documentation

Models currently supported

  1. Vanilla LSTM (LSTM): A basic LSTM that is suitable for multivariate time series forecasting and transfer learning.
  2. Full transformer (SimpleTransformer in model_dict): The full original transformer with all 8 encoder and decoder blocks. Requires passing the target in at inference.
  3. Simple Multi-Head Attention (MultiHeadSimple): A simple multi-head attention block and linear embedding layers. Suitable for transfer learning.
  4. Transformer with a linear decoder (CustomTransformerDecoder in model_dict): A transformer with n-encoder blocks (this is tunable) and a linear decoder.
  5. DA-RNN: (DARNN) A well rounded model with which utilizes a LSTM + attention.
  6. Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting (called DecoderTransformer in model_dict):
  7. Transformer XL:
  8. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (Informer)
  9. DeepAR

Forthcoming Models

We have a number of models we are planning on releasing soon. Please check our project board for more info

Integrations

Google Cloud Platform

Weights and Biases

Contributing

For instructions on contributing please see our contributions page and our project board.

Historical River Flow Data

Task 1 Stream Flow Forecasting

This task focuses on forecasting a stream's future flow/height (in either cfs or feet respectively) given factors such as current flow, temperature, and precipitation. In the future we plan on adding more variables that help with the stream flow prediction such as snow pack data and the surrounding soil moisture index.

Task 2 Flood severity forecasting

Task two focuses on predicting the severity of the flood based on the flood forecast, population information, and topography. Flood severity is defined based on several factors including the number of injuires, property damage, and crop damage.

If you use either the data or code from this repository please use the citation below. Additionally please cite the original authors of the models.

@misc{godfried2020flowdb,
      title={FlowDB a large scale precipitation, river, and flash flood dataset}, 
      author={Isaac Godfried and Kriti Mahajan and Maggie Wang and Kevin Li and Pranjalya Tiwari},
      year={2020},
      eprint={2012.11154},
      archivePrefix={arXiv},
      primaryClass={cs.AI}
}
Comments
  • Informer compatibility with interpretability methods

    Informer compatibility with interpretability methods

    Currently Informer does not work with the shap interpretability methods. Refactoring SHAP to work with these methods will likely require so significant refactoring. As with Informer we have the target being passed. We should also likely design a helper function to better help with this. history, _, forecast_start_idx = csv_test_loader.get_from_start_date(datetime_start) background_tensor = _prepare_background_tensor(csv_test_loader)

    enhancement 
    opened by isaacmg 7
  • Inference mode for time series models

    Inference mode for time series models

    Create a predict function which does inference for for time series models without requiring the target present. This module should initialize the model using the given configuration file (with a weight path). It should be able to consume a CSV file or query a SQL table #102 (thought this functionality is not required in the initial PR). It should ideally make use of the existing evaluator.py module but

    Acceptance Criteria

    • [ ] Passing tests
    deployment 
    opened by isaacmg 7
  • about dataset

    about dataset

    So how can i download the dataset of FlowDB Dataset? Gsutil is not working? Can you give some details for your dataset, and tell us how to use you model for gour FlowDB? Thanks!

    opened by Vipermdl 6
  • Does datetime_start parameter in inference_params is forecasting start date?

    Does datetime_start parameter in inference_params is forecasting start date?

    In your Infer.ipynb datetime_start parameter is forecasting start date? (Your predict_cfs bucknet had been expired.)

    'inference_params': {'dataset_params': {'file_path': 'gs://predict_cfs/day_addition/01064118KPWM_flow.csv', 'forecast_history': 8, 'forecast_length': 1, 'interpolate_param': {'method': 'back_forward', 'params': {}}, 'relevant_cols': ['cfs1', 'precip', 'temp', 'month'], 'scaling': RobustScaler(), 'sort_column': 'hour_updated', 'target_col': ['cfs1']}, 'datetime_start': '2018-05-31', 'decoder_params': {'decoder_function': 'simple_decode', 'unsqueeze_dim': 1}, 'hours_to_forecast': 336, 'num_prediction_samples': 30, 'test_csv_path': 'gs://predict_cfs/day_addition/01064118KPWM_flow.csv'}

    opened by JJNET 5
  • Poor informer performance

    Poor informer performance

    The performance of the Informer model still seems to be poor at least with respect to forecast the Virgin River Flow. There may still be bugs therefore we should investigate it on other datasets and additional unittests. Possibly we should also try to replicate the performance on the ETH datasets the model was trained on (related to #314 ) The model does not seem to learn anything from the temporal data input.

    opened by isaacmg 5
  • Adding GPU support to the Informer

    Adding GPU support to the Informer

    This PR aims to the resolve prior issues #343 as well as fix a new problem related to the label_len in the data-loader. This PR in addition includes documentation updates to the Informer and additional information on how to use relevant data-loaders and SHAP features.

    opened by isaacmg 5
  • DecoderTransformer: Distinguishing Know inputs from Observed inputs

    DecoderTransformer: Distinguishing Know inputs from Observed inputs

    Hello Isaac. First of all thank you for this brilliant project. I was able to run the Decoder Transformer on the EU Wind Energy dataset.

    One question though. The model's paper, when defining the problem, says that some exogenous time series are known until the forecast horizon. For example, I would like to add the wind forecast as a feature with a middle dimension equals to "forecast_length" and with the same time idx as the target. Is there a way to model this in your config_file or at a lower level within the Loader objects?

    Thank you

    Lorenzo Ostano

    opened by Vergangenheit 5
  • TypeError: Object of type Tensor is not JSON serializable when running train_transformer_style with takes_target as 1

    TypeError: Object of type Tensor is not JSON serializable when running train_transformer_style with takes_target as 1

    Traceback (most recent call last): File "flood_forecast/trainer.py", line 108, in main() File "flood_forecast/trainer.py", line 103, in main train_function(training_config["model_type"], training_config) File "flood_forecast/trainer.py", line 42, in train_function train_transformer_style(model=trained_model, File "/home/harsh/Documents/Coronawhy/flow-forecast/flood_forecast/pytorch_training.py", line 146, in train_transformer_style model.save_model(model_filepath, max_epochs) File "/home/harsh/Documents/Coronawhy/flow-forecast/flood_forecast/time_model.py", line 152, in save_model json.dump(self.params, p) File "/home/harsh/anaconda3/envs/flow-forecast/lib/python3.8/json/init.py", line 179, in dump for chunk in iterable: File "/home/harsh/anaconda3/envs/flow-forecast/lib/python3.8/json/encoder.py", line 431, in _iterencode yield from _iterencode_dict(o, _current_indent_level) File "/home/harsh/anaconda3/envs/flow-forecast/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict yield from chunks File "/home/harsh/anaconda3/envs/flow-forecast/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict yield from chunks File "/home/harsh/anaconda3/envs/flow-forecast/lib/python3.8/json/encoder.py", line 438, in _iterencode o = _default(o) File "/home/harsh/anaconda3/envs/flow-forecast/lib/python3.8/json/encoder.py", line 179, in default raise TypeError(f'Object of type {o.class.name} ' TypeError: Object of type Tensor is not JSON serializable

    opened by 97harsh 5
  • Add meta-data fusion method and documentation

    Add meta-data fusion method and documentation

    Based on #100 we want to fuse meta-data with temporal data to enable better time series forecasts.

    • [x] Create a design document of meta-data fusion methods and explain relevant approaches
    • [x] Review design document with @kritim13 and other teammates.
    • [x] Implement agreed upon approach
    • [x] Create a JSON config file and appropriate unit tests.
    • [x] Test end to end in the Kaggle Notebook.
    meta-data 
    opened by isaacmg 5
  • Get ASOS data on GCS for years 2014-2019

    Get ASOS data on GCS for years 2014-2019

    Get all the data on GCS for those dates.

    • [x] Create looping function to perform action
    • [x] Create list of ASOS stations already saved with path on GCS. Upload this file to GCS.
    • [x] Run and get all data on GCS for all gages
    opened by isaacmg 5
  • DecoderTransformer not implemented as paper at all

    DecoderTransformer not implemented as paper at all

    did I miss something? The decodertransformer which claims to implement the paper(Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting) in the document is not even close to what the paper proposed. There are no key component like conv1d layers for locality and logsparse. If we didn't implement that paper, then really shouldn't list it in the document.

    opened by mvccn 4
  • Example auto-encoder time series

    Example auto-encoder time series

    We could use a detailed end-to-end example of using an AutoEncoder to create representations of temporal data. This should likely be done on Kaggle then added to the flow tutorials repo as a lin.

    documentation 
    opened by isaacmg 0
  • Pyre type error fixed.

    Pyre type error fixed.

    "filename": "flood_forecast/preprocessing/process_usgs.py" "warning_type": "Invalid type [31]" "warning_message": " Expression (pandas.DataFrame, int, int, int) is not a valid type." "warning_line": 82 "fix": remove int,int,int

    opened by luca-digrazia 0
  • Bump shap from 0.40.0 to 0.41.0

    Bump shap from 0.40.0 to 0.41.0

    Bumps shap from 0.40.0 to 0.41.0.

    Release notes

    Sourced from shap's releases.

    v0.41.0

    Lots of bugs fixes and API improvements.

    Commits
    • 510c4b6 Merge pull request #2242 from ravwojdyla/allow-to-control-the-heatmap-size
    • dd967b6 Merge branch 'master' of https://github.com/slundberg/shap
    • a791685 fix std to account for averaging
    • 6995c03 Merge branch 'master' into allow-to-control-the-heatmap-size
    • b6e90c8 Merge pull request #2580 from alexisdrakopoulos/feat/refactor_exceptions
    • 4921c50 Merge pull request #2162 from TheZL/xgbmodel_buffer_lstrip_error_correction
    • a8dbefd Clean up the intro doc notebook
    • 84ddd09 Merge branch 'feat/refactor_exceptions' of github.com:alexisdrakopoulos/shap ...
    • 348dc7d accidental import
    • 2cfa489 Merge branch 'master' into xgbmodel_buffer_lstrip_error_correction
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
Releases(FF_FIXES_BRANCH_VER)
Owner
AIStream
AIStream develops open source deep learning solutions for real world problems
AIStream
On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification

Understanding Bayesian Classification This repository hosts the code to reproduce the results presented in the paper On Uncertainty, Tempering, and Da

Sanyam Kapoor 18 Nov 17, 2022
基于PaddleOCR搭建的OCR server... 离线部署用

开头说明 DangoOCR 是基于大家的 CPU处理器 来运行的,CPU处理器 的好坏会直接影响其速度, 但不会影响识别的精度 ,目前此版本识别速度可能在 0.5-3秒之间,具体取决于大家机器的配置,可以的话尽量不要在运行时开其他太多东西。需要配合团子翻译器 Ver3.6 及其以上的版本才可以使用!

胖次团子 131 Dec 25, 2022
Source code for Adaptively Calibrated Critic Estimates for Deep Reinforcement Learning

Adaptively Calibrated Critic Estimates for Deep Reinforcement Learning Official implementation of ACC, described in the paper "Adaptively Calibrated C

3 Sep 16, 2022
Automatic packaging of the open-composite libs for OvGME

OvGME Packager for OpenXR – OpenComposite for DCS Note This repository is currently unsupported and needs to be migrated to the upstream OpenComposite

12 Nov 03, 2022
Implementation of our recent paper, WOOD: Wasserstein-based Out-of-Distribution Detection.

WOOD Implementation of our recent paper, WOOD: Wasserstein-based Out-of-Distribution Detection. Abstract The training and test data for deep-neural-ne

8 Dec 24, 2022
The implementation for "Comprehensive Knowledge Distillation with Causal Intervention".

Comprehensive Knowledge Distillation with Causal Intervention This repository is a PyTorch implementation of "Comprehensive Knowledge Distillation wit

Xiang Deng 10 Nov 03, 2022
🐤 Nix-TTS: An Incredibly Lightweight End-to-End Text-to-Speech Model via Non End-to-End Distillation

🐤 Nix-TTS An Incredibly Lightweight End-to-End Text-to-Speech Model via Non End-to-End Distillation Rendi Chevi, Radityo Eko Prasojo, Alham Fikri Aji

Rendi Chevi 156 Jan 09, 2023
CLIP+FFT text-to-image

Aphantasia This is a text-to-image tool, part of the artwork of the same name. Based on CLIP model, with FFT parameterizer from Lucent library as a ge

vadim epstein 690 Jan 02, 2023
Rust bindings for the C++ api of PyTorch.

tch-rs Rust bindings for the C++ api of PyTorch. The goal of the tch crate is to provide some thin wrappers around the C++ PyTorch api (a.k.a. libtorc

Laurent Mazare 2.3k Dec 30, 2022
Advbox is a toolbox to generate adversarial examples that fool neural networks in PaddlePaddle、PyTorch、Caffe2、MxNet、Keras、TensorFlow and Advbox can benchmark the robustness of machine learning models.

Advbox is a toolbox to generate adversarial examples that fool neural networks in PaddlePaddle、PyTorch、Caffe2、MxNet、Keras、TensorFlow and Advbox can benchmark the robustness of machine learning models

AdvBox 1.3k Dec 25, 2022
Implementations of LSTM: A Search Space Odyssey variants and their training results on the PTB dataset.

An LSTM Odyssey Code for training variants of "LSTM: A Search Space Odyssey" on Fomoro. Check out the blog post. Training Install TensorFlow. Clone th

Fomoro AI 95 Apr 13, 2022
Second Order Optimization and Curvature Estimation with K-FAC in JAX.

KFAC-JAX - Second Order Optimization with Approximate Curvature in JAX Installation | Quickstart | Documentation | Examples | Citing KFAC-JAX KFAC-JAX

DeepMind 90 Dec 22, 2022
PoseCamera is python based SDK for human pose estimation through RGB webcam.

PoseCamera PoseCamera is python based SDK for human pose estimation through RGB webcam. Install install posecamera package through pip pip install pos

WonderTree 7 Jul 20, 2021
Deep Markov Factor Analysis (NeurIPS2021)

Deep Markov Factor Analysis (DMFA) Codes and experiments for deep Markov factor analysis (DMFA) model accepted for publication at NeurIPS2021: A. Farn

Sarah Ostadabbas 2 Dec 16, 2022
Algorithmic trading using machine learning.

Algorithmic Trading This machine learning algorithm was built using Python 3 and scikit-learn with a Decision Tree Classifier. The program gathers sto

Sourav Biswas 101 Nov 10, 2022
Source code for "Understanding Knowledge Integration in Language Models with Graph Convolutions"

Graph Convolution Simulator (GCS) Source code for "Understanding Knowledge Integration in Language Models with Graph Convolutions" Requirements: PyTor

yifan 10 Oct 18, 2022
Implementation of Axial attention - attending to multi-dimensional data efficiently

Axial Attention Implementation of Axial attention in Pytorch. A simple but powerful technique to attend to multi-dimensional data efficiently. It has

Phil Wang 250 Dec 25, 2022
This is the repository for Learning to Generate Piano Music With Sustain Pedals

SusPedal-Gen This is the official repository of Learning to Generate Piano Music With Sustain Pedals Demo Page Dataset The dataset used in this projec

Joann Ching 12 Sep 02, 2022
Many Class Activation Map methods implemented in Pytorch for CNNs and Vision Transformers. Including Grad-CAM, Grad-CAM++, Score-CAM, Ablation-CAM and XGrad-CAM

Class Activation Map methods implemented in Pytorch pip install grad-cam ⭐ Tested on many Common CNN Networks and Vision Transformers. ⭐ Includes smoo

Jacob Gildenblat 6.6k Jan 06, 2023
This is the repo for our work "Towards Persona-Based Empathetic Conversational Models" (EMNLP 2020)

Towards Persona-Based Empathetic Conversational Models (PEC) This is the repo for our work "Towards Persona-Based Empathetic Conversational Models" (E

Zhong Peixiang 35 Nov 17, 2022