BErt-like Neurophysiological Data Representation

Related tags

Data AnalysisBENDR
Overview

BENDR

BErt-like Neurophysiological Data Representation

A picture of Bender from Futurama

This repository contains the source code for reproducing, or extending the BERT-like self-supervision pre-training for EEG data from the article:

BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data

To run these scripts, you will need to use the DN3 project. We will try to keep this updated so that it works with the latest DN3 release. If you are just looking for the BENDR model, and don't need to reproduce the article results per se, BENDR will be (or maybe already is if I forgot to update it here) integrated into DN3, in which case I would start there.

Currently, we recommend version 0.2. Feel free to open an issue if you are having any trouble.

More extensive instructions are upcoming, but in essence you will need to either:

a)  Download the TUEG dataset and pre-train new encoder and contextualizer weights, _or_
b)  Use the [pre-trained model weights](https://github.com/SPOClab-ca/BENDR/releases/tag/v0.1-alpha)

Once you have a pre-trained model:

1) Add the paths of the pre-trained weights to configs/downstream.yml
2) Edit paths to local copies of your datasets in configs/downstream_datasets.yml
3) Run downstream.sh

Comments
  • about the loss function

    about the loss function

    Very appreciate for your contribution.i am really interested in the self training in EEG. The only question is about calculating loss function. In your paper, The calculation of the denominator uses cosine similarity between the output of the transformer and the 20 distractors and the input of the transformer. However, in the code, the calculation of the denominator uses cosine similarity between the input of the transformer and the 20 distractors, and the output of the transformer. In other word, the output and the input switch positions. Are both the calculation approaches the same? Or why did you change the calculation approache in the code? Thanks!

    opened by stickOverCarrot 2
  • About deploy downstream.yml and downstream_datasets.yml

    About deploy downstream.yml and downstream_datasets.yml

    Tranks for supplying your code. But when I follow your markdown, I meet some problems image

    This is my project files image

    This is my downstream.yml image

    This is my downstream_datasets.yml image

    opened by YoloEliwa 1
  • Pre-trained weights?

    Pre-trained weights?

    Not an issue per se, but you state the pre-trained weights for your paper are available in this repo, yet I have had a good look around and I haven't found them, nor a means of downloading them. Please can you let me know where I could find them? I'm really keen to try out this exciting architecture you've put together!

    opened by SgtWhiskeyjack 1
  • result_tracking module

    result_tracking module

    There's a reference that's in the module import: downstream.py from result_tracking import ThinkerwiseResultTracker that looks like some type of tracking code for experiments?

    opened by bencten 1
  • dropout should change

    dropout should change

    Iteration: 4%|▍ | 13/330 [00:36<16:00, 3.03s/batches, bac=0.5, Accuracy=0.51, loss=0.695, lr=1.47e-6]D:\Anaconda\envs\LGG\lib\site-packages\torch\nn\functional.py:1338: UserWarning: dropout2d: Received a 3D input to dropout2d and assuming that channel-wise 1D dropout behavior is desired - input is interpreted as shape (N, C, L), where C is the channel dim. This behavior will change in a future release to interpret the input as one without a batch dimension, i.e. shape (C, H, W). To maintain the 1D channel-wise dropout behavior, please switch to using dropout1d instead. warnings.warn("dropout2d: Received a 3D input to dropout2d and assuming that channel-wise "

    opened by zy2021314 0
  • A more detailed explanation

    A more detailed explanation

    We need to use your code for research, may I ask when you can provide detailed explanation, because we have some difficulties in understanding the code without detailed explanation.

    opened by EchizenMike 0
  • preload in downstream.yml

    preload in downstream.yml

    In the "downstream.yml" file, what is the function of the "preload"? What's mean if I specify "preload: True" or "preload: False"?

    Thank you in advance

    opened by frannfuri 0
Releases(v0.1-alpha)
Owner
GitHub Repository
Gaussian processes in TensorFlow

Website | Documentation (release) | Documentation (develop) | Glossary Table of Contents What does GPflow do? Installation Getting Started with GPflow

GPflow 1.7k Jan 06, 2023
Efficient matrix representations for working with tabular data

Efficient matrix representations for working with tabular data

QuantCo 70 Dec 14, 2022
Convert monolithic Jupyter notebooks into Ploomber pipelines.

Soorgeon Join our community | Newsletter | Contact us | Blog | Website | YouTube Convert monolithic Jupyter notebooks into Ploomber pipelines. soorgeo

Ploomber 65 Dec 16, 2022
MIR Cheatsheet - Survival Guidebook for MIR Researchers in the Lab

MIR Cheatsheet - Survival Guidebook for MIR Researchers in the Lab

SeungHeonDoh 3 Jul 02, 2022
Bamboolib - a GUI for pandas DataFrames

Community repository of bamboolib bamboolib is joining forces with Databricks. For more information, please read our announcement. Please note that th

Tobias Krabel 863 Jan 08, 2023
Flexible HDF5 saving/loading and other data science tools from the University of Chicago

deepdish Flexible HDF5 saving/loading and other data science tools from the University of Chicago. This repository also host a Deep Learning blog: htt

UChicago - Department of Computer Science 255 Dec 10, 2022
Calculate multilateral price indices in Python (with Pandas and PySpark).

IndexNumCalc Calculate multilateral price indices using the GEKS-T (CCDI), Time Product Dummy (TPD), Time Dummy Hedonic (TDH), Geary-Khamis (GK) metho

Dr. Usman Kayani 3 Apr 27, 2022
Mortgage-loan-prediction - Show how to perform advanced Analytics and Machine Learning in Python using a full complement of PyData utilities

Mortgage-loan-prediction - Show how to perform advanced Analytics and Machine Learning in Python using a full complement of PyData utilities. This is aimed at those looking to get into the field of D

Joachim 1 Dec 26, 2021
Pyspark Spotify ETL

This is my first Data Engineering project, it extracts data from the user's recently played tracks using Spotify's API, transforms data and then loads it into Postgresql using SQLAlchemy engine. Data

16 Jun 09, 2022
Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods

Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods Introduction Graph Neural Networks (GNNs) have demonstrated

37 Dec 15, 2022
Predictive Modeling & Analytics on Home Equity Line of Credit

Predictive Modeling & Analytics on Home Equity Line of Credit Data (Python) HMEQ Data Set In this assignment we will use Python to examine a data set

Dhaval Patel 1 Jan 09, 2022
collect training and calibration data for gaze tracking

Collect Training and Calibration Data for Gaze Tracking This tool allows collecting gaze data necessary for personal calibration or training of eye-tr

Pascal 5 Dec 17, 2022
In this tutorial, raster models of soil depth and soil water holding capacity for the United States will be sampled at random geographic coordinates within the state of Colorado.

Raster_Sampling_Demo (Resulting graph of this demo) Background Sampling values of a raster at specific geographic coordinates can be done with a numbe

2 Dec 13, 2022
Fit models to your data in Python with Sherpa.

Table of Contents Sherpa License How To Install Sherpa Using Anaconda Using pip Building from source History Release History Sherpa Sherpa is a modeli

134 Jan 07, 2023
BasstatPL is a package for performing different tabulations and calculations for descriptive statistics.

BasstatPL is a package for performing different tabulations and calculations for descriptive statistics. It provides: Frequency table constr

Angel Chavez 1 Oct 31, 2021
Import, connect and transform data into Excel

xlwings_query Import, connect and transform data into Excel. Description The concept is to apply data transformations to a main query object. When the

George Karakostas 1 Jan 19, 2022
Jupyter notebooks for the book "The Elements of Statistical Learning".

This repository contains Jupyter notebooks implementing the algorithms found in the book and summary of the textbook.

Madiyar 369 Dec 30, 2022
Demonstrate the breadth and depth of your data science skills by earning all of the Databricks Data Scientist credentials

Data Scientist Learning Plan Demonstrate the breadth and depth of your data science skills by earning all of the Databricks Data Scientist credentials

Trung-Duy Nguyen 27 Nov 01, 2022
An extension to pandas dataframes describe function.

pandas_summary An extension to pandas dataframes describe function. The module contains DataFrameSummary object that extend describe() with: propertie

Mourad 450 Dec 30, 2022
Dbt-core - dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

Dbt-core - dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

dbt Labs 6.3k Jan 08, 2023