PairNorm

Official pytorch source code for PairNorm paper (ICLR 2020)
This code requires pytorch_geometric>=1.3.2

usage

For SGC, we use original PairNorm. Notice norm_scale is data-dependent. One can choose it from {0.1, 1, 10, 50}.

python main.py --data cora --model SGC --nlayer 40 --missing_rate 100 --norm_mode PN --norm_scale 10

For GCN or GAT, we use PairNorm-SI or SCS.

python main.py --data cora --model DeepGCN --nlayer 10 --missing_rate 100 --norm_mode PN-SI --residual 0
python main.py --data cora --model DeepGAT --nlayer 10 --missing_rate 100 --norm_mode PN-SCS --residual 0

update: normalization and PN

we have found that PN works bad with symmetric normalized adjacency matrix, originally the experiments align with the paper used row-normalized adjacency matrix. What's more, we also found a small bug in the old experiments with using PN for GCN and GAT. The current version PN should works good for GCN and GAT also (haven't fully tested). Please start from using PN before testing PN-SI and PN-SCS.

For GCN or GAT, now using PN to start.

python main.py --data cora --model DeepGCN --nlayer 10 --missing_rate 100 --norm_mode PN --residual 0
python main.py --data cora --model DeepGAT --nlayer 10 --missing_rate 100 --norm_mode PN --residual 0

cite

If you use our code, please cite

@inproceedings{
zhao2020pairnorm,
title={PairNorm: Tackling Oversmoothing in {\{}GNN{\}}s},
author={Lingxiao Zhao and Leman Akoglu},
booktitle={International Conference on Learning Representations},
year={2020},
url={https://openreview.net/forum?id=rkecl1rtwB}
}

Source code for PairNorm (ICLR 2020)

Related tags

Overview

PairNorm

usage

update: normalization and PN

cite

Owner

Distributed Deep learning with Keras & Spark

Official code of the paper "ReDet: A Rotation-equivariant Detector for Aerial Object Detection" (CVPR 2021)

Code and data for "Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning" (EMNLP 2021).

On Evaluation Metrics for Graph Generative Models

Official repository of IMPROVING DEEP IMAGE MATTING VIA LOCAL SMOOTHNESS ASSUMPTION.

[CVPR 2020] 3D Photography using Context-aware Layered Depth Inpainting

Neural network graphs and training metrics for PyTorch, Tensorflow, and Keras.

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

Improving Factual Consistency of Abstractive Text Summarization

PyTorch implementation of "Representing Shape Collections with Alignment-Aware Linear Models" paper.

This repository lets you interact with Lean through a REPL.

[CVPR 2020] Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation

The Unsupervised Reinforcement Learning Benchmark (URLB)

K-PLUG: Knowledge-injected Pre-trained Language Model for Natural Language Understanding and Generation in E-Commerce (EMNLP Founding 2021)

Repository for RNNs using TensorFlow and Keras - LSTM and GRU Implementation from Scratch - Simple Classification and Regression Problem using RNNs

On the adaptation of recurrent neural networks for system identification

Official code repository for the publication "Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons"

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.

Official implementation of the paper Momentum Capsule Networks (MoCapsNet)

Code for the ICASSP-2021 paper: Continuous Speech Separation with Conformer.