An implementation of "MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing" (ICML 2019).

Overview

MixHop and N-GCN

PWC Arxiv codebeat badge repo sizebenedekrozemberczki

A PyTorch implementation of "MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing" (ICML 2019) and "A Higher-Order Graph Convolutional Layer" (NeurIPS 2018).


Abstract

Recent methods generalize convolutional layers from Euclidean domains to graph-structured data by approximating the eigenbasis of the graph Laplacian. The computationally-efficient and broadly-used Graph ConvNet of Kipf & Welling, over-simplifies the approximation, effectively rendering graph convolution as a neighborhood-averaging operator. This simplification restricts the model from learning delta operators, the very premise of the graph Laplacian. In this work, we propose a new Graph Convolutional layer which mixes multiple powers of the adjacency matrix, allowing it to learn delta operators. Our layer exhibits the same memory footprint and computational complexity as a GCN. We illustrate the strength of our proposed layer on both synthetic graph datasets, and on several real-world citation graphs, setting the record state-of-the-art on Pubmed.

This repository provides a PyTorch implementation of MixHop and N-GCN as described in the papers:

MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Hrayr Harutyunyan, Nazanin Alipourfard, Kristina Lerman, Greg Ver Steeg, and Aram Galstyan. ICML, 2019. [Paper]

A Higher-Order Graph Convolutional Layer. Sami A Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Hrayr Harutyunyan. NeurIPS, 2018. [Paper]

The original TensorFlow implementation of MixHop is available [Here].

Requirements

The codebase is implemented in Python 3.5.2. package versions used for development are just below.

networkx          2.4
tqdm              4.28.1
numpy             1.15.4
pandas            0.23.4
texttable         1.5.0
scipy             1.1.0
argparse          1.1.0
torch             1.1.0
torch-sparse      0.3.0

Datasets

The code takes the **edge list** of the graph in a csv file. Every row indicates an edge between two nodes separated by a comma. The first row is a header. Nodes should be indexed starting with 0. A sample graph for `Cora` is included in the `input/` directory. In addition to the edgelist there is a JSON file with the sparse features and a csv with the target variable.

The **feature matrix** is a sparse binary one it is stored as a json. Nodes are keys of the json and feature indices are the values. For each node feature column ids are stored as elements of a list. The feature matrix is structured as:

{ 0: [0, 1, 38, 1968, 2000, 52727],
  1: [10000, 20, 3],
  2: [],
  ...
  n: [2018, 10000]}

The **target vector** is a csv with two columns and headers, the first contains the node identifiers the second the targets. This csv is sorted by node identifiers and the target column contains the class meberships indexed from zero.

NODE ID Target
0 3
1 1
2 0
3 1
... ...
n 3

Options

Training an N-GCN/MixHop model is handled by the `src/main.py` script which provides the following command line arguments.

Input and output options

  --edge-path       STR    Edge list csv.         Default is `input/cora_edges.csv`.
  --features-path   STR    Features json.         Default is `input/cora_features.json`.
  --target-path     STR    Target classes csv.    Default is `input/cora_target.csv`.

Model options

  --model             STR     Model variant.                 Default is `mixhop`.               
  --seed              INT     Random seed.                   Default is 42.
  --epochs            INT     Number of training epochs.     Default is 2000.
  --early-stopping    INT     Early stopping rounds.         Default is 10.
  --training-size     INT     Training set size.             Default is 1500.
  --validation-size   INT     Validation set size.           Default is 500.
  --learning-rate     FLOAT   Adam learning rate.            Default is 0.01.
  --dropout           FLOAT   Dropout rate value.            Default is 0.5.
  --lambd             FLOAT   Regularization coefficient.    Default is 0.0005.
  --layers-1          LST     Layer sizes (upstream).        Default is [200, 200, 200]. 
  --layers-2          LST     Layer sizes (bottom).          Default is [200, 200, 200].
  --cut-off           FLOAT   Norm cut-off for pruning.      Default is 0.1.
  --budget            INT     Architecture neuron budget.    Default is 60.

Examples

The following commands learn a neural network and score on the test set. Training a model on the default dataset.

$ python src/main.py

Training a MixHop model for a 100 epochs.

$ python src/main.py --epochs 100

Increasing the learning rate and the dropout.

$ python src/main.py --learning-rate 0.1 --dropout 0.9

Training a model with diffusion order 2:

$ python src/main.py --layers 64 64

Training an N-GCN model:

$ python src/main.py --model ngcn

License


Comments
  • FileNotFoundError: [Errno 2] No such file or directory: './input/cora_edges.csv'

    FileNotFoundError: [Errno 2] No such file or directory: './input/cora_edges.csv'

    hello, when i run src/main.py,the error message appears: File "pandas_libs\parsers.pyx", line 361, in pandas._libs.parsers.TextReader.cinit File "pandas_libs\parsers.pyx", line 653, in pandas._libs.parsers.TextReader._setup_parser_source FileNotFoundError: [Errno 2] No such file or directory: './input/cora_edges.csv'

    do you know how to solve it?

    opened by tanjia123456 4
  • Citeseer and Pubmed Datasets

    Citeseer and Pubmed Datasets

    Hi Benedek,

    Thank you so much for the code. I want to run your code on Citeseer and Pubmed datasets. Would you mind providing Citeseer and Pubmed data in this format? By the way, after running MixHop model with default parameters I got the test accuracy 0.7867. Did the accuracy depend on the system that the code is running?

    Thanks in advance

    opened by bousejin 3
  • IndexError:

    IndexError:

    hello, when I run main.py, the error message occurs:

    File "D:\anaconda3.4\lib\site-packages\torch_sparse\spmm.py", line 30, in spmm out = matrix[col] IndexError: index 10241 is out of bounds for dimension 0 with size 10241

    the content of the spmm.py: `# import torch from torch_scatter import scatter_add

    def spmm(index, value, m, n, matrix): """Matrix product of sparse matrix with dense matrix. 稀疏矩阵与稠密矩阵的矩阵乘积

    Args:
        index (:class:`LongTensor`): The index tensor of sparse matrix.
        value (:class:`Tensor`): The value tensor of sparse matrix.
        m (int): The first dimension of corresponding dense matrix.
        n (int): The second dimension of corresponding dense matrix.
        matrix (:class:`Tensor`): The dense matrix.
      :rtype: :class:`Tensor`
    """
    
    assert n == matrix.size(0)
    
    row, col = index
    
    matrix = matrix if matrix.dim() > 1 else matrix.unsqueeze(-1)
    
    out = matrix[col]
    out = out * value.unsqueeze(-1)
    out = scatter_add(out, row, dim=0, dim_size=m)
    
    return out
    

    ` by the way, I use my own datasets, and the number of node is 10242. do you know how to solve it?

    opened by tanjia123456 2
  • some problem about codes

    some problem about codes

    When I run the code, some error occured as follows: MixHop-and-N-GCN-master\src\utils.py", line 45, in feature_reader out_features["indices"] = torch.LongTensor(np.concatenate([features.row.reshape(-1,1), features.col.reshape(-1,1)],axis=1).T) TypeError: can't convert np.ndarray of type numpy.int32. The only supported types are: float64, float32, float16, int64, int32, int16, int8, and uint8. I search it on the Internet and found that it seems to be a list of lists that are not of the same length. I was stuck in it and do not know how to correct it! Looking forward to your help!!Thanks!!

    opened by junkangwu 2
  • About

    About "torch_scatter"

    I cannot successfully install "torch_scatter". When I run the command line : pip3 install torch_scatter, an error always occurs, just like below. I tried to solve the problem, but I don't find the correct method. Could you help me? Thanks a lot!

    The error: ... cpu/scatter.cpp:1:29: fatal error: torch/extension.h: No such file or directory compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1


    Failed building wheel for torch-scatter Running setup.py clean for torch-scatter Failed to build torch-scatter Installing collected packages: torch-scatter Running setup.py install for torch-scatter ... error Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-ijd9s63n/torch-scatter/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-4tkx7v_s-record/install-record.txt --single-version-externally-managed --compile: running install running build running build_py creating build creating build/lib.linux-x86_64-3.5 creating build/lib.linux-x86_64-3.5/torch_scatter copying torch_scatter/mul.py -> build/lib.linux-x86_64-3.5/torch_scatter copying torch_scatter/mean.py -> build/lib.linux-x86_64-3.5/torch_scatter copying torch_scatter/sub.py -> build/lib.linux-x86_64-3.5/torch_scatter copying torch_scatter/min.py -> build/lib.linux-x86_64-3.5/torch_scatter copying torch_scatter/std.py -> build/lib.linux-x86_64-3.5/torch_scatter copying torch_scatter/init.py -> build/lib.linux-x86_64-3.5/torch_scatter copying torch_scatter/max.py -> build/lib.linux-x86_64-3.5/torch_scatter copying torch_scatter/div.py -> build/lib.linux-x86_64-3.5/torch_scatter copying torch_scatter/add.py -> build/lib.linux-x86_64-3.5/torch_scatter creating build/lib.linux-x86_64-3.5/test copying test/test_multi_gpu.py -> build/lib.linux-x86_64-3.5/test copying test/utils.py -> build/lib.linux-x86_64-3.5/test copying test/test_std.py -> build/lib.linux-x86_64-3.5/test copying test/init.py -> build/lib.linux-x86_64-3.5/test copying test/test_forward.py -> build/lib.linux-x86_64-3.5/test copying test/test_backward.py -> build/lib.linux-x86_64-3.5/test creating build/lib.linux-x86_64-3.5/torch_scatter/utils copying torch_scatter/utils/ext.py -> build/lib.linux-x86_64-3.5/torch_scatter/utils copying torch_scatter/utils/init.py -> build/lib.linux-x86_64-3.5/torch_scatter/utils copying torch_scatter/utils/gen.py -> build/lib.linux-x86_64-3.5/torch_scatter/utils running build_ext building 'torch_scatter.scatter_cpu' extension creating build/temp.linux-x86_64-3.5 creating build/temp.linux-x86_64-3.5/cpu x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/zgh/.local/lib/python3.5/site-packages/torch/lib/include -I/home/zgh/.local/lib/python3.5/site-packages/torch/lib/include/TH -I/home/zgh/.local/lib/python3.5/site-packages/torch/lib/include/THC -I/usr/include/python3.5m -c cpu/scatter.cpp -o build/temp.linux-x86_64-3.5/cpu/scatter.o -Wno-unused-variable -DTORCH_EXTENSION_NAME=scatter_cpu -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ cpu/scatter.cpp:1:29: fatal error: torch/extension.h: No such file or directory compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
    

    Command "/usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-ijd9s63n/torch-scatter/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-4tkx7v_s-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-ijd9s63n/torch-scatter/ You are using pip version 8.1.1, however version 19.1.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

    opened by zhangguanghui1 2
  • running error

    running error

    hello, when I run main.py, the error message occurs: File "/home/tj/anaconda3/lib/python3.6/site-packages/torch_sparse/init.py", line 22, in raise OSError(e) OSError: libcusparse.so.10.0: cannot open shared object file: No such file or directory

    my python version 3.6 cuda10.0 torch1.0.1 torch-sparse0.5.1 do you know how to solve it?

    opened by tanjia123456 1
  • Higher Powers Implementation

    Higher Powers Implementation

    image

    https://github.com/benedekrozemberczki/MixHop-and-N-GCN/blob/6e4ae00055fc1aecd972081ef9c152b0e9de37c1/src/layers.py#L54-L56

    To me it looks like this is implementing H(l+1) = σ(A^j * H(l) * W(l))

    image

    Can you explain where W_j and the concatenation are taking place?

    opened by datavistics 1
  • This paper was accepted in NeuralPS 2018?

    This paper was accepted in NeuralPS 2018?

    I didn't find this paper "A Higher-Order Graph Convolutional Layer" in NeuralPS 2018 accepted list. So I am not sure whether this paper has been accepted?

    opened by Jhy1993 1
Releases(v_00001)
Owner
Benedek Rozemberczki
Machine Learning Engineer at AstraZeneca | PhD from The University of Edinburgh.
Benedek Rozemberczki
StrongSORT: Make DeepSORT Great Again

StrongSORT StrongSORT: Make DeepSORT Great Again StrongSORT: Make DeepSORT Great Again Yunhao Du, Yang Song, Bo Yang, Yanyun Zhao arxiv 2202.13514 Abs

369 Jan 04, 2023
Music Source Separation; Train & Eval & Inference piplines and pretrained models we used for 2021 ISMIR MDX Challenge.

Introduction 1. Usage (For MSS) 1.1 Prepare running environment 1.2 Use pretrained model 1.3 Train new MSS models from scratch 1.3.1 How to train 1.3.

Leo 100 Dec 25, 2022
Starter code for the ICCV 2021 paper, 'Detecting Invisible People'

Detecting Invisible People [ICCV 2021 Paper] [Website] Tarasha Khurana, Achal Dave, Deva Ramanan Introduction This repository contains code for Detect

Tarasha Khurana 28 Sep 16, 2022
FairyTailor: Multimodal Generative Framework for Storytelling

FairyTailor: Multimodal Generative Framework for Storytelling

Eden Bens 172 Dec 30, 2022
Jupyter notebooks for using & learning Keras

deep-learning-with-keras-notebooks 這個github的repository主要是個人在學習Keras的一些記錄及練習。希望在學習過程中發現到一些好的資訊與範例也可以對想要學習使用 Keras來解決問題的同好,或是對深度學習有興趣的在學學生可以有一些方便理解與上手範例

ErhWen Kuo 2.1k Dec 27, 2022
Official implementation of "Refiner: Refining Self-attention for Vision Transformers".

RefinerViT This repo is the official implementation of "Refiner: Refining Self-attention for Vision Transformers". The repo is build on top of timm an

101 Dec 29, 2022
Utility tools for the "Divide and Remaster" dataset, introduced as part of the Cocktail Fork problem paper

Divide and Remaster Utility Tools Utility tools for the "Divide and Remaster" dataset, introduced as part of the Cocktail Fork problem paper The DnR d

Darius Petermann 46 Dec 11, 2022
Implementation of "GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings" in PyTorch

PyGAS: Auto-Scaling GNNs in PyG PyGAS is the practical realization of our G NN A uto S cale (GAS) framework, which scales arbitrary message-passing GN

Matthias Fey 139 Dec 25, 2022
Safe Model-Based Reinforcement Learning using Robust Control Barrier Functions

README Repository containing the code for the paper "Safe Model-Based Reinforcement Learning using Robust Control Barrier Functions". Specifically, an

Yousef Emam 13 Nov 24, 2022
A toolset of Python programs for signal modeling and indentification via sparse semilinear autoregressors.

SPAAR Description A toolset of Python programs for signal modeling via sparse semilinear autoregressors. References Vides, F. (2021). Computing Semili

Fredy Vides 0 Oct 30, 2021
Learning Tracking Representations via Dual-Branch Fully Transformer Networks

Learning Tracking Representations via Dual-Branch Fully Transformer Networks DualTFR ⭐ We achieves the runner-ups for both VOT2021ST (short-term) and

phiphi 19 May 04, 2022
Robust Partial Matching for Person Search in the Wild

APNet for Person Search Introduction This is the code of Robust Partial Matching for Person Search in the Wild accepted in CVPR2020. The Align-to-Part

Yingji Zhong 36 Dec 18, 2022
The official implementation of Equalization Loss v1 & v2 (CVPR 2020, 2021) based on MMDetection.

The Equalization Losses for Long-tailed Object Detection and Instance Segmentation This repo is official implementation CVPR 2021 paper: Equalization

Jingru Tan 129 Dec 16, 2022
Unofficial implementation of "Coordinate Attention for Efficient Mobile Network Design"

Unofficial implementation of "Coordinate Attention for Efficient Mobile Network Design". CoordAttention tensorflow slim

Billy 9 Aug 22, 2022
Generalized hybrid model for mode-locked laser diodes with an extended passive cavity

GenHybridMLLmodel Generalized hybrid model for mode-locked laser diodes with an extended passive cavity This hybrid simulation strategy combines a tra

Stijn Cuyvers 3 Sep 21, 2022
A repository for the updated version of CoinRun used to collect MUGEN, a multimodal video-audio-text dataset.

A repository for the updated version of CoinRun used to collect MUGEN, a multimodal video-audio-text dataset. This repo contains scripts to train RL agents to navigate the closed world and collect vi

MUGEN 11 Oct 22, 2022
SplineConv implementation for Paddle.

SplineConv implementation for Paddle This module implements the SplineConv operators from Matthias Fey, Jan Eric Lenssen, Frank Weichert, Heinrich Mül

北海若 3 Dec 29, 2021
Pytorch implementation for RelTransformer

RelTransformer Our Architecture This is a Pytorch implementation for RelTransformer The implementation for Evaluating on VG200 can be found here Requi

Vision CAIR Research Group, KAUST 21 Nov 22, 2022
An algorithm study of the 6th iOS 10 set of Boost Camp Web Mobile

알고리즘 스터디 🔥 부스트캠프 웹모바일 6기 iOS 10조의 알고리즘 스터디 입니다. 개인적인 사정 등으로 S034, S055만 참가하였습니다. 스터디 목적 상진: 코테 합격 + 부캠끝나고 아침에 일어나기 위해 필요한 사이클 기완: 꾸준하게 자리에 앉아 공부하기 +

2 Jan 11, 2022
PyTorchVideo is a deeplearning library with a focus on video understanding work

PyTorchVideo is a deeplearning library with a focus on video understanding work. PytorchVideo provides resusable, modular and efficient components needed to accelerate the video understanding researc

Facebook Research 2.7k Jan 07, 2023