Heterogeneous Graph Benchmark

Revisiting, benchmarking, and refining Heterogeneous Graph Neural Networks.

Roadmap

We organize our repo by task, and one sub-folder per task. Currently, we have four tasks, i.e., node classification (NC), link prediction (LP), knowledge-aware recommendation (Recom) and text classification (TC).

Revisiting

This part refers to Section 3 and Table 1 in our paper.

• HAN
• GTN
• RSHN
• HetGNN
• MAGNN-nc and MAGNN-lp

Benchmarking and Refining

This part refers to Section 4,5,6 in our paper.

• Node classification benchmark, NC-baseline
• Link prediction benchmark, LP-baseline
• Knowledge-aware recommendation benchmark, Recom-baseline

You should notice that the test data labels are randomly replaced to prevent data leakage issues. If you want to obtain test scores, you need to submit your prediction to our website.

For node classification and link prediction tasks, you can submit online. But for recommendation task, since the prediction files are too large to submit, you have to test offline by yourself.

If you want to show your method on our official leaderboard on HGB website, you should submit your code or paper to us. Once your code or paper is verified, your method will be displayed on the official leaderboard. (The request form is under development and will be available soon!)

More

This repo is actively under development. Therefore, there are some extra experiments in this repo beyond our paper, such as graph-based text classification. For more information, see our website. Welcome contribute new tasks, datasets, methods to HGB!

Moreover, we also have an implementation of Simple-HGN in cogdl.

Citation

• Title: Are we really making much progress? Revisiting, benchmarking and refining the Heterogeneous Graph Neural Networks.
• Authors: Qingsong Lv*, Ming Ding*, Qiang Liu, Yuxiang Chen, Wenzheng Feng, Siming He, Chang Zhou, Jianguo Jiang, Yuxiao Dong, Jie Tang.
• In proceedings: KDD 2021.