ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representation from common sense knowledge graphs.

Last update: Nov 21, 2022

Overview

ZSL-KG

ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representation from common sense knowledge graphs.

Reference paper: Zero-shot Learning with Common Sense Knowledge graphs.

Performance

Performance ZSL-KG compared to other existing graph-based zero-shot learning frameworks.

Method	Ontonotes (Strict)	BBN (Strict)	SNIPS-NLU (Acc.)	AWA2 (H)	aPY (H)	ImageNet (All T-1)	Avg.
GCNZ	41.5	21.5	82.5	73.3	58.1	1.0	46.3
SGCN	42.6	24.9	50.3	73.7	56.8	1.5	41.6
DGP	41.1	24.0	64.4	75.1	55.7	1.4	43.6
ZSL-KG	45.2	26.7	89.0	74.6	61.6	1.7	49.8

ZSL-KG outperforms existing graph-based frameworks on five out of six benchmark datasets.

For more details on the experiments, refer to nayak-arxiv20-code.

Installation

The package requires python >= 3.7. To install the package, type the following command:

pip install .

Example Usage

In our framework, we use AutoGNN to easily create graph neural networks for zero-shot learning.

from zsl_kg.class_encoder.auto_gnn import AutoGNN
from zsl_kg.common.graph import NeighSampler

trgcn = {
    "input_dim": 300,
    "output_dim": 2049,
    "type": "trgcn",
    "gnn": [
        {
            "input_dim": 300,
            "output_dim": 2048,
            "activation": nn.LeakyReLU(0.2),
            "normalize": True,
            "sampler": NeighSampler(100, mode="topk"),
            "fh": 100,
        },
        {
            "input_dim": 2048,
            "output_dim": 2049,
            "activation": None,
            "normalize": True,
            "sampler": NeighSampler(50, mode="topk"),
        },
    ],
}

class_encoder = AutoGNN(trgcn)

Our framework supports the following graph neural networks: gcn, gat, rgcn, lstm, trgcn. You can change the type to any of the available to graph neural networks to instantly create a new graph neural network.

For more examples, refer to nayak-arxiv20-code.

Run Tests

To run the tests, please type the following command:

pytest

Citation

Please cite the following paper if you are using our framework.

@article{nayak:arxiv20,
  Author = {Nayak, Nihal V. and Bach, Stephen H.},
  Title = {Zero-Shot Learning with Common Sense Knowledge Graphs},
  Volume = {arXiv:2006.10713 [cs.LG]},
  Year = {2020}}

Comments

Graph usage query in object classification AWA2

Hi, I'm not entirely sure if this is where I should be asking this but I was wondering if you could help me clarify a question.

1 When performing zero shot object classification in AWA2, what is the graph that is used? (I assume a knowledge graph such as ConceptNet?)

2 If this is a knowledge graph, than what is the "input" to the trgcn? As in where are these "concepts" coming from, is it from the class attributes?

opened by shadowbourne 2
Quick note

Dear Team behind @zsl-kg,

This framework is really cool, however, I was quite disappointed to see you are re-writing everything from scratch when you could have used an awesome framework like PyTorch Geometric.

Best regards, Thomas Chaton

opened by tchaton 1

Releases(v0.0.1)

v0.0.1(Aug 27, 2021)

This is the initial release. Contains all the necessary code to run experiments in nayak-arxiv20-code.
Source code(tar.gz)
Source code(zip)

Codes and models for the paper "Learning Unknown from Correlations: Graph Neural Network for Inter-novel-protein Interaction Prediction".

GNN_PPI Codes and models for the paper "Learning Unknown from Correlations: Graph Neural Network for Inter-novel-protein Interaction Prediction". Lear

2 Dec 14, 2022

Episodic Transformer (E.T.) is a novel attention-based architecture for vision-and-language navigation. E.T. is based on a multimodal transformer that encodes language inputs and the full episode history of visual observations and actions.

Episodic Transformers (E.T.) Episodic Transformer for Vision-and-Language Navigation Alexander Pashevich, Cordelia Schmid, Chen Sun Episodic Transform

62 Dec 24, 2022

Neighborhood Contrastive Learning for Novel Class Discovery

Neighborhood Contrastive Learning for Novel Class Discovery This repository contains the official implementation of our paper: Neighborhood Contrastiv

56 Dec 9, 2022

Awesome Deep Graph Clustering is a collection of SOTA, novel deep graph clustering methods

ADGC: Awesome Deep Graph Clustering ADGC is a collection of state-of-the-art (SOTA), novel deep graph clustering methods (papers, codes and datasets).

297 Dec 27, 2022

The code for the CVPR 2021 paper Neural Deformation Graphs, a novel approach for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.

Neural Deformation Graphs Project Page | Paper | Video Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction Aljaž Božič, Pablo P

134 Dec 16, 2022

Code for the AAAI 2022 paper "Zero-Shot Cross-Lingual Machine Reading Comprehension via Inter-Sentence Dependency Graph".

multilingual-mrc-isdg Code for the AAAI 2022 paper "Zero-Shot Cross-Lingual Machine Reading Comprehension via Inter-Sentence Dependency Graph". This r

5 Dec 7, 2022

Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

PyTorch Implementation of Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers 1 Using Colab Please notic

489 Jan 7, 2023

sense-py-AnishaBaishya created by GitHub Classroom

Compute Statistics Here we compute statistics for a bunch of numbers. This project uses the unittest framework to test functionality. Pass the tests T

1 Oct 21, 2021

This repo is the code release of EMNLP 2021 conference paper "Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories".

Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories This repo is the code release of EMNLP 2021 con

12 Nov 22, 2022

ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representation from common sense knowledge graphs.

Related tags

Overview

ZSL-KG

Performance

Installation

Example Usage

Run Tests

Citation

You might also like...

Codes and models for the paper "Learning Unknown from Correlations: Graph Neural Network for Inter-novel-protein Interaction Prediction".

Episodic Transformer (E.T.) is a novel attention-based architecture for vision-and-language navigation. E.T. is based on a multimodal transformer that encodes language inputs and the full episode history of visual observations and actions.

Neighborhood Contrastive Learning for Novel Class Discovery

Awesome Deep Graph Clustering is a collection of SOTA, novel deep graph clustering methods

The code for the CVPR 2021 paper Neural Deformation Graphs, a novel approach for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.

Code for the AAAI 2022 paper "Zero-Shot Cross-Lingual Machine Reading Comprehension via Inter-Sentence Dependency Graph".

Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

sense-py-AnishaBaishya created by GitHub Classroom

This repo is the code release of EMNLP 2021 conference paper "Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories".

Comments

Graph usage query in object classification AWA2

Quick note

Releases(v0.0.1)

v0.0.1(Aug 27, 2021)

Owner

Bats Research

A deep learning model for style-specific music generation.

Hub is a dataset format with a simple API for creating, storing, and collaborating on AI datasets of any size.

Pynomial - a lightweight python library for implementing the many confidence intervals for the risk parameter of a binomial model

Codes for "Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier"

Official Code for ICML 2021 paper "Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline"

A PyTorch toolkit for 2D Human Pose Estimation.

Pure python implementation reverse-mode automatic differentiation

A pytorch-version implementation codes of paper: "BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generation"

GLM (General Language Model)

Pytorch domain adaptation package

GPU-accelerated Image Processing library using OpenCL

Highway networks implemented in PyTorch.

Deep Halftoning with Reversible Binary Pattern

Apply AnimeGAN-v2 across frames of a video clip

Official PyTorch implementation of paper: Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentation (ICCV 2021 Oral Presentation)

potpourri3d - An invigorating blend of 3D geometry tools in Python.

Code release for NeRF (Neural Radiance Fields)

PyTorch 1.0 inference in C++ on Windows10 platforms

DeepAL: Deep Active Learning in Python

Similarity-based Gray-box Adversarial Attack Against Deep Face Recognition