LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Minimal deep learning library written from scratch in Python, using NumPy/CuPy.

Bagua is a flexible and performant distributed training algorithm development framework.

(ICCV 2021) ProHMR - Probabilistic Modeling for Human Mesh Recovery

A model to classify a piece of news as REAL or FAKE

A PyTorch implementation of "Predict then Propagate: Graph Neural Networks meet Personalized PageRank" (ICLR 2019).

A PyTorch implementation of deep-learning-based registration

CT Based COVID 19 Diagnose by Image Processing and Deep Learning

A TensorFlow implementation of the Mnemonic Descent Method.

A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains (IJCV submission)

Official implementations of PSENet, PAN and PAN++.

Domain Generalization for Mammography Detection via Multi-style and Multi-view Contrastive Learning

Accelerating BERT Inference for Sequence Labeling via Early-Exit

PIXIE: Collaborative Regression of Expressive Bodies

Airbus Ship Detection Challenge

A simple baseline for 3d human pose estimation in tensorflow. Presented at ICCV 17.

x-transformers-paddle 2.x version

disentanglement_lib is an open-source library for research on learning disentangled representations.

VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

PyTorch Implementations for DeeplabV3 and PSPNet

Facilitating Database Tuning with Hyper-ParameterOptimization: A Comprehensive Experimental Evaluation