LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Learning High-Speed Flight in the Wild

Current state of supervised and unsupervised depth completion methods

Weakly supervised medical named entity classification

Statistical-Rethinking-with-Python-and-PyMC3 - Python/PyMC3 port of the examples in " Statistical Rethinking A Bayesian Course with Examples in R and Stan" by Richard McElreath

code and data for paper "GIANT: Scalable Creation of a Web-scale Ontology"

Code for paper PairRE: Knowledge Graph Embeddings via Paired Relation Vectors.

Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression

Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark (ICCV 2021)

Pytorch Lightning code guideline for conferences

TensorFlow implementation of Adaptive Information Transfer Multi-task (AITM) framework. Code for the paper submitted to KDD21: Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning for Customer Acquisition.

Federated_learning codes used for the the paper "Evaluation of Federated Learning Aggregation Algorithms" and "A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison"

A GOOD REPRESENTATION DETECTS NOISY LABELS

Code to produce syntactic representations that can be used to study syntax processing in the human brain

RRxIO - Robust Radar Visual/Thermal Inertial Odometry: Robust and accurate state estimation even in challenging visual conditions.

BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis

A state-of-the-art semi-supervised method for image recognition

Pose estimation with MoveNet Lightning

PyTorch implementation of Progressive Growing of GANs for Improved Quality, Stability, and Variation.

A 2D Visual Localization Framework based on Essential Matrices [ICRA2020]

Diabetes-Feature-Engineering - A machine learning model that can predict whether people have diabetes when their characteristics are specified