LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Poisson Surface Reconstruction for LiDAR Odometry and Mapping

Pytorch implementation of Learning with Opponent-Learning Awareness

[CVPR 2021] Generative Hierarchical Features from Synthesizing Images

A machine learning malware analysis framework for Android apps.

Official project repository for 'Normality-Calibrated Autoencoder for Unsupervised Anomaly Detection on Data Contamination'

A port of muP to JAX/Haiku

TagLab: an image segmentation tool oriented to marine data analysis

Official PyTorch implementation of "Improving Face Recognition with Large AgeGaps by Learning to Distinguish Children" (BMVC 2021)

Cartoon-StyleGan2 🙃 : Fine-tuning StyleGAN2 for Cartoon Face Generation

PyTorch implementation of PP-LCNet: A Lightweight CPU Convolutional Neural Network

HNECV: Heterogeneous Network Embedding via Cloud model and Variational inference

Official implementation of Rich Semantics Improve Few-Shot Learning (BMVC, 2021)

A transformer which can randomly augment VOC format dataset (both image and bbox) online.

Analysing poker data from home games with friends

PyTorch implementation of Glow

LF-YOLO (Lighter and Faster YOLO) is used to detect defect of X-ray weld image.

An AI Assistant More Than a Toolkit

STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speech

Learning to Segment Instances in Videos with Spatial Propagation Network

OCRA (Object-Centric Recurrent Attention) source code