How Robust are Fact Checking Systems
on Colloquial Claims?

Official PyTorch implementation of our NAACL paper:
Byeongchang Kim*, Hyunwoo Kim*, Seokhee Hong, and Gunhee Kim. How Robust are Fact Checking Systems on Colloquial Claims? NAACL-HLT, 2021 [Paper] (* equal contribution)

If you use the materials in this repository as part of any published research, we ask you to cite the following paper:

@inproceedings{Kim:2021:colloquial,
  title={How Robust are Fact Checking Systems on Colloquial Claims?},
  author={Kim, Byeongchang and Kim, Hyunwoo and Hong, Seokhee and Kim, Gunhee},
  booktitle={NAACL-HLT},
  year={2021}
}

Colloquial Claims dataset

You can download the paper version of our Colloquial Claims dataset via following urls:
[train] [valid] [test]

You can read and explore the dataset as follows:

import json

turns = []
with open('colloquial_claims_train.jsonl', 'r') as fp:
    for line in fp:
        turns.append(json.loads(line))

print(turns[0].keys())
# dict_keys(['colloquial_claims', 'fever_claim', 'fever_label', 'evidences', 'gold_evidence_set', 'fever_id'])

Running style transfer pipeline

In progress