A 30000+ Chinese MRC dataset - Delta Reading Comprehension Dataset

Related tags

Text Data & NLPDRCD
Overview

Delta Reading Comprehension Dataset

台達閱讀理解資料集 Delta Reading Comprehension Dataset (DRCD) 屬於通用領域繁體中文機器閱讀理解資料集。 本資料集期望成為適用於遷移學習之標準中文閱讀理解資料集。 本資料集從2,108篇維基條目中整理出10,014篇段落,並從段落中標註出30,000多個問題

關於資料集之更詳細資訊請洽詢論文: For more information please refer to Paper https://arxiv.org/abs/1806.00920

Data format 資料格式

  • version : 資料集版本
  • data :
    • title : : 文章標題
    • id : : 文章編號
    • paragraphs :
      • id : : 文章編號_段落編號
      • context : : 段落內容
      • qas :
        • question : : 問題內容
        • id : : 文章編號_段落編號_問題編號
        • answers :
          • answer_start : text在文中位置
          • id : : "1"表示為人工標註的答案,"2"以上為人工答題的答案
          • text : : 答案內容

Example

{
"version": "1.3",
"data": [
  {
    "title": "基督新教",
    "id": "2128",
    "paragraphs": [
      {
        "context": "基督新教與天主教均繼承普世教會歷史上許多傳統教義,如三位一體、聖經作為上帝的啟示、原罪、認罪、最後審判等等,但有別於天主教和東正教,新教在行政上沒有單一組織架構或領導,而且在教義上強調因信稱義、信徒皆祭司, 以聖經作為最高權威,亦因此否定以教宗為首的聖統制、拒絕天主教教條中關於聖傳與聖經具同等地位的教導。新教各宗派間教義不盡相同,但一致認同五個唯獨:唯獨恩典:人的靈魂得拯救唯獨是神的恩典,是上帝送給人的禮物。唯獨信心:人唯獨藉信心接受神的赦罪、拯救。唯獨基督:作為人類的代罪羔羊,耶穌基督是人與上帝之間唯一的調解者。唯獨聖經:唯有聖經是信仰的終極權威。唯獨上帝的榮耀:唯獨上帝配得讚美、榮耀",
        "id": "2128-2",
        "qas": [
          {
            "id": "2128-2-1",
            "question": "新教在教義上強調信徒皆祭司以及什麼樣的理念?",
            "answers": [
              {
                "id": "1",
                "text": "因信稱義",
                "answer_start": 92
              }
            ]
          },
          {
            "id": "2128-2-2",
            "question": "哪本經典為新教的最高權威?",
            "answers": [
              {
                "id": "1",
                "text": "聖經",
                "answer_start": 105
              }
            ]
          },
          {
            "id": "2128-2-3",
            "question": "新教認同幾個唯獨?",
            "answers": [
              {
                "id": "1",
                "text": "五個",
                "answer_start": 171
              }
            ]
          },
          {
            "id": "2128-2-4",
            "question": "文中提及,人唯獨藉信心接受神的赦罪、拯救,此為哪一種唯獨?",
            "answers": [
              {
                "id": "1",
                "text": "唯獨信心",
                "answer_start": 206
              }
            ]
          }
        ]
      },
      {
        "context": "主教制源自天主教的主教制度,幾乎和天主教的主教制度一模一樣,唯一不同的是主教亦可以結婚。天主教的主教制是在使徒們去世後於第二、三世紀興起的主教制度,所以可以說主教制是整個基督宗教中歷史最悠久的神職人員制度。現在行主教制的新教教會已經很少,聖公會就是沿用主教制,從教會制度和禮儀上看來,聖公會基本上屬大公教會傳統。路德宗和衛理公會則由各區會自行選擇使用主教制還是長老制;在香港和澳門,路德會和衛理公會就選用了長老制。然而,在歐洲,例如瑞典、芬蘭、挪威、德國等地,他們則通常採用主教制。長老制,是一個以議會形式管理區會的制度。議會內的成員由各教會選出長老,代表該教會出席會議。顧名思義,長老會就是採用長老制的教會。採用長老制的教會有基督教改革宗長老會、台灣基督長老教會、韓國基督長老教會等。",
        "id": "2128-3",
        "qas": [
          {
            "id": "2128-3-1",
            "question": "新教的主教制度源自於哪一教?",
            "answers": [
              {
                "id": "1",
                "text": "天主教",
                "answer_start": 5
              }
            ]
          },
          {
            "id": "2128-3-2",
            "question": "文中提及,新教的主教可以做什麼?",
            "answers": [
              {
                "id": "1",
                "text": "結婚",
                "answer_start": 41
              }
            ]
          },
          {
            "id": "2128-3-3",
            "question": "哪個會屬於大公教會傳統?",
            "answers": [
              {
                "id": "1",
                "text": "聖公會",
                "answer_start": 142
              }
            ]
          },
          {
            "id": "2128-3-4",
            "question": "以議會形式管理區會的制度,名為?",
            "answers": [
              {
                "id": "1",
                "text": "長老制",
                "answer_start": 241
              }
            ]
          }
        ]
      }
    ]
  }
]
}

Copyright Notice 版權聲明

本資料集整理、改編自維基百科,其內容以CC-BY-SA 3.0條款發布。 台達電子對於本資料集內容之正確性不為任何擔保,且不就因使用或倚賴本資料集而引致的任何損失,承擔任何責任。 CC-BY-SA 3.0相關條款請參考以下連結 http://creativecommons.org/licenses/by-sa/3.0/

DRCD is compiled and adapted from Wikipedia and its content is published under the terms of CC-BY-SA 3.0. Delta Electronics, Inc. makes no representations or warranties of the correctness of the contents of DRCD and will not be liable for any loss or damage arising from the use or reliance on DRCD.

CC-BY-SA 3.0 can be found at http://creativecommons.org/licenses/by-sa/3.0/

Contact us 聯繫我們

You might also like...
DELTA is a deep learning based natural language and speech processing platform.
DELTA is a deep learning based natural language and speech processing platform.

DELTA - A DEep learning Language Technology plAtform What is DELTA? DELTA is a deep learning based end-to-end natural language and speech processing p

CDLA: A Chinese document layout analysis (CDLA) dataset
CDLA: A Chinese document layout analysis (CDLA) dataset

CDLA: A Chinese document layout analysis (CDLA) dataset 介绍 CDLA是一个中文文档版面分析数据集,面向中文文献类(论文)场景。包含以下10个label: 正文 标题 图片 图片标题 表格 表格标题 页眉 页脚 注释 公式 Text Title

A 10000+ hours dataset for Chinese speech recognition
A 10000+ hours dataset for Chinese speech recognition

A 10000+ hours dataset for Chinese speech recognition

Reading Wikipedia to Answer Open-Domain Questions
Reading Wikipedia to Answer Open-Domain Questions

DrQA This is a PyTorch implementation of the DrQA system described in the ACL 2017 paper Reading Wikipedia to Answer Open-Domain Questions. Quick Link

This is my reading list for my PhD in AI, NLP, Deep Learning and more.

This is my reading list for my PhD in AI, NLP, Deep Learning and more.

Code repository for "It's About Time: Analog clock Reading in the Wild"

it's about time Code repository for "It's About Time: Analog clock Reading in the Wild" Packages required: pytorch (used 1.9, any reasonable version s

🐍 A hyper-fast Python module for reading/writing JSON data using Rust's serde-json.
🐍 A hyper-fast Python module for reading/writing JSON data using Rust's serde-json.

A hyper-fast, safe Python module to read and write JSON data. Works as a drop-in replacement for Python's built-in json module. This is alpha software

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.

The model is designed to train a single and large neural network in order to predict correct translation by reading the given sentence.
The model is designed to train a single and large neural network in order to predict correct translation by reading the given sentence.

Neural Machine Translation communication system The model is basically direct to convert one source language to another targeted language using encode

Comments
  • Evaluation problem

    Evaluation problem

    According to the paper:

    F1 score and exact match from Rajpurkar et al. (2016) are used as the evaluation metrics. Both metrics ignore punctuations. In F1 score metric, we consider predictions and ground truth as bag of Chinese character.

    Is the ignored punctuations including fullwidth? Is there a original evaluation script for this dataset?

    opened by penut85420 2
  • Format conversion to SQuAD2.0

    Format conversion to SQuAD2.0

    When you used Bert-Chinese model to do the DRCD tasks like your paper told us, is there anything such as format conversion that we need to do first, and then we can use Bert-Chinese model to do DRCD tasks ?

    p.s. Format conversion means that convert DRCD format to SQuAD2.0 format.

    opened by allenyummy 1
  • Training set problem

    Training set problem

    In the paper, it said that "the training set contains 26,932 questions in 8,014 paragraphs". However, after calculating I found that I got 26936 question's id in the json file.

    opened by Liangtaiwan 1
  • Dev problem

    Dev problem

    The dev set answers are duplicate in the same question. Also, in SQuAD dataset, it has 3 different answers in dev set and test set, so the human performance is much higher than your dataset in EM performance. Are you going to provide more answer?

    opened by Liangtaiwan 1
Releases(v1.0)
Python module (C extension and plain python) implementing Aho-Corasick algorithm

pyahocorasick pyahocorasick is a fast and memory efficient library for exact or approximate multi-pattern string search meaning that you can find mult

Wojciech Muła 763 Dec 27, 2022
Generate custom detailed survey paper with topic clustered sections and proper citations, from just a single query in just under 30 mins !!

Auto-Research A no-code utility to generate a detailed well-cited survey with topic clustered sections (draft paper format) and other interesting arti

Sidharth Pal 20 Dec 14, 2022
CLIPfa: Connecting Farsi Text and Images

CLIPfa: Connecting Farsi Text and Images OpenAI released the paper Learning Transferable Visual Models From Natural Language Supervision in which they

Sajjad Ayoubi 66 Dec 14, 2022
Multilingual word vectors in 78 languages

Aligning the fastText vectors of 78 languages Facebook recently open-sourced word vectors in 89 languages. However these vectors are monolingual; mean

Babylon Health 1.2k Dec 17, 2022
Mlcode - Continuous ML API Integrations

mlcode Basic APIs for ML applications. Django REST Application Contains REST API

Sujith S 1 Jan 01, 2022
CorNet Correlation Networks for Extreme Multi-label Text Classification

CorNet Correlation Networks for Extreme Multi-label Text Classification Prerequisites python==3.6.3 pytorch==1.2.0 torchgpipe==0.0.5 click==7.0 ruamel

Guangxu Xun 38 Dec 31, 2022
NLP Core Library and Model Zoo based on PaddlePaddle 2.0

PaddleNLP 2.0拥有丰富的模型库、简洁易用的API与高性能的分布式训练的能力,旨在为飞桨开发者提升文本建模效率,并提供基于PaddlePaddle 2.0的NLP领域最佳实践。

6.9k Jan 01, 2023
Bidirectional LSTM-CRF and ELMo for Named-Entity Recognition, Part-of-Speech Tagging and so on.

anaGo anaGo is a Python library for sequence labeling(NER, PoS Tagging,...), implemented in Keras. anaGo can solve sequence labeling tasks such as nam

Hiroki Nakayama 1.5k Dec 05, 2022
🌸 fastText + Bloom embeddings for compact, full-coverage vectors with spaCy

floret: fastText + Bloom embeddings for compact, full-coverage vectors with spaCy floret is an extended version of fastText that can produce word repr

Explosion 222 Dec 16, 2022
💛 Code and Dataset for our EMNLP 2021 paper: "Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes"

Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes Official PyTorch implementation and EmoCause evaluatio

Hyunwoo Kim 50 Dec 21, 2022
A python wrapper around the ZPar parser for English.

NOTE This project is no longer under active development since there are now really nice pure Python parsers such as Stanza and Spacy. The repository w

ETS 49 Sep 12, 2022
Code for "Generating Disentangled Arguments with Prompts: a Simple Event Extraction Framework that Works"

GDAP The code of paper "Code for "Generating Disentangled Arguments with Prompts: a Simple Event Extraction Framework that Works"" Event Datasets Prep

45 Oct 29, 2022
A Lightweight NLP Data Loader for All Deep Learning Frameworks in Python

LineFlow: Framework-Agnostic NLP Data Loader in Python LineFlow is a simple text dataset loader for NLP deep learning tasks. LineFlow was designed to

TofuNLP 177 Jan 04, 2023
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context This repository contains the code in both PyTorch and TensorFlow for our paper

Zhilin Yang 3.3k Dec 28, 2022
A paper list for aspect based sentiment analysis.

Aspect-Based-Sentiment-Analysis A paper list for aspect based sentiment analysis. Survey [IEEE-TAC-20]: Issues and Challenges of Aspect-based Sentimen

jiangqn 419 Dec 20, 2022
Official PyTorch implementation of Time-aware Large Kernel (TaLK) Convolutions (ICML 2020)

Time-aware Large Kernel (TaLK) Convolutions (Lioutas et al., 2020) This repository contains the source code, pre-trained models, as well as instructio

Vasileios Lioutas 28 Dec 07, 2022
Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformers

beyond masking Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformers The code is coming Figure 1: Pipeline of token-based pre-

Yunjie Tian 23 Sep 27, 2022
UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language

UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language This repository contains UA-GEC data and an accompanying Python lib

Grammarly 227 Jan 02, 2023
This is a modification of the OpenAI-CLIP repository of moein-shariatnia

This is a modification of the OpenAI-CLIP repository of moein-shariatnia

Sangwon Beak 2 Mar 04, 2022
:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

(Framework for Adapting Representation Models) What is it? FARM makes Transfer Learning with BERT & Co simple, fast and enterprise-ready. It's built u

deepset 1.6k Dec 27, 2022