Chinese NER(Named Entity Recognition) using BERT(Softmax, CRF, Span)

Last update: Jan 03, 2023

Overview

Chinese NER using Bert

BERT for Chinese NER.

dataset list

cner: datasets/cner
CLUENER: https://github.com/CLUEbenchmark/CLUENER

model list

BERT+Softmax
BERT+CRF
BERT+Span

requirement

1.1.0 =< PyTorch < 1.5.0
cuda=9.0
python3.6+

input format

Input format (prefer BIOS tag scheme), with each character its label for one line. Sentences are splited with a null line.

美	B-LOC
国	I-LOC
的	O
华	B-PER
莱	I-PER
士	I-PER

我	O
跟	O
他	O

run the code

Modify the configuration information in run_ner_xxx.py or run_ner_xxx.sh .
sh scripts/run_ner_xxx.sh

note: file structure of the model

├── prev_trained_model
|  └── bert_base
|  |  └── pytorch_model.bin
|  |  └── config.json
|  |  └── vocab.txt
|  |  └── ......

CLUENER result

The overall performance of BERT on dev:

	Accuracy (entity)	Recall (entity)	F1 score (entity)
BERT+Softmax	0.7897	0.8031	0.7963
BERT+CRF	0.7977	0.8177	0.8076
BERT+Span	0.8132	0.8092	0.8112
BERT+Span+adv	0.8267	0.8073	0.8169
BERT-small(6 layers)+Span+kd	0.8241	0.7839	0.8051
BERT+Span+focal_loss	0.8121	0.8008	0.8064
BERT+Span+label_smoothing	0.8235	0.7946	0.8088

ALBERT for CLUENER

The overall performance of ALBERT on dev:

model	version	Accuracy(entity)	Recall(entity)	F1(entity)	Train time/epoch
albert	base_google	0.8014	0.6908	0.7420	0.75x
albert	large_google	0.8024	0.7520	0.7763	2.1x
albert	xlarge_google	0.8286	0.7773	0.8021	6.7x
bert	google	0.8118	0.8031	0.8074	-----
albert	base_bright	0.8068	0.7529	0.7789	0.75x
albert	large_bright	0.8152	0.7480	0.7802	2.2x
albert	xlarge_bright	0.8222	0.7692	0.7948	7.3x

Cner result

The overall performance of BERT on dev(test):

	Accuracy (entity)	Recall (entity)	F1 score (entity)
BERT+Softmax	0.9586(0.9566)	0.9644(0.9613)	0.9615(0.9590)
BERT+CRF	0.9562(0.9539)	0.9671(0.9644)	0.9616(0.9591)
BERT+Span	0.9604(0.9620)	0.9617(0.9632)	0.9611(0.9626)
BERT+Span+focal_loss	0.9516(0.9569)	0.9644(0.9681)	0.9580(0.9625)
BERT+Span+label_smoothing	0.9566(0.9568)	0.9624(0.9656)	0.9595(0.9612)

Chinese NER(Named Entity Recognition) using BERT(Softmax, CRF, Span)

Related tags

Overview

Chinese NER using Bert

dataset list

model list

requirement

input format

run the code

CLUENER result

ALBERT for CLUENER

Cner result

Owner

Weitang Liu

Transformer related optimization, including BERT, GPT

The training code for the 4th place model at MDX 2021 leaderboard A.

A Fast Sequence Transducer Implementation with PyTorch Bindings

Awesome Treasure of Transformers Models Collection

this repository has datasets containing information of Uber pickups in NYC from April 2014 to September 2014 and January to June 2015. data Analysis , virtualization and some insights are gathered here

Tools for curating biomedical training data for large-scale language modeling

Unsupervised text tokenizer focused on computational efficiency

The Easy-to-use Dialogue Response Selection Toolkit for Researchers

Training open neural machine translation models

Natural Language Processing at EDHEC, 2022

Dope Wars game engine on StarkNet L2 roll-up

Wind Speed Prediction using LSTMs in PyTorch

Build Text Rerankers with Deep Language Models

Code from the paper "High-Performance Brain-to-Text Communication via Handwriting"

Conversational-AI-ChatBot - Intelligent ChatBot built with Microsoft's DialoGPT transformer to make conversations with human users!

Behavioral Testing of Clinical NLP Models

Knowledge Graph,Question Answering System，基于知识图谱和向量检索的医疗诊断问答系统

InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective

CoNLL-English NER Task (NER in English)

Scene Text Retrieval via Joint Text Detection and Similarity Learning