CoSENT 比Sentence-BERT更有效的句向量方案

Last update: Dec 12, 2022

Related tags

Text Data & NLP CoSENT

Overview

CoSENT

比Sentence-BERT更有效的句向量方案

介绍

博客：https://kexue.fm/archives/8847
数据：https://github.com/bojone/BERT-whitening/tree/main/chn

效果

train训练、test测试：

	ATEC	BQ	LCQMC	PAWSX	STS-B	Avg
BERT+CoSENT	49.74	72.38	78.69	60.00	80.14	68.19
Sentence-BERT	46.36	70.36	78.72	46.86	66.41	61.74
RoBERTa+CoSENT	50.81	71.45	79.31	61.56	81.13	68.85
Sentence-RoBERTa	48.29	69.99	79.22	44.10	72.42	62.80

NLI训练、test测试：

	ATEC	BQ	LCQMC	PAWSX	STS-B	Avg
BERT+CoSENT	28.93	41.84	66.07	20.49	73.91	46.25
Sentence-BERT	28.19	42.73	64.98	15.38	**74.88	45.23
RoBERTa+CoSENT	31.84	46.65	68.43	20.89	74.37	48.43
Sentence-RoBERTa	31.87	45.60	67.89	15.64	73.93	46.99

环境

需要bert4keras >= 0.10.8。个人实验环境是tensorflow 1.15 + keras 2.3.1 + bert4keras 0.10.8。

交流

QQ交流群：808623966，微信群请加机器人微信号spaces_ac_cn

Owner

苏剑林(Jianlin Su)

科学爱好者

GitHub Repository

Harvis is designed to automate your C2 Infrastructure.

Harvis Harvis is designed to automate your C2 Infrastructure, currently using Mythic C2. 📌 What is it? Harvis is a python tool to help you create mul

99 Oct 06, 2022

A Facebook Messenger Chatbot using NLP

A Facebook Messenger Chatbot using NLP This project is about creating a messenger chatbot using basic NLP techniques and models like Logistic Regressi

6 Nov 20, 2022

JaQuAD: Japanese Question Answering Dataset

JaQuAD: Japanese Question Answering Dataset for Machine Reading Comprehension (2022, Skelter Labs)

84 Dec 27, 2022

Code for our ACL 2021 (Findings) Paper - Fingerprinting Fine-tuned Language Models in the wild .

🌳 Fingerprinting Fine-tuned Language Models in the wild This is the code and dataset for our ACL 2021 (Findings) Paper - Fingerprinting Fine-tuned La

5 Sep 13, 2022

Which Apple Keeps Which Doctor Away? Colorful Word Representations with Visual Oracles

Which Apple Keeps Which Doctor Away? Colorful Word Representations with Visual Oracles (TASLP 2022)

3 Apr 14, 2022

Easy-to-use CPM for Chinese text generation

CPM 项目描述 CPM（Chinese Pretrained Models）模型是北京智源人工智能研究院和清华大学发布的中文大规模预训练模型。官方发布了三种规模的模型，参数量分别为109M、334M、2.6B，用户需申请与通过审核，方可下载。由于原项目需要考虑大模型的训练和使用，需要安装较为复杂

382 Jan 07, 2023

A method for cleaning and classifying text using transformers.

NLP Translation and Classification The repository contains a method for classifying and cleaning text using NLP transformers. Overview The input data

0 Nov 15, 2022

Concept Modeling: Topic Modeling on Images and Text

Concept is a technique that leverages CLIP and BERTopic-based techniques to perform Concept Modeling on images.

120 Dec 27, 2022

Simple Text-To-Speech Bot For Discord

Simple Text-To-Speech Bot For Discord This is a very simple TTS bot for discord made with python. For this bot you need FFMPEG, see installation to se

1 Sep 26, 2022

Textpipe: clean and extract metadata from text

textpipe: clean and extract metadata from text textpipe is a Python package for converting raw text in to clean, readable text and extracting metadata

298 Nov 21, 2022

The RWKV Language Model

RWKV-LM We propose the RWKV language model, with alternating time-mix and channel-mix layers: The R, K, V are generated by linear transforms of input,

877 Jan 05, 2023

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis Jungil Kong, Jaehyeon Kim, Jaekyoung Bae In our paper, we p

1.1k Jan 02, 2023

Unofficial Python library for using the Polish Wordnet (plWordNet / Słowosieć)

Polish Wordnet Python library Simple, easy-to-use and reasonably fast library for using the Słowosieć (also known as PlWordNet) - a lexico-semantic da

12 Dec 23, 2022

Code and checkpoints for training the transformer-based Table QA models introduced in the paper TAPAS: Weakly Supervised Table Parsing via Pre-training.

End-to-end neural table-text understanding models.

914 Jan 07, 2023