项目说明:

百度2021年语言与智能技术竞赛机器阅读理解Pytorch版baseline
比赛链接:https://aistudio.baidu.com/aistudio/competition/detail/66?isFromLuge=true

官方的baseline版本是基于paddlepaddle框架的,我把它改写成了Pytorch框架,其中大部分代码沿用的是官方提供的代码,只是有一些框架部分进行了修改,另外增加了早停策略/对抗训练等优化措施,习惯用Pytorch版本的可以基于此进行优化.

环境

python=3.6
torch=1.7
transformers=4.5.0

训练示例

训练

python run.py
--max_len=256
--model_name_or_path=下载的预训练模型路径
--per_gpu_train_batch_size=7
--per_gpu_eval_batch_size=40
--learning_rate=1e-5
--linear_learning_rate=1e-4
--num_train_epochs=100
--output_dir="./output"
--weight_decay=0.01
--early_stop=2

预测

python predict.py
--max_len=400
--model_name_or_path=下载的预训练模型路径
--per_gpu_eval_batch_size=120
--output_dir="./output"
--fine_tunning_model=微调后的模型路径

实验结果

用的baseline模型是base版MacBERT(具体请看https://github.com/ymcui/MacBERT)

后续优化策略

数据清洗，据官方工作人员讲解到，训练集的准确率只能确保92%以上
更多的数据
更细粒度的数据增强
模型结构的优化

百度2021年语言与智能技术竞赛机器阅读理解Pytorch版baseline

Related tags

Overview

项目说明:

环境

训练示例

实验结果

后续优化策略

Owner

周俊贤

Pytorch implementation AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

Python package to generate image embeddings with CLIP without PyTorch/TensorFlow

Code for Domain Adaptive Video Segmentation via Temporal Consistency Regularization in ICCV 2021

Efficient-GlobalPointer - Pytorch Efficient GlobalPointer

Models Supported: AlbUNet [18, 34, 50, 101, 152] (1D and 2D versions for Single and Multiclass Segmentation, Feature Extraction with supports for Deep Supervision and Guided Attention)

Code release for Local Light Field Fusion at SIGGRAPH 2019

Raster Vision is an open source Python framework for building computer vision models on satellite, aerial, and other large imagery sets

Hyper-parameter optimization for sklearn

Open-Domain Question-Answering for COVID-19 and Other Emergent Domains

Implementation of our paper "Video Playback Rate Perception for Self-supervised Spatio-Temporal Representation Learning".

VM3000 Microphones

OverFeat is a Convolutional Network-based image classifier and feature extractor.

Pytorch implementation of our paper LIMUSE: LIGHTWEIGHT MULTI-MODAL SPEAKER EXTRACTION.

BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training

An efficient PyTorch implementation of the winning entry of the 2017 VQA Challenge.

Automated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks

Official implementation of Self-supervised Graph Attention Networks (SuperGAT), ICLR 2021.

Code for ECCV 2020 paper "Contacts and Human Dynamics from Monocular Video".

Efficient Training of Audio Transformers with Patchout

3D Human Pose Machines with Self-supervised Learning