项目说明:

百度2021年语言与智能技术竞赛机器阅读理解Pytorch版baseline
比赛链接:https://aistudio.baidu.com/aistudio/competition/detail/66?isFromLuge=true

官方的baseline版本是基于paddlepaddle框架的,我把它改写成了Pytorch框架,其中大部分代码沿用的是官方提供的代码,只是有一些框架部分进行了修改,另外增加了早停策略/对抗训练等优化措施,习惯用Pytorch版本的可以基于此进行优化.

环境

python=3.6
torch=1.7
transformers=4.5.0

训练示例

训练

python run.py
--max_len=256
--model_name_or_path=下载的预训练模型路径
--per_gpu_train_batch_size=7
--per_gpu_eval_batch_size=40
--learning_rate=1e-5
--linear_learning_rate=1e-4
--num_train_epochs=100
--output_dir="./output"
--weight_decay=0.01
--early_stop=2

预测

python predict.py
--max_len=400
--model_name_or_path=下载的预训练模型路径
--per_gpu_eval_batch_size=120
--output_dir="./output"
--fine_tunning_model=微调后的模型路径

实验结果

用的baseline模型是base版MacBERT(具体请看https://github.com/ymcui/MacBERT)

后续优化策略

数据清洗，据官方工作人员讲解到，训练集的准确率只能确保92%以上
更多的数据
更细粒度的数据增强
模型结构的优化

百度2021年语言与智能技术竞赛机器阅读理解Pytorch版baseline

Related tags

Overview

项目说明:

环境

训练示例

实验结果

后续优化策略

Owner

周俊贤

This is a official repository of SimViT.

An integration of several popular automatic augmentation methods, including OHL (Online Hyper-Parameter Learning for Auto-Augmentation Strategy) and AWS (Improving Auto Augment via Augmentation Wise Weight Sharing) by Sensetime Research.

This is the repository for the paper "Have I done enough planning or should I plan more?"

Learning Energy-Based Models by Diffusion Recovery Likelihood

A GOOD REPRESENTATION DETECTS NOISY LABELS

Tensorflow implementation of "BEGAN: Boundary Equilibrium Generative Adversarial Networks"

Let Python optimize the best stop loss and take profits for your TradingView strategy.

Towards Ultra-Resolution Neural Style Transfer via Thumbnail Instance Normalization

Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning

Tree Nested PyTorch Tensor Lib

Offline Reinforcement Learning with Implicit Q-Learning

This is an official implementation of the CVPR2022 paper "Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots".

Non-Imaging Transient Reconstruction And TEmporal Search (NITRATES)

Adversarial Learning for Modeling Human Motion

Source for the paper "Universal Activation Function for machine learning"

AI pipelines for Nvidia Jetson Platform

Self-Supervised Learning of Event-based Optical Flow with Spiking Neural Networks

Canonical Appearance Transformations

This reposityory contains the PyTorch implementation of our paper "Generative Dynamic Patch Attack".

A machine learning library for spiking neural networks. Supports training with both torch and jax pipelines, and deployment to neuromorphic hardware.