Utils to quickly evaluate many 🤗 models on the GLUE tasks

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Setup

wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/pytorch/text-classification/run_glue.py
conda env create -f environment.yaml
conda activate glue-utils

Evaluate multiple models on every glue task:

python runner.py \
    path/to/model_1/ \
    path/to/model_2/ \
    path/to/model_3/ \
    --gpu_idx 1

This will store results in path/to/model_N/glue/[task_name]/, for each model, for each task.

To quickly gather all results, run:

python gather_scores.py \
    path/to/model_1/glue/ \
    path/to/model_2/glue/ \
    path/to/model_3/glue/
Owner
Przemyslaw K. Joniak
Grad student at The University of Tokyo
Przemyslaw K. Joniak
Яндекс тренировки по алгоритмам. Июнь 2021

Young&&Yandex Тренировки по алгоритмам Если вы хотите попасть на летнюю стажировку в Яндекс, но пока не уверены в своих силах, приходите на наши трени

Podlevskiy Viktor 6 Sep 03, 2021
Repo to demo translating colab/jupyter notebook to streamlit webapp

Repo to demo translating colab/jupyter notebook to streamlit webapp

Marisa Smith 2 Feb 02, 2022
A short course on Julia and open-source software development

Advanced Scientific Computing: producing better code This course is taught as a 6-session "nanocourse" at Washington University in St. Louis. See the

Tim Holy 230 Jan 07, 2023
Fabric mod where anyone can PR anything, concerning or not. I'll merge everything as soon as it works.

Guess What Will Happen In This Fabric mod where anyone can PR anything, concerning or not (Unless it's too concerning). I'll merge everything as soon

anatom 65 Dec 25, 2022
This is a Saleae Logic custom high level analyzer that allows you to search and mark specific packets.

SaleaePacketParser This is a Saleae Logic custom high level analyzer that allows you to search and mark specific packets. Field "Search For" is used f

1 Dec 16, 2021
适用于HoshinoBot下的人生重来模拟器插件

LifeRestart for HoshinoBot 原作地址 python版原地址 本项目地址 安装方法 这是一个HoshinoBot的人生重来模拟器插件 这个项目使用的HoshinoBot的消息触发器,如果你了解其他机器人框架的api(比如nonebot)可以只修改消息触发器就将本项目移植到其他

黛笙笙 16 Sep 03, 2022
Python Repository for Bachelor Ski Sign.

BachelorSkiSign Python Repository for Bachelor Ski Sign. This application reads data from https://bachelorapi.azurewebsites.net/ It is written in Ciru

Winston 1 Jan 04, 2022
A price calculator for multiple things

Price Calculator A price calculator for multiple things Example I have 0.0567kg diamond. The price of diamond in kg is: $4500. Then it says: The price

Abel 1 Nov 26, 2021
py2dis - A disassembly engine & library for Python

py2dis - A disassembly engine & library for Python. py2dis is a disassembly library for Python that does not use any modules/libraries other than colo

3 Feb 04, 2022
Is a util for xferring skinning from one mesh to another

maya_pythonplugins skinTo: Is a util for xferring skinning from one mesh to another args: :param maxInfluences: is the number of max influences on the

James Dunlop 2 Jan 24, 2022
A tool for RaceRoom Racing Experience which shows you launch data

R3E Launch Tool A tool for RaceRoom Racing Experience which shows you launch data. Usage Run the tool, change the Stop Speed to whatever you want, and

Yuval Rosen 2 Feb 01, 2022
x-tools is a collection of tools developed in Python

x-tools X-tools is a collection of tools developed in Python Commands\

5 Jan 24, 2022
Example of my qtile config using the gruvbox colorscheme.

QTILE config Example of my qtile config using the gruvbox colorscheme. unicodes.py unicodes.py returns a widget.TextBox with a unicode. Currently it c

Imanuel Febie 31 Jan 02, 2023
A simple assembly- and brainfuck-inspired stack-based language

asm-stackfuck A simple assembly- and brainfuck-inspired stack-based language. The language has a few goals: Be stack-based Look like assembly Have a s

Nils Trinity 1 Feb 06, 2022
Open source stenotype engine

Plover Bringing stenography to everyone. Homepage Releases Wiki Blog Google Group Discord Chat About Installation Getting help Contributing Donations

Open Steno Project 2k Jan 09, 2023
Anki for desktop computers

Anki This repo contains the source code for the computer version of Anki. If you'd like to try development builds of Anki but don't feel comfortable b

Ankitects 12.9k Jan 09, 2023
AminoAutoRegFxck/AutoReg For AminoApps.com

AminoAutoRegFxck AminoAutoRegFxck/AutoReg For AminoApps.com Termux apt update -y apt upgrade -y pkg install python git clone https://github.com/LilZev

3 Jan 18, 2022
An Advent calendar of small programming puzzles for a variety of skill sets and skill levels.

Advent of Code 2021 The Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be sol

Evan Cope 0 Feb 13, 2022
GitHub Actions Version Updater Updates All GitHub Action Versions in a Repository and Creates a Pull Request with the Changes.

GitHub Actions Version Updater GitHub Actions Version Updater is GitHub Action that is used to update other GitHub Actions in a Repository and create

Maksudul Haque 42 Dec 22, 2022