Utils to quickly evaluate many 🤗 models on the GLUE tasks

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Setup

wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/pytorch/text-classification/run_glue.py
conda env create -f environment.yaml
conda activate glue-utils

Evaluate multiple models on every glue task:

python runner.py \
    path/to/model_1/ \
    path/to/model_2/ \
    path/to/model_3/ \
    --gpu_idx 1

This will store results in path/to/model_N/glue/[task_name]/, for each model, for each task.

To quickly gather all results, run:

python gather_scores.py \
    path/to/model_1/glue/ \
    path/to/model_2/glue/ \
    path/to/model_3/glue/
Owner
Przemyslaw K. Joniak
Grad student at The University of Tokyo
Przemyslaw K. Joniak
Yandex Media Browser

Браузер медиа для плагина Yandex Station Включайте музыку, плейлисты и радио на Яндекс.Станции из Home Assistant! Скриншот Корневой раздел: Библиотека

Alexander Ryazanov 35 Dec 19, 2022
A timer for bird lovers, plays a random birdcall while displaying its image and info.

Birdcall Timer A timer for bird lovers. Siriema hatchling by Junior Peres Junior Background My partner needed a customizable timer for sitting and sta

Marcelo Sanches 1 Jul 08, 2022
Python code for YouTube videos.

#This is a open source project. Python 3 These files are mainly intended to accompany my series of YouTube tutorial videos here, https://www.youtube.c

Joe James 1.3k Dec 26, 2022
Distributed behavioral experiments

Autopilot Docs Paper Forum Hardware Autopilot is a Python framework for performing complex, hardware-intensive behavioral experiments with swarms of n

70 Dec 14, 2022
This Python3 script will monitor Upwork RSS feed and then email you the results.

Upwork RSS Parser This Python3 script will monitor Upwork RSS feed and then email you the results. Table of Contents General Info Technologies Used Fe

Chris 5 Nov 29, 2021
Personal Chat Assistance

Python-Programming Personal Chat Assistance {% import "bootstrap/wtf.html" as wtf %} titleEVT/title script src="https://code.jquery.com/jquery-3.

PRASH_SMVIT 2 Nov 14, 2021
Sample python script for monitoring Rocketchat database and get statistics of users.

rocketchat-DB-monitoring Sample python script for monitoring Rocketchat database and get statistics of users. 1. Update python: yum check-update && yu

Mojtaba Taleghani 1 Apr 12, 2022
An easy way to access the Scratch API!

The majority of people are likely here because they want to easily access the Scratch API!

rgantzos 0 May 04, 2022
Manjaro CN Repository

Manjaro CN Repository Automatically built packages based on archlinuxcn/repo and manjarocn/docker. Install Add manjarocn to /etc/pacman.conf: Please m

Manjaro CN 28 Jun 26, 2022
My solutions for Advent of Code 2021 🌟🎄

🌟 Advent of Code 2021 🎄 My solutions for Advent of Code 2021. About · What is Advent of Code? · Contents · Usage · Table of puzzles (TODO: add final

Amanda P. Pinha 2 Dec 05, 2022
Understanding the field usage of any object in Salesforce

Understanding the field usage of any object in Salesforce One of the biggest problems that I have addressed while working with Salesforce is to unders

Sebastian Undurraga 1 Dec 14, 2021
Automated moth pictures for biodiversity research

Automated moth pictures for biodiversity research

Ludwig Kürzinger 1 Dec 16, 2021
A custom advent of code I am completing

advent-of-code-custom A custom advent of code I am doing in python. The link to the problems I am solving is here: https://github.com/seldoncode/Adven

Rocio PV 2 Dec 11, 2021
More granular intermediaries for legacy Minecraft versions

Orinthe/Intermediary mappings This repository contains the match information between different versions of Minecraft created by the Orinthe project, a

4 Jan 11, 2022
This repository requires you to solve a problem by writing some basic python code.

Can You Solve a Problem? A beginner friendly repository that requires you to solve familiar problems with python. This could be as simple as implement

Precious Kolawole 11 Nov 30, 2022
Online HackerRank problem solving challenges

LinkedListHackerRank Online HackerRank problem solving challenges This challenge is part of a tutorial track by MyCodeSchool You are given the pointer

Sefineh Tesfa 1 Nov 21, 2021
Estimating the potential photovoltaic production of buildings (in Berlin)

The following people contributed equally to this repository (in alphabetical order): Daniel Bumke JJX Corstiaen Versteegh This repository is forked on

Daniel Bumke 6 Feb 18, 2022
Unzip Japanese Shift-JIS zip archives on non-Japanese systems.

Unzip JP GUI Unzip Japanese Shift-JIS zip archives on non-Japanese systems. This script unzips the file while converting the file names from Shift-JIS

Emile Bangma 9 Dec 07, 2022
This code extracts line width of phonons from specular energy density (SED) calculated with LAMMPS.

This code extracts line width of phonons from specular energy density (SED) calculated with LAMMPS.

Masato Ohnishi 3 Jun 15, 2022
Collection of functions for working with interlaced content in VapourSynth.

vsfieldkit Collection of functions for working with interlaced content in VapourSynth. It does not have any hard dependencies outside of VapourSynth.

Justin Turner Arthur 11 May 27, 2022