Utils to quickly evaluate many 🤗 models on the GLUE tasks

Overview

Utils to quickly evaluate many 🤗 models on the GLUE tasks

Setup

wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/pytorch/text-classification/run_glue.py
conda env create -f environment.yaml
conda activate glue-utils

Evaluate multiple models on every glue task:

python runner.py \
    path/to/model_1/ \
    path/to/model_2/ \
    path/to/model_3/ \
    --gpu_idx 1

This will store results in path/to/model_N/glue/[task_name]/, for each model, for each task.

To quickly gather all results, run:

python gather_scores.py \
    path/to/model_1/glue/ \
    path/to/model_2/glue/ \
    path/to/model_3/glue/
Owner
Przemyslaw K. Joniak
Grad student at The University of Tokyo
Przemyslaw K. Joniak
A reference implementation for processing the content.log files found at opendata.dwd.de/weather

A reference implementation for processing the content.log files found at opendata.dwd.de/weather.

Deutscher Wetterdienst (DWD) 6 Nov 26, 2022
OB_Template is a vault template reference for using Obsidian.

Obsidian Template OB_Template is a vault template reference for using Obsidian. If you've tested out Obsidian. and worked through the "Obsidian Help"

323 Dec 27, 2022
this is a basic python project that I made using python

this is a basic python project that I made using python. This project is only for practice because my python skills are still newbie.

Elvira Firmansyah 2 Dec 14, 2022
PORTSCANNING-IN-PYTHON - A python threaded portscanner to scan websites and ipaddresses

PORTSCANNING-IN-PYTHON This is a python threaded portscanner to scan websites an

1 Feb 16, 2022
Pixelarticons - Pixel Art Icons made simple for Flutter, powered by pixelarticons and fontify

Pixelarticons - Pixel Art Icons made simple for Flutter, powered by pixelarticons and fontify

lask 16 Dec 12, 2022
fast_bss_eval is a fast implementation of the bss_eval metrics for the evaluation of blind source separation.

fast_bss_eval Do you have a zillion BSS audio files to process and it is taking days ? Is your simulation never ending ? Fear no more! fast_bss_eval i

Robin Scheibler 99 Dec 13, 2022
Similarity checking of sign languages

Similarity checking of sign languages This repository checks for similarity betw

Tonni Das Jui 1 May 13, 2022
Utility to play with ADCS, allows to request tickets and collect information about related objects

certi Utility to play with ADCS, allows to request tickets and collect information about related objects. Basically, it's the impacket copy of Certify

Eloy 185 Dec 29, 2022
python scripts and other files to generate induction encoder PCBs in Kicad

induction_encoder python scripts and other files to generate induction encoder PCBs in Kicad Targeting the Renesas IPS2200 encoder chips.

Taylor Alexander 8 Feb 16, 2022
Localization and multifractal properties of the long-range Kitaev chain in the presence of an Aubry-André-Harper modulation

This repository contains the code for the paper Localization and multifractal properties of the long-range Kitaev chain in the presence of an Aubry-André-Harper modulation.

Joana Fraxanet 2 Apr 17, 2022
Exploiting Linksys WRT54G using a vulnerability I found.

Exploiting Linksys WRT54G Exploit # Install the requirements. pip install -r requirements.txt ROUTER_HOST=192.169.1.1 ROUTER_USERNAME=admin ROUTER_P

Elon Gliksberg 31 May 29, 2022
Absolute solvation free energy calculations with OpenFF and OpenMM

ABsolute SOLVantion Free Energy Calculations The absolv framework aims to offer a simple API for computing the change in free energy when transferring

7 Dec 07, 2022
Create N Share is a No Code solution which gives users the ability to create any type of feature rich survey forms with ease.

create n share Note : The Project Scaffold will be pushed soon. Create N Share is a No Code solution which gives users the ability to create any type

Chiraag Kakar 11 Dec 03, 2022
Bible-App : Simple Tool To Show Bible Books

Bible App Simple Tool To Show Bible Books Socials: Language:

ميخائيل 5 Jan 18, 2022
🤖🧭Creates google-like navigation menu using python-telegram-bot wrapper

python telegram bot menu pagination Makes a google style pagination line for a list of items. In other words it builds a menu for navigation if you ha

Sergey Smirnov 9 Nov 27, 2022
MeepoBenchmark - This project aims at providing the scripts, logs, and analytic results for Meepo Blockchain

MeepoBenchmark - This project aims at providing the scripts, logs, and analytic results for Meepo Blockchain

Peilin Zheng 3 Aug 16, 2022
The worst and slowest programming language you have ever seen

VenumLang this is a complete joke EXAMPLE: fizzbuzz in venumlang x = 0

Venum 7 Mar 12, 2022
Object-oriented programming (OOP) is a method of structuring a program by bundling related properties and behaviors into individual objects. In this tutorial, you’ll learn the basics of object-oriented programming in Python.

06_Python_Object_Class Introduction 👋 Objected oriented programming as a discipline has gained a universal following among developers. Python, an in-

Milaan Parmar / Милан пармар / _米兰 帕尔马 239 Dec 20, 2022
Kivy program for identification & rotation sensing of objects on multi-touch tables.

ObjectViz ObjectViz is a multitouch object detection solution, enabling you to create physical markers out of any reliable multitouch solution. It's e

TangibleDisplay 8 Apr 04, 2022
A hackers attempt at an MVP anki plugin

my anki plugin if you have found this by accident, you should probably run away this is nothing more than a hackers attempt at an MVP anki plugin I re

Chris Hall 1 Nov 02, 2021