The evaluator covering all of the metrics required by tasks within the DUE Benchmark.

Related tags

Testingevaluator
Overview

DUE Evaluator

The repository contains the evaluator covering all of the metrics required by tasks within the DUE Benchmark, i.e., set-based F1 (for KIE), ANLS (used in document VQA), accuracy (including variant used in WTQ), as well as group-based ANLS we proposed for KIE problems with structured output.

Usage

The deval command will be available after the package installation. Every time, it is required to provide input and output files (both in the DU-Schema format) using -o and -r parameters.

Other settings are task-specific and limited to metric (-m) and optional case-insensitiveness (-i). Recommended values of these are:

Dataset Metric Case insensitive
DocVQA, InfographicsVQA ANLS Yes
Kleister Charity, DeepForm F1 Yes
PapersWithCode GROUP-ANLS Yes
WikiTableQuestions WTQ No (handled by metric itself)
TabFact F1 (obtained value will be equal to Accuracy) No
Owner
DUE Benchmark
The benchmark consisting of both available and reformulated datasets to measure the end-to-end capabilities of systems in real-world scenarios.
DUE Benchmark
GitHub Repository
A simple Python script I wrote that scrapes NASA's James Webb Space Telescope tracker website using Selenium and returns its current status and location.

A simple Python script I wrote that scrapes NASA's James Webb Space Telescope tracker website using Selenium and returns its current status and location.

9 Feb 10, 2022
The Penetration Testers Framework (PTF) is a way for modular support for up-to-date tools.

The PenTesters Framework (PTF) is a Python script designed for Debian/Ubuntu/ArchLinux based distributions to create a similar and familiar distribution for Penetration Testing

trustedsec 4.5k Dec 28, 2022
bulk upload files to libgen.lc (Selenium script)

LibgenBulkUpload bulk upload files to http://libgen.lc/librarian.php (Selenium script) Usage ./upload.py to_upload uploaded rejects So title and autho

8 Jul 07, 2022
Active Directory Penetration Testing methods with simulations

AD penetration Testing Project By Ruben Enkaoua - GL4Di4T0R Based on the TCM PEH course (Heath Adams) Index 1 - Setting Up the Lab Intallation of a Wi

GL4DI4T0R 3 Aug 12, 2021
Doing dirty (but extremely useful) things with equals.

Doing dirty (but extremely useful) things with equals. Documentation: dirty-equals.helpmanual.io Source Code: github.com/samuelcolvin/dirty-equals dir

Samuel Colvin 602 Jan 05, 2023
🐍 Material for PyData Global 2021 Presentation: Effective Testing for Machine Learning Projects

Effective Testing for Machine Learning Projects Code for PyData Global 2021 Presentation by @edublancas. Slides available here. The project is develop

Eduardo Blancas 73 Nov 06, 2022
A automated browsing experience.

browser-automation This app is an automated browsing technique where one has to enter the required information, it's just like searching for Animals o

Ojas Barawal 3 Aug 04, 2021
A single module to link Python ecosystem to the Web

A single module to link Python ecosystem to the Web. Have a quick look at the Gallery first to get convinced ! FAQ For any questions, please use Stack

66 Dec 21, 2022
Python Rest Testing

pyresttest Table of Contents What Is It? Status Installation Sample Test Examples Installation How Do I Use It? Running A Simple Test Using JSON Valid

Sam Van Oort 1.1k Dec 28, 2022
Test django schema and data migrations, including migrations' order and best practices.

django-test-migrations Features Allows to test django schema and data migrations Allows to test both forward and rollback migrations Allows to test th

wemake.services 382 Dec 27, 2022
Automated tests for OKAY websites in Python (Selenium) - user friendly version

Okay Selenium Testy Aplikace určená k testování produkčních webů společnosti OKAY s.r.o. Závislosti K běhu aplikace je potřeba mít v počítači nainstal

Viktor Bem 0 Oct 01, 2022
hCaptcha solver and bypasser for Python Selenium. Simple website to try to solve hCaptcha.

hCaptcha solver for Python Selenium. Many thanks to engageub for his hCaptcha solver userscript. This script is solely intended for the use of educati

Maxime Dréan 59 Dec 25, 2022
Automatic SQL injection and database takeover tool

sqlmap sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of

sqlmapproject 25.7k Jan 04, 2023
The async ready version of the AniManga library created by centipede000.

Async-Animanga An Async/Aiohttp compatible library. Async-Animanga is an async ready web scraping library that returns Manga information from animepla

3 Sep 22, 2022
PyAutoEasy is a extension / wrapper around the famous PyAutoGUI, a cross-platform GUI automation tool to replace your boooring repetitive tasks.

PyAutoEasy PyAutoEasy is a extension / wrapper around the famous PyAutoGUI, a cross-platform GUI automation tool to replace your boooring repetitive t

Dingu Sagar 7 Oct 27, 2022
a plugin for py.test that changes the default look and feel of py.test (e.g. progressbar, show tests that fail instantly)

pytest-sugar pytest-sugar is a plugin for pytest that shows failures and errors instantly and shows a progress bar. Requirements You will need the fol

Teemu 963 Dec 28, 2022
Bayesian A/B testing

bayesian_testing is a small package for a quick evaluation of A/B (or A/B/C/...) tests using Bayesian approach.

Matus Baniar 35 Dec 15, 2022
Selenium Page Object Model with Python

Page-object-model (POM) is a pattern that you can apply it to develop efficient automation framework.

Mohammad Ifran Uddin 1 Nov 29, 2021
Just a small test with lists in cython

Test for lists in cython Algorithm create a list of 10^4 lists each with 10^4 floats values (namely: 0.1) - 2 nested for iterate each list and compute

Federico Simonetta 32 Jul 23, 2022
Make Selenium work on Github Actions

Make Selenium work on Github Actions Scraping with BeautifulSoup on GitHub Actions is easy-peasy. But what about Selenium?? After you jump through som

Jonathan Soma 33 Dec 27, 2022