This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
Scan Site - Tools For Scanning Any Site and Get Site Information

Site Scanner Tools For Scanning Any Site and Get Site Information Example Require - pip install colorama - pip install requests How To Use Download Th

NumeX 5 Mar 19, 2022
Directory Traversal in Afterlogic webmail aurora and pro

CVE-2021-26294 Exploit Directory Traversal in Afterlogic webmail aurora and pro . Description: AfterLogic Aurora and WebMail Pro products with 7.7.9 a

Ashish Kunwar 8 Nov 09, 2022
GDID (Google Dorks for Information Disclosure)

GDID (Google Dorks for Information Disclosure) Script made for your recon automation in Bug Bounty or Pentest. It will help you to find Information Di

Nischacid 5 Mar 10, 2022
Log4jScanner is a Log4j Related CVEs Scanner, Designed to Help Penetration Testers to Perform Black Box Testing on given subdomains.

Log4jScanner Log4jScanner is a Log4j Related CVEs Scanner, Designed to Help Penetration Testers to Perform Black Box Testing on given subdomains. Disc

Pushpender Singh 35 Dec 12, 2022
IDA scripts for hypervisor (Hyper-v) analysis and reverse engineering automation

Re-Scripts IA32-VMX-Helper (IDA-Script) IA32-MSR-Decoder (IDA-Script) IA32 VMX Helper It's an IDA script (Updated IA32 MSR Decoder) which helps you to

Behrooz Abbassi 16 Oct 08, 2022
🔍 IRIS: An open-source intelligence framework

IRIS is an open-source OSINT framework, consisting of modules to find information about a target by scraping sites and fetching data from APIs.

IRIS 79 Dec 20, 2022
Spring-0day/CVE-2022-22965

CVE-2022-22965 Spring Framework/CVE-2022-22965 Vulnerability ID: CVE-2022-22965/CNVD-2022-23942/QVD-2022-1691 Reproduce the vulnerability docker pull

iak 4 Apr 05, 2022
KeyLogger

By-Emirhan KeyLogger Hangi Sistemlerde Çalışır? | On Which Systems Does It Work? KALİ LİNUX UBUNTU PARDUS MİNT TERMUX ARCH YÜKLEME & ÇALIŞTIRMA KOMUTL

2 Feb 24, 2022
Discord-keylogger - Discord keylogger With Python

Discord-keylogger Usage python dlogger.py -t [Time interval in sec] if not speci

Satwik Sinha 1 Jan 30, 2022
A script to search, scrape and scan for Apache Log4j CVE-2021-44228 affected files using Google dorks

Log4j dork scanner This is an auto script to search, scrape and scan for Apache Log4j CVE-2021-44228 affected files using Google dorks. Installation:

Jagar 5 Dec 27, 2022
A Python Tool that uses Shodan API's to perform quick recon for vulnerabilities

Shodan Quick Recon A Python Tool that uses Shodan API's to perform quick recon for vulnerabilities Configuration You must edit the python code, and in

Black Hat Ethical Hacking 5 Aug 09, 2022
This repository consists of the python scripts for execution and automation of vivid tasks.

Scripting.py is a repository being maintained to keep log of the python scripts that I create for automating and executing some of my boring manual task.

Prakriti Regmi 1 Feb 07, 2022
Exploit for GitLab CVE-2021-22205 Unauthenticated Remote Code Execution

Vuln Impact An issue has been discovered in GitLab CE/EE affecting all versions starting from 11.9. GitLab was not properly validating image files tha

Hendrik Agung 2 Dec 30, 2021
Use FOFA automatic vulnerability scanning tool

AutoSRC Use FOFA automatic vulnerability scanning tool Usage python3 autosrc.py -e FOFA EMAIL -k TOKEN Screenshots License MIT Dev 6613GitHub6613

PwnWiki 48 Oct 25, 2022
BloodyAD is an Active Directory Privilege Escalation Framework

BloodyAD Framework BloodyAD is an Active Directory Privilege Escalation Framework, it can be used manually using bloodyAD.py or automatically by combi

757 Jan 07, 2023
Anti-Nuke capabilities, powerful moderation features, auto punishments, captcha-verification and more.

Server-Security-Discord-Bot Anti-Nuke capabilities, powerful moderation features, auto punishments, captcha-verification and more. Installation Instal

20 Apr 07, 2022
Buff A simple BOF library I wrote under an hour to help me automate with BOF attack

What is Buff? A simple BOF library I wrote under an hour to help me automate with BOF attack. It comes with fuzzer and a generic method to generate ex

0x00 3 Nov 21, 2022
A dynamic multi-STL, multi-process OpenSCAD build system with autoplating support

scad-build This is a multi-STL OpenSCAD build system based around GNU make. It supports dynamic build targets, intelligent previews with user-defined

Jordan Mulcahey 1 Dec 21, 2021
RCE Exploit for Gitlab < 13.9.4

GitLab-Wiki-RCE RCE Exploit for Gitlab 13.9.4 RCE via unsafe inline Kramdown options when rendering certain Wiki pages Allows any user with push acc

Enox 52 Nov 09, 2022
一款辅助探测Orderby注入漏洞的BurpSuite插件,Python3编写,适用于上xray等扫描器被ban的场景

OrderbyHunter 一款辅助探测Orderby注入漏洞的BurpSuite插件,Python3编写,适用于上xray等扫描器被ban的场景 1. 支持Get/Post型请求参数的探测,被动探测,对于存在Orderby注入的请求将会在HTTP Histroy里标红 2. 自定义排序参数list

Automne 21 Aug 12, 2022