This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
The best Python Backdoor👌

Backdoor The best Python Backdoor Files Server file is used in all of cases If client is Windows, the client need execute EXE file If client is Linux,

13 Oct 28, 2022
Wordlist attacks on Bitwarden data.json files

BitwardenDecryptBrute This is a slightly modified version of BitwardenDecrypt. In addition to the decryption this version can do wordlist attacks for

42 Nov 09, 2022
利用NTLM Hash读取Exchange邮件

GetMail 利用NTLM Hash读取Exchange邮件:在进行内网渗透时候,我们经常拿到的是账号的Hash凭据而不是明文口令。在这种情况下采用邮件客户端或者WEBMAIL的方式读取邮件就很麻烦,需要进行破解,NTLM的破解主要依靠字典强度,破解概率并不是很大。

<a href=[email protected]"> 388 Dec 27, 2022
Patching - Interactive Binary Patching for IDA Pro

Patching - Interactive Binary Patching for IDA Pro Overview Patching assembly code to change the behavior of an existing program is not uncommon in ma

589 Dec 30, 2022
Python Library For Ethical Hacker

Python Library For Ethical Hacker

11 Nov 03, 2022
automatically crawl every URL and find cross site scripting (XSS)

scancss Fastest tool to find XSS. scancss is a fastest tool to detect Cross Site scripting (XSS) automatically and it's also an intelligent payload ge

Md. Nur habib 30 Sep 24, 2022
This exploit allows to connect to the remote RemoteMouse 3.008 service to virtually press arbitrary keys and execute code on the machine.

RemoteMouse-3.008-Exploit The RemoteMouse application is a program for remotely controlling a computer from a phone or tablet. This exploit allows to

Podalirius 25 Dec 04, 2022
Worm/Trojan/Ransomware/apt/Rootkit/Virus Database

Pestilence - The Malware Database [] Screenshot Pestilence is a project created to make the possibility of malware analysis open and available to the

*ERR0R* 47 Dec 21, 2022
Early days of an Asset Discovery tool.

Please star this project! Written in Python Report Bug . Request Feature DISCLAIMER This project is in its early days, everything you see here is almo

grag1337 3 Dec 20, 2022
Simples brute forcer de diretorios para web pentest.

🦑 dirbruter Simples brute forcer de diretorios para web pentest. ❕ Atenção Não ataque sites privados. Isto é illegal. 🖥️ Pré-requisitos Ultima versã

Dio brando 6 Jan 22, 2022
This is an injection tool that can inject any xposed modules apk into the debug android app

This is an injection tool that can inject any xposed modules apk into the debug android app, the native code in the xposed module can also be injected.

Windy 32 Nov 05, 2022
Log4j exploit catcher, detect Log4Shell exploits and try to get payloads.

log4j_catcher Log4j exploit catcher, detect Log4Shell exploits and try to get payloads. This is a basic python server that listen on a port and logs i

EntropyQueen 17 Dec 20, 2021
SPV SecurePasswordVerification

SPV SecurePasswordVerification Its is python module for doing a secure password verification without sharing the password directly. Features The passw

Merwin 1 Feb 12, 2022
All in One CRACKER911181's Tool. This Tool For Hacking and Pentesting.🎭

This is A Python & Bash Programming Based Termux-Tool Created By CRACKER911181. This Tool Created For Hacking and Pentesting. If You Use This Tool To Evil Purpose,The Owner Will Never be Responsible

CRACKER911181 1 Jan 10, 2022
SonicWALL SSL-VPN Web Server Vulnerable Exploit

SonicWALL SSL-VPN Web Server Vulnerable Exploit

44 Nov 15, 2022
Tool to scan for RouterOS (Mikrotik) forensic artifacts and vulnerabilities.

RouterOS Scanner Forensics tool for Mikrotik devices. Search for suspicious properties and weak security points that need to be fixed on the router. T

Microsoft 823 Dec 21, 2022
Use scrapli to retrieve security zone information from a Juniper SRX firewall

Get Security Zones with Scrapli Overview This example will show how to retrieve security zone information on Juniper's SRX firewalls. In addition to t

Calvin Remsburg 2 Jun 19, 2022
Windows Virus who destroy some impotants files on C:\windows\system32\

psychic-robot Windows Virus who destroy some importants files on C:\windows\system32\ Signatures of psychic-robot.PY (python file) : Bkav Pro : ASP.We

H-Tech-Dev36 1 Jan 06, 2022
The Easiest Way To Gallery Hacking

The easiest way to HACK A GALLARY, Get every part of your friends' gallery ( 100% Working ) | Tool By John Kener 🇱🇰

John Kener 34 Nov 30, 2022