This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
CamOver is a camera exploitation tool that allows to disclosure network camera admin password.

CamOver is a camera exploitation tool that allows to disclosure network camera admin password. Features Exploits vulnerabilities in most popul

EntySec 247 Jan 02, 2023
Dlint is a tool for encouraging best coding practices and helping ensure Python code is secure.

Dlint Dlint is a tool for encouraging best coding practices and helping ensure Python code is secure. The most important thing I have done as a progra

Dlint 127 Dec 27, 2022
🎻 Modularized exploit generation framework

Modularized exploit generation framework for x86_64 binaries Overview This project is still at early stage of development, so you might want to come b

ᴀᴇꜱᴏᴘʜᴏʀ 30 Jan 17, 2022
A high-performance DNS stub resolver for bulk lookups and reconnaissance (subdomain enumeration)

MassDNS A high-performance DNS stub resolver MassDNS is a simple high-performance DNS stub resolver targeting those who seek to resolve a massive amou

B. Blechschmidt 2.5k Jan 07, 2023
This tool was created in order to automate some basic OSINT tasks for penetration testing assingments.

This tool was created in order to automate some basic OSINT tasks for penetration testing assingments. The main feature that I haven't seen much anywhere is the downloadd google dork function where t

Tobias 5 May 31, 2022
A python module for retrieving and parsing WHOIS data

pythonwhois A WHOIS retrieval and parsing library for Python. Dependencies None! All you need is the Python standard library. Instructions The manual

Sven Slootweg 384 Dec 23, 2022
IDA Pro Python plugin to analyze and annotate Linux kernel alternatives

About This is an IDA Pro (Interactive Disassembler) plugin allowing to automatically analyze and annotate Linux kernel alternatives (content of .altin

Open Source Security, Inc. 16 Oct 12, 2022
Script Crack Facebook Premium 🚶‍♂

premium Script Crack Facebook Premium 🚶‍♂ In Script Install Script $ pkg update && pkg upgrade $ termux-setup-storage $ pkg install python $ pkg inst

Yumasaa 2 Dec 19, 2021
NexScanner is a tool which allows you to scan a website and find the admin login panel and sub-domains

NexScanner NexScanner is a tool which helps you scan a website for sub-domains and also to find login pages in the website like the admin login panel

8 Sep 03, 2022
Ini membuat tema berbasis bendera Indonesia with Python + Linux.py

tema Ubah Tema Termux Menjadi Linux Ubah Font Termux Jadi Linux dibuat oleh wahyudioputra INSTALL pkg update && pkg upgrade pkg install python pkg ins

wahyudioputra 2 Nov 30, 2021
Let's you scan the entire internet in a couple of hours and identify all Minecraft servers on IPV4

Minecraft-Server-Scanner Let's you scan the entire internet in a couple of hours and identify all Minecraft servers on IPV4 Installation and running i

116 Jan 08, 2023
A python script to brute-force guess the passwords to Instagram accounts

Instagram-Brute-Force The purpose of this script is to brute-force guess the passwords to Instagram accounts. Specifics: Comes with 2 separate modes i

Moondog 2 Nov 16, 2021
Facebook Fast Cracking Tool With Python

Pro-Crack Facebook Fast Cracking Tool This is a multi-password‌ cracking tool that can help you hack facebook accounts very quickly Installation On Te

ReD H4CkeR 5 Feb 19, 2022
Backdoor is a term that refers to the access of the software or hardware of a computer system without being detected.

This program is an non-object oriented opensource, hidden and undetectable backdoor/reverse shell/RAT for Windows made in Python 3 which contains many features such as multi-client support and cross-

35 Apr 17, 2022
Gitlab RCE - Remote Code Execution

Gitlab RCE - Remote Code Execution RCE for old gitlab version = 11.4.7 & 12.4.0-12.8.1 LFI for old gitlab versions 10.4 - 12.8.1 This is an exploit f

153 Nov 09, 2022
Tool for finding PHP source code vulnerabilities.

vulnz Tool for finding php source code vulnerabilities. Scans PHP source code and prints out potentially dangerous lines. This tool is useful for secu

Mateo Hanžek 1 Jan 14, 2022
Phoenix Framework is an environment for writing, testing and using exploit code.

Phoenix Framework is an environment for writing, testing and using exploit code. 🖼 Screenshots 🎪 Community PwnWiki Forums 🔑 Licen

42 Aug 09, 2022
Python decompiler for Python 1.5-2.4 (for historical archive)

This preserves the early code of a Python decompiler for Python versions 1.5 to 2.4. I have been able to install this using pyenv using Python 2.3.7 u

R. Bernstein 2 Jan 04, 2022
LaxrFar Python Obfuscator

LaxrFar Python Obfuscator Usage First do the things from "Upload to Webserver" o

LaxrFar 5 Jul 19, 2022
Vuln Scanner With Python

VulnScanner Features Web Application Firewall (WAF) detection. Cross Site Scripting (XSS) tests. SQL injection time based test. SQL injection error ba

< / N u l l S 0 U L > 1 Dec 25, 2021