This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
labsecurity is a tool that brings together python scripts made for ethical hacking, in a single tool, through a console interface

labsecurity labsecurity is a tool that brings together python scripts made for ethical hacking, in a single tool, through a console interface. Warning

Dylan Meca 16 Dec 08, 2022
Fuck - Multi Brute Force 🚶‍♂

f-mbf Fuck - Multi Brute Force 🚶‍♂ Install Script $ pkg update && pkg upgrade $ pkg install python2 $ pkg install git $ pip2 install requests $ pip2

Yumasaa 1 Dec 03, 2021
CVE-2021-40346 integer overflow enables http smuggling

CVE-2021-40346-POC CVE-2021-40346 integer overflow enables http smuggling Reference: https://jfrog.com/blog/critical-vulnerability-in-haproxy-cve-2021

donky16 34 Nov 15, 2022
Anti Supercookie - Confusing the ISP & Escaping the Supercookie

Confusing the ISP & Escaping the Supercookie

Baris Dincer 2 Nov 22, 2022
Multi Brute Force Facebook - Crack Facebook With Login - Free For Now

✭ SAKERA CRACK Made With ❤️ By Denventa, Araya, Dapunta Author: - Denventa - Araya Dev - Dapunta Khurayra X ⇨ Fitur Login [✯] Login Cookies ⇨ Ins

Dapunta ID 26 Jan 01, 2023
DependConfusion-X Tool is written in Python3 that scans and monitors list of hosts for Dependency Confusion

DependConfusion-X Tool is written in Python3 which allows security researcher/bug bounty hunter to scan and monitor list of hosts for Dependency Confusion.

Ali Fathi Ali Sawehli 4 Dec 21, 2021
Yuyu Scanner is a Web Reconnaissance & Web Analysis Scanner to find assets and information about targets.

Yuyu Scanner Yuyu Scanner is a Web Reconnaissance & Web Analysis Scanner to find assets and information about targets. installation ! run as root

Justakazh 20 Nov 24, 2022
A tool to find good RCE From my series: A powerful Burp extension to make bounties rain

A tool to find good RCE From my series: A powerful Burp extension to make bounties rain

52 Dec 16, 2022
The First Python Compatible Camera Hacking Tool

ZCam Hack webcam using python by sending malicious link. FEATURES : [+] Real-time Camera hacking [+] Python compatible [+] URL Shortener using bitly [

Sanketh J 109 Dec 28, 2022
This is an advanced backdoor, created with Python

Backdoor This is a Backdoor, created with Python 3. Types of Commands: Downloading / Uploading files. Launching / Deleting / Reading file's content. S

swagkarna 28 Oct 28, 2022
Crowbar - A windows post exploitation tool

Crowbar - A windows post exploitation tool Status - ✔️ This project is now considered finished. Any updates from now on will most likely be new script

29 Nov 20, 2022
Caretaker 2 Jun 06, 2022
Security audit Python project dependencies against security advisory databases.

Security audit Python project dependencies against security advisory databases.

52 Dec 17, 2022
A Python Bytecode Disassembler helping reverse engineers in dissecting Python binaries

A Python Bytecode Disassembler helping reverse engineers in dissecting Python binaries by disassembling and analyzing the compiled python byte-code(.pyc) files across all python versions (including P

neeraj 95 Dec 26, 2022
威胁情报播报

Threat-Broadcast 威胁情报播报 运行环境 项目介绍 从以下公开的威胁情报来源爬取并整合最新信息: 360:https://cert.360.cn/warning 奇安信:https://ti.qianxin.com/advisory/ 红后:https://redqueen.tj-u

东方有鱼名为咸 148 Nov 09, 2022
POC of CVE-2021-26084, which is Atlassian Confluence Server OGNL Pre-Auth RCE Injection Vulneralibity.

CVE-2021-26084 Description POC of CVE-2021-26084, which is Atlassian Confluence Server OGNL(Object-Graph Navigation Language) Pre-Auth RCE Injection V

antx 9 Aug 31, 2022
A tool combined with the advantages of masscan and nmap

A tool combined with the advantages of masscan and nmap

59 Dec 24, 2022
Facebook account cloning/hacking advanced tool + dictionary attack added | Facebook automation tool

loggef Facebook automation tool, Facebook account hacking and cloning advanced tool + dictionary attack added Warning Use this tool for educational pu

Md Josif Khan 149 Aug 10, 2022
Vuln Scanner With Python

VulnScanner Features Web Application Firewall (WAF) detection. Cross Site Scripting (XSS) tests. SQL injection time based test. SQL injection error ba

< / N u l l S 0 U L > 1 Dec 25, 2021
This project is for finding a solution to use Security Onion Elastic data with Jupyter Notebooks.

This project is for finding a solution to use Security Onion Elastic data with Jupyter Notebooks. The goal is to successfully use this notebook project below with Security Onion for beacon detection

4 Jun 08, 2022