This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
A simple multi-threaded distributed SSH brute-forcing tool written in Python.

OrbitalDump A simple multi-threaded distributed SSH brute-forcing tool written in Python. How it Works When the script is executed without the --proxi

K4YT3X 408 Jan 03, 2023
Open-source jailbreaking tool for many iOS devices

Open-source jailbreaking tool for many iOS devices *Read disclaimer before using this software. checkm8 permanent unpatchable bootrom exploit for hund

6.7k Jan 05, 2023
A script based on sqlmap that uses sql injection vulnerabilities to traverse the existence of a file

A script based on sqlmap that uses sql injection vulnerabilities to traverse the existence o

2 Nov 09, 2022
Python script to tamper with pages to test for Log4J Shell vulnerability.

log4jShell Scanner This shell script scans a vulnerable web application that is using a version of apache-log4j 2.15.0. This application is a static

GoVanguard 8 Oct 20, 2022
阿里云accesskey利用工具

aliyun-accesskey-Tools 此工具用于查询ALIYUN_ACCESSKEY的主机,并且远程执行命令。 对于ALIYUN_ACCESSKEY利用方式可参考文章:记一次阿里云主机泄露Access Key到Getshell 工具截图 安装模块 pip install -r require

一灯老和尚 826 Jan 01, 2023
Laravel RCE (CVE-2021-3129)

CVE-2021-3129 - Laravel RCE About The script has been made for exploiting the Laravel RCE (CVE-2021-3129) vulnerability. This script allows you to wri

Joshua van der Poll 21 Dec 27, 2022
S2-061 的payload,以及对应简单的PoC/Exp

S2-061 脚本皆根据vulhub的struts2-059/061漏洞测试环境来写的,不具普遍性,还望大佬多多指教 struts2-061-poc.py(可执行简单系统命令) 用法:python struts2-061-poc.py http://ip:port command 例子:python

dreamer 46 Oct 20, 2022
Repository for a project of the course EP2520 Building Networked Systems Security

EP2520_ACME_Project Repository for a project of the course EP2520 Building Networked Systems Security in Royal Institute of Technology (KTH), Stockhol

1 Dec 11, 2021
Instagram brute force tool that uses tor as its proxy connections

Insta-crack This is a instagram brute force tool that uses tor as its proxy connections, keep in mind that you should not do anything illegal with thi

Liam 3 Jan 28, 2022
Password database With special stuff

This is a Password database I made for myself, as I want to keep all my passwords in the same place. but still protected, shall anyone get access to the file. And so I made this simple password datab

9 Oct 30, 2022
Safe Policy Optimization with Local Features

Safe Policy Optimization with Local Feature (SPO-LF) This is the source-code for implementing the algorithms in the paper "Safe Policy Optimization wi

Akifumi Wachi 6 Jun 05, 2022
Better-rtti-parser - IDA script to parse RTTI information in executable

RTTI parser Parses RTTI information from executable. Example HexRays decompiler view Before: After: Functions window Before: After: Structs window Ins

101 Jan 04, 2023
Separation of Mainlobes and Sidelobes in the Ultrasound Image Based on the Spatial Covariance (MIST) and Aperture-Domain Spectrum of Received Signals

Separation of Mainlobes and Sidelobes in the Ultrasound Image Based on the Spatial Covariance (MIST) and Aperture-Domain Spectrum of Received Signals

Rehman Ali 3 Jan 03, 2023
Bilgi Sistemleri Projesi için yapılan keylogger

Keylogger Bilgi Sistemleri Projesi için yapılan keylogger Projede kullanılan kütüphanelere sahip olmasanız da python dosyası çalıştığında kendisi gere

Tarik Bulut 1 Jan 07, 2022
Linus-png.github.io - Versionsverwaltung & Open Source Hausaufgabe

Let's Git - Versionsverwaltung & Open Source Hausaufgabe Herzlich Willkommen zu

1 Jan 24, 2022
This exploit allows to connect to the remote RemoteMouse 3.008 service to virtually press arbitrary keys and execute code on the machine.

RemoteMouse-3.008-Exploit The RemoteMouse application is a program for remotely controlling a computer from a phone or tablet. This exploit allows to

Podalirius 25 Dec 04, 2022
LittleBrother is a simple parental control application monitoring specific processes on Linux hosts to monitor and limit the play time of children.

Parental Control Application LittleBrother Overview LittleBrother is a simple parental control application monitoring specific processes (read "games"

40 Dec 21, 2022
macOS persistence tool

PoisonApple Command-line tool to perform various persistence mechanism techniques on macOS. This tool was designed to be used by threat hunters for cy

Cyborg Security, Inc 212 Dec 29, 2022
Domain abuse scanner covering domainsquatting and phishing keywords.

🦷 monodon 🐋 Domain abuse scanner covering domainsquatting and phishing keywords. Setup Monodon is a Python 3.7+ programm. To setup on a Linux machin

2 Mar 15, 2022
telegram bug that discloses user's hidden phone number (still unpatched) (exploit included)

CVE-2019-15514 Type: Information Disclosure Affected Users, Versions, Devices: All Telegram Users Still not fixed/unpatched. brute.py is available exp

Gray Programmerz 66 Dec 08, 2022