Injector/automatic translator (using deepL API) for Tsukihime Remake

Overview

deepLuna

Extractor/Editor/Translator/Injector for Tsukihime Remake

About

deepLuna, from "deepL", the machine translation service, and "Luna", the name of the Roman Moon goddess, is a Python script with a GUI interface that works as an extractor/editor/translator (with deepL API)/injector for the text of the Tsukihime Remake game on Switch and possibly on PS4, while taking into account the whole internal structure of the script.

Installation

You just need a regular Python distribution, for example the last one from https://www.python.org/downloads/, and then launch the script deepLuna.py using a Python IDE, for example IDLE that you get by default, or some other (Spyder, Pyzo, etc.).

Usage

Preprocessing steps

  1. Dump/Find the NSP Tsukihime Switch ROM as well as the relevant keys (prod.keys and title.keys)
  2. Extract the NSP and the resulting (biggest) NCA file using for example hactool (https://github.com/SciresM/hactool)
  3. In the romfs folder, copy the script_text.mrg and allscr.mrg files and paste them in the same folder as deepLuna.py.

Extracting the text (first compulsory step)

After launching deepLuna, click on "Extract text" and wait until the extraction finishes.

IMPORTANT NOTE: This might take a bit of time, around a minute, and this is normal if you have the feeling that the graphical interface is lagging. The script is actually working at the command line level, so you can follow what is going on in the python console or in the console of the IDE you're using.

The main editor window should open. If the script complains (an error window will open), make sure that you placed both script files in the same folder as deepLuna and check that their respective names are "script_text.mrg" and "allscr.mrg" (these are the original names of the files).

Editing the text

Once the first extraction has been done, when launching deepLuna simply click on "Open translation". This might also take some seconds.

The main editor window contains three main elements:

  • On the left there is a tree view of the text files of the game, and it is the only active element at the beginning. This section is the organized by heroine and days ("Arcueid" and "Ciel"). The remaining three sections concern the bad end section "Teach Me, Ciel-sensei!", the part that is common to both Arcueid's and Ciel's route called "Common" and finally the section "Hidden text" that shows some text grabbed by the script that is not used ingame. The first thing you have to do is choose one of the scenes that appear as elements of the form "TEXT_WITH_UNDERSCORES_AND_NUMBERS", and then double-click on it. IMPORTANT NOTE: When clicking on one of the element, due to the structure of the script, you might have to wait a few seconds before the next content loads - this is perfectly normal.
  • The second element from the left, a scrolling list, activates. It shows the content of the selected scene in the form "a : b" where a is the page and b is the line on the page a. Double-click on any element. You might also note at this point that the two fields above activated, and will show up the percentage of translation done on the selected scene as well as for the global file, computed both in real time. We will come back to them in the section below.
  • Finally, you have the main editing part: a field that shows the original japanese selected line called "Original text", a field called "Translated text" below where you have to input your translation of the line above by replacing the word "TRANSLATION" that is there, and finally some buttons that will allow you to perform all the relevant actions.

IMPORTANT NOTE: When in some line you meet the pound/hashtag character "#", PLEASE do not delete it and leave as it is in the translation! This character symbolizes the fact that the line is shown ingame with a small break where the "#" is, and from a low-level perspective, symbolizes that the line is cut into two separate strings with altogether different pointers, so erasing it WILL break everything.

Commands

Validate: When you finish inputting the translation for a line, click on this button. The corresponding offset will become green to confirm the translation and the percentage of advancement of translation will update for the scene as well as for the total game.

Save: When you finish your translation, simply click on this button to save everything. IMPORTANT NOTE: This operation might also take a few seconds, this is also normal.

Search: Opens a window to search for some text. Enter the text in the first field, and click on "Search". Move between the results with "Previous"/"Next". If you want to replace some text (works only for the translated text for obvious reasons), simply use the last field in the window and then click on "Replace"/"Replace all" (the number of results will update automatically). This allows you to search for text only in the selected scene, not in the whole game script.

Translating the text

Select an offset or a range of offset (using Shift+double-click), then click on the "Translate" button in the main screen, a new window will open. The first time you execute the script, you'll have to configure three options (after that, they will be saved in a config.ini file created in deepLuna folder after the initial launch and load automatically everytime you launch deepLuna):

a) "deepL API link": Input here your API link(s) from deepL (replace the 0 that is there). If you have several of them, just separate them by a comma - the script will try them successively when a translation request fails. Currently works only for free deepL accounts (possible future update for a pro deepL API link). NOTE: If you copy/paste your API link and it doesn't work, make sure that there is not an additional space character that was copied at the end of the link, this might happen.

b) "Language": Put the initials of the language into which you want to translate the script (e.g. EN for english, FR for french, DE for german, IT for italian, RU for russian, PL for polish, ES for spansih, etc.)

c) "Block translation": If you keep this box unchecked, deepLuna will send each line separately to deepL for translation. If you check the box, deepLuna will glue all the lines from the selected range of offsets together, and send the whole block to deepL before splitting them again. From experience, the latter option might improve the quality of translation since you give more context to deepL, but on the other hand produces sometimes completely unrelated translated sentences in the middle if the block is really big. You have to play around to see what option is the best depending on context and chosen sentences.

IMPORTANT NOTE: The furigana are thrown away during translation at the moment since their structure prevent deepL from giving a consistent translation, and also because their meaning and positioning sometimes doesn't match at all the original word (Nasu writing style is really something...). Maybe I'll try to implement something to deal with this problem in the future, but for the moment it is as it is.

Now, it suffices to click on "OK" and wait until translation finishes. If something fails (too much deepL requests for example), the program will tell you and if you've chosen the line-by-line translation approach, will point you out the line where it stopped.

Injecting the text back

Easy as pie, simply click on the button "Insert" and wait a little bit (it might take up to ~20 seconds on some older computers). deepLuna will generate a new script file of the form "script_text_translatedDATETIME.mrg", with DATETIME being the date and time of insertion (to avoid overwriting files and keeping backups). NOTE: The script inserts everytime the full translation.

NEW: Exporting functions

Since the v2.0, deepLuna makes your life easier if you're working as a group on the translation, which is usually the case. How to keep everything up to date for all the people? Whenever you finish working on some scene, just press on "Export", and deepLuna will generate an update file in the update folder, in the same directory. If you look inside, you will see that this is a simple text file with the same name as the scene you were working on, and if you open it, you'll realise that this is exactly the case - the exported file contains the whole scene in a very nice and readable way, with the original sentences where they were, and the freshly translated sentences where the japanese ones were supposed to be. It suffices now to send this file to your colleagues, and they will simply have to put it in the "update" folder. When they launch deepLuna the next time they want to use it, it will update itself and include the new file in its own database.

If you need for some reason to export your whole personal database, this is also doable - simply click on "Export all", and a new window will open, asking you for confirmation. Note that this operation might take up to 10 mn on some older computers.

On another note, this update function makes it possible to have an alternative use of deepLuna: simply export the scene file, then edit it directly with your favorite text editor, and finally share it with your teammates or use the "Insert" function to inject it in the script yourself.

Additional comments

Feel free to raise an issue if something happens or if you wish to have some specific functionality, I'm always open to new possibilities and improvements! On another note, if you end up using this tool in some way or the other for your translation, please simply credit the script name itself, deepLuna, in some place or the other on your website/patched game/other.

This readme will get a more substantial update in the following days with screenshots and/or video tutorials.

Acknoledgments

I am really thankful to Hintay for his wonderful tools at https://github.com/Hintay/PS-HuneX_Tools. I relied on them heavily (unpack_allsrc.py, prep_tpl.py and the mzx decompression script) and included a modified version of them in deepLuna to make it perfectly operational.

Comments
  • Encoding question

    Encoding question

    Sorry to bother you again, and for the dumb question too; but is there a way to have the encoding supported for other languages? I was testing translations in my native language - which makes a lot of use of commas and accents - however, when executed the letters are always distorted since the format is not supported

    opened by Zac91281 13
  • ModuleNotFoundError (I do have the module installed)

    ModuleNotFoundError (I do have the module installed)

    Traceback (most recent call last): File "C:\Users\arthu\Desktop\deepLuna-main\deepLuna-main\deepLuna.py", line 18, in from PIL import ImageTk, Image ModuleNotFoundError: No module named 'PIL'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "C:\Users\arthu\Desktop\deepLuna-main\deepLuna-main\deepLuna.py", line 20, in install("Pillow") NameError: name 'install' is not defined

    The problem here is that when I try to run the script on IDE this errors occurs and I just don't know what to do since I already have everything needed installed.

    opened by Arthurfogo7 10
  • Strange Error

    Strange Error

    I've been doing the deepLuna translation in increments and have had minimal issues. I got up to 44% and now whenever I try to translate again, no matter the line or chunk of lines, I'm getting this.

    Exception in Tkinter callback
    Traceback (most recent call last):
      File "C:\Users\Jeremy\AppData\Local\Programs\Python\Python39\lib\tkinter\__init__.py", line 1892, in __call__
        return self.func(*args)
      File "F:\tsukihime\deepLuna-main\deepLuna-main\deepLuna.py", line 910, in translate_game
        if self.i < cs[-1]:
    IndexError: tuple index out of range
    
    opened by jeremoople 7
  • Not opening

    Not opening

    Whenever I try to open deepLuna, python opens and closes it instantly. I'm using the latest version and the files are placed with the needed extracted files from the game. My knowledge is pretty lay, so I don't know what to do.

    opened by Zac91281 5
  • Decode error on new build

    Decode error on new build

    I decide to check new build, and it's give a error when i try to press "Extract Text" or "Open translation". Traceback (most recent call last): File \luna\ui\start_window.py", line 166, in extract_database tl_db = TranslationDb.from_mrg( File \luna\translation_db.py", line 387, in from_mrg strings_by_offset[i] = data.decode('utf-8') UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

    opened by SkyAshpd 4
  • All of the .py files keep immediately closing?

    All of the .py files keep immediately closing?

    So for some reason every time I try to open deepLuna.py it keeps immediately closing with no text or anything at all. I'm quite confused as I have python downloaded.

    opened by Rescuro 2
  • Install error

    Install error

    Every time I try to run deepLuna.py in IDLE or Pyzo, I get this error:

    Traceback (most recent call last):
      File "F:\tsukihime\deepLuna-main\deepLuna-main\deepLuna.py", line 18, in <module>
        from PIL import ImageTk, Image
    ModuleNotFoundError: No module named 'PIL'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "F:\tsuikihime\deepLuna-main\deepLuna-main\deepLuna.py", line 20, in <module>
        install("Pillow")
    NameError: name 'install' is not defined
    >>> 
    

    I'm not used to installing programs using Python as opposed to like, a setup wizard, so I could very well just be making a basic newbie mistake, but still, I don't know where to go from here.

    opened by jeremoople 2
  • Issue with inserting translation

    Issue with inserting translation

    When I press the insert button on the translation window, I get this error in the console window:

    Saving file, please wait...
    Exception in Tkinter callback
    Traceback (most recent call last):
      File "C:\Users\fergu\AppData\Local\Programs\Python\Python38-32\lib\tkinter\__init__.py", line 1883, in __call__
        return self.func(*args)
      File "C:\Users\fergu\AppData\Roaming\yuzu\load\01001DC01486A000\deepLuna\romFS\deepLuna.py", line 1518, in function_insert_translation
        self.enregistrer_fichier()
      File "C:\Users\fergu\AppData\Roaming\yuzu\load\01001DC01486A000\deepLuna\romFS\deepLuna.py", line 1718, in enregistrer_fichier
        self.table_file = self.reinsert_daytable(self.table_scr_file,self.table_file)
      File "C:\Users\fergu\AppData\Roaming\yuzu\load\01001DC01486A000\deepLuna\romFS\deepLuna.py", line 1510, in reinsert_daytable
        if self.splitDayTable[self.i][0] == self.newMainTable[self.j][0] and self.splitDayTable[self.i][2] != 'TRANSLATION':
    TypeError: 'type' object is not subscriptable
    

    Any ideas as to what I can do to fix the issue? I'm currently using python 3.8.5.

    opened by shingle-bells 1
  • Fix cursor accounting for _long_ split lines

    Fix cursor accounting for _long_ split lines

    Previously, when breaking lines, we would take the length of the final section of the broken line and add that to the cursor position, e.g.

    ZM("20 characters of text");
    // Cursor position: 20
    MSAD("20 characters of text")
    // Cursor position: 40
    MSAD("5 characters of text")
    // Cursor position: 45
    -> "20 characters of text20characters of text5 characters of text"
    

    however, this is wrong for lines where a MSAD causes a line break -

    ZM("20 characters of text");
    // Cursor position: 20
    MSAD("40 characters of text")
    // Gets wrapped to 35 chars + 5 chars
    // Cursor position: 20 + 5 (WRONG!)
    
    opened by rschlaikjer 0
  • Injecting Question

    Injecting Question

    Greetings. I know it isn't quite a issue, but I don't know where else to ask. But should the edited script go in some specific folder? Mine is within the DeepLuna folder, already replacing the original file, but the translation is not appearing. image

    opened by Zaldi556 0
Releases(v5.2)
  • v5.2(Jan 16, 2022)

  • v4.6(Jan 9, 2022)

    • deepLuna now uses new 'Literate' import/export script format
    • Internal database structure now JSON-based
    • Various performance improvements
    • Added luna_cli and luna_linter to facilitate scripting of simple workflows
    Source code(tar.gz)
    Source code(zip)
  • v3.1.1(Oct 18, 2021)

    • Major performance improvements, in particular importing update files and loading scenes takes now only split seconds
    • A new comment field has been added (see Readme for more details)
    • A new "swap character" option has been added for languages with special characters (see Readme for more details)
    Source code(tar.gz)
    Source code(zip)
    deepLuna_v3.1.1.zip(224.48 KB)
  • v2.3(Oct 6, 2021)

  • v2.2.1(Sep 14, 2021)

  • v2.2(Sep 10, 2021)

  • v2.1(Sep 7, 2021)

    Major upgrade of deepLuna:

    • Extraction algorithm and main interface completely remodelled
    • Taking now into account the underlying day/heroine structure of the files
    • Proposes a new simple and intuitive function to share your updated translation with you teammates
    • Added many small improvements to the interface that enhances the global user experience
    • Corrected many different bugs
    Source code(tar.gz)
    Source code(zip)
    deepLuna_v2.1.zip(215.62 KB)
  • v1.0(Sep 6, 2021)

    The is the initial release of deepLuna. It contains the following elements:

    • a linear extraction module of the Tsukihime Remake script coming from the script_text.mrg file
    • a graphical interface edition module with some basic usual functions, such as search and replace
    • a machine translation module using deepL API links as a professional translation possible helper
    • an injection module that injects the translated text in the script_text.mrg file
    Source code(tar.gz)
    Source code(zip)
    deepLuna_v1.0.zip(203.09 KB)
For specific function. For my own convenience. Remind owner to share data to another DITO user.

For specific function. For my own convenience. Remind owner to share data to another DITO user.

Meigo 1 Dec 14, 2021
BLYRIC is a Twitter bot that tweets a song lyric every night.

BLYRIC BLYRIC, a bot that tweets a song lyric every night. Follow on Twitter: @blyric_ Overview BLYRIC is a Twitter bot that tweets a song quote every

Bruno Kenzo Hyodo 6 Oct 05, 2022
A python package that allows you to place automated trades using the TD Ameritrade API.

Template Repo Table of Contents Overview Setup Usage Support These Projects Overview Setup Setup - Requirements Install:* For this particular project,

Alex Reed 4 Jan 25, 2022
A Anything goes Discord bot written in python and uses the wrapper Discord.py

GerardTheWizard A Anything goes Discord bot written in python and uses the wrapper Discord.py What can he do? Allow users to level up through typing,

1 May 05, 2022
Petit webhook manager by moi (wassim)

Webhook Manager By wassim oubliez pas de ⭐ le projet Installations il te faut python sinon quand tu va lancer le start.bat sa va tout installer tout s

wassim 9 Jul 08, 2021
Collect links to profiles by username through search engines

Marple Summary Collect links to profiles by username through search engines (currently Google and DuckDuckGo). Quick Start ./marple.py soxoj Results:

125 Dec 19, 2022
A tool for exporting Telegram group chats into static websites, preserving chat history like mailing list archives.

tg-archive is a tool for exporting Telegram group chats into static websites, preserving chat history like mailing list archives. Preview The @fossuni

Kailash Nadh 400 Dec 27, 2022
Python wrapper for Interactive Brokers Client Portal Web API

EasyIB: Unofficial Wrapper for Interactive Brokers API EasyIB is an unofficial python wrapper for Interactive Brokers Client Portal Web API. Features

39 Dec 13, 2022
Remedy when Amazon ECR is not running basic scans for container CVEs.

Welcome to your CDK Python project! This is a blank project for Python development with CDK. The cdk.json file tells the CDK Toolkit how to execute yo

4n6ir 4 Nov 05, 2022
派蒙Bot / PaimonBot

派蒙Bot / PaimonBot 基于Mrs4s / go-cqhttp 和 nonebot / nonebot2 的原神QQ群聊机器人 特别鸣谢 MingxuanGame 的人物武器名字自动纠正功能 小灰灰 的人物武器信息api 环境 请务必使用Python3.7以上版本!! 尽量在linux下

晓轩 96 Dec 16, 2022
A free, minimal, lightweight, cross-platform, easily expandable Twitch IRC/API bot.

parky's twitch bot A free, minimal, lightweight, cross-platform, easily expandable Twitch IRC/API bot. Features 🔌 Connect to Twitch IRC chat! 🔌 Conn

Andreas Schneider 10 Dec 30, 2022
Accurately dump Commodore 64 tapes

TrueTape64 A cheap, easy to build adapter to interface a Commodore 1530 (C2N) Datasette to your PC to dump and preserve your aging Commodore 64 softwa

francesco 38 Dec 03, 2022
Automatic generation of crypto-arts based on image layers

NFT Generator Автоматическая генерация крипто-артов на основе слоев изображения. Установка pip3 install -r requirements.txt rm -rf result/* Как это ра

Zproger 31 Dec 29, 2022
Clippin n grafting Backend

Clipping' n Grafting Presenting you, 🎉 Clippin' n Grafting 🎉 , your very own ecommerce website displaying all your artsy-craftsy stuff. Not only the

Google-Developer-Student-Club-ISquareIT (GDSC I²IT) 2 Oct 22, 2021
Yet another random discord bot.

YARDB (r!) Yet another fully functional and random discord bot. I might add more features if I'm bored also don't criticize on my code. Commands: 4 Di

kayle 1 Oct 21, 2021
Pybt: a BaoTa panel python sdk

About Pybt is a BaoTa panel python sdk. Pybt 是一个宝塔面板API的Python版本sdk封装库。 公司很多服务器都装了宝塔面板,通过宝塔来部署、安装、维护一些服务,服务器的数量上以后,导致了维护的不方便,这个时候就想使用宝塔提供的API来开发一个运维平台

Adam Zhang 9 Dec 05, 2022
Python API for Photoshop.

Python API for Photoshop. The example above was created with Photoshop Python API.

Hal 372 Jan 02, 2023
Resources for the AMLD 2022 workshop "DevOps on AWS"

MLOPS on AWS | AMLD 2022 This repository contains all the resources necessary to follow along and reproduce the workshop "MLOps on AWS: a Hands-On Tut

xtream 8 Jun 16, 2022
A simple Python wrapper for the Amazon.com Product Advertising API ⛺

Amazon Simple Product API A simple Python wrapper for the Amazon.com Product Advertising API. Features An object oriented interface to Amazon products

Yoav Aviram 789 Dec 26, 2022
Bitstamp API wrapper for Python

NOTICE: THIS REPOSITORY IS NO LONGER ACTIVELY MAINTAINED It is highly unlikely that I will respond to PRs and questions about usage. This library was

Jack Preston 53 Mar 09, 2022