🎮 🎧 🚀 Generic Visual Translator 🚀 🎧 🎮

GVT is a generic translation tool for parts of text on the PC screen with Text to Speech functionality. I wanted to create it because the existing tools that I experimented with did not satisfy me in ease-to-use experience and configuration. Personally I used it with Lost Ark (example included generated by 2k monitor) to translate simple dialogues of quests in Italian.

📝 Requirements

Tested Operating Systems : Windows 10/11 Python Version: 3.9.6

• Easynmt
• OpenCV2
• Easyocr
• Numpy
• Deepl (Unofficial API)
• Pyttsx3
• Pywin32
• WXWidgets
• Pygame
• Keyboard

The requirements.txt file has been created with the versions currently installed on my pc, but it is not excluded that GVT could work also with newer or older versions of the same libraries

Requirements installation command pip install -r requirements.txt

💪 How it works

GVT simply translates a user-defined region of the screen and then recites it using Windows 10/11 TTS (Not tested on Windows 7) showing the translated text instead of the one on the screen.

Before using it, you need to configure the config.yaml file in the same folder.

Then you can run GVT using run.bat or with the command python main.py.

## 👀 File description config.yaml.

🚀 Getting started

This is an example based on the LostArk video game

• Clone this repository on your pc or download the folder and enter in it
• Launch LostArk and reach a dialogue scene
• Run runCoordHelper.bat or the command python GetCoords.py
• Press Z on the upper left point of the text box
• Press Z on the lower right point of the text box
• Copy the coordinates from the console instead of the empty fields in the config.yaml file under the main_region and close the console
• Find the dot or icon that appears whenever the text to be translated also appears, in the case of LostArk it is the Leave button at the bottom right
• Press the Shift + Win + S buttons on Windows 10 or 11 and select this image and save it later in the ** activators ** folder with a recognizable name
• Run runCoordHelper.bat again or the command` python GetCoords.py
• Use the same method as above to get the coordinates of a not too narrow box surrounding the ** activator ** in-game image
• Copy the coordinates from the console and paste it instead of the empty fields in the config.yaml file under the activator_region and close the console
• Set the source_language with the acronym of the language you want to translate from, and the target_language for the language you want to translate the game into (use https://github.com/ptrstn/deepl-translate for the reference table and languages supported by deepl or go here https://huggingface.co/Helsinki-NLP for opus models)
• Set the dialog progress key if desired, otherwise leave it at None. Note: Leave to None if your game have a heavy anti-cheat system that not allow anything except you to press the keys of your keyboard
• Set show_text and tts_enabled according to what you want enabled/disabled
• If you have set tts_enabled to True, run runVoiceList.bat or python voice_list.py to find out the number matched to the voices installed in your Windows distribution (is the one in the square parentheses) and set the variable tts_voice_number to the desired number.

Here is an example of the complete file 📋

game_name:  Lost_Ark
source_language: en
target_language: it
translation_method: deepl
translation_internal_method: offline
gpu_enabled: True
time_between_captures: 1
skip_key: "g"
show_text: False
time_to_wait_for_word: 0.3
tts_enabled: True
tts_voice_number: 0

main_region: 
  X: 567
  Y: 1304
  extensionOfX: 2068
  extensionOfY: 1439
activator_region:
  name: "lost_ark.png"
  X: 2
  Y: 1308
  extensionOfX: 2559
  extensionOfY: 1439

• Execute run.bat

💭 To Do

• Add the capability to define more regions and activator at once
• Add the capability to support multiple game just chosing it from a menu