AI Dungeon Catalog Archive Toolkit

Related tags

DownloaderAIDCAT
Overview

AI Dungeon Content Archive Toolkit (AID CAT)

AID CAT is a command-line utility that will allow you to download JSON backups of:

  1. Your private and published AI Dungeon scenarios, adventures, posts, bookmarks, worlds, friends, followers, and following.
  2. Any user's published scenarios, adventures, and posts, as well as their friends, followers, and following.
  3. Obfuscate all adventures' titles, descriptions, tags, memory, actions, undone actions, and world info with junk data.
    WARNING! Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE!

Things AID CAT will NEVER do:

  1. Download another user's private content.
  2. Share your information with anyone else. All data transferred is strictly between your device and the aidungeon.io GraphQL backend.

Go check out KoboldAI!

Saved AI Dungeon games exported using AID CAT can now be imported and played with KoboldAI! KoboldAI is a browser front-end for playing with multiple local & remote AI models. KoboldAI supports interact with the AI models via Tensorflow and includes a easy-to-use automated install process. Currently supported models are GPT Neo 1.3B/2.7B, GPT-2 Med/Large/XL, Megatron (via InferKit API) as well as the ability to run your fine-tuned custom GPT-Neo (e.g., Neo-horni) and custom GPT-2 (e.g., CloverEdition) models.

Latest Release

2021-06-10: v0.7.0

Added the ability to disable Safe Mode
Hardened the obfuscation script against network instability
Optimized the obfuscation script to run faster
**WARNING! Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE!**

Instructions below copied with edits from original script author's site (referenced files uploaded here for archival purposes):

The wAIfupocalypse is upon us

Let's save your stories!

Recently, Latitude has announced changes that will end AI cooms as we know them today. Going beyond just the explore page, employees may now ban you for content in your unpublished stories. For this reason, I have programmed a script that can automatically download all of the stories and scenarios in your account. Here are the instructions:

  1. Before you can use the script, you must download and install the Python 3 runtime. Make sure to select "Add Python 3.X to PATH" on the first page of the installer.
  2. The script needs your access token so it can access your private stories. The token will only be used for that purpose, and it will not be stored. See how to access it in Firefox or Chrome.
  3. Download the aidcat.py script here. Move it to where you want your stories saved and run it. (usually by double-clicking, or typing python aidcat.py or python3 aidcat.py in the command prompt while in the directory containing the script file).
  4. Enter your login token and press enter. While you wait, think back to the good times you had with your fictional friends.
  5. When the download is complete, a file called stories.json will be created containing all your stories.

I have also created a script that will turn that JSON file into HTML files so it's easier to read. Download it here and run it in the same folder as the JSON file. You might also want to get this stylesheet (put in same folder as html files) to make things look a little nicer.

For mobile users

You can run these scripts using Termux (Android) or Pythonista (iOS). Since it's difficult to access your login token on a mobile browser, here is a script to help you with that: login.py (1.1 KiB)

Termux Instructions

  1. pkg update && pkg install python && curl -O https://raw.githubusercontent.com/CuriousNekomimi/AIDCAT/main/aidcat.py && curl -O https://raw.githubusercontent.com/CuriousNekomimi/AIDCAT/main/login.py
  2. python login.py (To get your x-access-token)
  3. python aidcat.py (To run AIDCAT)

Changelog

2021-05-30: v0.6.9
Added the ability to obfuscate all adventures' title, description, tags, memory, actions, undone actions, and world info with junk data.
**WARNING! Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE!**
Added an info page to sources documenting Latitude.
 
2021-05-11: v0.6.7
HUGE thanks to Eta for making these improvements and for refactoring the program to be object oriented:
- The phrase "any key" was changed to "Enter." (Actually waiting for any keypress is significantly more complicated).
- Option 1 of the auth menu previously said [1] Change your access access token, which was fixed.
- Added a missing error message on "Our Content" page.
- PEP 8 compliance:
  - The header docstring was changed to use """ instead of '''
  - Fixed whitespace around operators in code and method declarations
  - Fixed indentation in a couple places
  - Added whitespace between the # and the start of each comment
  - Added a newline at the end of the file
  - Added two lines of whitespace before and after function definitions
  - (Non-PEP 8) Changed menu choices to all be double quoted strings, for consistency
- Removed default values from actually-mandatory method parameters like save_json's content_type
- Separated the long as heck query strings from the rest of the code and moved them to the bottom of the file, just above if __name__ == '__main__', to improve readability
- Added a User class
  - This class encompasses the operations make_query (new), save_json, get_scenarios/subscenarios/adventures/posts/worlds/social, and get_saves (new, which calls all of the bookmarked operations together)
  - It holds the fields content_cache, and each of the query_s as instance variables (no longer global)
    - This means these can be maintained independently per user, which will help a lot with the batch download feature later on (it allows that to be multithreaded, for example)
- All of the menu code was changed to use User objects representing different users instead of passing around a username parameter
- The menu code was additionally refactored to cut down on duplicated code by a lot

2021-05-10: v0.5.7
Thanks to Eta for these suggestions:
UUID (access token) validation using python's built-in uuid library.
Cleaned up boolean checks.
Better file management. Saving files no longer risks a memory leak if the programmer forgets to call close().

2021-05-09: v0.5.6
Various improvements to the code in terms of efficiency and syntax, thanks Eta!

2021-05-09: v0.5.4
Changed aidcat.py to UTF-8 encoding. If you were seeing weird issues with screens, like corrupted text, this should fix that issue.
Thanks to Eta for the great suggestions for improving efficiency and helping with refactoring overall, including:
Menu headers are now all block quotes.
Reduced the number of escaped characters ( e.g. \' ).
More conventional print() statements.

Future Plans:
Proper GUID validation
HTML format export/converter. (Next priority)
Possibly add the ability to batch download all your friends' and followings' published content.
Ability to get the user's x-access-token from within the application. (Rather than digging around in the browser.)

2021-05-09: v0.5.2 (Completely new program written from scratch)
Command-line user interface with menus.
Download your scenarios, subscenarios, adventures, posts, and worlds (including official purchased worlds).
Download your saved scenarios, adventures, and posts (bookmarks).
Download a list of your friends, following, and followers.
Download other user's published scenarios, subscenarios, and posts.
Download a list of other user's friends, followers, and following.
Saved files tagged with ISO 8601 format times, so previous files aren't accidentally overwritten.
NOTE: Windows (Command prompt/PowerShell) doesn't display UTF-8 encoded characters. If someone has fancy UTF characters in their username, they will show up as empty boxes when pasted in, but the program will still work.

Future plans:
HTML format export/converter. (Next priority)
Possibly add the ability to batch download all your friends' and followings' published content.
Ability to get the user's x-access-token from within the application. (Rather than digging around in the browser.)

2021-05-02:
Added: Ability to save subscenarios (thanks to the original script author)!
Added: Ability to save bookmarks (thanks to the original script author)!
Added: Choice to save scenario bookmarks.
Added: Choice to save adventure bookmarks.

2021-05-02:
Added: Ability to choose whether or not to download scenarios, adventures, or worlds. Useful for users who have thousands of adventures and don't want to download them every time.
Added: More error catching and a reminder to report errors to https://github.com/CuriousNekomimi/AIDungeonRescue/issues.
Note: The original script author has released an update to download scenario options. I'll work on incorporating that code.

2021-05-02:
Added: Ability to save worlds!
Added: Ability to save x-access token for later use (token.txt).
Added: Scenarios, adventures, and worlds, now save to separate json files.
Added: Improved readability of saved scenario, adventure, and world json files with newlines and indents.
Added: Pause for user interaction after running so the user knows the program executed successfully.
Added: Notification that files saved successfully.
Added: Try/Except to main() for several methods.
Refactored: 'stories' to 'adventures'.
Comments
  • Downloading more than 50 actions

    Downloading more than 50 actions

    I ran this almost two weeks ago to export my stories, including one with over 15,000 actions (yes, I know). Everything was exported including that full story, resulting in a 3.88 MB file.

    Since you now have support for importing into KoboldAI, I figured I'd get the latest version of this tool and re-run it to download my AI Dungeon stories. This results in a much smaller file, only 293 KB. I checked and found that it now seems to cap out at the 50 most recent actions of the story.

    I still have the older version of this tool, so I ran it again to see if the change was on your end or AI Dungeon, and it downloaded the full 3.88 MB file again. So that tells me it's a change made to this tool.

    I'm guessing it's to keep things from getting to bloated before importing into KoboldAI or something, only keeping a certain number of relevant actions which would be potentially referenced by it? Is there an option somewhere now to download the full adventures again, or at least can you provide clarification on the reason for this?

    bug 
    opened by DagothRa 3
  • Tweaked and cleaned

    Tweaked and cleaned

    -Used requests sessions instead of urllib. -Put images in their own file. -Separated the UI from the program logic. -Created a module to transform .json to html (I used my tweaked version from Scriptanon's)

    I'm going to start making the automated tests.

    opened by Moist-Cat 2
  • Infinite Loop on Latitude corrupted backwards compatibility

    Infinite Loop on Latitude corrupted backwards compatibility

    Search the term "Latitude " (with the space), which will lead to an error message. It is inside a true while loop with a continue that triggers no advancement nor does it lead to a break, thus causing infinite looping.

    opened by YuliaS11 1
  • Tools to help migrate from AID to KAI

    Tools to help migrate from AID to KAI

    I created some simple tools to help integrate AIDCAT and KAI so people could migrate from AID faster. The scripts I made interpret AIDCAT's output in a "scenario explorer" that lets the person select their scenario, type the placeholders just like in AID and then save the generated story into a file ready to be used in KoboldAI. Same for my own scenarios (it has its own script) and another tool that converts AIDCAT's extracted stories from AID into something readable by KoboldAI.

    I plan to join these three tools into one, but for now they're separated. This might be useful for people using AIDCAT. Here's the link: https://github.com/thaalesalves/aid-scripts/tree/main/other

    opened by thaalesalves 1
  • Saving bookmarked scenarios fails if any sub-scenario fetch returns None.

    Saving bookmarked scenarios fails if any sub-scenario fetch returns None.

    For reasons unknown to me, certain sub-scenarios in my bookmarks return {Scenario: None}. This satisfies the if 'scenario' in result['data'] check but causes an exception when the script tries to progress and halts.

    I was able to do a quick fix locally by changing if 'data' in result and 'scenario' in result['data']: to if 'data' in result and 'scenario' in result['data'] and result['data']['scenario'] != None:

    opened by throwaway47323674 1
  • genhtml_edit.py throws an error when trying to open a file

    genhtml_edit.py throws an error when trying to open a file

    Welcome to the AIDCAT HTML Generator! First we'll ask for your Adventure file, press Enter to continue... Traceback (most recent call last): File "C:\Users\Me\Documents\AIDungeonRescue-main\aidcat\genhtml_edit.py", line 51, in infile = json.load(file) File "C:\Users\Me\AppData\Local\Programs\Python\Python37\lib\json_init_.py", line 293, in load return loads(fp.read(), UnicodeDecodeError: 'cp932' codec can't decode byte 0x9d in position 119195: illegal multibyte sequence

    opened by cjk137 1
  • Improved Token and Error Handling

    Improved Token and Error Handling

    This pull request improves token handling, adding autocorrection for missing hyphens and server validation to check if the token is associated with an account. It will also now display the account name associated with an entered token. Error handling is also improved, and a keyboard interrupt can now properly exit out of the program (without any error text, either).

    opened by Eta0 0
  • Broken with 'Walls Approach' update

    Broken with 'Walls Approach' update

    https://latitude.io/blog/the-walls-approach

    It seems they're now encrypting user stories. When I try to download them with AIDCAT, I get 400 Bad Request. I might have missed my chance, unless this script can be updated to decrypt the stories.

    opened by ImportTaste 16
Releases(v0.7.0)
  • v.0.6.9(May 30, 2021)

    Added the ability to obfuscate all adventures' title, description, tags, memory, actions, undone actions, and world info with junk data. WARNING: Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE! Added an info page to sources documenting Latitude.

    Source code(tar.gz)
    Source code(zip)
  • v0.6.7(May 17, 2021)

  • v0.6.6(May 17, 2021)

    Descriptive exceptions and traceback for HTTP connections errors and file saving. Possible fix for Mac devices not saving files to the current directory.

    Source code(tar.gz)
    Source code(zip)
  • v0.6.5(May 16, 2021)

    Re-refactored actionWindow back to actions . It turns out that actionWindow only returns the last 50 actions. I sincerely apologize for any inconvenience this may have caused!

    Source code(tar.gz)
    Source code(zip)
  • v0.6.2_html_utf8(May 15, 2021)

  • v0.6.2_html(May 14, 2021)

  • v0.6.2(May 13, 2021)

    Eta0: This pull request improves token handling, adding autocorrection for missing hyphens and server validation to check if the token is associated with an account. It will also now display the account name associated with an entered token. Error handling is also improved, and a keyboard interrupt can now properly exit out of the program (without any error text, either).

    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(May 11, 2021)

    HUGE thanks to Eta0 for making these improvements and for refactoring the program to be object oriented:

    • The phrase "any key" was changed to "Enter." (Actually waiting for any keypress is significantly more complicated).
    • Option 1 of the auth menu previously said [1] Change your access access token, which was fixed.
    • Added a missing error message on "Our Content" page.
    • PEP 8 compliance:
      • The header docstring was changed to use """ instead of '''
      • Fixed whitespace around operators in code and method declarations
      • Fixed indentation in a couple places
      • Added whitespace between the # and the start of each comment
      • Added a newline at the end of the file
      • Added two lines of whitespace before and after function definitions
      • (Non-PEP 8) Changed menu choices to all be double quoted strings, for consistency
    • Removed default values from actually-mandatory method parameters like save_json's content_type
    • Separated the long as heck query strings from the rest of the code and moved them to the bottom of the file, just above if name == 'main', to improve readability
    • Added a User class
      • This class encompasses the operations make_query (new), save_json, get_scenarios/subscenarios/adventures/posts/worlds/social, and get_saves (new, which calls all of the bookmarked operations together)
      • It holds the fields content_cache, and each of the query_s as instance variables (no longer global)
        • This means these can be maintained independently per user, which will help a lot with the batch download feature later on (it allows that to be multithreaded, for example)
    • All of the menu code was changed to use User objects representing different users instead of passing around a username parameter
    • The menu code was additionally refactored to cut down on duplicated code by a lot
    Source code(tar.gz)
    Source code(zip)
  • v0.5.7(May 10, 2021)

Owner
Mimi
Mimi
Fully automated download and parsing for Texas A&M University's Registrar's grade distribution PDFs for years 2014+.

Fully automated download and parsing for Texas A&M University's Registrar's grade distribution PDFs for years 2014+. Adds the parsing results to a mySQL database.

TAMU Grade Distribution 1 Sep 28, 2022
A YouTube downloader app built with Django.

YouTube Downloader ⭐️ Star this project ⭐️ Requirements Python3+ Git Installation Install the dependencies and start the server. git clone https://git

Gabriel Tavares 26 Aug 19, 2022
A Telegram bot to download TikTok videos without any watermark.

TikTok Downloader Bot A Telegram bot to download TikTok videos without any watermark. Host on Heroku Youtube: Deployment Tutorial Demo: JayBee TikTok

Joy Biswas 184 Jan 04, 2023
Tool to get Canvas cover videos from Spotify tracks.

Spotify Canvas Downloader Tool to get Canvas cover videos from Spotify tracks. ✨ Try it out Building Clone the repository git clone https://github.com

Gabriel 35 Dec 28, 2022
Python youtube playlist downloader

Youtube-Playlist-Downloader-python 👍 This program is a simple Youtube playlist downloader where you input the playlist link, and then the desired pat

Pepczenko 2 Dec 25, 2021
Youtube Downloader Telegram Bot 😉

Youtube Dl bot 😉 Prerequisite ffmpeg install dependencies pip3 install -r requirements.txt Setup Bot - Change configuration config.py File - insta

Aryan Vikash 285 Dec 06, 2022
Youtube Downloader is a Graphic User Interface(GUI) that lets users download a Youtube Video or Audio through a URL

Youtube Downloader This Python and Tkinter based GUI allows users to directly download the Best Resolution Videos and Audios from Youtube. Pa-fy Insta

Samarth Kumar 2 Jun 25, 2022
YouTube Downloader Bot With Python

TG YᴏᴜTᴜʙᴇ Uᴘʟᴏᴀᴅᴇʀ * Commands YouTube for Audio & Video and sends it to telegram after receiving valid URL [Do not forwarded any just copy and paste

Pʀᴇᴅᴀᴛᴏʀ 5 Oct 21, 2022
1Fichier Download Manager.

1fichier-dl 1Fichier Download Manager. Features ⭐ Manage your downloads ⭐ Bypass time limits Credits All icons, including the app icon, were provided

manuGMG 470 Oct 08, 2022
Pantheon - The fastest YouTube downloader.

A Youtube downloader written in Python3, using HTTP requests and an API.

Billy 38 Nov 21, 2022
Bulk Downloader for Reddit

saveddit is a bulk media downloader for reddit pip3 install saveddit Setting up authorization Register an application with Reddit Write down your clie

Pranav 136 Jan 03, 2023
Downloads data from OSM API and uploads it to the mapping sandbox.

OpenStreetMap To Sandbox This is a script to download data from OSM API and upload it to the mapping sandbox. Note that it clears all data in the sand

Ilya Zverev 5 Nov 27, 2022
PyDownloader - Downloads files and folders at high speed (based on your interent speed).

PyDownloader - Downloads files and folders at high speed (based on your interent speed).

Armen._.G 4 Feb 24, 2022
A tool to download program information from Bugcrowd, for use by researchers to compare programs they are eligible to participate in

Description bcstats is a tool which allows Bugcrowd researchers to download information about all accessible programs (public and private) into a sing

19 Oct 13, 2022
Twayback: Downloading deleted Tweets from the Wayback Machine, made easy

Finding and downloading deleted Tweets takes a lot of time. Thankfully, with this tool, it becomes a piece of cake! 🎂

126 Dec 27, 2022
Automatically download multiple papers by keywords in CVPR

Automatically download multiple papers by keywords in CVPR

46 Jun 08, 2022
Scripts to download files and folders programmatically from Google Drive

Google Drive Downloader Scripts Every time I need to download a lot of files from Google Drive (e.g. a dataset), it's always incredibly frustrating an

Ivan Evtimov 6 Jul 22, 2021
Apple Music Animated Artwork Fetcher

A python script for downloading the animated artwork of an Apple Music album.

bunny 46 Jan 03, 2023
Easy automated ebook downloader using openbooks as the backend

Easy automated ebook downloader using openbooks as the backend

27 Nov 06, 2022
Download images where login is required using har python and js

이미지 다운로드(har, python, js 사용) 로그인이 필요한 사이트에서 DevTools로 이미지를 다운받는 방법은 조금 까다로웠다. 가장 쉽게 할 수 있는 방법을 찾아보았다. 사용법 F12를 눌러 DevTools를 실행 Network 탭으로 이동 페이지 새로고침

0 Jul 22, 2022