AI Dungeon Catalog Archive Toolkit

Related tags

DownloaderAIDCAT
Overview

AI Dungeon Content Archive Toolkit (AID CAT)

AID CAT is a command-line utility that will allow you to download JSON backups of:

  1. Your private and published AI Dungeon scenarios, adventures, posts, bookmarks, worlds, friends, followers, and following.
  2. Any user's published scenarios, adventures, and posts, as well as their friends, followers, and following.
  3. Obfuscate all adventures' titles, descriptions, tags, memory, actions, undone actions, and world info with junk data.
    WARNING! Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE!

Things AID CAT will NEVER do:

  1. Download another user's private content.
  2. Share your information with anyone else. All data transferred is strictly between your device and the aidungeon.io GraphQL backend.

Go check out KoboldAI!

Saved AI Dungeon games exported using AID CAT can now be imported and played with KoboldAI! KoboldAI is a browser front-end for playing with multiple local & remote AI models. KoboldAI supports interact with the AI models via Tensorflow and includes a easy-to-use automated install process. Currently supported models are GPT Neo 1.3B/2.7B, GPT-2 Med/Large/XL, Megatron (via InferKit API) as well as the ability to run your fine-tuned custom GPT-Neo (e.g., Neo-horni) and custom GPT-2 (e.g., CloverEdition) models.

Latest Release

2021-06-10: v0.7.0

Added the ability to disable Safe Mode
Hardened the obfuscation script against network instability
Optimized the obfuscation script to run faster
**WARNING! Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE!**

Instructions below copied with edits from original script author's site (referenced files uploaded here for archival purposes):

The wAIfupocalypse is upon us

Let's save your stories!

Recently, Latitude has announced changes that will end AI cooms as we know them today. Going beyond just the explore page, employees may now ban you for content in your unpublished stories. For this reason, I have programmed a script that can automatically download all of the stories and scenarios in your account. Here are the instructions:

  1. Before you can use the script, you must download and install the Python 3 runtime. Make sure to select "Add Python 3.X to PATH" on the first page of the installer.
  2. The script needs your access token so it can access your private stories. The token will only be used for that purpose, and it will not be stored. See how to access it in Firefox or Chrome.
  3. Download the aidcat.py script here. Move it to where you want your stories saved and run it. (usually by double-clicking, or typing python aidcat.py or python3 aidcat.py in the command prompt while in the directory containing the script file).
  4. Enter your login token and press enter. While you wait, think back to the good times you had with your fictional friends.
  5. When the download is complete, a file called stories.json will be created containing all your stories.

I have also created a script that will turn that JSON file into HTML files so it's easier to read. Download it here and run it in the same folder as the JSON file. You might also want to get this stylesheet (put in same folder as html files) to make things look a little nicer.

For mobile users

You can run these scripts using Termux (Android) or Pythonista (iOS). Since it's difficult to access your login token on a mobile browser, here is a script to help you with that: login.py (1.1 KiB)

Termux Instructions

  1. pkg update && pkg install python && curl -O https://raw.githubusercontent.com/CuriousNekomimi/AIDCAT/main/aidcat.py && curl -O https://raw.githubusercontent.com/CuriousNekomimi/AIDCAT/main/login.py
  2. python login.py (To get your x-access-token)
  3. python aidcat.py (To run AIDCAT)

Changelog

2021-05-30: v0.6.9
Added the ability to obfuscate all adventures' title, description, tags, memory, actions, undone actions, and world info with junk data.
**WARNING! Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE!**
Added an info page to sources documenting Latitude.
 
2021-05-11: v0.6.7
HUGE thanks to Eta for making these improvements and for refactoring the program to be object oriented:
- The phrase "any key" was changed to "Enter." (Actually waiting for any keypress is significantly more complicated).
- Option 1 of the auth menu previously said [1] Change your access access token, which was fixed.
- Added a missing error message on "Our Content" page.
- PEP 8 compliance:
  - The header docstring was changed to use """ instead of '''
  - Fixed whitespace around operators in code and method declarations
  - Fixed indentation in a couple places
  - Added whitespace between the # and the start of each comment
  - Added a newline at the end of the file
  - Added two lines of whitespace before and after function definitions
  - (Non-PEP 8) Changed menu choices to all be double quoted strings, for consistency
- Removed default values from actually-mandatory method parameters like save_json's content_type
- Separated the long as heck query strings from the rest of the code and moved them to the bottom of the file, just above if __name__ == '__main__', to improve readability
- Added a User class
  - This class encompasses the operations make_query (new), save_json, get_scenarios/subscenarios/adventures/posts/worlds/social, and get_saves (new, which calls all of the bookmarked operations together)
  - It holds the fields content_cache, and each of the query_s as instance variables (no longer global)
    - This means these can be maintained independently per user, which will help a lot with the batch download feature later on (it allows that to be multithreaded, for example)
- All of the menu code was changed to use User objects representing different users instead of passing around a username parameter
- The menu code was additionally refactored to cut down on duplicated code by a lot

2021-05-10: v0.5.7
Thanks to Eta for these suggestions:
UUID (access token) validation using python's built-in uuid library.
Cleaned up boolean checks.
Better file management. Saving files no longer risks a memory leak if the programmer forgets to call close().

2021-05-09: v0.5.6
Various improvements to the code in terms of efficiency and syntax, thanks Eta!

2021-05-09: v0.5.4
Changed aidcat.py to UTF-8 encoding. If you were seeing weird issues with screens, like corrupted text, this should fix that issue.
Thanks to Eta for the great suggestions for improving efficiency and helping with refactoring overall, including:
Menu headers are now all block quotes.
Reduced the number of escaped characters ( e.g. \' ).
More conventional print() statements.

Future Plans:
Proper GUID validation
HTML format export/converter. (Next priority)
Possibly add the ability to batch download all your friends' and followings' published content.
Ability to get the user's x-access-token from within the application. (Rather than digging around in the browser.)

2021-05-09: v0.5.2 (Completely new program written from scratch)
Command-line user interface with menus.
Download your scenarios, subscenarios, adventures, posts, and worlds (including official purchased worlds).
Download your saved scenarios, adventures, and posts (bookmarks).
Download a list of your friends, following, and followers.
Download other user's published scenarios, subscenarios, and posts.
Download a list of other user's friends, followers, and following.
Saved files tagged with ISO 8601 format times, so previous files aren't accidentally overwritten.
NOTE: Windows (Command prompt/PowerShell) doesn't display UTF-8 encoded characters. If someone has fancy UTF characters in their username, they will show up as empty boxes when pasted in, but the program will still work.

Future plans:
HTML format export/converter. (Next priority)
Possibly add the ability to batch download all your friends' and followings' published content.
Ability to get the user's x-access-token from within the application. (Rather than digging around in the browser.)

2021-05-02:
Added: Ability to save subscenarios (thanks to the original script author)!
Added: Ability to save bookmarks (thanks to the original script author)!
Added: Choice to save scenario bookmarks.
Added: Choice to save adventure bookmarks.

2021-05-02:
Added: Ability to choose whether or not to download scenarios, adventures, or worlds. Useful for users who have thousands of adventures and don't want to download them every time.
Added: More error catching and a reminder to report errors to https://github.com/CuriousNekomimi/AIDungeonRescue/issues.
Note: The original script author has released an update to download scenario options. I'll work on incorporating that code.

2021-05-02:
Added: Ability to save worlds!
Added: Ability to save x-access token for later use (token.txt).
Added: Scenarios, adventures, and worlds, now save to separate json files.
Added: Improved readability of saved scenario, adventure, and world json files with newlines and indents.
Added: Pause for user interaction after running so the user knows the program executed successfully.
Added: Notification that files saved successfully.
Added: Try/Except to main() for several methods.
Refactored: 'stories' to 'adventures'.
Comments
  • Downloading more than 50 actions

    Downloading more than 50 actions

    I ran this almost two weeks ago to export my stories, including one with over 15,000 actions (yes, I know). Everything was exported including that full story, resulting in a 3.88 MB file.

    Since you now have support for importing into KoboldAI, I figured I'd get the latest version of this tool and re-run it to download my AI Dungeon stories. This results in a much smaller file, only 293 KB. I checked and found that it now seems to cap out at the 50 most recent actions of the story.

    I still have the older version of this tool, so I ran it again to see if the change was on your end or AI Dungeon, and it downloaded the full 3.88 MB file again. So that tells me it's a change made to this tool.

    I'm guessing it's to keep things from getting to bloated before importing into KoboldAI or something, only keeping a certain number of relevant actions which would be potentially referenced by it? Is there an option somewhere now to download the full adventures again, or at least can you provide clarification on the reason for this?

    bug 
    opened by DagothRa 3
  • Tweaked and cleaned

    Tweaked and cleaned

    -Used requests sessions instead of urllib. -Put images in their own file. -Separated the UI from the program logic. -Created a module to transform .json to html (I used my tweaked version from Scriptanon's)

    I'm going to start making the automated tests.

    opened by Moist-Cat 2
  • Infinite Loop on Latitude corrupted backwards compatibility

    Infinite Loop on Latitude corrupted backwards compatibility

    Search the term "Latitude " (with the space), which will lead to an error message. It is inside a true while loop with a continue that triggers no advancement nor does it lead to a break, thus causing infinite looping.

    opened by YuliaS11 1
  • Tools to help migrate from AID to KAI

    Tools to help migrate from AID to KAI

    I created some simple tools to help integrate AIDCAT and KAI so people could migrate from AID faster. The scripts I made interpret AIDCAT's output in a "scenario explorer" that lets the person select their scenario, type the placeholders just like in AID and then save the generated story into a file ready to be used in KoboldAI. Same for my own scenarios (it has its own script) and another tool that converts AIDCAT's extracted stories from AID into something readable by KoboldAI.

    I plan to join these three tools into one, but for now they're separated. This might be useful for people using AIDCAT. Here's the link: https://github.com/thaalesalves/aid-scripts/tree/main/other

    opened by thaalesalves 1
  • Saving bookmarked scenarios fails if any sub-scenario fetch returns None.

    Saving bookmarked scenarios fails if any sub-scenario fetch returns None.

    For reasons unknown to me, certain sub-scenarios in my bookmarks return {Scenario: None}. This satisfies the if 'scenario' in result['data'] check but causes an exception when the script tries to progress and halts.

    I was able to do a quick fix locally by changing if 'data' in result and 'scenario' in result['data']: to if 'data' in result and 'scenario' in result['data'] and result['data']['scenario'] != None:

    opened by throwaway47323674 1
  • genhtml_edit.py throws an error when trying to open a file

    genhtml_edit.py throws an error when trying to open a file

    Welcome to the AIDCAT HTML Generator! First we'll ask for your Adventure file, press Enter to continue... Traceback (most recent call last): File "C:\Users\Me\Documents\AIDungeonRescue-main\aidcat\genhtml_edit.py", line 51, in infile = json.load(file) File "C:\Users\Me\AppData\Local\Programs\Python\Python37\lib\json_init_.py", line 293, in load return loads(fp.read(), UnicodeDecodeError: 'cp932' codec can't decode byte 0x9d in position 119195: illegal multibyte sequence

    opened by cjk137 1
  • Improved Token and Error Handling

    Improved Token and Error Handling

    This pull request improves token handling, adding autocorrection for missing hyphens and server validation to check if the token is associated with an account. It will also now display the account name associated with an entered token. Error handling is also improved, and a keyboard interrupt can now properly exit out of the program (without any error text, either).

    opened by Eta0 0
  • Broken with 'Walls Approach' update

    Broken with 'Walls Approach' update

    https://latitude.io/blog/the-walls-approach

    It seems they're now encrypting user stories. When I try to download them with AIDCAT, I get 400 Bad Request. I might have missed my chance, unless this script can be updated to decrypt the stories.

    opened by ImportTaste 16
Releases(v0.7.0)
  • v.0.6.9(May 30, 2021)

    Added the ability to obfuscate all adventures' title, description, tags, memory, actions, undone actions, and world info with junk data. WARNING: Obfuscating adventures wipes all adventures on an account and CANNOT BE UNDONE! Added an info page to sources documenting Latitude.

    Source code(tar.gz)
    Source code(zip)
  • v0.6.7(May 17, 2021)

  • v0.6.6(May 17, 2021)

    Descriptive exceptions and traceback for HTTP connections errors and file saving. Possible fix for Mac devices not saving files to the current directory.

    Source code(tar.gz)
    Source code(zip)
  • v0.6.5(May 16, 2021)

    Re-refactored actionWindow back to actions . It turns out that actionWindow only returns the last 50 actions. I sincerely apologize for any inconvenience this may have caused!

    Source code(tar.gz)
    Source code(zip)
  • v0.6.2_html_utf8(May 15, 2021)

  • v0.6.2_html(May 14, 2021)

  • v0.6.2(May 13, 2021)

    Eta0: This pull request improves token handling, adding autocorrection for missing hyphens and server validation to check if the token is associated with an account. It will also now display the account name associated with an entered token. Error handling is also improved, and a keyboard interrupt can now properly exit out of the program (without any error text, either).

    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(May 11, 2021)

    HUGE thanks to Eta0 for making these improvements and for refactoring the program to be object oriented:

    • The phrase "any key" was changed to "Enter." (Actually waiting for any keypress is significantly more complicated).
    • Option 1 of the auth menu previously said [1] Change your access access token, which was fixed.
    • Added a missing error message on "Our Content" page.
    • PEP 8 compliance:
      • The header docstring was changed to use """ instead of '''
      • Fixed whitespace around operators in code and method declarations
      • Fixed indentation in a couple places
      • Added whitespace between the # and the start of each comment
      • Added a newline at the end of the file
      • Added two lines of whitespace before and after function definitions
      • (Non-PEP 8) Changed menu choices to all be double quoted strings, for consistency
    • Removed default values from actually-mandatory method parameters like save_json's content_type
    • Separated the long as heck query strings from the rest of the code and moved them to the bottom of the file, just above if name == 'main', to improve readability
    • Added a User class
      • This class encompasses the operations make_query (new), save_json, get_scenarios/subscenarios/adventures/posts/worlds/social, and get_saves (new, which calls all of the bookmarked operations together)
      • It holds the fields content_cache, and each of the query_s as instance variables (no longer global)
        • This means these can be maintained independently per user, which will help a lot with the batch download feature later on (it allows that to be multithreaded, for example)
    • All of the menu code was changed to use User objects representing different users instead of passing around a username parameter
    • The menu code was additionally refactored to cut down on duplicated code by a lot
    Source code(tar.gz)
    Source code(zip)
  • v0.5.7(May 10, 2021)

Owner
Mimi
Mimi
DYA ( Ditch YouTube API ) is a package created to power the user with YouTube Data API functionality without any API Key

Ditch YouTubeAPI (BETA) DYA ( Ditch YouTube API ) is a package created to power the user with YouTube Data API functionality without any API Key Detai

Sougata Jana 23 Dec 22, 2022
Download Apple Music Cover Artwork in the best Quality by providing an Apple Music Link. It downloads the jpg, png and webp version since they often differ from another.

amogus.py - Version 0.0.5 amogus - Apple Music Hi-Res Artwork Fetcher this is my first real python tool so sorry if its bad amogus is a Python script

reaper 46 Jan 09, 2023
A YouTube downloader which allows you to choose which video you want

Youtube Video Downloader Download multiple videos in one go! How to Use 1.First type the video you want to download 2.On clicking the Search button yo

2 Dec 17, 2021
ComicDownloader - Downloads Comics from readcomiconline.li

ComicDownloader Downloads Comics from readcomiconline.li To use this script from

2 Nov 08, 2022
A small distributed download manager to help bypass device-specific bandwidth limitations.

Distributed Download Manager A small distributed download manager to help bypass device-specific bandwidth limitations. Architecture The download mana

Anand Balaji 3 Sep 23, 2022
A discord bot for downloading youtube video and audio files

disctube disctube is a discord bot for downloading video and audio files from youtube using python pytube. disclaimer i am not the best python program

razor420 3 Feb 03, 2022
Download YouTube videos/music and images in MP4, JPG with this tool.

ABOUT THE TOOL Download YouTube videos, music and images in MP4, JPG with this tool, with an easy to understand interface. This tool works with both,

TrollSkull 5 Jan 02, 2023
A cross-platform python based utility to download courses from udemy for personal offline use.

udemy-dl A cross-platform python based utility to download courses from udemy for personal offline use. Warning Udemy has started to encrypt many of t

Nasir Khan 4.6k Dec 31, 2022
A Fast as F*** Downloader

FAFD A Fast as F*** Downloader Github Usages You'll want to use a URL like this: https://github.com/RPowell-C/FAFD/raw/main/FAFD.py It's easier DONT F

1 Jan 19, 2022
Making the process of downloading youtube videos faster and more convinient.

Easy-YT Making the process of downloading youtube videos faster and more convinient. What can it do? This python script can be used to download youtub

Meynam 39 Nov 15, 2021
YouPlay is a python based tool for downloading YouTube videos through its URL

YouPlay is a python based tool for downloading YouTube videos through its URL. It is capable to download videos from YouTube playlists too and can extract the audio file only from the video. It can r

Nitin Choudhury 10 Sep 15, 2022
This package helps you to directly download an APK from Google Play by providing the package id of the app

Apk Downloader About | Features | Technologies | Requirements | Starting | License | Author 🎯 About This package helps you to directly download an AP

Daniel Agyapong 9 Dec 11, 2022
A Spotify downloader needing only a python interpreter and ffmpeg.

ZSpotify A Spotify downloader needing only a python interpreter and ffmpeg. Discord Server - Matrix Server - Gitea Mirror - Main Site Requirements: B

2.4k Dec 14, 2021
A downloader for Cave Story written in Python

Cave Story Downloader This is a downloader for Cave Story written in Python. Thi

Imsad2 2 Feb 16, 2022
Convert BMS songs to osu! With options to convert keysounds and convert to 7key.

bmx2osu Convert BMS to osu! With options to: convert keysounds to one song file using BMX2WAV include 7k version change Overall Difficulty and HP Drai

7 Nov 28, 2022
This repository contains code for a youtube-dl GUI written in PyQt.

youtube-dl-GUI This repository contains code for a youtube-dl GUI written in PyQt. It is based on youtube-dl which is a Video downloading script maint

M.Yasoob Ullah Khalid ☺ 191 Jan 02, 2023
Will load an SRC page, logged in with Firefox's cookies imported, and delete all comments from every run

SRCCommentsAutoDeleter Will load an SRC page, logged in with a support browser's cookies, and delete all comments from every run Config is all done in

3 Oct 29, 2021
VD Song Bot - A telegram bot that can download songs

VD Song Bot A telegram bot that can download songs Reach me on Telegram @MusicVNDbot Deploy to Heroku The easiest way to deploy this Song Bot Mandator

Venuja Thilakarathna 2 Feb 19, 2022
Python based YouTube video Downloader GUI Application.

Youtube video Downloader Python based Youtube video Downloader GUI Application. Installation Python Dependencies Import pytube pip install pytube Im

Naem Azam 1 Jan 03, 2022
GTK4 + Python tutorial with code examples

Taiko's GTK4 Python tutorial Wanna make apps for Linux but not sure how to start with GTK? This guide will hopefully help! The intent is to show you h

190 Jan 08, 2023