A basic animation modding workflow for FFXIV

Overview

AnimAssist

Provides a quick and easy way to mod animations in FFXIV.

You will need:

  • Before anything, the VC++2012 32-bit Redist from here. Havok will not launch in any way without it. You need VSU_4\vcredist_x86.exe from the link.
  • The executables from this repository. Click releases, download the latest one, and extract the folder somewhere on your system.
  • One of:
    • Python. I wrote anim.py on 3.8.10.
    • BeautifulSoup. pip install bs4 at a terminal or command line.
  • OR:
    • anim.exe, packaged in the release.
  • 3DS Max. No other editors are supported with this method.
  • The HavokMax plugin by PredatorCZ: https://github.com/PredatorCZ/HavokMax
    • You can place this plugin in your 3DS Max plugins folder.
  • FFXIV Explorer: https://github.com/goaaats/ffxiv-explorer-fork
    • OR: Some other method of obtaining raw files from the game. They must be raw files, and not converted in any way.
  • Godbert: https://github.com/xivapi/SaintCoinach

Ok, how do I do this?

Warning: this is fairly technical. It's command line. It's not as easy as using something like Textools or the quick launcher.

Obtaining the files you need

FFXIV's files are only browsable with third party tools, like FFXIV Explorer above, which is what I use. However, it's not always easy to tell which file you need. This is where tools like Godbert come in handy, since there aren't really any easy to use tools for identifying animations or skeletons. I'm going to target an emote for now. I used the Fist Bump emote back when I wrote my C++ converter, and I've been using it again, so we'll use that here. I'll be using Godbert to view game sheets.

Popping open the emote sheet, we can see emote names, their text commands, and a list of "ActionTimeline" columns in the row. sheet

As you can see, not all ActionTimeline entries are filled out. The main one we care about is the first entry - emote/fistbump. While this is actually referring to the timeline, the animation for a timeline is almost always located at the same folder, but for the skeleton that the animation is for. A lot of animations in XIV are built for male Hyur (skeleton 0101) and other races just use that. Hard to tell without using a live path logger.

Anyways, let's just mess with male Hyur's fist bump. We know where it is now. All animation related things, barring cutscene-specific animations, are located in 040000. Pop open FFXIV Explorer, navigate to your game folder, and pop open 040000.win32.index. Let's grab the male Hyur skeleton first:

male hyur skele

FFXIV keeps human skeletons in chara/human/c{skele_id}/skeleton/base/b0001/skl_c{skele_id}b0001.sklb. So, we went there, and we have an sklb file. Extract this as "raw" using File > Extract Raw. Save it somewhere nice. Note that FFXIV Explorer will recreate the FFXIV folder paths. If you select C:\Users\User\Desktop\tmp, it will save the file in C:\Users\User\Desktop\tmp\chara\human\c0101\skeleton\base\b0001\skl_c0101b0001.sklb. That's just the way it is.

Anyways, since we're still on male Hyur, go to chara/human/c0101/animation/bt_common/emote and do the same thing as above, but with fistbump.pap. This is your animation file. Place them both somewhere save, as you'll need both of them later to put the animation back into the game.

Using AnimAssist

AnimAssist requires two files: anim.exe and animassist.exe. Or, if you're just running the Python script itself, anim.py and animassist.exe. I'll be using anim.exe rather than using the python script directly.

The command for turning the skeleton and animation files into something you can edit is:

anim.exe extract -s skeleton_file.sklb -p pap_file.pap -o output_file.hkx

where output_file.hkx will be a Havok file that you can open in 3DS Max using HavokMax.

This is what it looks like when I run this command on the files we just obtained: command

Note the skeleton warning. Skeletons mismatch don't produce an error, and will break literally everything ever.

Opening the file properly

Open 3DS Max and check the Plug-in Manager to ensure HavokMax is properly installed.

From the Max menu, click Import and select your output_file.hkx from the previous step to open it in 3DS Max. Select disable scale, invert top, and set Back: Top. Click import, and you'll get a wonderful fistbumping skeleton, read directly from what was once game files.

skeleton

Saving the file properly

You have some sweet changes to some animation, but now we need to put it back into the game.

From the Max menu, click Export and type any filename you want, and save it in a cool, dry place. In the "Save as type" drop-down, select "Havok Export (*.HKA, *.HKX, *.,HKT)". We will be saving an hkx file once more. Export your file, again, selecting disable scale, invert top, and set Back: Top. You must select those, as well as Export animation and include skeleton. Optimize tracks is optional.

Using AnimAssist (cont)

You have a file containing a skeleton and an animation. But it's Havok. We need AnimAssist to turn it into an animation file for the game.

That command is:

anim.exe pack -s original_skeleton_file.sklb -p original_pap_file.pap -a modified_animation.hkx -o new_ffxiv_animation.pap

where new_ffxiv_animation.pap is an animation file, ready to be imported into the game.

pack The output will look like this. Again, note the skeleton warning.

That's great. How do I put it into the game? What?

There are two methods.

FFXIV Textools

The love hate relationship we all have. Assuming you wanted to replace the animation you originally exported (in my case, chara/human/c0101/animation/a0001/bt_common/emote/fistbump.pap),

Steps:

  • Open Textools
  • Select Tools > Raw File Operations > Import Raw File
  • Enter the path to the game's file in the box. Note: You can right-click on files in FFXIV Explorer to copy their path.
  • Select the new pap file from the file selector button TexTools should import the file. If it doesn't, that kinda sucks.

Penumbra

I use Penumbra to develop things like that modify game files because it's far easier. Penumbra installation will not be covered here, but it is assumed you have a mod directory set up and other Penumbra mods work properly. Steps:

  • Create a new mod using the + button in the bottom left of the Installed Mods menu in Penumbra.
  • Name the mod.
  • Navigate to the mod's folder.
  • Recreate the directory structure up to the expected file. If you exported with FFXIV Explorer, you can copy that folder structure, but be careful you don't overwrite your new pap file.
  • Place your new pap file in the expected location by the game. For example, when I used AnimAssist to create my new pap file, I named it fistbump_new.pap, but when I copy it into the mod folder, I need to name it fistbump.pap because that's what the game is expecting.
  • Go to Settings, and click Rediscover Mods.
  • Go back to Installed Mods, and your replacement file should appear in the list.
  • Click Enable on the mod's checkbox.
  • You may have to swap races or jobs to get the game to reload an animation file live.

Congrats. You have an animation.

comparison

Known Issues

Pap files with multiple animations will ask which animation to extract, but I didn't test this. I am not going to be adding support for recreating a pap file with multiple animations in it.

This has been barely tested, and was developed over 3 days. I was still fixing stuff when I wrote this README.

Technical Details

I was originally writing these tools to be used with the Havok content tools for asset merging and conversion manually, as well as manual XML editing. But I was like, I could just automate that too.

Python handles:

  • Parsing XIV sklb (skeleton) and pap (animation) files
  • Parsing Havok XML packfiles for the skeleton remapping functionality
  • Calling the C++ executable

C++ handles:

  • Conversion between Havok proprietary filetypes

The overall flow goes as follows:

  • Skeleton and animation are obtained from the game in the form of the sklb and pap files.
    • Both file types have a header, and some optional info in the beginning before the Havok data.
    • This Havok data is directly readable by just about any Havok tool. They are Havok Binary Tagfiles, which means they aren't specific to any platform.
  • The following occurs when "extracting":
    • The sklb and pap are read, and their Havok data written to temporary files.
    • Those temporary files are combined in C++ by invoking animassist.exe.
    • A Havok Binary Packfile is written out with both the skeleton and animation present. HavokMax does not support binary or XML tagfiles, so this is how it had to be handled.
  • The following occurs when "packing":
    • The original sklb and pap are read, with the sklb Havok data being written to a temporary file.
    • The temporary sklb is converted in C++ to a Havok XML Packfile. (The output of HavokMax is the same type)
    • Python parses both XMLs with beautifulsoup to determine how to remap the bone indices, since HavokMax rewrites the transform track order differently from how the skeleton actually is, and we cannot include a skeleton as XIV already has it loaded.
    • With the transform tracks remapped properly, that text is written out, and converted to a Havok Binary Tagfile using C++ again, but this time, it removes the skeleton from the file. The game is expecting a Binary Tagfile, so I just didn't want to rock the boat too much. Other formats might work.
    • Python then reads the data preceding the Havok data and the data after the Havok data in the original pap, fixing the offset to the data after the Havok data based on the size of the new Havok data.
    • Python writes pre-Havok data, new Havok data, and post-Havok data to a new pap file. Done.

Building and Contributions

Note

I feel bad that this workflow was thought up, designed, and then kept a secret. That's not what the XIV developer community is about. While there are things people don't share for certain reasons - be it legal, be it automation, monetization should not be one of them. That's my belief. Note that this project's license is WTFPL.

Contributions

I would prefer forks rather than contributions to this repository. Please do whatever you want to make modding more fun and easy for others, but I ask you make it public.

This repository would not have been possible without PredatorCZ's immense work on HavokLib and HavokMax.

Building

Building animassist.exe requires the Havok 2014 SDK and an env var of HAVOK_SDK_ROOT set to the directory.

You can find the Havok SDK to compile with in the description of this video. Please note that is NOT a download I control, just a random one from online.

Comments
  • Anim not... working?

    Anim not... working?

    I'm trying to convert a skeleton and a animation file to .hkx but I got this error everytime I try to run the code. Am I missing something?

    With the file "anim.py" on the folder: Screenshot_4

    Without the file "anim.py" on the folder: Screenshot_5

    opened by jovemlex 3
  • Can not use anim to pack the hxk file

    Can not use anim to pack the hxk file

    QQ图片20220413075102 Hello I have some problems using it. when I use anim to pack the hxk it show me this message. I don't know much about python. Can you help me figure out how to solve it?

    opened by yangjimzero 0
  • Error with Edge of Shadow

    Error with Edge of Shadow

    Edge of Shadow and also Oka both cause 3ds max to crash and fail on import. Any change on how to fix this?

    If anyone has the XML animation file for edge of shadow I'd greatly appreciate it.

    opened by XCZA 0
  • Animations not working on male viera

    Animations not working on male viera

    Male viera potentially different from other skeletons. Animations that work on other races have issues on Male Viera where the skeleton goes all over the place.

    Any ideas on a fix? Male Viera uses c1701 however file explorer doesn't seem to have that.

    opened by TetsunoWondel 0
  • Animation not importing correctly

    Animation not importing correctly

    Using max 2020 and the latest v1.10 import plugin(AnimAssist). Q1: 01 Animation files: \chara\human\c0101\animation\a0001\bt_common\ability\2sw_dark\abl029.pap abl029.zip

    Q2: Honeycam 2022-03-22 10-25-10 Animation files: \chara\human\c0101\animation\a0001\bt_2sw_emp\resident\idle.pap idle.zip

    opened by xyunhe0729 0
Releases(1.0.1)
Owner
liam
I think C# is a good language
liam
Lenovo Yoga Ideapad Autocharge

Description This program uses the conservation_mode of Lonovo Ideapad / Yoga not

1 Jan 09, 2022
Back-end API for the reternal framework

RE:TERNAL RE:TERNAL is a centralised purple team simulation platform. Reternal uses agents installed on a simulation network to execute various known

Joey Dreijer 7 Apr 15, 2022
OLDBot (Online Lessons Discord Bot)

This program is designed to facilitate online lessons. With this you don't need to get up early. Just config and watch the program resolve itself. It automatically enters to the lesson at the specifi

Da4ndo 1 Nov 21, 2021
Why write code when you can import it directly from GitHub Copilot?

Copilot Importer Why write code when you can import it directly from GitHub Copilot? What is Copilot Importer? The copilot python module will dynamica

Mythic 41 Jan 04, 2023
Mini-calculadora escrita como exemplo para uma palestra relâmpago sobre `git bisect`

Calculadora Mini-calculadora criada para uma palestra relâmpado sobre git bisect. Tem até uma colinha! Exemplo de uso Modo interativo $ python -m calc

Eduardo Cuducos 3 Dec 14, 2021
A turtlebot auto controller allows robot to autonomously explore environment.

A turtlebot auto controller allows robot to autonomously explore environment.

Yuliang Zhong 1 Nov 10, 2021
Easy Alias's for bash

easy-alias Easy Alias's for bash Setup Your system needs to have 'echo' which every 21st century computer has You dont need any python requirments but

Hashm 2 Jan 18, 2022
Terminal compatible with ansi-bbs. Meant to be a prototype, but published because why not.

pybbsterm: Terminal emulator for calling BBSs. Use cases (non-exhaustive) Explore terminal protocols. Connect to BBSs. Highlights Python 3.8+ code. Bu

Roc Vallès i Domènech 9 Apr 29, 2022
Module for working with the site dnevnik.ru with python

dnevnikru Module for working with the site dnevnik.ru with python Dnevnik object accepts login and password from the dnevnik.ru account Methods: homew

Aleksandr 21 Nov 21, 2022
RFDesign - Protein hallucination and inpainting with RoseTTAFold

RFDesign: Protein hallucination and inpainting with RoseTTAFold Jue Wang (juewan

139 Jan 06, 2023
Proyecto desarrollado para el programa #FutureDevelopers, tabla periódica interactiva.

Tabla_Periodica Proyecto desarrollado para el programa #FutureDevelopers, tabla periódica interactiva. Descripcion primer entregable: Tabla periodica

1 Dec 04, 2021
An evolutionary multi-agent platform based on mesa and NEAT

An evolutionary multi-agent platform based on mesa and NEAT

Valerio1988 6 Dec 04, 2022
A scuffed remake of Kahoot... Made by Y9 and Y10 SHSB

A scuffed remake of Kahoot... Made by Y9 and Y10 SHSB

Tobiloba Kujore 3 Oct 28, 2022
🍕 A small app with capabilities ordering food and listing them with pub/sub pattern

food-ordering A small app with capabilities ordering food and listing them. Prerequisites Docker Run Tests docker-compose run --rm web ./manage.py tes

Muhammet Mücahit 1 Jan 14, 2022
pybicyclewheel calulates the required spoke length for bicycle wheels

pybicyclewheel pybicyclewheel calulates the required spoke length for bicycle wheels. (under construcion) - homepage further readings wikipedia bicyc

karl 0 Aug 24, 2022
Replay Felica Exchange For Python

FelicaReplay Replay Felica Exchange Description Standalone Replay Module Usage Save FelicaRelay (=2.0) output to file, then python replay.py [FILE].

3 Jul 14, 2022
Python library to natively send files to Trash (or Recycle bin) on all platforms.

Send2Trash -- Send files to trash on all platforms Send2Trash is a small package that sends files to the Trash (or Recycle Bin) natively and on all pl

Andrew Senetar 224 Jan 04, 2023
Script Repository for the ICGM-CNRS FRANCE

Here you will find my Python Work repesitory for the ICGM institute - Montpellier - France.

CABOS Matthieu 1 Apr 13, 2022
Convert Roman numerals to modern numerals and vice-versa

Roman Numeral Conversion Utilities This is a utility module for converting from and to Roman numerals. It supports numbers upto 3,999,999, using the v

Fictive Kin 1 Dec 17, 2021
Print 'text color' and 'text format' on Term with Python

term-printer Print 'text color' and 'text format' on Term with Python ※ It may not work depending on the OS and shell used. PIP $ pip install term-pri

ななといつ 10 Nov 12, 2022