Find index entries in $INDEX_ALLOCATION attributes

Last update: Nov 05, 2022

Overview

INDXRipper

Find index entries in $INDEX_ALLOCATION attributes

Timeline created using mactime.pl on the combined output of INDXRipper and fls.
See: sleuthkit

Motivation

$INDEX_ALLOCATION attributes are used by NTFS directories to store index entries for the files in the directory.

A file's index entry contains the following information:

File name
File size
Allocated size of file (size on disk)
A set of MACB timestamps

The slack space of these attributes may contain index entries of deleted files, even after thier MFT record was recycled. Finding these index entries may help you prove a file existed on a system.

How it works

INDXRipper scans the MFT for records of directories that have an $INDEX_ALLOCATION attribute. If it finds such a record, it searches the attribute for file references to this record. Since the index entries in the attribute represent children of the directory, the $FILE_NAME attributes in them must contain this file reference.

This way, It is able to find entries most other tools aren't.
Finding the full paths of directories is done by using the parent directory reference in $FILE_NAME attributes inside the MFT records.

Features and Details

These are pretty standard but here's a list anyways

Applies fixups for index records and mft records.
Supports $INDEX_ALLOCATION and $FILE_NAME attributes in extension records
Supports unicode filenames
Index entries from orphan directories are listed under "/$Orphan"
Provides bodyfile output for supertimeline creation
Works on live windows systems using the "\\.\" notation
All times outputted are UTC times

Installation

Python 3.8 or above is required.
Use the package manager pip to install construct.

pip install construct==2.10.56

Alternatively, you can use the Windows standalone executable.

Usage

# process dead disk image, get all index entries
python INDXRipper.py ntfs.part.001 output.csv

# process live system, slack space only, bodyfile output, append "C:" to all the paths
python INDXRipper.py -m C: --slack-only --bodyfile \\.\C: output.bodyfile

Limitations

Entries that are partially overitten may not be found. If they are found, though, the tool may give you false information

What this tool doesn't do

This tool doesn't process $INDEX_ROOT attributes. You won't see an output for every file on the volume
This tool doesn't carve $INDEX_ALLOCATION attributes. It won't find attributes that thier MFT entry was recycled.

License

MIT

Bleach is an allowed-list-based HTML sanitizing library that escapes or strips markup and attributes

Bleach Bleach is an allowed-list-based HTML sanitizing library that escapes or strips markup and attributes. Bleach can also linkify text safely, appl

2.5k Dec 29, 2022

Tweak the form field rendering in templates, not in python-level form definitions. CSS classes and HTML attributes can be altered.

django-widget-tweaks Tweak the form field rendering in templates, not in python-level form definitions. Altering CSS classes and HTML attributes is su

1.8k Jan 2, 2023

Allow foreign key attributes in list_display with '__'

django-related-admin Allow foreign key attributes in Django admin change list list_display with '__' This is based on DjangoSnippet 2996 which was mad

62 Nov 18, 2022

Tweak the form field rendering in templates, not in python-level form definitions. CSS classes and HTML attributes can be altered.

django-widget-tweaks Tweak the form field rendering in templates, not in python-level form definitions. Altering CSS classes and HTML attributes is su

1.8k Jan 6, 2023

Fader Networks: Manipulating Images by Sliding Attributes - NIPS 2017

FaderNetworks PyTorch implementation of Fader Networks (NIPS 2017). Fader Networks can generate different realistic versions of images by modifying at

753 Dec 23, 2022

Bleach is an allowed-list-based HTML sanitizing library that escapes or strips markup and attributes

Bleach Bleach is an allowed-list-based HTML sanitizing library that escapes or strips markup and attributes. Bleach can also linkify text safely, appl

2.5k Dec 29, 2022

This is an example of how to automate Ridit Analysis for a dataset with large amount of questions and many item attributes

1 Nov 17, 2021

dta Convert Dict To Attributes!

dta (Dict to Attributes) dta is very small dict (or json) to attributes converter. It is only have 1 files and applied to every python versions.

0 Dec 31, 2021

SAN for Product Attributes Prediction

SAN Heterogeneous Star Graph Attention Network for Product Attributes Prediction This repository contains the official PyTorch implementation for ADVI

9 Dec 12, 2022

It is a Blender Tool which can convert the Object Data Attributes in face corner to the UVs or Vertex Color.

Blender_ObjectDataAttributesConvertTool It is a Blender Tool which can convert the Object Data Attributes in face corner to the UVs or Vertex Color. D

2 Jan 8, 2022

Two predictive attributes (Speed and Angle) and one attribute target (Power)

Two predictive attributes (Speed and Angle) and one attribute target (Power). A container crane has the function of transporting containers from one point to another point. The difficulty of this task lies in the fact that the container is connected to the bridge crane by cables causing an opening angle while the container is being transported, interfering with the operation at high speeds due to oscillation that occurs at the end point, which could cause accidents.

1 Jan 11, 2022

Pcos-prediction - Predicts the likelihood of Polycystic Ovary Syndrome based on patient attributes and symptoms

PCOS Prediction 🥼 Predicts the likelihood of Polycystic Ovary Syndrome based on

1 Jan 10, 2022

A very simple tool to rewrite parameters such as attributes and constants for OPs in ONNX models. Simple Attribute and Constant Modifier for ONNX.

sam4onnx A very simple tool to rewrite parameters such as attributes and constants for OPs in ONNX models. Simple Attribute and Constant Modifier for

6 May 15, 2022

Foreign exchange rates, Bitcoin price index and currency conversion using ratesapi.io

forex-python Forex Python is a Free Foreign exchange rates and currency conversion. Note: Install latest forex-python==1.1 to avoid RatesNotAvailableE

540 Jan 5, 2023

The Python Package Index

Warehouse Warehouse is the software that powers PyPI. See our development roadmap, documentation, and architectural overview. Getting Started You can

3.1k Jan 1, 2023

Get Air Quality Index for your city/country 😷

Air Quality Index CLI Get Air Quality index for your City. Installation $ pip install air-quality-cli Contents Air Quality Index CLI Installation Cont

40 Oct 21, 2022

A Python media index

pyvideo https://pyvideo.org is simply an index of Python-related media records. The raw data being used here comes out of the pyvideo/data repo. Befor

235 Dec 24, 2022

Scan, index, and archive all of your paper documents

[ en | de | el ] Important news about the future of this project It's been more than 5 years since I started this project on a whim as an effort to tr

7.8k Jan 6, 2023

A Python media index

pyvideo https://pyvideo.org is simply an index of Python-related media records. The raw data being used here comes out of the pyvideo/data repo. Befor

235 Dec 24, 2022

Comments

Issue running on SIFT: ModuleNotFoundError: No module named 'construct'
When running INDXParser.py against both a single E01 and a series of split E01 files, I am presented with the following error:

INDXRipper: error: invalid volume boot record

All of the following command produce the same error Single E01:

python3.9 ../tools/INDXRipper/INDXRipper.py /mnt/hgfs/testing/Disk1.E01 outfile

Split E01, with offset (2048)

python3.9 ../tools/INDXRipper/INDXRipper.py -o 2048 /mnt/hgfs/testing/Disk2-Collection/Disk2.E01 outfile

Split E01, with calculated offset (2048x512)

python3.9 ../tools/INDXRipper/INDXRipper.py -o 1048576 /mnt/hgfs/testing/Disk2-Collection/Disk2.E01 outfile
help wanted
opened by ezaspy 8

construct.core.StreamError: Error in path (parsing)

I have followed the updated instructions for setting up a venv and have ran into the following error:

$ sudo venv/bin/python tools/INDXRipper/INDXRipper.py -w csv -o 576716800 /mnt/i30_disk1.E01/ewf1 outfile
Traceback (most recent call last):
  File "/opt/elrond/elrond/tools/INDXRipper/INDXRipper.py", line 309, in <module>
    main()
  File "/opt/elrond/elrond/tools/INDXRipper/INDXRipper.py", line 300, in main
    vbr = get_boot_sector(raw_image, args.o * args.b)
  File "/opt/elrond/elrond/tools/INDXRipper/ntfs.py", line 160, in get_boot_sector
    return BOOT_SECTOR.parse_stream(raw_image)
  File "/opt/elrond/elrond/venv/lib/python3.9/site-packages/construct/core.py", line 300, in parse_stream
    return self._parsereport(stream, context, "(parsing)")
  File "/opt/elrond/elrond/venv/lib/python3.9/site-packages/construct/core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
  File "/opt/elrond/elrond/venv/lib/python3.9/site-packages/construct/core.py", line 2120, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/opt/elrond/elrond/venv/lib/python3.9/site-packages/construct/core.py", line 312, in _parsereport
    obj = self._parse(stream, context, path)
  File "/opt/elrond/elrond/venv/lib/python3.9/site-packages/construct/core.py", line 4101, in _parse
    stream_read(stream, pad, path)
  File "/opt/elrond/elrond/venv/lib/python3.9/site-packages/construct/core.py", line 91, in stream_read
    raise StreamError("stream read less than specified amount, expected %d, found %d" % (length, len(data)), path=path)
construct.core.StreamError: Error in path (parsing)
stream read less than specified amount, expected 3, found 0

I have two offsets for the disk:

2048 (x512) = 1048576
1126400 (x512) = 576716800

And I can't not run it as sudo, as the mount point permissions for /mnt/i30_disk1/ewf1 cannot be altered:

chmod: changing permissions of '/mnt/i30_disk1.E01': Function not implemented
chmod: changing permissions of '/mnt/i30_disk1.E01/ewf1': Function not implemented

opened by ezaspy 2

Releases(v5.2.7)

v5.2.7(Oct 6, 2022)
What's new?

Better error handling

Source code(tar.gz)
Source code(zip)
INDXRipper-5.2.7-py3.9-amd64.zip(5.80 MB)
Test.Image.zip(267.65 KB)
v5.2.6(May 27, 2022)
What's new?

A slight performance improvement

A test image for tool validation

Source code(tar.gz)
Source code(zip)
INDXRipper-5.2.6-py3.9-amd64.zip(5.73 MB)
Test.Image.zip(267.65 KB)
v5.2.5(Mar 14, 2022)
What's new?

Fixed an issue with certain cluster sizes

Fixed #5

Source code(tar.gz)
Source code(zip)
INDXRipper-5.2.5-py3.9-amd64.zip(6.47 MB)
v5.2.4(Feb 20, 2022)
What's new?

Minor changes

Source code(tar.gz)
Source code(zip)
INDXRipper-5.2.4-py3.9-amd64.zip(6.47 MB)
v5.2.3(Feb 12, 2022)
What's new?

Better path reconstruction

Source code(tar.gz)
Source code(zip)
INDXRipper-5.2.3-py3.9-amd64.zip(6.47 MB)
v5.2.2(Feb 11, 2022)
What's new?

A better carving method

Deleted directories are not parsed by default

Escape quotes in CSV output

Better handling of invalid fixups in index records

A validity check on index records in deleted directories

Source code(tar.gz)
Source code(zip)
INDXRipper-5.2.2-py3.9-amd64.zip(6.47 MB)
v5.0.1(Jan 12, 2022)
What's new?

Fixed #6

Source code(tar.gz)
Source code(zip)
INDXRipper-5.0.1-py3.9-amd64.zip(6.47 MB)
v5.0.0(Jan 7, 2022)
What's new?

Replaced some stupid functionality with some not-so-stupid functionality

Huge performance boost

Source code(tar.gz)
Source code(zip)
INDXRipper-5.0.0-py3.9-amd64.zip(6.47 MB)

Owner

Forensic Analyst

GitHub Repository

Find index entries in $INDEX_ALLOCATION attributes

Related tags

Overview

INDXRipper

Motivation

How it works

Features and Details

Installation

Usage

Limitations

What this tool doesn't do

License

You might also like...

Bleach is an allowed-list-based HTML sanitizing library that escapes or strips markup and attributes

Tweak the form field rendering in templates, not in python-level form definitions. CSS classes and HTML attributes can be altered.

Allow foreign key attributes in list_display with '__'

Tweak the form field rendering in templates, not in python-level form definitions. CSS classes and HTML attributes can be altered.

Fader Networks: Manipulating Images by Sliding Attributes - NIPS 2017

Bleach is an allowed-list-based HTML sanitizing library that escapes or strips markup and attributes

This is an example of how to automate Ridit Analysis for a dataset with large amount of questions and many item attributes

dta Convert Dict To Attributes!

SAN for Product Attributes Prediction

It is a Blender Tool which can convert the Object Data Attributes in face corner to the UVs or Vertex Color.

Two predictive attributes (Speed and Angle) and one attribute target (Power)

Pcos-prediction - Predicts the likelihood of Polycystic Ovary Syndrome based on patient attributes and symptoms

A very simple tool to rewrite parameters such as attributes and constants for OPs in ONNX models. Simple Attribute and Constant Modifier for ONNX.

Foreign exchange rates, Bitcoin price index and currency conversion using ratesapi.io

The Python Package Index

Get Air Quality Index for your city/country 😷

A Python media index

Scan, index, and archive all of your paper documents

A Python media index

Comments

Issue running on SIFT: ModuleNotFoundError: No module named 'construct'

construct.core.StreamError: Error in path (parsing)

Releases(v5.2.7)

v5.2.7(Oct 6, 2022)

What's new?

v5.2.6(May 27, 2022)

What's new?

v5.2.5(Mar 14, 2022)

What's new?

v5.2.4(Feb 20, 2022)

What's new?

v5.2.3(Feb 12, 2022)

What's new?

v5.2.2(Feb 11, 2022)

What's new?

v5.0.1(Jan 12, 2022)

What's new?

v5.0.0(Jan 7, 2022)

What's new?

Owner

HTTP load generator, ApacheBench (ab) replacement, formerly known as rakyll/boom

A pytest plugin to skip `@pytest.mark.slow` tests by default.

The evaluator covering all of the metrics required by tasks within the DUE Benchmark.

pywinauto is a set of python modules to automate the Microsoft Windows GUI

The successor to nose, based on unittest2

Parameterized testing with any Python test framework

Turn any OpenAPI2/3 and Postman Collection file into an API server with mocking, transformations and validations.

Pytest-rich - Pytest + rich integration (proof of concept)

Make Selenium work on Github Actions

🏃💨 For when you need to fill out feedback in the last minute.

Generate random test credit card numbers for testing, validation and/or verification purposes.

A utility for mocking out the Python Requests library.

Just for testing video streaming using pytgcalls.

Main purpose of this project is to provide the service to automate the API testing process

Enabling easy statistical significance testing for deep neural networks.

Whatsapp messages bulk sender using Python Selenium.

This is a Python script for Github Bot which uses Selenium to Automate things.

Repository for JIDA SNP Browser Web Application: Local Deployment

自动化爬取并自动测试所有swagger-ui.html显示的接口

Simple assertion library for unit testing in python with a fluent API