CRC Reverse Engineering Tool in Python

Overview

CRC Beagle

CRC Beagle is a tool for reverse engineering CRCs. It is designed for commnication protocols where you often have several messages of the same length. This allows CRC Beagle to use the CRC Differential Technique described by Gregory Ewing described in Reverse-Engineering a CRC Algorithm

The advantage of this technique is it allows recovery of an "effective equivalent" CRC even in cases where the algorithm uses non-standard parameters for XOR-in or XOR-out (a common obfuscation technique).

The CRC RevEng tool by Greg Cook is a more mature tool, I haven't implemented as much. I started CRC Beagle to (a) use Python which I find much easier to modify, and (b) when CRC RevEng failed to recover a CRC for a device I was looking at, and it was difficult to understand why.

CRC Beagle has some other handy features, such as giving you the code you need to create valid CRCs with a copy-paste. It also checks inputs when running on 8-bit CRCs to see if it's just a simple checksum and not a real CRC.

Hopefully you find CRC Beagle useful, but this is hardly a novel creation, so the credit goes to those who built up the foundation.

Using CRC Beagle

The basic usage is shown in the file demo.py:

from crcbeagle import crcbeagle

crcb = crcbeagle.CRCBeagle()

crcb.search([[165,  16,  2,  7,  85,  163,  209,  114,  21,  131,  143,  144,  52,  187,  183,  142,  180,  39,  169,  76],
        [165,  16,  2,  7,  140,  39,  242,  202,  181,  209,  220,  248,  156,  112,  66,  128,  236,  187,  35,  176],
        [165,  16,  2,  7,  113,  105,  30,  118,  164,  96,  43,  198,  84,  170,  123,  76,  107,  225,  133,  194]],
        
       [[253,  14],
        [90,  38],
        [248,  236]]
)

This generates an output like this:

Input parameters:
    16-bit CRC size
    3 total messages, with:
       3 messages with 20 byte payload
NOTE: Output parameters will be specific to this message size only. Pass different length messages if possible.

Working on messages of 20 length:
  Found single likely solution for differences of len=20, yah!
  Found single XOR-out value for len = 20: 0xCACA
********** example usage *************
import struct
from crccheck.crc import Crc16Base
crc = Crc16Base
def my_crc(message):
  crc._poly = 0x1021
  crc._reflect_input = False
  crc._reflect_output = False
  crc._initvalue = 0x0
  crc._xor_output = 0xCACA
  output_int = crc.calc(message)
  output_bytes = struct.pack("<H", output_int)
  output_list = list(output_bytes)
  return (output_int, output_bytes, output_list)

m = [165, 16, 2, 7, 85, 163, 209, 114, 21, 131, 143, 144, 52, 187, 183, 142, 180, 39, 169, 76]
output = my_crc(m)
print(hex(output[0]))
**************************************
If you have multiple message lengths this solution may be valid for this only.

Important Limitations

The CRC differential technique packs all of the "constant bytes" into the XOR-out parameters.

Constants that occur at the start of the CRC are transformed by the CRC operation. This transformation depends on the number of cyclic shifts - that means the constant changes for different lengths of messages, since the number of cyclic shifts changes every time you 'add' a byte to the CRC.

If you can find the 'actual' XOR-in settings, or how many bytes the operation takes, you will have a more generic function.

However in practice I find that many communication protocols only transmit certain length messages. Thus having different XOR-out values for each message length isn't a major problem for the purpose of interoperating with the original system.

This tool doesn't try to be too clever and just spits out settings for each message length you gave it.

How it Works

While you can just brute-force CRC parameters with a given message, this has some complexities in practice. You may not know what exactly is covered by the CRC - for example most protocols have some 'start of frame' characters. They may also add padding to the message before being passed to the CRC algorhtm.

As described by Gregory Ewing described in Reverse-Engineering a CRC Algorithm, you can take advantage of the fact CRC can be broken down into several components. A typical CRC operation of message m1 could be considered as:

CRC(m1) = CRC(fixedin) ^ CRC(m1') ^ fixedout

Where m1' is a variable portion of m1. Some of the fixedin comes from the CRC algorithm, some of it could come from the CRC of fixed parameters.

This means if you take the XOR of the CRC portion of two messages:

CRC(m1) ^ CRC(m2) = [CRC(fixedin) ^ CRC(m1') ^ fixedout] ^ [CRC(fixedin) ^ CRC(m2') ^ fixedout]

You cancel the common terms, and are left with:

CRC(m1) ^ CRC(m2) = CRC(m1') ^ CRC(m2')

The advantage of this is that we have removed the fixed portion. This is much easier to brute-force since we now only have to worry about what the polynomial of CRC() was (and a few other issues such as bit/byte ordering).

We can take two messages which we have a known CRC for, xor the messages together, and then we can try to simply find the CRC polynomial (ignoring the input & output settings). Any constant terms we can ignore, whether they come from the CRC parameter or the CRC usage (such as including a constant header byte).

With the polynomial known, all the fixed input data CRC(fixedin) becomes a constant we can roll into a single variable. Note that this constant changes with different message lengths, but you can still achieve interoperability in most cases.

Owner
Colin O'Flynn
Colin is a huge nerd.
Colin O'Flynn
Direct Multi-view Multi-person 3D Human Pose Estimation

Implementation of NeurIPS-2021 paper: Direct Multi-view Multi-person 3D Human Pose Estimation [paper] [video-YouTube, video-Bilibili] [slides] This is

Sea AI Lab 253 Jan 05, 2023
Bad Apple printed out on the console with Python!

bad-apple Bad Apple printed out on the console with Python! Preface A word of disclaimer, while the final code is somewhat original, this project is a

CalvinLoke 186 Dec 01, 2022
Mommas-cookbook - A Repository About Mom's Recipes

Mommas Cookbook A Repository for Mom's Recipes Contents bacalhau à Gomes de Sá Beef-Rendang bacalhau à Gomes de Sá, recommended by @s0undt3ch One of t

1 Jan 08, 2022
Vehicle Identification Speed Detection (VISD) extracts vehicle information like License Plate number, Manufacturer and colour from a video and provides this data in the form of a CSV file

Vehicle Identification Speed Detection (VISD) extracts vehicle information like License Plate number, Manufacturer and colour from a video and provides this data in the form of a CSV file. VISD can a

6 Feb 22, 2022
Download and archive entire usenet newsgroups over NNTP.

Usenet Archiving Tool This code is for archiving Usenet discussions, not downloading files. Newsgroup posts are saved under the authors name and email

Corey White 2 Dec 23, 2021
Vita Specific Patches and Application for Doki Doki Literature Club (Steam Version) using Ren'Py PSVita

Doki-Doki-Literature-Club-Vita Vita Specific Patches and Application for Doki Doki Literature Club (Steam Version) using Ren'Py PSVita Contains: Modif

Jaylon Gowie 25 Dec 30, 2022
A lightweight and unlocked launcher for Lunar Client made in Python.

LCLPy LCL's Python Port of Lunar Client Lite. Releases: https://github.com/Aetopia/LCLPy/releases Build Install PyInstaller. pip install PyInstaller

21 Aug 03, 2022
This is a fork of the BakeTool with some improvements that I did to have better workflow.

blender-bake-tool This is a fork of the BakeTool with some improvements that I did to have better workflow. 99.99% of work was done by BakeTool team.

Acvarium 3 Oct 04, 2022
Anki Addon idea by gbrl.sc to see previous ratings of a card in the reviewer

Card History At A Glance Stop having to press card browser and ctrl+i for every card and then WINCING to see it's history of reviews FEATURES Visualiz

Jerry Zhou 11 Dec 19, 2022
LinuxHelper - A collection of utilities for non-technical Linux users accessible via a GUI

Linux Helper A collection of utilities for non-technical Linux users accessible via a GUI This app is still in very early development, expect bugs and

Seth 7 Oct 03, 2022
Runs macOS on linux with qemu.

mac-on-linux-with-qemu Runs macOS on linux with qemu. Pre-requisites qemu-system-x86_64 dmg2img pulseaudio python[click] Usage After cloning the repos

Arindam Das 177 Dec 26, 2022
A simple streamlit webapp with multiple functionality

A simple streamlit webapp with multiple functionality

Omkar Pramod Hankare 2 Nov 24, 2021
Repository voor verhalen over de woningbouw-opgave in Nederland

Analyse plancapaciteit woningen In deze notebook zetten we cijfers op een rij om de woningbouwplannen van Nederlandse gemeenten in kaart te kunnen bre

Follow the Money 10 Jun 30, 2022
Exercise to teach a newcomer to the CLSP grid to set up their environment and run jobs

Exercise to teach a newcomer to the CLSP grid to set up their environment and run jobs

Alexandra 2 May 18, 2022
Transparently load variables from environment or JSON/YAML file.

A thin wrapper over Pydantic's settings management. Allows you to define configuration variables and load them from environment or JSON/YAML file. Also generates initial configuration files and docum

Lincoln Loop 90 Dec 14, 2022
Virtual webcam that takes real webcam footage and replaces the background in order to have Virtual Backgrounds in MS Teams for Linux where the feature is unimplemented.

Background Remover The Need It's been good long while since Microsoft first released a Teams version for Linux and yet, one of Teams' coolest features

Dylan Turner 80 Dec 20, 2022
A submodule of rmcrkd/ODE-Uniqueness

Heston-ODE This repo contains the Heston-related code that accompanies the article One-sided maximal uniqueness for a class of spatially irregular ord

0 Jan 05, 2022
CRC Reverse Engineering Tool in Python

CRC Beagle CRC Beagle is a tool for reverse engineering CRCs. It is designed for commnication protocols where you often have several messages of the s

Colin O'Flynn 51 Jan 05, 2023
Persistent/Immutable/Functional data structures for Python

Pyrsistent Pyrsistent is a number of persistent collections (by some referred to as functional data structures). Persistent in the sense that they are

Tobias Gustafsson 1.8k Dec 31, 2022
Developed a website to analyze and generate report of students based on the curriculum that represents student’s academic performance.

Developed a website to analyze and generate report of students based on the curriculum that represents student’s academic performance. We have developed the system such that, it will automatically pa

VIJETA CHAVHAN 3 Nov 08, 2022