The malicious code injected doesn't clean up the stack after itself which is what prevents it from being injected into arbitrary locations. This also would be the easiest way to detect pickles you've injected into. A "correct" pickle will only leave one value on the stack when everything is done, the pointer to the final object. I've never seen a real pickle not comply with this, so using pickletools.dis or your symbolic interpreter you can detect pickles you've injected into because it leaves two values on the stack whether you inject at the beginning or end.

You can make the injections more covert by adding a pop instruction to the end so that it cleans up after itself. You would then also be able to inject into an arbitrary location like I do in https://github.com/coldwaterq/pickle_injector/blob/main/inject.py.

For replacing the output you would add the pop instruction to the beginning of your payload / end of the real pickle, to throw away everything created and replace it with what you create.

File "/Users/abenavides/workspace/enricher/fury_fda-models-hub-enricher/venvpython3/lib/python3.8/site-packages/fickling/pickle.py", line 106, in new raise NotImplementedError(f"TODO: Add support for Opcode {info.name}") NotImplementedError: TODO: Add support for Opcode BININT

I was trying out something sophisticated with a simple model pre-trained on MNIST. But I git this error.

Traceback (most recent call last):
  File ".\pytorch_poc.py", line 147, in <module>
    exfil_model.pickled.insert_python_exec(
  File ".\pytorch_poc.py", line 58, in pickled
    self._pickled = Pickled.load(pickle_file)
  File "C:\Python38\lib\site-packages\fickling\pickle.py", line 343, in load
    opcodes.append(Opcode(info=info, argument=arg, data=data, position=pos))
  File "C:\Python38\lib\site-packages\fickling\pickle.py", line 105, in __new__
    raise NotImplementedError(f"TODO: Add support for Opcode {info.name}")
NotImplementedError: TODO: Add support for Opcode BINFLOAT

I guess, the project still needs work to allow to make a full-fledged ML-based attack. Any plans for when this will be completed?

Bumps pypa/gh-action-pip-audit from 1.0.2 to 1.0.3.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps pypa/gh-action-pip-audit from 1.0.0 to 1.0.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Resolves issue #30.

When injecting new code, preserve leading PROTO and FRAME opcodes.

Also adds an analysis to detect invalid PROTO opcodes that can be an indicator of tampering.

I'm trying to give the stable diffusion community the ability to trade Textual Inversion embeddings (basically, fine-tuning the model) between each other. When I run fickle against one my embeddings, I see this:

(base) [email protected]:~/Downloads/archive$ fickling -t data.pkl

PROTO EMPTY_DICT Pushed {} BINPUT Memoized 0 -> {} MARK Pushed MARK BINUNICODE Pushed 'string_to_token' BINPUT Memoized 1 -> 'string_to_token' EMPTY_DICT Pushed {} BINPUT Memoized 2 -> {} BINUNICODE Pushed '' BINPUT Memoized 3 -> '' GLOBAL Traceback (most recent call last): File "/home/berble/.local/bin/fickling", line 8, in sys.exit(main()) File "/home/berble/.local/lib/python3.8/site-packages/fickling/cli.py", line 82, in main print(unparse(trace.run())) File "/home/berble/.local/lib/python3.8/site-packages/fickling/tracing.py", line 54, in run self.on_statement(added) File "/home/berble/.local/lib/python3.8/site-packages/fickling/tracing.py", line 38, in on_statement print(f"\t{unparse(statement).strip()}") File "/home/berble/.local/lib/python3.8/site-packages/astunparse/init.py", line 13, in unparse Unparser(tree, file=v) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 38, in init self.dispatch(tree) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch meth(tree) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 113, in _ImportFrom interleave(lambda: self.write(", "), self.dispatch, t.names) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 19, in interleave f(next(seq)) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch meth(tree) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 856, in _alias if t.asname: AttributeError: 'alias' object has no attribute 'asname'

Any idea where I could start looking? We'd really like to be able to share embeddings safely!

Here's a base64-encoded copy of my data.pkl:

gAJ9cQAoWA8AAABzdHJpbmdfdG9fdG9rZW5xAX1xAlgBAAAAKnEDY3RvcmNoLl91dGlscwpfcmVi dWlsZF90ZW5zb3JfdjIKcQQoKFgHAAAAc3RvcmFnZXEFY3RvcmNoCkxvbmdTdG9yYWdlCnEGWAEA AAAwcQdYAwAAAGNwdXEIS010cQlRSwEpKYljY29sbGVjdGlvbnMKT3JkZXJlZERpY3QKcQopUnEL dHEMUnENc1gPAAAAc3RyaW5nX3RvX3BhcmFtcQ5jdG9yY2gubm4ubW9kdWxlcy5jb250YWluZXIK UGFyYW1ldGVyRGljdApxDymBcRB9cREoWAgAAAB0cmFpbmluZ3ESiFgLAAAAX3BhcmFtZXRlcnNx E2gKKVJxFGgDY3RvcmNoLl91dGlscwpfcmVidWlsZF9wYXJhbWV0ZXIKcRVoBCgoaAVjdG9yY2gK RmxvYXRTdG9yYWdlCnEWWAEAAAAxcRdYBgAAAGN1ZGE6MHEYTQADdHEZUUsASwFNAAOGcRpNAANL AYZxG4loCilScRx0cR1ScR6IaAopUnEfh3EgUnEhc1gIAAAAX2J1ZmZlcnNxImgKKVJxI1gbAAAA X25vbl9wZXJzaXN0ZW50X2J1ZmZlcnNfc2V0cSRjX19idWlsdGluX18Kc2V0CnElXXEmhXEnUnEo WA8AAABfYmFja3dhcmRfaG9va3NxKWgKKVJxKlgWAAAAX2lzX2Z1bGxfYmFja3dhcmRfaG9va3Er TlgOAAAAX2ZvcndhcmRfaG9va3NxLGgKKVJxLVgSAAAAX2ZvcndhcmRfcHJlX2hvb2tzcS5oCilS cS9YEQAAAF9zdGF0ZV9kaWN0X2hvb2tzcTBoCilScTFYGgAAAF9sb2FkX3N0YXRlX2RpY3RfcHJl X2hvb2tzcTJoCilScTNYGwAAAF9sb2FkX3N0YXRlX2RpY3RfcG9zdF9ob29rc3E0aAopUnE1WAgA AABfbW9kdWxlc3E2aAopUnE3WAUAAABfa2V5c3E4fXE5aANOc3VidS4=

Hello! Great tool, I like that it also includes a way to check for potentially malicious opcodes in pickle files.

I injected a payload into a stylegan2-ada pickle file and it behaves as expected. :)

Now, when running both --check-safety or --trace commands the following error is shown:

!fickling --check-safety /tmp/network-snapshot-000250.backdoor.pkl

Traceback (most recent call last):
  File "/usr/local/bin/fickling", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/fickling/cli.py", line 79, in main
    return [1, 0][check_safety(pickled)]
  File "/usr/local/lib/python3.7/dist-packages/fickling/analysis.py", line 38, in check_safety
    shortened, already_reported = shorten_code(node)
  File "/usr/local/lib/python3.7/dist-packages/fickling/analysis.py", line 23, in shorten_code
    code = unparse(ast_node).strip()
  File "/usr/local/lib/python3.7/dist-packages/astunparse/__init__.py", line 13, in unparse
    Unparser(tree, file=v)
  File "/usr/local/lib/python3.7/dist-packages/astunparse/unparser.py", line 38, in __init__
    self.dispatch(tree)
  File "/usr/local/lib/python3.7/dist-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/usr/local/lib/python3.7/dist-packages/astunparse/unparser.py", line 113, in _ImportFrom
    interleave(lambda: self.write(", "), self.dispatch, t.names)
  File "/usr/local/lib/python3.7/dist-packages/astunparse/unparser.py", line 19, in interleave
    f(next(seq))
  File "/usr/local/lib/python3.7/dist-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/usr/local/lib/python3.7/dist-packages/astunparse/unparser.py", line 856, in _alias
    if t.asname:
AttributeError: 'alias' object has no attribute 'asname'

Let me know if there is anything more needed to debug the issue.

Greetings!

When attempting to use fickling on PyTorch models I get this error. I believe these models were just the weights. So i'm currious if this is hard to fix, and if you don't have time to fix it, any guidance you can give about the code base to help me attempt to patch it.

Two very simple additions: The INT opcode is used to declare constant integer values. The DICT opcode reads values from the stack until it reaches a MARK opcode, alternating between keys and values.

Take the following script:

import ast
import pickletools
from fickling.pickle import Pickled

if __name__ == '__main__':
	
	pickled = b"(I1\nI2\nd."

	for op, arg, pos in pickletools.genops(pickled):
		print(f"{pos}: {op.name} {arg}")

	ast_data = Pickled.load(pickled).ast
	print(ast.dump(ast_data))

Before, it would output:

0: MARK None
1: INT 1
4: INT 2
7: DICT None
8: STOP None
Traceback (most recent call last):
  File "test.py", line 12, in <module>
    ast_data = Pickled.load(pickled).ast
  File "/home/carlos/.local/lib/python3.7/site-packages/fickling/pickle.py", line 343, in load
    opcodes.append(Opcode(info=info, argument=arg, data=data, position=pos))
  File "/home/carlos/.local/lib/python3.7/site-packages/fickling/pickle.py", line 105, in __new__
    raise NotImplementedError(f"TODO: Add support for Opcode {info.name}")
NotImplementedError: TODO: Add support for Opcode INT

Now it outputs:

0: MARK None
1: INT 1
4: INT 2
7: DICT None
8: STOP None
Module(body=[Assign(targets=[Name(id='result', ctx=Store())], value=Dict(keys=[Constant(value=1)], values=[Constant(value=2)]))])

Thanks for this great tool! This Opcode appears to be necessary to fickle some uses of PyTorch modules such as torch.nn.Linear, which I'd love to have support for.

Here is an example which triggers the following error:

# example.py
import pickle
from fickling.pickle import Pickled
from torch import nn

filename = "model.pt"
model = nn.Linear(2, 1)

with open(filename, "wb") as model_file:
    pickle.dump(model, model_file)

with open(filename, "rb") as model_file:
    pickled = Pickled.load(model_file)

$ python example.py
Traceback (most recent call last):
  File "~/ml-attacks/src/pickle_deserialization/example.py", line 12, in <module>
    pickled = Pickled.load(model_file)
  File "~/ml-attacks/.venv/lib/python3.9/site-packages/fickling/pickle.py", line 343, in load
    opcodes.append(Opcode(info=info, argument=arg, data=data, position=pos))
  File "~/ml-attacks/.venv/lib/python3.9/site-packages/fickling/pickle.py", line 105, in __new__
    raise NotImplementedError(f"TODO: Add support for Opcode {info.name}")
NotImplementedError: TODO: Add support for Opcode EMPTY_SET

Is anything else needed?

Bumps pypa/gh-action-pip-audit from 1.0.2 to 1.0.4.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Hello,

I've been playing around with some alternative ways to execute Python via pickles, and discovered both runpy._run_code and torch.jit.unsupported_tensor_ops.execWrapper can be used to call into exec without fickling detecting it. I have some demo code here that will create pickles using these techniques: https://bitbucket.org/hiddenlayersec/sai/src/master/pytorch_inject/torch_picke_inject.py

runpy._run_code produces no warnings, and execWrapper generates a "Call to execWrapper(...) can execute arbitrary code and is inherently unsafe" warning.

It might be worth adding explicit checks for both of these methods and detecting as overtly bad.

Many thanks btw for the awesome library!

Best regards,

Tom

I'm not so familiar with pickling and these scans. However, I wondered if maybe there are heuristics or signatures for certain types of pickle files that could be evaluated.

If you knew for example that a pickle file should be for a stable diffusion model, some properties could be examined that might help to verify a bit more.

If so, could set up something like a /signatures directoy and let people pull request in definitions, then could scan -security -sig='signatures/typename'

This can be closed, just wanted to pass the idea by in case it could be useful

Right now, pytorch_poc.py injects malicious code contained within the pickle files of the PyTorch standard model format. This and the TorchScript serialization format are ZIP archives with pickle files. It would be great to expand upon those and provide users with easy-to-use functions that can directly manipulate these files since they're relatively common.