• Fixed an issue that was preventing me from importing on python3
• Added type hint removal
• Automatically applied if AST fails to parse the module
• Found In src/python_minifier/transforms/remove_hints.py
• Removal code is generated from selected files from 'strip-hint' package
• All of the original comments from 'strip-hints' are maintained
• A script is included to regenerate 'remove_hints.py' from tlatest master as desired (e.g. if python3 grammar changes to include more funkt syntax in type hints, which has happened several times 😂)

Comments / feedback appreciated! Love your package. ❤

It's probably a hard feature request, but would be great run "pyminify -r hello.py" to minify the file & the libraries its using

e.g. for projects with a lot of dependencies which want to have a .exe or even a .apk, as kivy projects (they tend to be much bigger than normal apks)

is it viable?

I was trying to minify a file that contains lines as below in utf-8 format: print("≡")

Currently it fails with UnicodeEncodeError: 'charmap' codec can't encode character '\u2030' in position 8: character maps to <undefined>

A fix would be if the encoding is specified when the file is opened with open('filename', 'w', encoding='utf-8') as f:

Maybe allow something like? pyminifer --encoding 'utf-8' file.py

When I try to run the converted code it returns the following error: (unicode error) 'utf-8' codec can't decode byte 0xa9 in position 29: invalid start byte

I had a string that had a copyright symbol, I've remove the symbol and it works.

Thanks

An issue related to this is already closed. But it seems that it is not yet completely fixed.

The error occurs on emojis.

This is the exact command I used:

pyminify folder --in-place --remove-literal-statements

Versions:

• python 3.10.7
• python-minifier 2.6.0

Hi, very nice library/application! I'm using it to reduce the size of MyPaint's appimage files (by a fair chunk actually).

Prelude

Version used: 2.3.0

When pre-minifying the libraries I ran into an issue with files that had already been minified before, where pyminify produced invalid parameters. As far as I can tell, this only happens when running with python 2.7, but not 3.5+ (tested with 2.7.5 and 2.7.12).

Problem

The same name is assigned to the *args and the **kwds parameters, yielding invalid code.

Minimal example

Input:

class Q:
	def f(C,a,*A,**B):D='.';super(Q,C).f(a,*A,**B);D or D

when minified, yields the invalid output:

class Q:
	def f(D,a,*C,**C):A='.';super(Q,D).f(a,*B,**C);A or A

where *C really should be *B (classic off by one, but due to some py2/3 semantic difference?)

The minimal example is itself a (valid) minification of:

class Q:

    def f(self, a, *b, **c):
        super(Q, self).f(a, *b, **c)
        '.' or '.'

Hello!

So I noticed that python-minifier, when for example encounters a builtin function used multiple times, assigns it to a variable at the very beginning: E=print However, when it does so multiple times, they are put each in their own line. On Windows, a new line consists of a carriage return and line break. It is 2 bytes on its own. Using a semicolon would take only 1 byte.

How would one go about making this change? I would make a pull request myself if I find any time to do it myself

I opened this issue to discuss, and point out possible drawbacks of this

(I know that by default it inserts only LINE FEED character, but when porting code across multiple machines this might or might not break?)

sys.stdin.read() reads a UTF-8 string, but a file is opened in binary mode (and therefore reads bytes). Either bytes should be read from sys.stdin.buffer, or the file should be opened in text mode.

Hi!

First of all, thank you very much for writing this software. It is very useful to decrease the size of Python's standard library when exposing it for web-apps running with Pyodide, a full CPython 3.8 compiled to web-assembly and running in the browser.

Anyway, it seems I found a bug, because the --remove-literal-statements does not seem to have any effect on the provided sources. These two calls return exactly the same result. Docstrings are everywhere.

$ pyminify  0.py >a.txt 
$ pyminify --remove-literal-statements 0.py  >b.txt 

Attached are the results I got. I'm on pyminify 2.4.1 as installed from PyPI, the used Python version for execution is 3.9.1, but I also tested it with 3.8.2 with the same result.

0.py => this is the __init.py__ of Python 3.8.2's collections standard library package a.txt b.txt

Hi, thanks for this really useful tool.

I have been testing it with some of my files and I see that for some files, the function names get changed but for some they don't. Unfortunately I can't paste the one that doesn't get the names changed. Do you have any reason in mind as to why that could be happening? If not, I will put more work into trying to create a working example that I can share.

For this file, however, the functions do get renamed:

# test_file.py
global_var = 3

def test_function(prep_text, entites):
    '''
    test comment
    '''
    # single line comment
    test_var_name = {
        'blabla': '1',
        'blabla2': '2'
    }

    myset = {'one', 'two', 'three'}
    print(myset)

    return test_var_name


def test_func_1(var1):
    print('bla')

def test_func_2(var2):
    test_func_1(var1)

def test_func_3(var1):
    test_func_2('blabla')

$ pyminify --remove-literal-statements --rename-globals --no-hoist-literals test_file.py
C=print
D=3
def E(prep_text,entites):A={'blabla':'1','blabla2':'2'};B={'one','two','three'};C(B);return A
def A(var1):C('bla')
def B(var2):A(var1)
def F(var1):B('blabla')

Example: class MyTuple(NamedTuple):url:0;domain:0;sitekey:0;kind:0;action:0

Should be: class MyTuple(NamedTuple):url:str;domain:str;sitekey:str;kind:CaptchaKindEnum;action:str

Is there any way to keep this from happening?

First of all, huge thanks for providing this fantastic library! 🙌

We've discovered a small issue with type declarations and transitive class inheritance. If we use this sample code:

from typing import Optional, TypedDict
class A(TypedDict):
    arg1: str
class B(A):
    arg2: Optional[int]
class C(B):
    arg3: Optional[str]

... it would get minified to:

from typing import Optional,TypedDict
class A(TypedDict):arg1:str
class B(A):arg2:0
class C(B):arg3:0

... which is invalid Python code (tested under 3.8, 3.10, but also applies to other versions):

$ python test.py
  ...
  File "~/.pyenv/versions/3.10.4/lib/python3.10/typing.py", line 176, in _type_check
    raise TypeError(f"{msg} Got {arg!r:.100}.")
TypeError: TypedDict('Name', {f0: t0, f1: t1, ...}); each t must be a type Got 0.

The issue is that the library is currently not able to detect transitive inheritance dependencies (for good reasons, as this is a hard problem - probably infeasible to solve in the general case, as class hierarchies may be distributed across different source files, and minification is only performed within the scope of a single source file.)

One solution is to use the remove_annotations=False flag to retain all annotations, but in a larger codebase it can actually be beneficial to remove type declarations, especially from function/method args.

The code in question is this piece: https://github.com/dflook/python-minifier/blob/9748fabffcc5954a326f1e95697d00615a8937b5/src/python_minifier/transforms/remove_annotations.py#L100-L106 The ideal case would be if we can introduce a config flag to disable the transformation that is happening for class attributes in this block.

@dflook Would it make sense to distinguish between (1) function annotations and (2) class annotations, and introduce a new flag remove_annotations_class? We could then either introduce a new SuiteTransformer for class annotations, or alternatively leave the current structure and skip executing visit_AnnAssign if remove_annotations_class is False. Happy to work towards a PR, but would like to get your thoughts and guidance first.. 👍 Thanks!

Hi There,

I have found an issue. The description says that Python 2.7 is supported, but it gives and error on "print" function

Missing parentheses in call to 'print'. Did you mean print(boolObjL)?

Parentheses is not required for print in Python 2.7.

When I try to shorten this code with all options enabled it only removes the spaces.

equation = input().split("=")[1]
grid = [["."]*10 for i in range(10)]
for y in range(10):
    x = int(eval(equation))
    if 0 <= x <= 9:
        grid[y][x] = "o"

print('\n'.join("".join(j for j in i) for i in grid))

Result: Reduction from 217 to 192

equation=input().split('=')[1]
grid=[['.']*10 for i in range(10)]
for y in range(10):
	x=int(eval(equation))
	if 0<=x<=9:grid[y][x]='o'
print('\n'.join((''.join((j for j in i))for i in grid)))

When the eval() is removed from the code it gets shortened properly (203 to 162)

D=range
E=input().split('=')[1]
A=[['.']*10 for A in D(10)]
for C in D(10):
	B=int()
	if 0<=B<=9:A[C][B]='o'
print('\n'.join((''.join((A for A in B))for B in A)))

Modern Python projects define their build requirements in pyproject.toml, e.g.,

[build-system]
requires = [
  "setuptools>=42",
  "wheel",
]
build-backend = "setuptools.build_meta"

It'd be nice addition of python-minifier could be defined as the builder, e.g.,

[build-system]
requires = [
  "setuptools>=42",
  "wheel",
  "python-minifier>=xyz",
]
build-backend = "python_minifier.build_meta"

to spit out minified package code.

This improves the remove_literal_statements-feature and replaces doc-strings in case the module uses doc by the string 'doc-string stripped by python-minifier', which is much shorter in most cases.

Draft for issue #38.