contextlib2 is a backport of the standard library's contextlib module to earlier Python versions.

Python 3.7 added async/await support to the contextlib module (e.g. asyncontextmanager and AsyncExitStack), and then Python 3.10 added more (e.g. aclosing).

This raised the question of how to backport those features to 3.6+ (the first version with support for yield inside async def).

The answer turned out to be "procrastinate on the question for 4+ years, so it became feasible to simply drop support for Python 3.5 and earlier, and use the async generator code as-is".

(Issue description updated Jun 2021 to describe what actually happened, rather than the more complicated alternatives I was considering to allow adding the new async features without dropping support for Python 3.5 and earlier)

Just normal build, install and test cycle used on building package from non-root account:

• "setup.py build"
• "setup.py install --root </install/prefix>"
• "pytest with PYTHONPATH pointing to setearch and sitelib inside </install/prefix>

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-contextlib2-0.6.0.post1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-contextlib2-0.6.0.post1-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ PYTHONDONTWRITEBYTECODE=1
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.9, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/tkloczko/rpmbuild/BUILD/contextlib2-0.6.0.post1
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, httpbin-1.0.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, case-1.5.3, isort-1.3.0, aspectlib-1.5.2, asyncio-0.15.1, toolbox-0.5, xprocess-0.17.1, aiohttp-0.3.0, checkdocs-2.7.0, mock-3.6.1, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, cases-3.6.1, flaky-3.7.0, hypothesis-6.14.0, benchmark-3.4.1, xdist-2.3.0, Faker-8.8.1
collected 81 items

. .                                                                                                                                                                  [  1%]
test_contextlib2.py ...............................................FFFFFFF.........................                                                                  [100%]

================================================================================= FAILURES =================================================================================
__________________________________________________________________ TestRedirectStream.test_instance_docs ___________________________________________________________________

self = <test_contextlib2.TestRedirectStream object at 0x7ff603109fd0>

    @requires_docstrings
    def test_instance_docs(self):
        # Issue 19330: ensure context manager instances have good docstrings
        cm_docstring = self.redirect_stream.__doc__
>       obj = self.redirect_stream(None)
E       TypeError: 'NoneType' object is not callable

test_contextlib2.py:842: TypeError
_______________________________________________________________ TestRedirectStream.test_no_redirect_in_init ________________________________________________________________

self = <test_contextlib2.TestRedirectStream object at 0x7ff6028e6d00>

    def test_no_redirect_in_init(self):
>       orig_stdout = getattr(sys, self.orig_stream)
E       TypeError: getattr(): attribute name must be string

test_contextlib2.py:846: TypeError
______________________________________________________________ TestRedirectStream.test_redirect_to_string_io _______________________________________________________________

self = <test_contextlib2.TestRedirectStream object at 0x7ff602845340>

    def test_redirect_to_string_io(self):
        f = io.StringIO()
        msg = "Consider an API like help(), which prints directly to stdout"
>       orig_stdout = getattr(sys, self.orig_stream)
E       TypeError: getattr(): attribute name must be string

test_contextlib2.py:853: TypeError
______________________________________________________________ TestRedirectStream.test_enter_result_is_target ______________________________________________________________

self = <test_contextlib2.TestRedirectStream object at 0x7ff602645520>

    def test_enter_result_is_target(self):
        f = io.StringIO()
>       with self.redirect_stream(f) as enter_result:
E       TypeError: 'NoneType' object is not callable

test_contextlib2.py:862: TypeError
__________________________________________________________________ TestRedirectStream.test_cm_is_reusable __________________________________________________________________

self = <test_contextlib2.TestRedirectStream object at 0x7ff602845760>

    def test_cm_is_reusable(self):
        f = io.StringIO()
>       write_to_f = self.redirect_stream(f)
E       TypeError: 'NoneType' object is not callable

test_contextlib2.py:867: TypeError
_________________________________________________________________ TestRedirectStream.test_cm_is_reentrant __________________________________________________________________

self = <test_contextlib2.TestRedirectStream object at 0x7ff6021ed070>

    def test_cm_is_reentrant(self):
        f = io.StringIO()
>       write_to_f = self.redirect_stream(f)
E       TypeError: 'NoneType' object is not callable

test_contextlib2.py:879: TypeError
____________________________________________________________ TestRedirectStream.test_cm_is_exitstack_compatible ____________________________________________________________

self = <test_contextlib2.TestRedirectStream object at 0x7ff6028667f0>

    def test_cm_is_exitstack_compatible(self):
        with ExitStack() as stack:
            # This shouldn't raise an exception.
>           stack.enter_context(self.redirect_stream(io.StringIO()))
E           TypeError: 'NoneType' object is not callable

test_contextlib2.py:892: TypeError
========================================================================= short test summary info ==========================================================================
FAILED test_contextlib2.py::TestRedirectStream::test_instance_docs - TypeError: 'NoneType' object is not callable
FAILED test_contextlib2.py::TestRedirectStream::test_no_redirect_in_init - TypeError: getattr(): attribute name must be string
FAILED test_contextlib2.py::TestRedirectStream::test_redirect_to_string_io - TypeError: getattr(): attribute name must be string
FAILED test_contextlib2.py::TestRedirectStream::test_enter_result_is_target - TypeError: 'NoneType' object is not callable
FAILED test_contextlib2.py::TestRedirectStream::test_cm_is_reusable - TypeError: 'NoneType' object is not callable
FAILED test_contextlib2.py::TestRedirectStream::test_cm_is_reentrant - TypeError: 'NoneType' object is not callable
FAILED test_contextlib2.py::TestRedirectStream::test_cm_is_exitstack_compatible - TypeError: 'NoneType' object is not callable
======================================================================= 7 failed, 73 passed in 7.07s =======================================================================

Dropped support for Python 2.6 as it reached EOL in 2013 and not supported by both tox and setuptools anymore Enabled back Travis CI testing in pypy3 env, using stable python 3.7 Updated setup.py according to all the changes Fixed several formatting issues to better comply with PEP8

The contextmanager decorated function can be used as decorator for sync functions. But asynccontextmanager decorated function cannot be used as decorator for async function. It would be nice to allow it

In Python 2.7, StreamReader and StreamWriter are context managers that wrap around a stream and handle encoding issue. However, the type of one of these objects is instance, not file. As a result, ExitStack.enter_context with AttributeError when one of these is added to the stack:

MWE:

# -*- coding: utf-8 -*-
from contextlib2 import ExitStack
from codecs import getreader

with open('test_file', 'wb') as f:
    f.write(u'いちりきつくっ'.encode('UTF-8'))

reader = getreader('utf-8')
with reader(open('test_file', 'r')) as f:
    print(f.read())  # いちりきつくっ

try:
    with ExitStack() as stack:
        f = stack.enter_context(reader(open('test_file', 'r'))) # Error
        print(f.read())
finally:
    import os
    os.unlink('test_file')    # Delete the file

Here is the result:

AttributeError: type object 'instance' has no attribute '__exit__'

Not sure what happens on Python 3, since Python 3's native unicode support obviates the need for the StreamReader in the first place as far as I can tell.

I don't know if this is a good solution, but if you use cm.__class__ instead of type(cm), it seems to work OK:

    def enter_context(self, cm):
        """Enters the supplied context manager

        If successful, also pushes its __exit__ method as a callback and
        returns the result of the __enter__ method.
        """
        # We look up the special methods on the type to match the with statement
        for _cm_type in type(cm), cm.__class__:
            try:
                _exit = _cm_type.__exit__
                result = _cm_type.__enter__(cm)
            except AttributeError as e:
                error = e
                continue

            error = None
            break

        if error is not None:
            raise error        

        self._push_cm_exit(cm, _exit)
        return result

(Derived from https://github.com/python/mypy/issues/5028#issuecomment-869001339)

The current mypy CI check just tests that the stub file is well formed, it doesn't test that the module API actually matches the type hinted stub API. Running mypy.stubcheck instead will fix that testing gap, but the failures need to be fixed first:

[[email protected] contextlib2]$ python3 -m mypy.stubtest contextlib2
error: contextlib2.ContextDecorator.refresh_cm is not present in stub
Stub:
MISSING
Runtime: at line 76 in file /home/ncoghlan/devel/contextlib2/contextlib2/__init__.py
<function ContextDecorator.refresh_cm at 0x7fd497b61dc0>

error: contextlib2.ContextStack is not present in stub
Stub:
MISSING
Runtime: at line 783 in file /home/ncoghlan/devel/contextlib2/contextlib2/__init__.py
<class 'contextlib2.ContextStack'>

error: contextlib2.nullcontext is not a function
Stub: at line 122
Overload(def [_T] (enter_result: _T`-1) -> typing.ContextManager[_T`-1], def () -> typing.ContextManager[None])
Runtime: at line 755 in file /home/ncoghlan/devel/contextlib2/contextlib2/__init__.py
<class 'contextlib2.nullcontext'>

v0.6.0 has been released in https://github.com/jazzband/contextlib2/tree/v0.6.0 and https://pypi.org/project/contextlib2/0.6.0/ but https://github.com/jazzband/contextlib2/blob/v0.6.0/NEWS.rst#060-unreleased makes it seems otherwise.

Hi,

Thanks for your work on contextlib2 and Python in general!

Now that contextlib2 has only supported Python 3.x for a while, what do you think about this change that stops it from building universal wheels? It ought to be harmless, since there is a python_require specification, but still it might confuse some tools that see both "python_version": "py2.py3" and "requires_python": ">=3.6" in the PyPI metadata :)

Thanks for your time, and keep up the great work!

G'luck, Peter

• Sync module and test suite from CPython 3.10
• Switch test suite over to using unittest autodiscovery (since there are now two test files)
• Capture the differences from the Python 3.10 version as a patch file
• Ship the tox config (and other dev files) in the sdist

In all of the setup.py scripts I have used before, the import for setup is like this:

try: from setuptools import setup except ImportError: from distutils.core import setup

Which allows it to fall back to distutils.core if setuptools is not available.

Is there a reason that this is missing from this projects setup.py?

Thanks.

Hello,

The next stable release of Debian includes Python 3.11 (and might make Python 3.11 the default version)

Running the unit tests from version 0.6.0 or version 21.6.0 under Python 3.11 produces the following errors:

======================================================================
ERROR: test_typo_enter (test.test_contextlib.TestContextDecorator.test_typo_enter)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.oVLsYe/build.IvE/src/test/test_contextlib.py", line 498, in test_typo_enter
    with mycontext():
TypeError: 'mycontext' object does not support the context manager protocol

======================================================================
ERROR: test_typo_exit (test.test_contextlib.TestContextDecorator.test_typo_exit)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.oVLsYe/build.IvE/src/test/test_contextlib.py", line 510, in test_typo_exit
    with mycontext():
TypeError: 'mycontext' object does not support the context manager protocol (missed __exit__ method)

----------------------------------------------------------------------

Any guidance on how we can patch contextlib2 version 21.6.0 to fix this?

Thanks!

https://github.com/python/cpython/commit/6cb145d23f5cf69b6d7414877d142747cd3d134c

A TypeError is now raised instead of an AttributeError in contextlib.ExitStack.enter_context() and contextlib.AsyncExitStack.enter_async_context() for objects which do not support the context manager or asynchronous context manager protocols correspondingly. (Contributed by Serhiy Storchaka in bpo-44471.)

however this is only to match the change in behaviour of the async with and with statements in 3.11:

A TypeError is now raised instead of an AttributeError in with and async with statements for objects which do not support the context manager or asynchronous context manager protocols correspondingly. (Contributed by Serhiy Storchaka in bpo-12022.)

so would need conditional checking.

perhaps:

if sys.version_info >= (3, 11):
    from contextlib import ExitStack, AsyncExitStack
else:
    class ExitStack(...):
        ...

    class AsyncExitStack(...):
        ...

would be best?