nb. the pattern I'm showing here is basically flask's blueprints

It seems that currently Klein doesn't have any builtin way to help factoring out view functions to separate modules.

For example, if you had the following app:

app = Klein()

@app.route('/users')
def list_all_users(request):
    return 'all users'

then you could divide it into two modules like this:

# main.py
app = Klein()

# users.py - version 1
from main import app

@app.route('/users')
# ...

This is bad because it leads to several problems, the most immediate being circular import issues, but also this is only a separate module technically, not logically: you can't really re-use that module with another app, or test it in isolation.

An easy way to help that would be to use dependency injection:

# main.py
from users import add_users_routes

app = Klein()
add_users_routes(app)

# users.py - version 2

def list_all_users(request): 
    return 'all_users'

def add_users_routes(app):
    app.route('/users')(list_all_users)

Now users is a separate logical module, but it's a bit awkward, with the central add_users_routes function. We could use Klein's .resource() to help that:

# users.py - version 3
users = Klein()

@users.route('/')
def list_all_users(request):
    return 'all users'

def add_users_routes(app):
    @app.route('/users', branch=True)
    def branch_to_users(request):
        return users.resource()

This is already pretty nice, could maybe use some helpers so that you wouldn't need to implement that branch function, but is reusable and possible to test in isolation. The problem however is, routing to a Klein resource returns control to twisted.web, which then calls .render() on the nested resource, which causes a werkzeug map bind, etc - it's a performance hit. A simple hello-world-grade benchmark shown that a root Klein can serve ~2200 requests per second, routing one branch deep: ~1700, two branches deep: ~1400 (experience with flask's blueprints shows that 2 levels of nesting are enough in practice)

I'm aware Klein is a web framework, and web applications aren't typically constrained by strict real-time requirements, and they're often easy to scale, but I think Klein needs some guideline on structuring applications, and while we're at it, might as well make it fast :)

Basically I think we want the syntax of users.py version 2, with the performance of version 3. Here's how flask does it (translated to Klein):

# main.py
from users import users_blueprint
app = Klein()
users_blueprint.register(app, url_prefix='/users')

# users.py
users_blueprint = Blueprint()

@users_blueprint.route('/')
# ...

The Blueprint is simply a container that records all @route() calls it was used for, and does them on the app when .register is called. This makes it only a setup-time thing, with no runtime (performance) effects.

I've put together a proof-of-concept implementation for reference, see https://github.com/tehasdf/klein/tree/blueprints (here's an example.rst in the repo)

The following Pull Request focuses on exploring different options while registering new routes on Klein. The example provided shows primarily two things:

• How to use decorators to avoid duplication of code i.e. request.setHeader('Content-Type', 'application/json') and an example of key authorization
• Show other ways to register routes dynamically wrapping the klein instance in a custom class as instance attribute

Usage stats via pypistats python_minor klein --last-month:

| category | percent | downloads | |----------|--------:|----------:| | 2.7 | 46.53% | 10,612 | | 3.7 | 21.11% | 4,815 | | 3.6 | 21.09% | 4,809 | | 3.8 | 7.10% | 1,619 | | 3.5 | 2.30% | 525 | | null | 1.64% | 374 | | 3.4 | 0.23% | 52 | | 2.6 | 0.00% | 1 | | Total | | 22,807 |

Date range: 2020-05-01 - 2020-05-31

(Using Klein master and Twisted 13.2.0)

Hi, just started using Klein for a small project, and I noticed that whenever I return static.File from a route-decorated function or method, I will intermittently see "Producer was not unregistered" when accessing files in the directory being served by static.File.

Plain old twisted.web works, this error only shows up when using Klein.

Simple example, run with "twistd --nodaemon web --class=testing.resource":

testing.py

from klein import resource, route

from twisted.web import static


@route('/files/', branch=True)
def files(request):
    return static.File('/Some/Directory/')

And lastly, the traceback from the logfile:

2013-11-13 10:47:26-0600 [HTTPChannel,0,127.0.0.1] 127.0.0.1 - - [13/Nov/2013:16:47:25 +0000] "GET /files/robots.txt HTTP/1.1" 200 27 "http://127.0.0.1:8080/files/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.71 (KHTML, like Gecko) Version/6.1 Safari/537.71" 2013-11-13 10:47:26-0600 [HTTPChannel,0,127.0.0.1] Unhandled Error Traceback (most recent call last): Failure: exceptions.RuntimeError: Producer was not unregistered for /files/robots.txt

The fact that "::" works for ipv4 connections as well surprises me but seems to be true on linux. I'm not sure what layer is making that work.

Hello,

This pull request is to introduce support of the dynamic extension module loading into the klein.ext namespace.

I've also moved _call method from module static namespace to Klein object methods. This is because I'll have to inject to this exact spot in my future extension (I am planning to create an extension that introduces injector support to Klein route handlers).

Enable Mypy-Zope so that we can properly type check the use of Zope Interface classes and remove the various hacks we have in place to make Mypy happy without the plugin.

I'm the maintainer of Arch Linux package python-klein, and I run a periodic script to check compatibility among Python packages. Recently I notice tests in python-klein no longer work with python-treq 20.4.1.

Steps to reproduce:

1. Clone commit c91085d3b3a4b3521e1527fff82fa970c004eb2a of this repo

2. Change the following line to treq==20.4.1 https://github.com/twisted/klein/blob/c91085d3b3a4b3521e1527fff82fa970c004eb2a/tox.ini#L46

3. Run tox -e coverage-py38-twcurrent

Actual results:

[FAIL]
Traceback (most recent call last):
  File "/home/yen/tmp/klein/.tox/coverage-py38-twcurrent/lib/python3.8/site-packages/klein/test/test_form.py", line 473, in test_customParameterValidation
    self.assertTrue(errorText.endswith("'not a number'"))
  File "/home/yen/tmp/klein/.tox/coverage-py38-twcurrent/lib/python3.8/site-packages/twisted/trial/_synctest.py", line 397, in assertTrue
    super(_Assertions, self).assertTrue(condition, msg)
  File "/usr/lib/python3.8/unittest/case.py", line 765, in assertTrue
    raise self.failureException(msg)
twisted.trial.unittest.FailTest: False is not true

klein.test.test_form.TestForms.test_customParameterValidation
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/home/yen/tmp/klein/.tox/coverage-py38-twcurrent/lib/python3.8/site-packages/klein/test/test_form.py", line 409, in test_handlingGET
    self.assertEqual(calls, [(u"hello, big world", 4321)])
  File "/home/yen/tmp/klein/.tox/coverage-py38-twcurrent/lib/python3.8/site-packages/twisted/trial/_synctest.py", line 434, in assertEqual
    super(_Assertions, self).assertEqual(first, second, msg)
  File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.8/unittest/case.py", line 1118, in assertListEqual
    self.assertSequenceEqual(list1, list2, msg, seq_type=list)
  File "/usr/lib/python3.8/unittest/case.py", line 1100, in assertSequenceEqual
    self.fail(msg)
twisted.trial.unittest.FailTest: Lists differ: [('hello, big+world', 4321.0)] != [('hello, big world', 4321)]

First differing element 0:
('hello, big+world', 4321.0)
('hello, big world', 4321)

- [('hello, big+world', 4321.0)]
?              ^            --

+ [('hello, big world', 4321)]
?              ^


klein.test.test_form.TestForms.test_handlingGET

Expected results: tests pass

This PR is a Klein 20.6 release.

The primary goal for this release is to ship all work that's currently been done with support for Python 2.7 before we drop support for Python 2.7, so that we leave the 2.7 users with the latest available progress.

In addition to incrementing the version, this PR adds a script for automating the release process.

The new headers API allows headers to be accessed and set using either bytes or text for the header names, as long as the same type is returned or provided for corresponding values.

@moshez points out that this is icky and suggests that we use separate API for bytes and text, instead of having this polymorphism.

The polymorphic API follows the pattern used by twisted.web.http_headers.Headers does.

This ticket is here to solicit input on the path forward.

I'd like to propose Klein.url_for() method for reverse URL mapping.

def url_for(self, request, endpoint, values = None, method = None, force_external = False, append_unknown = True)

request is the Request object. It is used to obtain correct hostname for external links. Other arguments are just like in Werkzeug's MapAdapter.build().

It can be used like this:

from klein import route, run, url_for

@route('/user/<username>')
def user_page(request, username):
    return "Welcome to {}'s homepage".format(username)

@route('/')
def homepage(request):
    users = ['Alice', 'Bob', 'Carl']
    return '<br/>'.join([
        '<a href="{link}">{username}</a>'.format(
            link=url_for(request, 'user_page', {'username': username}),
            username=username)
        for username in users
    ])

run('localhost', 8080)

Actually, Klein already has url_for() method on IKleinRequest that is assumed to be used like this:

@app.route("/")
def homepage(request):
    krequest = IKleinRequest(request)
    url = krequest.url_for('user_page', {'username': username})

But this looks like a piece of confusing magic. How does request know about url-mapping? Why are we asking the request to do url mapping, not the app? And this is working by exploiting the fact that Request is Componentized that can be problematic as described in #31. Having url_for() method on app seems more reasonable.

Questions:

• request argument to url_for is needed only if force_external=True. Most of the links are internal, so it might be more convenient to make request arg optional. May be have two methods: url_for(endpoint, ...) that always returns internal link and external_url_for(request, endpoint, ...) for external ones?

If overall concept will be accepted I will add a docstring and example in docs.

When I import klein (i.e.: from klein import Klein), I get an error that concludes in the following lines:

 File "/usr/local/lib/python3.7/dist-packages/twisted/internet/_sslverify.py", line 1807, in fromOpenSSLCipherString
    SSL.TLS_METHOD,
AttributeError: module 'OpenSSL.SSL' has no attribute 'TLS_METHOD'

Appears related to this twisted issue.

Death to twisted loser slave devs. Death to those monkeys for naming their garbage programs with disgusting & racist & childish (I'm referring to that stupid 6-letter word which comes after that stupid kiddish program starting with the word "py"). Hope your throats get slit, you daft imbeciles. Death to you and every idiot who uses that disgusting 6-letter childish word.

Death to open source. Death to you garbage mainstream propaganda slaves. Death to you twisted devs. Hope you have a tragic life and painful death.

Death to twisted loser slave devs.

Death to those monkeys for naming their garbage programs with disgusting & racist & childish (I'm referring to that stupid 6-letter word which comes after that stupid kiddish program starting with the word "py"). Hope your throats get slit, you daft imbeciles. Death to you and every idiot who uses that disgusting 6-letter childish word. Death to open source. Death to you garbage mainstream propaganda slaves. Death to you twisted devs. Hope you have a tragic life and painful death.

Death to twisted loser slave devs.

Death to those monkeys for naming their garbage programs with disgusting & racist & childish (I'm referring to that stupid 6-letter word which comes after that stupid kiddish program starting with the word "py").

Hope your throats get slit, you daft imbeciles. Death to you and every idiot who uses that disgusting 6-letter childish word. Death to open source. Death to you garbage mainstream propaganda slaves. Death to you twisted devs. Hope you have a tragic life and painful death.

Gentoo bug: https://bugs.gentoo.org/865677

Full build log: https://865677.bugs.gentoo.org/attachment.cgi?id=800049

================================== FAILURES ===================================
_______________________ KleinResourceTests.test_addSlash _______________________

self = <klein.test.test_resource.KleinResourceTests testMethod=test_addSlash>

    def test_addSlash(self) -> None:
        app = self.app
        request = MockRequest(b"/foo")
    
        @app.route("/foo/")
        def foo(request: IRequest) -> KleinRenderable:
            return "foo"
    
        d = _render(self.kr, request)
    
        self.assertFired(d)
        self.assertEqual(
            request.setHeader.call_count,
            3,
        )
>       request.setHeader.assert_has_calls(
            [
                call(b"Content-Type", b"text/html; charset=utf-8"),
                call(b"Content-Length", b"258"),
                call(b"Location", b"http://localhost:8080/foo/"),
            ]
        )

app        = <klein._app.Klein object at 0x7f463a37d610>
d          = <Deferred at 0x7f4637efc730 current result: None>
foo        = <function KleinResourceTests.test_addSlash.<locals>.foo at 0x7f4639dfa5e0>
request    = <MockRequest at 0x7f463983cf70 method=GET uri=/foo clientproto=HTTP/1.1>
self       = <klein.test.test_resource.KleinResourceTests testMethod=test_addSlash>

src/klein/test/test_resource.py:642: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Mock id='139939562899008'>
calls = [call(b'Content-Type', b'text/html; charset=utf-8'), call(b'Content-Length', b'258'), call(b'Location', b'http://localhost:8080/foo/')]
any_order = False

    def assert_has_calls(self, calls, any_order=False):
        """assert the mock has been called with the specified calls.
        The `mock_calls` list is checked for the calls.
    
        If `any_order` is False (the default) then the calls must be
        sequential. There can be extra calls before or after the
        specified calls.
    
        If `any_order` is True then the calls can be in any order, but
        they must all appear in `mock_calls`."""
        expected = [self._call_matcher(c) for c in calls]
        cause = next((e for e in expected if isinstance(e, Exception)), None)
        all_calls = _CallList(self._call_matcher(c) for c in self.mock_calls)
        if not any_order:
            if expected not in all_calls:
                if cause is None:
                    problem = 'Calls not found.'
                else:
                    problem = ('Error processing expected calls.\n'
                               'Errors: {}').format(
                                   [e if isinstance(e, Exception) else None
                                    for e in expected])
>               raise AssertionError(
                    f'{problem}\n'
                    f'Expected: {_CallList(calls)}'
                    f'{self._calls_repr(prefix="Actual").rstrip(".")}'
                ) from cause
E               AssertionError: Calls not found.
E               Expected: [call(b'Content-Type', b'text/html; charset=utf-8'),
E                call(b'Content-Length', b'258'),
E                call(b'Location', b'http://localhost:8080/foo/')]
E               Actual: [call(b'Content-Type', b'text/html; charset=utf-8'),
E                call(b'Content-Length', b'239'),
E                call(b'Location', b'http://localhost:8080/foo/')]

all_calls  = [call(b'Content-Type', b'text/html; charset=utf-8'),
 call(b'Content-Length', b'239'),
 call(b'Location', b'http://localhost:8080/foo/')]
any_order  = False
calls      = [call(b'Content-Type', b'text/html; charset=utf-8'),
 call(b'Content-Length', b'258'),
 call(b'Location', b'http://localhost:8080/foo/')]
cause      = None
expected   = [call(b'Content-Type', b'text/html; charset=utf-8'),
 call(b'Content-Length', b'258'),
 call(b'Location', b'http://localhost:8080/foo/')]
problem    = 'Calls not found.'
self       = <Mock id='139939562899008'>

/usr/lib/python3.8/unittest/mock.py:950: AssertionError
=============================== warnings summary ===============================
../../../../../../../usr/lib/python3.8/site-packages/twisted/web/test/test_web.py:21
  /usr/lib/python3.8/site-packages/twisted/web/test/test_web.py:21: DeprecationWarning: twisted.test.proto_helpers.EventLoggingObserver was deprecated in Twisted 19.7.0: Please use twisted.internet.testing.EventLoggingObserver instead.
    from twisted.test.proto_helpers import EventLoggingObserver

src/klein/_form.py:132: 1 warning
src/klein/test/test_form.py: 14 warnings
  /var/tmp/portage/dev-python/klein-21.8.0/work/klein-21.8.0/src/klein/_form.py:132: DeprecationWarning: assoc is deprecated and will be removed after 2018/01.
    return attr.assoc(

src/klein/test/test_form.py:38
  /var/tmp/portage/dev-python/klein-21.8.0/work/klein-21.8.0/src/klein/test/test_form.py:38: PytestCollectionWarning: cannot collect test class 'TestObject' because it has a __init__ constructor (from: src/klein/test/test_form.py)
    class TestObject:

src/klein/test/test_form.py: 17 warnings
  /var/tmp/portage/dev-python/klein-21.8.0/work/klein-21.8.0/src/klein/_form.py:340: DeprecationWarning: assoc is deprecated and will be removed after 2018/01.
    yield attr.assoc(

src/klein/test/test_resource.py::KleinResourceTests::test_explicitStaticBranch
  /var/tmp/portage/dev-python/klein-21.8.0/work/klein-21.8.0/src/klein/test/test_resource.py:596: ResourceWarning: unclosed file <_io.BufferedReader name='/var/tmp/portage/dev-python/klein-21.8.0/work/klein-21.8.0/src/klein/test/__init__.py'>
    expected = open(

src/klein/test/test_resource.py::KleinResourceTests::test_staticRoot
  /var/tmp/portage/dev-python/klein-21.8.0/work/klein-21.8.0/src/klein/test/test_resource.py:578: ResourceWarning: unclosed file <_io.BufferedReader name='/var/tmp/portage/dev-python/klein-21.8.0/work/klein-21.8.0/src/klein/test/__init__.py'>
    expected = open(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED src/klein/test/test_resource.py::KleinResourceTests::test_addSlash - A...
============ 1 failed, 241 passed, 36 warnings in 108.70s (0:01:48) ============
 * ERROR: dev-python/klein-21.8.0::gentoo failed (test phase):
 *   pytest failed with python3.8