WSGI middleware for sessions and caching

Related tags

Cachingbeaker
Overview

Cache and Session Library

About

Beaker is a web session and general caching library that includes WSGI middleware for use in web applications.

As a general caching library, Beaker can handle storing for various times any Python object that can be pickled with optional back-ends on a fine-grained basis.

Beaker was built largely on the code from MyghtyUtils, then refactored and extended with database support.

Beaker includes Cache and Session WSGI middleware to ease integration with WSGI capable frameworks, and is automatically used by Pylons and TurboGears.

Features

  • Fast, robust performance
  • Multiple reader/single writer lock system to avoid duplicate simultaneous cache creation
  • Cache back-ends include dbm, file, memory, memcached, Redis, MongoDB, and database (Using SQLAlchemy for multiple-db vendor support)
  • Signed cookies to prevent session hijacking/spoofing
  • Cookie-only sessions to remove the need for a db or file backend (ideal for clustered systems)
  • Extensible Container object to support new back-ends
  • Caches can be divided into namespaces (to represent templates, objects, etc.) then keyed for different copies
  • Create functions for automatic call-backs to create new cache copies after expiration
  • Fine-grained toggling of back-ends, keys, and expiration per Cache object

Documentation

Documentation can be found on the Official Beaker Docs site.

Source

The latest developer version is available in a GitHub repository.

Contributing

Bugs can be filed on GitHub, should be accompanied by a test case to retain current code coverage, and should be in a pull request when ready to be accepted into the beaker code-base.

Comments
  • Fix path for CookieSession

    Fix path for CookieSession

    CookieSession does not properly apply the cookie_path configuration option, causing the resulting cookie to be set for the default path instead.

    This happens directly after initialization since this does not set the internal dict item associated with the _path key. It also happens after calling delete since this will clear the dict items.

    The attribute _path should be used in _create_cookie instead as is done in _set_cookie_values.

    opened by FredrikAppelros 11
  • session information not carried across requests

    session information not carried across requests

    The setup is a traditional web client (angularjs) making requests to a web server (python 2.6.9, gunicorn 19.0.0, beaker 1.6.4).

    app handling code looks pretty much like so:

    from beaker.middleware import SessionMiddleware
    session_opts = {
      "session.type": "cookie",
      "session.validate_key": "some secret",
      "session.cookie_expires": 3600*24*90,
      "session.auto": True,
    }
    app_handler = ...
    application = SessionMiddleware(app_handler, session_opts)
    

    The web client makes one HTTP GET and two HTTP POST requests sequentially against the web server. The problem is that the last request doesn't seem to carry the session info that the first request added to the session. Here's what I see in the server logs:

    • GET / : sets some session info via session["info"] = "..."
    • POST /foo:
      • beaker.session.id cookie is set (just a long b64 hash)
      • session["info"] has my information, as expected
    • POST /bar:
      • beaker.session.id cookie is set and seems to have a different value than the hash just before
      • session does not have "info" key, the only thing it's got is an "_id" entry.

    So somehow the session info is cleared after the "POST /foo", which makes no sense to me. Am I missing something? Also, is beaker.session.id supposed to change all the time? I seem to always see a different value.

    Thanks!

    opened by expntly 11
  • Possibly unsafe use of pickle()

    Possibly unsafe use of pickle()

    I've recently come across http://vudang.com/2013/01/python-web-framework-from-lfr-to-rce/, which details possible vulnerabilities related to unpickling cookies.

    I've noticed that pickle is used in https://github.com/rcarmo/beaker/blob/master/beaker/session.py#L288, and was wondering if you had any plans to change that.

    opened by rcarmo 11
  • Encrypted cookies don't work in Python 3.2

    Encrypted cookies don't work in Python 3.2

    It appears that none of the beaker.crypto code is ready for Python 3. In particular, pbkdf2.py does a bunch of isinstance() checks which enforce that the encryption keys are 'str' objects, and calls encode() on them if not. This is of course precisely backwards when working with str and bytes in Python 3.

    I've tried making a few patches to the code, but this only serves to reveal additional issues, and I don't have the time to prepare a complete patch for now.

    In the meantime, please update the docs to indicate that encrypted cookies (that is, cookie-only sessions) are not supported when running under Python 3.

    opened by kiwidrew 7
  • Please deprecate bitbucket repository

    Please deprecate bitbucket repository

    https://bitbucket.org/bbangert/beaker/overview

    still says "Beaker Official Mirror". There is no immediately obvious reference to the github repository there. I signed up and submitted a pull request months ago, only to realise now that the canonical source is here on github.

    opened by mitchellrj 7
  • Ability to pass expiretime into RedisNamespaceManager

    Ability to pass expiretime into RedisNamespaceManager

    Hey,

    I was trying to figure out what is the best way to clean up old / unused session data from Redis backend. I found that the basic mechanics for setting expiration time is there, but there's no way to use it from layers above. With that change, I'm able to just pass expiretime into Middleware, and voila - keys in reds are having expiration time set! :)

    Example usage of middleware:

    app = SessionMiddleware(app, config, expiretime=5000)
    

    Would be great if you could merge this change.

    Thanks!

    Kacper.

    opened by pagenoare 6
  • KeyError on session encrypt_key update

    KeyError on session encrypt_key update

    Hello, When I restart an application and generate a new encrypt_key without deleting the existing cookies from logged-in browsers I get the following error:

      File "/usr/lib/python2.7/dist-packages/beaker/session.py", line 651, in __getattr__
        return getattr(self._session(), attr)
      File "/usr/lib/python2.7/dist-packages/beaker/session.py", line 644, in _session
        self.__dict__['_sess'] = CookieSession(req, **params)
      File "/usr/lib/python2.7/dist-packages/beaker/session.py", line 531, in __init__
        self['_accessed_time'] > self.timeout:
    KeyError: '_accessed_time'
    

    Maybe Beaker should detect this condition and ignore the old cookie, or delete it, or raise a specific exception. Thanks!

    opened by FedericoCeratto 6
  • Samesite flag not set

    Samesite flag not set

    The samesite flag is not properly set upon creation of a session.

    session_opts = {
        ...
        'session.key': 'oncall-auth',
        'session.samesite': 'Lax',
        ...
    }
    application = SessionMiddleware(application, session_opts)
    

    This results in this cookie:

    oncall-auth=<cookie>; httponly; Path=/
    

    After logout (cookie delete) the flag is set:

    oncall-auth=<cookie>; expires=Wed, 13-Dec-2017 15:20:35 GMT; httponly; Path=/; SameSite=Lax
    

    This might be caused, because method _set_cookie_values (in session.py) is only called in _update_cookie_out and _delete_cookie. This but renders the samesite cookie useless, so it would be nice if this could be fixed.

    opened by tvlieg 4
  • create new `safe_write` function, buffer pickling on all platforms

    create new `safe_write` function, buffer pickling on all platforms

    This is a continuation of my efforts to deal with #134.

    While buffering to a separate file and then moving it into location works for posix based systems, the lack of atomic move support on Windows and the fact that it will fail with an exception if the file already exists makes the solution not viable for that platform.

    There are some complex ways that this can be dealt with, but they would have a negative performance hit (and I don't care enough about people who deploy to windows systems to devote the time to this).

    Short of that, this PR reduces the time spent writing the file which in turn reduces the chances of corruption occurring. It does this by buffering the pickled output to a string rather than using the pickler's built in "write to file" system (which writes as the pickling happens).

    opened by tedivm 4
  • MySQL Server has Gone away

    MySQL Server has Gone away

    I have been getting these~ OperationalError: (_mysql_exceptions.OperationalError) (2006, 'MySQL server has gone away')

    Is there a flag to set when to recycle mysql connections? I know that you can do this with Flask SQLAlchemy with app.config['SQLALCHEMY_POOL_RECYCLE'].

    opened by EndenDragon 4
  • ext.Database doesn't close database connections on load

    ext.Database doesn't close database connections on load

    I'm using pyramid and pyramid_beaker with python 3.6 and the below configuration.

    session.type = ext:database
    session.auto = True
    session.key = fake_local
    session.data_dir = %(here)s/data/sessions/data
    session.secret = some_secret
    session.cookie_expires=true
    session.url=mysql+pymysql://user:[email protected]:3406/db
    session.timeout=86400
    #session.lock_dir=/tmp/cache/lock
    session.httponly = True
    session.secure = True
    session.domain = 'example.com'
    session.path = '/'
    session.sa.pool_size = 10
    session.sa.max_overflow=10
    

    I initialize the session factory via the recommended config.include('pyramid_beaker'). After loading the page a few times SQLAlchemy is no longer able to connect to the database and throws the below exception because connections are not being closed.

    sqlalchemy.exc.TimeoutError: QueuePool limit of size 10 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/3o7r)

    If I switch to using session.type = file the problem goes away. After some debugging I tracked it to the do_open() method in beaker.ext.DatabaseNamespaceManager. It appears that sqlalchemy.select().execute().fetchone() doesn't close the database connection in all cases.

    I have submitted PR #147 with a patch for the issue.

    opened by chrismcmacken 3
  • Deserialization of Untrusted Data

    Deserialization of Untrusted Data

    Dependabot flags <= 1.11.0 as having the following security risk:

    The Beaker library through 1.11.0 for Python is affected by the deserialization of untrusted data, which could lead to arbitrary code execution.

    With weakness: https://cwe.mitre.org/data/definitions/502.html

    I use this as a dependency of Turbogears2 and need this resolved in order to continue using that framework.

    opened by camilla007 0
  • allow ignoring some arguments for caching key

    allow ignoring some arguments for caching key

    This PR is not as complete or extensive as #202, but provides minimal implementation that fulfills the use case I'm facing. It partially addresses #201.

    This allows to decorate a function with some parameters to be ignored when creating the cache key that will match following calls. It allows to group together some operations where a parameter is important for the calculation of the function, but is not strictly required to map the call to the cached result. This is particularly useful in situation where the parameter cannot be cached (not serializable).

    opened by fmigneault 1
  • Resource consumption dos from way back

    Resource consumption dos from way back

    Going way back to 2019 I had submitted a vulnerability to Ubiquiti Edgemax devices, it was a denial of service by filling up the beaker.session.id stored locally on the device until it had a resource consumption issue.

    It just thought about it again for whatever reason and maybe I should have submitted something to the source. I am imagining it was just a problem because the device had limited space.

    That being said has this been an issue for any other devices using beaker with limited resources or has this been patched in the past.

    Thanks for humoring me.

    Here is a link to the proof of concept i created to test for this problem.

    https://github.com/grampae/meep

    opened by grampae 0
  • Incapable to disable cache after modified region settings

    Incapable to disable cache after modified region settings

    When using cache_region decorator, beaker will use the following snippet to determine if it should instantiate a new cache for each function that was decorated with it.

    https://github.com/bbangert/beaker/blob/889d3055a4ca31b55a0b0681b00f2973b3250d88/beaker/cache.py#L545-L570

    When cache[0] is None, the decorator generates the missing cache if the corresponding region was defined with enabled = true and stores it in that local list.

    I have a situation where I'm running unit tests on an application where I want to evaluate different combinations of cache regions with enabled/disable states. The problem with that local list storing the enabled cache is that if one test case later that must have it disabled, the if not cache[0]: check completely bypasses whichever new settings I set via cache_regions, since the cache is already created.

    Also, since the Cache reference is stored in that local list (rather than global/package one), I cannot even wipe it.

    Calling cache_managers.clear(), which has the same reference to that local cache added from the following lines will NOT clear cache[0].

    https://github.com/bbangert/beaker/blob/889d3055a4ca31b55a0b0681b00f2973b3250d88/beaker/cache.py#L308-L314

    Is there a way to avoid this? I struggle trying to monkey-patch the cache object in cache[0] for my test cases.

    opened by fmigneault 0
  • Beaker Maintainership

    Beaker Maintainership

    Hi, for the past few years I have been taking care of maintaining Beaker, merging PRs, cleaning up API, adding features and making releases.

    Recently due to personal reasons I have found myself with less time to spend on that kind of work, and while I still plan to contribute to it, I think someone else should take my place in taking the lead so that Beaker can get the attention it deserves and I stop slowing down its releases :D

    If any contributor is willing to take over that role, I think that @bbangert won't have much to complain about and could probably help providing the permissions necessary to do the job.

    Feel free to mention your candidature here in this issue. Obviously this doesn't mean that Beaker isn't maintained anymore, I'll still be doing all I can until someone is willing to take over. I just think that there are probably people out there who can do a better job than what I have been doing in the past few months and it makes sense to avoid slowing down the project.

    opened by amol- 0
  • Redis ext cannot handle passwords with special URL characters.

    Redis ext cannot handle passwords with special URL characters.

    If your password for Redis AUTH has # or : in it, it will need to be url encoded. however, redis-py will not automatically decode the url with the current version.

    Redis URLs with passwords are in the form of redis(s)://user:[email protected]:port/db

    If say you had redis://:my#[email protected]:6379/1 this forms an invalid URL

    The solution would be to do something like redis://:my%[email protected]:6379/1

    redis-py supports this, however, versions 3.5.3 and below, from_url must be passed decode_components=True ... it defaults to False for now.

    It looks like version 4.x will get rid of this and always decode the url, so we could end up with some compatibility issues if we try and pass it.

    I was going to put forth a PR, but SyncDict was passing arguments in unnamed so could cause some issues there down the road. from_url probably won't change as far as URL being the first param.

    The workaround is to construct your own StrictRedis ( really just Redis now ) and pass it. Already doing that do deal with SSL when the cert can't be verified, but an issue.

    Looks a little something like this...I'm using pyramid and pyramid_beaker. How you get at your settings will be up to you.

        if settings['beaker.session.type'] == 'ext:redis' and settings['beaker.session.url'] == 'custom': 
            from redis import StrictRedis
            db = StrictRedis(host='127.0.0.1', port=6379, password = 'my#pass', db = 1)
            db.ping()
            db.set('_startup','_test')
            
            settings['beaker.session.url'] = db
    
           session_factory = session_factory_from_settings(settings)
           config.set_session_factory(session_factory)
    
    

    Obviously another workaround is to not have a password with symbols :smile:

    opened by twiggy 0
Releases(1.12.0)
  • 1.12.0(Dec 7, 2022)

    • Enabled testing on Python 3.10 and 3.11
    • Fixed issue #122 - Session ignores deserializer json
    • Remove ID generation fallback for when the uuid module is not found
    • Port testing from nose to pytest
    • Fixed issue #180 - KeyError when loading deleted session
    Source code(tar.gz)
    Source code(zip)
  • 1.11.0(Aug 26, 2019)

    • Fixed cookie path option not being properly set (self._path was removed, only self.path exists)
    • Documented SameSite option
    • Fixed cookie expiration being localised when it shouldn't.
    Source code(tar.gz)
    Source code(zip)
  • 1.10.1(Feb 21, 2019)

  • 1.10.0(Jun 4, 2018)

    Release 1.10.0 (2018-06-04)

    • Redis namespace manager now supports providing a TTL for session entries that had a timeout provided. This will remove the need to manually clear expired sessions from the redis storage.
    • nsscrypto backend is now properly identified as providing AES support.
    • When a crypto backend doesn't support AES it will no longer crash if the encrypt_key is None.
    • Session cookies will now provide support for SameSite through the samesite option. By default this will be Lax, but can be set to Strict or None to disable it.
    Source code(tar.gz)
    Source code(zip)
  • 1.9.1(Apr 9, 2018)

    Release 1.9.1 ( 2018-04-09 )

    • When decorating a function with @cache_region decorator the function generated to update the cached value will be named like the decorated function. So that during debugging it's easy to know which function is involved.
    • Removed usage of async as a variable in code, this fixes a compatibility problem with Python 3.7 where it's a keyword.
    • Fixed a race condition in FileNamespaceManager.
    • ext.database backend will now properly close connections.
    • Do not leave bhind corrupted files if FileNamespaceManager is interrupted while writing a new file. Replacing content of a file or writing a new one is now always an atomic operation.
    • DBMNamespaceManager and FileSynchronizer will now deal with directories disappearing while they try to write to them.
    • The Redis and MongoDB backends are not exposed in documentation.
    Source code(tar.gz)
    Source code(zip)
  • 1.9.0(Jun 18, 2017)

    Release 1.9.0 (2017-06-19)

    • Beaker now provides builtin ext:mongodb and ext:redis namespace managers. Both come with a Synchronizer implemented on the storage backend instead of relying on file one.
    • Fixed an issue where cookie options like Secure, Domain and so on where lost.
    • Improved support for cache entries expiration. NamespaceManagers that support it will expire their key automatically.
    • Pycryptodome can be used instead of pycrypto.
    • An issue with Cookie module import on case insensitive file systems should have been resolved.
    • Cryptography module is now as a crypto function provider instead of pycrypto
    Source code(tar.gz)
    Source code(zip)
  • 1.8.1(Oct 23, 2016)

    Release 1.8.1 (2016-10-24)

    • Sessions have a new option save_accessed_time which defaults to true for backwards compatibility. Set to false to tell beaker not to update _accessed_time if the session hasn't been changed, for non-cookie sessions stores. This lets you avoid needless datastore writes. _accessed_time will always be updated when the session is intentionally saved.
    • data_serializer parameter in Session accepts a custom object with dumps and loads methods.
    • Fixed a TypeError in exception reporting when failing to load a NamespaceManager
    • Allow to change Cookie Expiration from a value back to None, previously it had no effect.
    • Allow SessionMiddleware to setup a custom Session class through the session_class argument.
    • Added invalidate_corrupt option to CookieSessions too for valid cookies containing invalid data.
    Source code(tar.gz)
    Source code(zip)
Asyncio cache manager for redis, memcached and memory

aiocache Asyncio cache supporting multiple backends (memory, redis and memcached). This library aims for simplicity over specialization. All caches co

aio-libs 764 Jan 02, 2023
johnny cache django caching framework

Johnny Cache is a caching framework for django applications. It works with the django caching abstraction, but was developed specifically with the use

Jason Moiron 304 Nov 07, 2022
Caching for HTTPX

Caching for HTTPX. Note: Early development / alpha, use at your own risk. This package adds caching functionality to HTTPX Adapted from Eric Larson's

Johannes 51 Dec 04, 2022
A caching extension for Flask

A fork of the Flask-cache extension which adds easy cache support to Flask.

Pallets Community 773 Jan 08, 2023
Extensible memoizing collections and decorators

cachetools This module provides various memoizing collections and decorators, including variants of the Python Standard Library's @lru_cache function

Thomas Kemmer 1.5k Jan 05, 2023
A caching extension for Flask

Flask-Caching Adds easy cache support to Flask. This is a fork of the Flask-Cache extension. Flask-Caching also includes the cache module from werkzeu

Peter Justin 774 Jan 02, 2023
Automatic Flask cache configuration on Heroku.

flask-heroku-cacheify Automatic Flask cache configuration on Heroku. Purpose Configuring your cache on Heroku can be a time sink. There are lots of di

Randall Degges 39 Jun 05, 2022
PyCache - simple key:value server written with Python

PyCache simple key:value server written with Python and client is here run server python -m pycache.server or from pycache.server import start_server

chick_0 0 Nov 01, 2022
Peerix is a peer-to-peer binary cache for nix derivations

Peerix Peerix is a peer-to-peer binary cache for nix derivations. Every participating node can pull derivations from each other instances' respective

92 Dec 13, 2022
A decorator for caching properties in classes.

cached-property A decorator for caching properties in classes. Why? Makes caching of time or computational expensive properties quick and easy. Becaus

Daniel Roy Greenfeld 658 Dec 01, 2022
Robust, highly tunable and easy-to-integrate in-memory cache solution written in pure Python, with no dependencies.

Omoide Cache Caching doesn't need to be hard anymore. With just a few lines of code Omoide Cache will instantly bring your Python services to the next

Leo Ertuna 2 Aug 14, 2022
A Python wrapper around the libmemcached interface from TangentOrg.

pylibmc is a Python client for memcached written in C. See the documentation at sendapatch.se/projects/pylibmc/ for more information. New in version 1

Ludvig Ericson 458 Dec 30, 2022
No effort, no worry, maximum performance.

Django Cachalot Caches your Django ORM queries and automatically invalidates them. Documentation: http://django-cachalot.readthedocs.io Table of Conte

NoriPyt 979 Jan 03, 2023
Render template parts with extended cache control.

Render template parts with extended cache control. Installation Install django-viewlet in your python environment $ pip install django-viewlet Support

5 Monkeys 59 Apr 05, 2022
Automatic caching and invalidation for Django models through the ORM.

Cache Machine Cache Machine provides automatic caching and invalidation for Django models through the ORM. For full docs, see https://cache-machine.re

846 Nov 26, 2022
RecRoom Library Cache Tool

RecRoom Library Cache Tool A handy tool to deal with the Library cache file. Features Parse Library cache Remove Library cache Parsing The script pars

Jesse 5 Jul 09, 2022
Aircache is an open-source caching and security solution that can be integrated with most decoupled apps that use REST APIs for communicating.

AirCache Aircache is an open-source caching and security solution that can be integrated with most decoupled apps that use REST APIs for communicating

AirCache 2 Dec 22, 2021
WSGI middleware for sessions and caching

Cache and Session Library About Beaker is a web session and general caching library that includes WSGI middleware for use in web applications. As a ge

Ben Bangert 500 Dec 29, 2022
Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.

DiskCache is an Apache2 licensed disk and file backed cache library, written in pure-Python, and compatible with Django.

Grant Jenks 1.7k Jan 05, 2023
Persistent caching for python functions

Cashier Persistent caching for python functions Simply add a decorator to a python function and cache the results for future use. Extremely handy when

Anoop Thomas Mathew 82 Mar 04, 2022