A Python dictionary implementation designed to act as an in-memory cache for FaaS environments

Related tags

Data Structuresfaas-cache-dict
Overview

faas-cache-dict

A Python dictionary implementation designed to act as an in-memory cache for FaaS environments.

Formally you would describe this a memory constrained LRU TTL cache dict.

This is implementation only uses core Python stdlib with no external dependencies.

Background

This was originally designed to be used as an in-memory cache for AWS Lambda.

In most FaaS environments, successive quick calls of the function persists variables in the global scope. We can leverage this to cache data for future calls.

FaaS runtimes have limited RAM capacities so this library allows you to set a max mebibyte size for the dict. It also allows setting an optional max items length.

Items are kept in order with the LRU at the HEAD of the list.

Items are deleted if they expire, or from the head (LRU) if the cache is out of space.

Usage

Limitations

  • Performance degrades with size, you will need to test this for your use case. Though in most circumstances this will be much faster than performing a network call to a cache.
  • The library should be thread-safe, but limited testing has gone into this.

Support

CPython 3.7 or greater. No extra dependencies are required.

Contributions

This code is distributed under an open license. Feel free to fork it or preferably open a PR.

Inspirations

Thanks to mobilityhouse/ttldict for their implementation which served as a starting point.

You might also like...
Leetcode solutions - All algorithms implemented in Python 3 (for education)

Leetcode solutions - All algorithms implemented in Python 3 (for education)

Vineet Dhaimodker 3 Oct 21, 2022
This Repository consists of my solutions in Python 3 to various problems in Data Structures and Algorithms
This Repository consists of my solutions in Python 3 to various problems in Data Structures and Algorithms

Problems and it's solutions. Problem solving, a great Speed comes with a good Accuracy. The more Accurate you can write code, the more Speed you will

SAMIR PAUL 1.3k Jan 1, 2023
Python library for doing things with Grid-like structures

gridthings Python library for doing things with Grid-like structures Development This project uses poetry for dependency management, pre-commit for li

Matt Kafonek 2 Dec 21, 2021
A Python library for electronic structure pre/post-processing
A Python library for electronic structure pre/post-processing

PyProcar PyProcar is a robust, open-source Python library used for pre- and post-processing of the electronic structure data coming from DFT calculati

Romero Group 124 Dec 7, 2022
Programming of a spanning tree algorithm with Python : In depth first with a root node.

ST Algorithm Programming of a spanning tree algorithm with Python : In depth first with a root node. Description This programm reads informations abou

Mathieu Lamon 1 Dec 16, 2021
Data Structure With Python

Data-Structure-With-Python- Python programs also include in this repo Stack A stack is a linear data structure that stores items in a Last-In/First-Ou

Sumit Nautiyal 2 Jan 9, 2022
🔬 Fixed struct serialization system, using Python 3.9 annotated type hints

py-struct Fixed-size struct serialization, using Python 3.9 annotated type hints This was originally uploaded as a Gist because it's not intended as a

Alba Mendez 4 Jan 14, 2022
Final Project for Practical Python Programming and Algorithms for Data Analysis
Final Project for Practical Python Programming and Algorithms for Data Analysis

Final Project for Practical Python Programming and Algorithms for Data Analysis (PHW2781L, Summer 2020) Redlining, Race-Exclusive Deed Restriction Lan

Aislyn Schalck 1 Jan 27, 2022
Basic sort and search algorithms written in python.

Basic sort and search algorithms written in python. These were all developed as part of my Computer Science course to demonstrate understanding so they aren't 100% efficent

Ben Jones 0 Dec 14, 2022
Releases(0.2.3)

  • 0.2.3(Apr 27, 2022)

    Included

    • Fix min version installable on Py 3.8 + (due to pickle protocol versioning)

    Todo:

    • Discover better documents solution

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.2.2(Apr 25, 2022)

    Included

    • Add purge() to empty all contents of cache dict

    Todo:

    • Discover better documents solution

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.2.1(Apr 22, 2022)

    Included

    • Fix set_ttl() should be able to make a value immortal by passing None
    • Fix is_expired() no longer errors if the key has been purged, now returns True if missing
    • Add ability to change the max (LRU) length of the cache once created, purging the LRU as required.
    • Add extensive new documentation

    Todo:

    • Discover better documents solution

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.2.0(Apr 22, 2022)

    Included

    • Add FileBackedFaaSCache
    • Add support for pickling faas cache objects
    • Bump minimum supported version to Python 3.8
    • Fix minor docs

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.1.3(Feb 15, 2022)

    Included

    • Cleanup deprecated leftovers from objsize migration

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.1.2(Feb 15, 2022)

    Included

    • Replaced primitive deep size algorithm with objsize

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.6(Feb 15, 2022)

    Included

    • Fixed "falsey" key causing badly reported item expiry. Thanks @lokhman
    • Fixed get_deep_size incorrectly passing params on deeply nested dicts.
    • Fix handling of attempted nested deletion of already deleted items.
    • Refactor constants to own file and general code cleanup.

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.5(Dec 20, 2021)

    Included

    • Fix usage in README

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.4(Dec 20, 2021)

    Included

    • Add module level python version lockout
    • Improve object __repr__
    • Add usage docstring

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.3(Dec 20, 2021)

    Included

    • Added new support for memory constraint shortcuts, 1M for 1 Mebibyte, 1K, 1G and 1T also supported respectively.
    • Added support for float TTL and float memory constraints.
    • Fixed nasty bug which could cause livelock if memory constraint made too small after data had been added.
    • Added max items to object __repr___.
    • Standardised user input linting.
    • Unit tests for the new functionality.
    • Excluded scripts folder from the distributable.
    • Refactoring of project to split out logic into smaller files.

    Todo:

    • Add usage documentation.

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.2(Dec 17, 2021)

    Initial release.

    Included

    • Support implemented for TTL, LRU and max mebibyte size.
    • Unit tests.

    Todo:

    • Add usage documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)
Owner
Juan
Security Engineer, Python charmer, JSON wizard.
Juan
GitHub Repository
Final Project for Practical Python Programming and Algorithms for Data Analysis

Final Project for Practical Python Programming and Algorithms for Data Analysis (PHW2781L, Summer 2020) Redlining, Race-Exclusive Deed Restriction Lan

Aislyn Schalck 1 Jan 27, 2022
Solutions for leetcode problems.

Leetcode-solution This is an repository for storring new algorithms that I am learning form the LeetCode for future use. Implemetations Two Sum (pytho

Shrutika Borkute 1 Jan 09, 2022
Python tree data library

Links Documentation PyPI GitHub Changelog Issues Contributors If you enjoy anytree Getting started Usage is simple. Construction from anytree impo

776 Dec 28, 2022
dict subclass with keylist/keypath support, normalized I/O operations (base64, csv, ini, json, pickle, plist, query-string, toml, xml, yaml) and many utilities.

python-benedict python-benedict is a dict subclass with keylist/keypath support, I/O shortcuts (base64, csv, ini, json, pickle, plist, query-string, t

Fabio Caccamo 799 Jan 09, 2023
A simple tutorial to use tree-sitter to parse code into ASTs

A simple tutorial to use py-tree-sitter to parse code into ASTs. To understand what is tree-sitter, see https://github.com/tree-sitter/tree-sitter. Tr

Nghi D. Q. Bui 7 Sep 17, 2022
Persistent dict, backed by sqlite3 and pickle, multithread-safe.

sqlitedict -- persistent dict, backed-up by SQLite and pickle A lightweight wrapper around Python's sqlite3 database with a simple, Pythonic dict-like

RARE Technologies 954 Dec 23, 2022
This repo is all about different data structures and algorithms..

Data Structure and Algorithm : Want to learn data strutrues and algorithms ??? Then Stop thinking more and start to learn today. This repo will help y

Priyanka Kothari 7 Jul 10, 2022
This repo represents all we learned and are learning in Data Structure course.

DataStructure Journey This repo represents all we learned and are learning in Data Structure course which is based on CLRS book and is being taught by

Aprime Afr (Alireza Afroozi) 3 Jan 22, 2022
IADS 2021-22 Algorithm and Data structure collection

A collection of algorithms and datastructures introduced during UoE's Introduction to Datastructures and Algorithms class.

Artemis Livingstone 20 Nov 07, 2022
An command-line utility that schedules your exams preparation routines

studyplan A tiny utility that schedules your exams preparation routines. You only need to specify the tasks and the deadline. App will output a iCal f

Ilya Breitburg 3 May 18, 2022
schemasheets - structuring your data using spreadsheets

schemasheets - structuring your data using spreadsheets Create a data dictionary / schema for your data using simple spreadsheets - no coding required

Linked data Modeling Language 23 Dec 01, 2022
Multidict is dict-like collection of key-value pairs where key might be occurred more than once in the container.

multidict Multidict is dict-like collection of key-value pairs where key might be occurred more than once in the container. Introduction HTTP Headers

aio-libs 325 Dec 27, 2022
A Python implementation of red-black trees

Python red-black trees A Python implementation of red-black trees. This code was originally copied from programiz.com, but I have made a few tweaks to

Emily Dolson 7 Oct 20, 2022
A high-performance immutable mapping type for Python.

immutables An immutable mapping type for Python. The underlying datastructure is a Hash Array Mapped Trie (HAMT) used in Clojure, Scala, Haskell, and

magicstack 996 Jan 02, 2023
A Munch is a Python dictionary that provides attribute-style access (a la JavaScript objects).

munch munch is a fork of David Schoonover's Bunch package, providing similar functionality. 99% of the work was done by him, and the fork was made mai

Infinidat Ltd. 643 Jan 07, 2023
Google, Facebook, Amazon, Microsoft, Netflix tech interview questions

Algorithm and Data Structures Interview Questions HackerRank | Practice, Tutorials & Interview Preparation Solutions This repository consists of solut

Quan Le 8 Oct 04, 2022
Svector (pronounced Swag-tor) provides extension methods to pyrsistent data structures

Svector Svector (pronounced Swag-tor) provides extension methods to pyrsistent data structures. Easily chain your methods confidently with tons of add

James Chua 5 Dec 09, 2022
This repository is a compilation of important Data Structures and Algorithms based on Python.

Python DSA 🐍 This repository is a compilation of important Data Structures and Algorithms based on Python. Please make seperate folders for different

Bhavya Verma 27 Oct 29, 2022
My solutions to the competitive programming problems on LeetCode, USACO, LintCode, etc.

This repository holds my solutions to the competitive programming problems on LeetCode, USACO, LintCode, CCC, UVa, SPOJ, and Codeforces. The LeetCode

Yu Shen 32 Sep 17, 2022
pyprobables is a pure-python library for probabilistic data structures

pyprobables is a pure-python library for probabilistic data structures. The goal is to provide the developer with a pure-python implementation of common probabilistic data-structures to use in their

Tyler Barrus 86 Dec 25, 2022