Functional interface for concurrent futures, including asynchronous I/O.

Overview

image image image image image image image image image

Futured provides a consistent interface for concurrent functional programming in Python. It wraps any callable to return a concurrent.futures.Future, wraps any async coroutine to return an asyncio.Future, and provides concurrent iterators and context managers for futures.

Usage

threaded, processed

Transform any callable into one which runs in a thread or process pool, and returns a future.

from futured import threaded, processed
import httpx

fetch = threaded(httpx.Client().get)
fetch(url)  # return Future

fs = (fetch(url + path) for path in paths)
threaded.results(fs)  # generate results from futures
threaded.results(fs, timeout=...)  # generate results as completed

fetch.map(urls)  # generate results in order
fetch.map(urls, timeout=...)  # generate results as completed
fetch.mapzip(urls)  # generate (url, result) pairs as completed

Thread and process pool executors may be used as context managers, customized with options, and reused with different callables.

threaded(max_workers=...)(func, ...)
processed(max_workers=...)(func, ...)

futured classes have a waiting context manager which collects results from tasks. Futures can be registered at creation, or appended to the list of tasks.

with threaded.waiting(*fs) as tasks:
    tasks.append(future)
tasks  # list of completed results

futured classes provide a tasks interface which generalizes futures.as_completed and futures.wait, while allowing the set of tasks to be modified, e.g., for retries.

threaded.tasks(fs, timeout=...)  # mutable set of running tasks which iterate as completed

asynced

The same interface works for asyncio.

from futured import asynced
import httpx

fetch = asynced(httpx.AsyncClient().get)
fetch(url)  # return coroutine

asynced.results(fs)  # generate results from futures
asynced.results(fs, timeout=...)  # generate results as completed

fetch.map(urls)  # generate results in order
fetch.map(urls, timeout=...)  # generate results as completed
fetch.mapzip(urls)  # generate (url, result) pairs as completed

asynced provides utilities for calling coroutines from a synchronous context. waiting is similar to trio's nursery, but returns results from a synchronous with block.

asynced.run(async_func, ...)  # call and run until complete
asynced.run(async_gen, ...)  # call and run synchronous iterator
with asynced.waiting(*fs) as tasks:  # concurrent coroutines completed in a block
asynced.tasks(fs, timeout=...)  # mutable set of running tasks which iterate as completed

decorators

Naturally futured wrappers can be used as decorators, but arguments can also be partially bound.

@threaded
def slow():
   ...

fetch = threaded(httpx.Client().get, url)
fetch(params=...)

Methods are supported, as well as a decorated utility for automatically subclassing.

from futured import decorated

FutureClient = decorated(httpx.Client, request=threaded)

 # equivalent to
class FutureClient(httpx.Client):
    request = threaded(httpx.Client.request)

command

command wraps subprocess.Popen to provide a Future compatible interface.

from futured import futured, command

command('ls').result()  # return stdout or raises stderr
command('ls').pipe('wc')  # pipes into next command, or | ('wc',... )
for line in command('ls'):  # iterable lines
command.coroutine('ls')  # return coroutine

futured(command, 'ls')  # supports `map` interface
asynced(command.coroutine, 'ls')  # supports `map` interface with timeout

forked

forked allows iteration in separate child processes.

from futured import forked

for value in forked(values, max_workers=...):
    # in a child process
 # in parent after children have exited

Installation

% pip install futured

Tests

100% branch coverage.

% pytest [--cov]

Changes

1.3

  • Python >=3.7 required
  • Python 3.10 event loop changes
  • Streams replaced with tasks

1.2

  • Python >=3.6 required

1.1

  • Stream completed futures from a pending pool

1.0

  • Executed functions are context managers
  • starmap supported

0.3

  • forked has optional maximum number of workers
  • waiting context manager
  • command pipes (|)
  • distributed.Client support

0.2

  • command.coroutine creates asyncio subprocesses
  • futured.mapzip generates results zipped with arguments
  • asynced.run supports asynchronous iterators
Easily Generate Revolut Business Cards

RevBusinessCardGen Easily Generate Revolut Business Cards Prerequisites Before you begin, ensure you have met the following requirements: You have ins

Younes™ 35 Dec 14, 2022
API wrapper for VCS hosting system.

PythonVCS API wrapper for VCS hosting system. Supported platforms Gitea Github, Gitlab, Bitbucket support will not, until that packages is not updated

MisileLaboratory 1 Apr 02, 2022
Syntax highlighting for yarn.lock and bun.lockb files

Yarn.lock Syntax Highlighting Syntax highlighting for yarn.lock and bun.lockb files Installation Plugin is not publushed yet on Package Control, to in

Alexander Kuznetsov 4 Jul 06, 2022
An example project that shows how to check if a certain macro is active in a file.

PlatformIO Check Compiler Flags Example Description Demonstrates the usage of an extra script and a special compilter invocation to get the active mac

Maximilian Gerhardt 1 Oct 28, 2021
HSPICE can not perform Monte Carlo (MC) simulations while considering aging effects

HSPICE can not perform Monte Carlo (MC) simulations while considering aging effects. I developed a python wrapper that automatically performs MC and aging simulations using HPSICE to save engineering

Habib Kazemi 2 Nov 22, 2021
Example code for the book Fluent Python, 1st Edition (O'Reilly, 2015)

Fluent Python, First Edition: example code This repository is archived and will not be updated.

Fluent Python 5.4k Jan 09, 2023
🛠️ Plugin to integrate Chuy with Poetry

Archived This is bundled with Chuy since v1.3.0. Poetry Chuy Plugin This plugin integrates Chuy with Poetry. Note: This only works in Poetry 1.2.0 or

Eliaz Bobadilla 4 Sep 24, 2021
Abilian Core: an enterprise application development platform based on the Flask micro-framework, the SQLAlchemy ORM

About Abilian Core is an enterprise application development platform based on the Flask micro-framework, the SQLAlchemy ORM, good intentions and best

Abilian open source projects 47 Apr 14, 2022
This is a Docker-based pipeline for preparing sextractor-ready multiwavelength images

Pipeline for creating NB422-detected (ODI) catalog The repository contains a Docker-based pipeline for preprocessing observational data. The pipeline

1 Sep 01, 2022
Run CodeServer on Google Colab using Inlets in less than 60 secs using your own domain.

Inlets Colab Run CodeServer on Colab using Inlets in less than 60 secs using your own domain. Features Optimized for Inlets/InletsPro Use your own Cus

2 Dec 30, 2021
A simple program which gets a file(CSV/Excel) with tasks and creates different variants

TestMakerProject A simple program which gets a file(CSV/Excel) with tasks and creates different variants The main program is QTengine.py You only have

George 3 Nov 18, 2021
An alternative site to emplea.do due to inconsistent service of the app.

feline a agile and fast alternative to emplea.do License: MIT Settings Moved to settings. Basic Commands Setting Up Your Users To create a normal user

Codetiger 8 Nov 10, 2021
End-to-End text sumarization, QAs generation using flask.

Help-Me-Read A web application created with Flask + BootStrap + HuggingFace 🤗 to generate summary and question-answer from given input text. It uses

Ankush Kuwar 12 Nov 13, 2022
Just imagine normal bancho, but you can have multiple profiles and funorange speed up maps ranked

Local osu! server Just imagine normal bancho, but you can have multiple profiles and funorange speed up maps ranked (coming soon)! Windows Setup Insta

Cover 25 Nov 15, 2022
validation for pre-commit.ci configuration

pre-commit-ci-config validation for pre-commit.ci configuration installation pip install pre-commit-ci-config api pre_commit_ci_config.SCHEMA a cfgv s

pre-commit.ci 17 Jul 11, 2022
Neogex is a human readable parser standard, being implemented in Python

Neogex (New Expressions) Parsing Standard Much like Regex, Neogex allows for string parsing and validation based on a set of requirements. Unlike Rege

Seamus Donnellan 1 Dec 17, 2021
An account generator for guilded.gg that I made a while back and decided to bring back up

An account generator for guilded.gg that I made a while back and decided to bring back up

8 Nov 17, 2022
Draw random mazes in python

a-maze Draw random mazes in python This program generates and draws a rectangular maze, with an entrance on one side and one on the opposite side. The

Andrea Pasquali 1 Nov 21, 2021
My custom Fedora ostree build with sway/wayland.

Ramblurr's Sway Desktop This is an rpm-ostree based minimal Fedora developer desktop with the sway window manager and podman/toolbox for doing develop

Casey Link 1 Nov 28, 2021
Run PD patches in NRT using Python

The files in this repository demonstrate how to use Pure Data (Pd) patches designed to run in Non-Real-Time mode to batch-process (synthesize, analyze, etc) sounds in series using Python.

Jose Henrique Padovani 3 Feb 08, 2022