Async boto3 with Autogenerated Data Classes

Overview

awspydk

Async boto3 with Autogenerated JIT Data Classes


Motivation

This library is forked from an internal project that works with a lot of backend AWS APIs, and I got tired of having to constantly parse the returned responses before being able to work with them for (most of the time), a few seconds. Any API-driven application that uses boto3 tends to suffer from being syncronous only. So this library solves a few major problems:

  • Enables both sync and async from the same client.

  • Client can be called implicitly without needing initialization, i.e. you can directly use AwsClient without needing to initialize.

  • Dynamically generates class functions based on the boto3.client child functions, and allows you to call them. This is useful in repl or ipython environments where type-hints are always helpful, especially when the names are so long.

  • Automatically initializes the aws client if its not initialized from the defaults, simply by calling it.

  • Translates all results into Automatically Generated Dataclasses through lazycls.

    • Can be disabled by setting aws.config.AutoCls = False

Quickstart

pip install --upgrade awspydk
from aws import AwsClient

# Sync Method
buckets = AwsClient.v1.s3_list_buckets(as_cls=True)

# Async Method
buckets = await AwsClient.v1.async_s3_list_buckets(as_cls=True)

"""
Both yield the same results.
The underlying classes are auto-generated from Pydantic BaseModels, so anything you can do with Pydantic Models, you can do with these.

{
    'Buckets': [
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 8, 25, 16, 42, 46, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 17, 54, 56, tzinfo=tzutc()), Name='...',
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 3, 4, 20, 10, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 1, 20, 50, 33, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 4, 2, 28, tzinfo=tzutc()), Name='...')
    ],
    'Owner': AwsS3Owner(DisplayName='...', ID='...')
}
"""

## Change Regions
AwsClient.reset(region='us-west-1')

## Change the defaut clients created
from aws.config import DefaultClients

## Modify to only create ec2 client
DefaultClients = {
    'ec2': 'ec2'
}

## Reset implicitly
AwsClient.reset()


BotoKwargs = {
    'AWS_PROFILE': ...,
}

## Reset Explicitly
AwsClient.reset(clients=DefaultClients, boto_kwargs=BotoKwargs)

Client Defaults

These are found in aws.config

AwsRegion = envToStr('AWS_REGION', 'us-east-1')
AutoCls = envToBool('AWSSDK_AUTOCLS', 'true')

## These are the default clients that will be autogenerated.
## Key is the shorthand, value is the actual AWS API Name in boto3
DefaultClients = {
    'ec2': 'ec2', 
    'ecr': 'ecr', 
    'r53' :'route53', 
    'acm': 'acm',
    'elb': 'elb',
    'elbv2': 'elbv2',
    'asg': 'autoscaling',
    's3': 's3'
}

## These are the default resources that will be autogenerated.
## Key is the shorthand, value is the actual AWS Resource API Name in boto3
DefaultResources = {
    'Ec2': 'ec2',
    'S3': 's3',
    'Iam': 'iam'
}

# These are the default filter args for querying
DefaultFilterArgs = {
    'string_only': True,
    'remove_null': True
}
Owner
Chief Architect @ Growth Engine
A Collection Manager for the objkt.com Minting Factory

Objkt Collection Manager A Collection Manager for the objkt.com Minting Factory. This contract can create a collection on objkt.com and mint into it.

Asbjorn Enge 5 Nov 22, 2022
Python SDK for LUSID by FINBOURNE, a bi-temporal investment management data platform with portfolio accounting capabilities.

LUSID® Python SDK This is the Python SDK for LUSID by FINBOURNE, a bi-temporal investment management data platform with portfolio accounting capabilit

FINBOURNE 6 Dec 24, 2022
A program to convert YouTube channel registration information into Json files for ThirdTube.

ThirdTubeImporter A program to convert YouTube channel registration information into Json files for ThirdTube. Usage Japanese https://takeout.google.c

Hidegon 2 Dec 18, 2021
Telegram bot to download almost all from Instagram

Instagram Manager Bot The most advanced Instagram Downloader Bot. Please fork this repository don't import code Made with Python3 (C) @subinps Copyrig

SUBIN 300 Dec 30, 2022
Allows you to easily share bookmarks from Raindrop.io in Telegram chats.

Allows you to easily share bookmarks from Raindrop.io in Telegram chats. As well as save links/photos/longreads from Telegram right into Raindrop.io. Join us, we have a nice 'reader mode' :)

Oleh 36 Dec 19, 2022
Client library for accessing IQM quantum computers

IQM Client Client-side library for connecting to an IQM quantum computer. Installation IQM client is not intended to be used directly by human users.

IQM 10 Dec 21, 2022
Unofficial WebApp for WhatsApp Web created in PyQt6

Unofficial WebApp for WhatsApp Web created in PyQt6 using PyQt6-WebEngine

Rafael Tosta Santos 126 Dec 20, 2022
Group Management Bot

❤️ 𝗦𝗛𝗔𝗗𝗜𝗬𝗢 ❤️ A Powerful, Smart And Advance Group Manager ... Written with AioGram , Pyrogram and Telethon... ⭐️ Thanks to everyone who starred

Abdisamad Omar Mohamed 4 Dec 01, 2021
A free sniper bot built to work with PancakeSwap: Router V2

Pancakeswap Sniper Bot PancakeSwap sniper bot. Automated sniping bot to snipe crypto coin launches. How it works The sniping bot can be used in three

89 Aug 06, 2022
Check AWS S3 instances for read/write/delete access

s3sec Test AWS S3 buckets for read/write/delete access This tool was developed to quickly test a list of s3 buckets for public read, write and delete

0xmoot 114 Dec 06, 2022
The world's first public V2ray manager Telegram bot

📌 DarkV2ray-Manager-Bot 0.1 UPDATE 11/11/2021 Telegram bot v2ray Test user expired date data limit paylode && sni usage user on/off heroku bot hostin

@Dk_king_offcial 1 Nov 11, 2021
Balanced API library in python.

Balanced Online Marketplace Payments v1.x requires Balanced API 1.1. Use v0.x for Balanced API 1.0. Installation pip install balanced Usage View Bala

Balanced 70 Oct 04, 2022
My beancount practice as a template

my-beancount-template 个人 Beancount 方案的模板仓库 相关博客 复式记账指北(一):What and Why? 复式记账指北(二):做账方法论 复式记账指北(三):如何打造不半途而废的记账方案 配置 详细配置请参考博客三。必须修改的配置有: Bot功能:data/be

KAAAsS 29 Nov 29, 2022
Configure your linux server and check for vulnerabilities with serverlla

serverlla Configure your linux server and check for vulnerabilities with serverlla. Serverlla has a menu with options and allows you to configure your

Dylan Meca 10 Feb 01, 2022
An script where it logs in your instagram account and follows people and likes their posts

InstaFollower An script where it logs in your instagram account and follows people and likes their posts (uses the tags to fetch people) Requirements:

Bless 3 Nov 29, 2022
A method to check whether a Discord user is using the client or not.

Discord Captcha Method This is an example, of a verification trough a check, if the user loads the picture send with the verification-message. This ma

Julien 2 Jan 19, 2022
Ethereum Gas Fee for the MacBook Pro touchbar (using BetterTouchTool)

Gasbar Ethereum Gas Fee for the MacBook Pro touchbar (using BetterTouchTool) Worried about Ethereum gas fees? Me too. I'd like to keep an eye on them

TSS 51 Nov 14, 2022
Python Telegram Bot Template

Python Telegram Bot Template Concepts Customizable python-telegram-bot template implementing code refactoring to streamline development process: handl

Jansen A. Simanullang 1 Jan 25, 2022
🔏 Discordちゃんねる ◆wGFzKUzY7E

使い方 discord.pyをインストール. python -m pip install -r requirements.txtを実行. bot.pyと同じ階層に.tokenを用意. bot.pyを実行. ※現状、使用しているライブラリの関係でWindowsOSは未対応です。 コマンド ニックネーム

Gattxxa 3 Feb 02, 2022