Async boto3 with Autogenerated Data Classes

Overview

awspydk

Async boto3 with Autogenerated JIT Data Classes


Motivation

This library is forked from an internal project that works with a lot of backend AWS APIs, and I got tired of having to constantly parse the returned responses before being able to work with them for (most of the time), a few seconds. Any API-driven application that uses boto3 tends to suffer from being syncronous only. So this library solves a few major problems:

  • Enables both sync and async from the same client.

  • Client can be called implicitly without needing initialization, i.e. you can directly use AwsClient without needing to initialize.

  • Dynamically generates class functions based on the boto3.client child functions, and allows you to call them. This is useful in repl or ipython environments where type-hints are always helpful, especially when the names are so long.

  • Automatically initializes the aws client if its not initialized from the defaults, simply by calling it.

  • Translates all results into Automatically Generated Dataclasses through lazycls.

    • Can be disabled by setting aws.config.AutoCls = False

Quickstart

pip install --upgrade awspydk
from aws import AwsClient

# Sync Method
buckets = AwsClient.v1.s3_list_buckets(as_cls=True)

# Async Method
buckets = await AwsClient.v1.async_s3_list_buckets(as_cls=True)

"""
Both yield the same results.
The underlying classes are auto-generated from Pydantic BaseModels, so anything you can do with Pydantic Models, you can do with these.

{
    'Buckets': [
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 8, 25, 16, 42, 46, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 17, 54, 56, tzinfo=tzutc()), Name='...',
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 3, 4, 20, 10, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 1, 20, 50, 33, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 4, 2, 28, tzinfo=tzutc()), Name='...')
    ],
    'Owner': AwsS3Owner(DisplayName='...', ID='...')
}
"""

## Change Regions
AwsClient.reset(region='us-west-1')

## Change the defaut clients created
from aws.config import DefaultClients

## Modify to only create ec2 client
DefaultClients = {
    'ec2': 'ec2'
}

## Reset implicitly
AwsClient.reset()


BotoKwargs = {
    'AWS_PROFILE': ...,
}

## Reset Explicitly
AwsClient.reset(clients=DefaultClients, boto_kwargs=BotoKwargs)

Client Defaults

These are found in aws.config

AwsRegion = envToStr('AWS_REGION', 'us-east-1')
AutoCls = envToBool('AWSSDK_AUTOCLS', 'true')

## These are the default clients that will be autogenerated.
## Key is the shorthand, value is the actual AWS API Name in boto3
DefaultClients = {
    'ec2': 'ec2', 
    'ecr': 'ecr', 
    'r53' :'route53', 
    'acm': 'acm',
    'elb': 'elb',
    'elbv2': 'elbv2',
    'asg': 'autoscaling',
    's3': 's3'
}

## These are the default resources that will be autogenerated.
## Key is the shorthand, value is the actual AWS Resource API Name in boto3
DefaultResources = {
    'Ec2': 'ec2',
    'S3': 's3',
    'Iam': 'iam'
}

# These are the default filter args for querying
DefaultFilterArgs = {
    'string_only': True,
    'remove_null': True
}
Owner
Chief Architect @ Growth Engine
Solves bombcrypto newest captcha

Solves Bombcrypto newest captcha A very compact implementation using just cv2 and ctypes, ready to be deployed to your own project. How does it work I

19 May 06, 2022
Check your accounts/tokens fast with our checker!

Discord_Account_Checker How to use? Installing library's pip install -r reqs.txt Loading accounts Load your accounts to accounts.txt file. Launch pyth

1 Jan 11, 2022
Gnosis-py includes a set of libraries to work with Ethereum and Gnosis projects

Gnosis-py Gnosis-py includes a set of libraries to work with Ethereum and Gnosis projects: EthereumClient, a wrapper over Web3.py Web3 client includin

Gnosis 93 Dec 23, 2022
GitHub Usage Report

github-analytics from github_analytics import analyze pr_analysis = analyze.PRAnalyzer( "organization/repo", "organization", "team-name",

Shrivu Shankar 1 Oct 26, 2021
A powerful discord bot for forming team.

Discord_SquadBot A powerful discord bot for forming team. Pre-requirement Python 3.7 and latest Discord.py module is required. Installation guideline

Jacky Yu 2 Jan 29, 2022
Automating whatsapp with python

whatsapp-automation Automating whatsapp with python used on this project pyautogui pywhatkit pyttsx3 SpeechRecognition colorama embedded in python tim

Miguel 60 Nov 21, 2022
Basic Discord python bot

#How to Create a Discord Bot Account In order to work with the Python library and the Discord API, we must first create a Discord Bot account. Here ar

Tustus 1 Oct 13, 2021
“ HOLA HUMANS 👋 I'M DAISYX 2.0 ❤️ „ LATEST VERSION OF DAISYX.. Source Code of @Daisyxbot

❤️ DaisyX 2.0 ❤️ A Powerful, Smart And Simple Group Manager ... Written with AioGram , Pyrogram and Telethon... ⭐️ Thanks to everyone who starred Dais

TeamDaisyX 153 Dec 06, 2022
DeKrypt 24 Sep 21, 2022
The Foursquare API client for Python

foursquare Python client for the foursquare API. Philosophy: Map foursquare's endpoints one-to-one Clean, simple, Pythonic calls Only handle raw data,

Mike Lewis 400 Dec 19, 2022
An API Client package to access the APIs for NBA.com

nba_api An API Client package to access the APIs for NBA.com Development Version: v1.1.9 nba_api is an API Client for www.nba.com. This package is mea

Swar Patel 1.4k Jan 01, 2023
A jokes api python module

A jokes api python module

Fayas Noushad 3 Nov 28, 2021
AWSXenos will list all the trust relationships in all the IAM roles and S3 buckets

AWS External Account Scanner Xenos, is Greek for stranger. AWSXenos will list all the trust relationships in all the IAM roles, and S3 buckets, in an

AirWalk 57 Nov 07, 2022
A discord token nuker With loads of options that will screw an account up real bad

A discord token nuker With loads of options that will screw an account up real bad, also has inbuilt massreport, GroupChat Spammer and Token/Password/Creditcard grabber and so much more!

XPTGR 0 Aug 07, 2022
Python library wrapping and enhancing the Invenio RDM REST API.

Iridium The metal Iridium is used to refine and enhance metal alloys. Similarly, this package provides an enhanced coating around the Invenio RDM APIs

Materials Data Science and Informatics 2 Mar 29, 2022
Notion4ever - Python tool for export all your content of Notion page using official Notion API

NOTION4EVER Notion4ever is a small python tool that allows you to free your cont

50 Dec 30, 2022
Troposphere and shellscript based AWS infrastructure automation creates an awsapigateway lambda with a go backend

Automated-cloudformation-infra Troposphere and shellscript based AWS infrastructure automation. Feel free to clone and edit for personal usage. The en

1 Jan 03, 2022
Facebook fishing on telegram bot

Facebook-fishing Facebook fishing on telegram bot تثبيت الاداة pkg update -y pkg upgrade -y pkg install git -y pkg install python -y git clone https:/

sadamalsharabi 7 Oct 18, 2022
RedFish API Toolkit

RedFish API Toolkit RedFish API Toolkit Build Status GitHub Actions Travis CI Requirements requirements.txt requirements-dev.txt Dependencies Document

Larry Smith Jr. 1 Nov 20, 2021