Dumps to CSV all the resources in an organization's member accounts

Overview

AWS Org Inventory

Dumps to CSV all the resources in an organization's member accounts.

Set your environment's AWS_PROFILE and AWS_DEFAULT_REGION variables.

The AWS_PROFILE should be configured to use a role in the organization management account that can assume OrganizationAccountAccessRole in the member accounts.

Redirect the dumper's output to save the file.

The dumper uses Botocove to query each member account.

Why?

This tool fills in the gaps in AWS Config's inventory.

Sadly AWS Config supports only a subset of all the resource types you may have.

AWS Config's organization aggregators are great, but they may not update instantly with new resources.

Installation

The package is published to PyPy as aws-org-inventory, so you can install it with pip or anything equivalent.

pip install aws-org-inventory

Basic example

Configure environment:

export AWS_PROFILE=OrgMgmtRole
export AWS_DEFAULT_REGION=eu-west-1

Dump inventory of CloudWatch log groups:

aws-org-inventory logs describe_log_groups logGroups

Dump inventory of support cases:

aws-org-inventory support describe_cases cases

Dump inventory of EC2 key pairs:

aws-org-inventory ec2 describe_key_pairs KeyPairs

Try doing those with AWS Config!

General use

To derive arguments for other use cases, check the boto service documentation.

The value passed to the boto3.client method that would instantiate a client for the service is parameter 1.

Find the method of that client that lists or describes the resource type that you want to dump.

The name of the method is parameter 2.

Find in the method's response syntax the top-level key for the list of objects.

The name of the key is parameter 3.

Error output

On stderr you will always see a summary of the botocove result and any exceptions. These exceptions may reveal problems such as an incorrect command invocation, a misconfigured AWS account, or a bug in the program (feel free to report those!)

If Botocove fails to get a session for an account, it will output the ID to stderr like this.

Invalid session Account IDs as list: ['111111111111']

That account's resources will not be included in the main output.

Development

Use Poetry to build and push a new version to PyPI.

poetry build
poetry publish

TODO

TODO: query multiple regions (see aws-boto-multiregion-client for example)

TODO: ensure that org management account is included in results

TODO: give example of how to use AwsOrgInventory class in other applications

TODO: improve CLI

TODO: Use boto's service model to automate the parameters given a resource type

TODO: improve error handling

Comments
  • Fix PyPI package description

    Fix PyPI package description

    "The author of this package has not provided a project description."

    I think I can fix it with the readme option.

    https://github.com/python-poetry/poetry/issues/1979

    https://python-poetry.org/docs/pyproject/#readme

    opened by iainelder 1
  • Bump ipython from 7.28.0 to 7.31.1

    Bump ipython from 7.28.0 to 7.31.1

    Bumps ipython from 7.28.0 to 7.31.1.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Define a config file format

    Define a config file format

    Support configuration for multiple organizations.

    Each organization should have a:

    • user-defined label
    • management account profile
    • list of regions
    • list of organizations units (OUs) to exclude
    opened by iainelder 0
  • Include organization management account in results

    Include organization management account in results

    botocove by default returns results from member accounts, but not the organization management account.

    You can pass a list of account IDs instead.

    This might work, but it feel like it's duplicate what botocove already does internally.

    Contact the botocove maintainers to see if we can update the tool to return results for the management account.

    opened by iainelder 0
  • Derive transformation from boto model

    Derive transformation from boto model

    services = boto3.Session().get_available_services()
    
    def get_shape(service, method):
    
        meta = boto3.client(service).meta
        api = meta.method_to_api_mapping[method]
        output_shape = meta.service_model.operation_model(api).output_shape
        key = next(k for k, v in output_shape.members.items())
        return output_shape.members[key]
    

    See boto-multiregion-client

    Build a table of: key, list / or scalar for top key, scalar or dict for members of list

    opened by iainelder 0
  • Improve compatibility with csvsql

    Improve compatibility with csvsql

    Using aws-org-inventory version 0.2.0.

    cases.csv was created using this command:

    poetry run aws-org-inventory support describe_cases cases > cases.csv
    
    $ csvsql --snifflimit 0 --query 'SELECT 1 FROM cases' -- cases.csv
    /home/isme/.local/pipx/venvs/csvkit/lib/python3.8/site-packages/agate/utils.py:276: UnnamedColumnWarning: Column 0 has no name. Using "a".
    OperationalError: (sqlite3.OperationalError) duplicate column name: status
    [SQL: 
    CREATE TABLE cases (
    	a FLOAT, 
    	"Id" FLOAT, 
    	"Arn" VARCHAR, 
    	"Email" VARCHAR, 
    	"Name" VARCHAR, 
    	"Status" VARCHAR, 
    	"AssumeRoleSuccess" BOOLEAN, 
    	"caseId" VARCHAR, 
    	"displayId" FLOAT, 
    	subject VARCHAR, 
    	status VARCHAR, 
    	"serviceCode" VARCHAR, 
    	"categoryCode" VARCHAR, 
    	"severityCode" VARCHAR, 
    	"submittedBy" VARCHAR, 
    	"timeCreated" TIMESTAMP, 
    	"recentCommunications" VARCHAR, 
    	"ccEmailAddresses" VARCHAR, 
    	language VARCHAR, 
    	CHECK ("AssumeRoleSuccess" IN (0, 1))
    )
    
    ]
    (Background on this error at: http://sqlalche.me/e/13/e3q8)
    

    Several issues to fix here (maybe split them out):

    • Avoid duplicate column names by removing less-useful columns added by botocove (Arn, Email, Name, Status) so that only Id remains
    • Rename Id column to InventoryAccountId (not just AccountId because it may still clash with stack set APIs)
    • Remove or replace the unnamed index column (this forced me to use snifflimit)
      • replace with AccountId and Id of resource (may be hard to determine resource Id in the general case)
      • can you have a data frame without an index, or at least hide it on export?
      • replace with a monotonically increasing id
    opened by iainelder 0
Releases(v0.5.2.post1)
Owner
Iain Samuel McLean Elder
AWS Certified Solutions Architect. Freelance consultant.
Iain Samuel McLean Elder
Authenticate your League of legends account on riot client in a few lines of code.

lol-authenticator v1.0.0 Content index Project Setup Dependencies Project Setup Dependencies Python v3.9.6 If you don't have Python installed on your

Cássio Fontoura 5 Aug 28, 2022
Trading strategy for the Freqtrade crypto bot

NostalgiaForInfinity Trading strategy for the Freqtrade crypto bot Change strategy Add strategies to the user_data/strategies folder and also in the d

iterativ 1.5k Jan 01, 2023
:snake: A simple library to fetch data from the iTunes Store API made for Python >= 3.5

itunespy itunespy is a simple library to fetch data from the iTunes Store API made for Python 3.5 and beyond. Important: Since version 1.6 itunespy no

Fran González 56 Dec 22, 2022
Discord CTF helper bot for CyberErudites

Eruditus - CTF helper bot Eruditus - CTF helper bot About Eruditus is a Discord CTF helper bot built with Python, it was initially designed to be used

Hafidh 34 Dec 30, 2022
Spodcast is a caching Spotify podcast to RSS proxy

Spodcast Spodcast is a caching Spotify podcast to RSS proxy. Using Spodcast you can follow Spotify-hosted netcasts/podcasts using any player which sup

Frank de Lange 260 Jan 01, 2023
PYAW allows you to call assembly from python

PYAW allows you to call assembly from python

2 Dec 13, 2021
Um script simples para consultar dados, com API's simples.

Info sobre o Script Esta é uma das mais simples ferramentas para consultar dados. Daqui um tempo eu farei um UPGRADE no painel, irei adicionar um banc

Crowley 6 Apr 11, 2022
Evernote SDK for Python

Evernote SDK for Python Evernote API version 1.28 This SDK is intended for use with Python 2.X For Evernote's beta Python 3 SDK see https://github.com

Evernote 612 Dec 30, 2022
A Pancakeswap v2 trading client (and bot) with limit orders, stop-loss, custom gas strategies, a GUI and much more.

Pancakeswap v2 trading client A Pancakeswap trading client (and bot) with limit orders, stop-loss, custom gas strategies, a GUI and much more. If you

571 Mar 15, 2022
A Rich renderable for viewing Multiple Sequence Alignments in the terminal.

rich-msa A simple module to render colorful Multiple Sequence Alignment with rich in the terminal. 🔧 Installing Install the rich-msa package directly

Martin Larralde 64 Dec 04, 2022
A simple discord bot that generates facts!

fact-bot A simple discord bot that generates facts! How to make a bot Go to https://discord.com/developers/applications Then click on 'New Application

1 Jan 05, 2022
Public API client for GETTR, a "non-bias [sic] social network," designed for data archival and analysis.

GoGettr GoGettr is an API client for GETTR, a "non-bias [sic] social network." (We will not reward their domain with a hyperlink.) GoGettr is built an

Stanford Internet Observatory 72 Dec 14, 2022
The purpose of this bot is to take soundcloud track requests, that are posted in the stream-requests channel, and add them to a playlist, to make the process of scrolling through the requests easier for Root

Discord Song Collector Dont know if anyone is actually going to read this, but the purpose of this bot is to check every message in the stream-request

2 Mar 01, 2022
New discord token grabber, password and general information

New discord token grabber, password and general information

Monstered 6 Nov 09, 2022
ToqueIO Nuke tools - A collection of tools designed to assist in enhancing your workflows within nuke

ToqueIO Nuke tools - A collection of tools designed to assist in enhancing your workflows within nuke

4 Feb 19, 2022
A VCVideoPlayer Bot for Telegram made with 💞 By @ProErrorXD

VC Video Player How To Host ✨ Heroku Deploy ✨ The easiest way to deploy this Bot is via Heroku. Credit 🔥 |🇮🇳 Louis |🇮🇳 Sammy |🇮🇳 Blaze Marsha

丂ムᄊᄊƳ 95 May 17, 2022
“ Hey there 👋 I'm Daisy „ AI based Advanced Group Management Bot Suit For All Your Needs ❤️.. Source Code of @Daisyxbot

Project still under heavy development Everything will be changed in the release “ Hey there 👋 I'm Daisy „ AI based Advanced telegram Group Management

TeamDaisyX 43 Nov 12, 2022
GUI Pancakeswap2 and Uniswap3 trading client (and bot)

GUI Pancakeswap2 and Uniswap3 trading client (and bot) (MOST ADVANCE TRADING BOT SUPPORT WINDOWS LINUX MAC) (AUTO BUY TOKEN ON LUNCH AFTER ADD LIQUIDI

16 Dec 23, 2021
VC-Music , Playing music without bot.

VC-Userbot A Telegram Userbot to play or streaming Audio and Video songs / files in Telegram Voice Chats. It's made with PyTgCalls and Pyrogram Requir

RioProjectX 8 Aug 04, 2022
Using Streamlit to build a simple UI on top of the OpenSea API

OpenSea API Explorer Using Streamlit to build a simple UI on top of the OpenSea API. 🤝 Contributing Contributions, issues and feature requests are we

Gavin Capriola 1 Jan 04, 2022