Dumps to CSV all the resources in an organization's member accounts

Last update: Dec 24, 2021

Overview

AWS Org Inventory

Dumps to CSV all the resources in an organization's member accounts.

Set your environment's AWS_PROFILE and AWS_DEFAULT_REGION variables.

The AWS_PROFILE should be configured to use a role in the organization management account that can assume OrganizationAccountAccessRole in the member accounts.

Redirect the dumper's output to save the file.

The dumper uses Botocove to query each member account.

Why?

This tool fills in the gaps in AWS Config's inventory.

Sadly AWS Config supports only a subset of all the resource types you may have.

AWS Config's organization aggregators are great, but they may not update instantly with new resources.

Installation

The package is published to PyPy as aws-org-inventory, so you can install it with pip or anything equivalent.

pip install aws-org-inventory

Basic example

Configure environment:

export AWS_PROFILE=OrgMgmtRole
export AWS_DEFAULT_REGION=eu-west-1

Dump inventory of CloudWatch log groups:

aws-org-inventory logs describe_log_groups logGroups

Dump inventory of support cases:

aws-org-inventory support describe_cases cases

Dump inventory of EC2 key pairs:

aws-org-inventory ec2 describe_key_pairs KeyPairs

Try doing those with AWS Config!

General use

To derive arguments for other use cases, check the boto service documentation.

The value passed to the boto3.client method that would instantiate a client for the service is parameter 1.

Find the method of that client that lists or describes the resource type that you want to dump.

The name of the method is parameter 2.

Find in the method's response syntax the top-level key for the list of objects.

The name of the key is parameter 3.

Error output

On stderr you will always see a summary of the botocove result and any exceptions. These exceptions may reveal problems such as an incorrect command invocation, a misconfigured AWS account, or a bug in the program (feel free to report those!)

If Botocove fails to get a session for an account, it will output the ID to stderr like this.

Invalid session Account IDs as list: ['111111111111']

That account's resources will not be included in the main output.

Development

Use Poetry to build and push a new version to PyPI.

poetry build
poetry publish

TODO

TODO: query multiple regions (see aws-boto-multiregion-client for example)

TODO: ensure that org management account is included in results

TODO: give example of how to use AwsOrgInventory class in other applications

TODO: improve CLI

TODO: Use boto's service model to automate the parameters given a resource type

TODO: improve error handling

Comments

Fix PyPI package description

"The author of this package has not provided a project description."

I think I can fix it with the readme option.

https://github.com/python-poetry/poetry/issues/1979

https://python-poetry.org/docs/pyproject/#readme

opened by iainelder 1
Bump ipython from 7.28.0 to 7.31.1
Bumps ipython from 7.28.0 to 7.31.1.

Commits

e321e76 release 7.31.1

67ca2b3 Merge pull request from GHSA-pq7m-3gw7-gq5x

2794330 back to dev

be343e7 release 7.31.0

0fcf2c4 Merge pull request #13428 from meeseeksmachine/auto-backport-of-pr-13427-on-7.x

b8db9b1 Backport PR #13427: wn 731

7f253dc Merge pull request #13412 from bnavigator/backport-inspect

4f26796 fix xxlimited_35 import name

77ca4a6 don't run nose-based iptest on py310, only pytest

533e509 back to decorator skip

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Define a config file format
Support configuration for multiple organizations.

Each organization should have a:

user-defined label

management account profile

list of regions

list of organizations units (OUs) to exclude
opened by iainelder 0
Include organization management account in results

botocove by default returns results from member accounts, but not the organization management account.

You can pass a list of account IDs instead.

This might work, but it feel like it's duplicate what botocove already does internally.

Contact the botocove maintainers to see if we can update the tool to return results for the management account.

opened by iainelder 0

Derive transformation from boto model

services = boto3.Session().get_available_services()

def get_shape(service, method):

    meta = boto3.client(service).meta
    api = meta.method_to_api_mapping[method]
    output_shape = meta.service_model.operation_model(api).output_shape
    key = next(k for k, v in output_shape.members.items())
    return output_shape.members[key]

See boto-multiregion-client

Build a table of: key, list / or scalar for top key, scalar or dict for members of list

opened by iainelder 0

Improve compatibility with csvsql

Using aws-org-inventory version 0.2.0.

cases.csv was created using this command:

poetry run aws-org-inventory support describe_cases cases > cases.csv

$ csvsql --snifflimit 0 --query 'SELECT 1 FROM cases' -- cases.csv
/home/isme/.local/pipx/venvs/csvkit/lib/python3.8/site-packages/agate/utils.py:276: UnnamedColumnWarning: Column 0 has no name. Using "a".
OperationalError: (sqlite3.OperationalError) duplicate column name: status
[SQL: 
CREATE TABLE cases (
	a FLOAT, 
	"Id" FLOAT, 
	"Arn" VARCHAR, 
	"Email" VARCHAR, 
	"Name" VARCHAR, 
	"Status" VARCHAR, 
	"AssumeRoleSuccess" BOOLEAN, 
	"caseId" VARCHAR, 
	"displayId" FLOAT, 
	subject VARCHAR, 
	status VARCHAR, 
	"serviceCode" VARCHAR, 
	"categoryCode" VARCHAR, 
	"severityCode" VARCHAR, 
	"submittedBy" VARCHAR, 
	"timeCreated" TIMESTAMP, 
	"recentCommunications" VARCHAR, 
	"ccEmailAddresses" VARCHAR, 
	language VARCHAR, 
	CHECK ("AssumeRoleSuccess" IN (0, 1))
)

]
(Background on this error at: http://sqlalche.me/e/13/e3q8)

Several issues to fix here (maybe split them out):

Avoid duplicate column names by removing less-useful columns added by botocove (Arn, Email, Name, Status) so that only Id remains
Rename Id column to InventoryAccountId (not just AccountId because it may still clash with stack set APIs)
Remove or replace the unnamed index column (this forced me to use snifflimit)
- replace with AccountId and Id of resource (may be hard to determine resource Id in the general case)
- can you have a data frame without an index, or at least hide it on export?
- replace with a monotonically increasing id

opened by iainelder 0

Releases(v0.5.2.post1)

v0.5.2.post1(Mar 19, 2022)

Source code(tar.gz)
Source code(zip)
v0.5.2(Mar 15, 2022)

Source code(tar.gz)
Source code(zip)
v0.5.1(Mar 15, 2022)

Source code(tar.gz)
Source code(zip)

Owner

Iain Samuel McLean Elder

AWS Certified Solutions Architect. Freelance consultant.

GitHub Repository

Easy-apply-bot - A LinkedIn Easy Apply bot to help with my job search.

easy-apply-bot A LinkedIn Easy Apply bot to help with my job search. Getting Started First, clone the repository somewhere onto your computer, or down

5 Dec 09, 2022

A frame to create discord bots (for myself) that uses cogs, JSON, activities, and more.

dpy-frame A frame to create discord bots (for myself) that uses cogs, JSON, activities, and more. NOTE: Documentation is incomplete, so please wait un

1 Nov 06, 2021

ANKIT-OS/TG-SESSION-HACK-BOT: A Special Repository.Telegram Bot Which Can Hack The Victim By Using That Victim Session

🔰 ᵀᴱᴸᴱᴳᴿᴬᴹ ᴴᴬᶜᴷ ᴮᴼᵀ 🔰 The owner would not be responsible for any kind of bans due to the bot. • ⚡ INSTALLING ⚡ • • 🛠️ Lᴀɴɢᴜᴀɢᴇs Aɴᴅ Tᴏᴏʟs 🔰 • If

2 Dec 24, 2021

Discord bot for Shran development

shranbot A discord bot named Herbert West that will monitor the Shran development discord server. Using dotenv shranbot uses a .env file to load secre

1 Jul 29, 2022

Python SDK for LUSID by FINBOURNE, a bi-temporal investment management data platform with portfolio accounting capabilities.

LUSID® Python SDK This is the Python SDK for LUSID by FINBOURNE, a bi-temporal investment management data platform with portfolio accounting capabilit

6 Dec 24, 2022

Python library wrapping and enhancing the Invenio RDM REST API.

Iridium The metal Iridium is used to refine and enhance metal alloys. Similarly, this package provides an enhanced coating around the Invenio RDM APIs

2 Mar 29, 2022

基于nonebot2的twitter推送插件

HanayoriBot(Twitter插件) ✨ 基于NoneBot2的Twitter推送插件，自带百度翻译接口 ✨ 简介本插件基于NoneBot2与go-cqhttp,可以及时将Twitter用户的最新推文推送至群聊，并且自带基于百度翻译的推文翻译接口，及时跟进你所关注的Vtuber的外网动态。

16 Feb 12, 2022

Shred your reddit comment and post history

trasheddit Shred your reddit comment and post history (x89/Shreddit replacement) Usage Simple Example Download trasheddit: git clone https://github.co

2 Jan 05, 2022

A file-based quote bot written in Python

Let's Write a Python Quote Bot! This repository will get you started with building a quote bot in Python. It's meant to be used along with the Learnin

1 Nov 01, 2021

The text based version of my App Blocker that I planning on converting to GUI soon.

App-Blocker The text based version of my App Blocker that I planning on converting to GUI soon. Currently I am just uploading the appblocker.py file,

0 Sep 13, 2022

EduuRobot Telegram bot source code.

EduuRobot A multipurpose Telegram Bot made with Pyrogram and asynchronous programming. Requirements Python 3.6+ An Unix-like operating system (Running

119 Dec 23, 2022

An Open Source ALL-In-One Telegram RoBot, that can do lot of things.

URL Uploader Bot An Open Source ALL-In-One Telegram RoBot, that can do lot of things. My Features Installation The Easy Way You can also tap the Deplo

1 Oct 23, 2021

A simple bot that lives in your Telegram group, logging messages to a Postgresql database and serving statistical tables and plots to users as Telegram messages.

telegram-stats-bot Telegram-stats-bot is a simple bot that lives in your Telegram group, logging messages to a Postgresql database and serving statist

22 Dec 26, 2022

Discord bot that displays the current Swatch Internet Time (.beat) as a status.

Internet-Time-Display Discord bot that displays the current Swatch Internet Time (.beat) as a status. Visit the website! Add the bot to your server! A

2 Mar 15, 2022

A python based Telegram Bot for Compressing Videos with negligible Quality change

𝕍𝕚𝕕𝕖𝕠 ℂ𝕆𝕄ℙℝ𝔼𝕊𝕊𝕆ℝ 𝔹𝕆𝕋 ᴍᴜʟᴛɪғᴜɴᴄᴛɪᴏɴ ǫᴜᴀʟɪᴛʏ ᴄᴏᴍᴘʀᴇssᴏʀ A Telegram Video CompressorBot it compress videos with negligible Quality change.

154 Dec 04, 2022

Boilerplate template for the discord-py-interactions library

discord-py-interactions_boilerplate Boilerplate template for the discord-py-interactions library Currently, this boilerplate supports discord-py-inter

7 Dec 03, 2022

This is a free python bot program that crosses you to farm with auto click in space crypto NFT game, having fun :) Creator: Marlon Zanardi

🚀 Space Crypto auto click bot ready-to-use 🚀 This is a free python bot program that crosses you to farm with auto click in space crypto NFT game, ha

170 Dec 20, 2022