Yet Another Workflow Parser for SecurityHub

Related tags

Data Analysisyawps
Overview

YAWPS

Yet Another Workflow Parser for SecurityHub

"Screaming pepper" by Rum Bucolic Ape is licensed with CC BY-ND 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-nd/2.0/

Purpose

Currently SecurityHub has a ChatBot integration thats a bit lacking. All of securityhub goes to chatbot, which means a singular flooding channel of alerts.

With cloud-custodians recent support for securityhub and organizations we have a good way to send all alerts for an entire org to slack. But that means every account goes to a single channel.

This repo is part of a multi-series talk/demo on how to intelligently route account messages to differing Slack channels.

In the scenario where a team owns an account it would be nice to let cloud-custodian generate meaningful securityhub notifications that go to specific team channels.

For this talk we will simply tag AWS accounts with 2 tags account_name (a human readable name) and slack_channel (a slack channel to direct those security hub notifications to).

A blog post and KubeCon talk will be coming soon with more information

Prerequisites

The only real pre-requisite here is a working multi-account SecurityHub

Configuration

Environment Variable Description
SLACK_FALLBACK_CHANNEL Channel to fallback to if the slack_channel tag is not provided on the account
SLACK_TOKEN the path in SSM to the slack token`
SLACK_TOKEN_SSM_PATH if a SLACK_TOKEN is not found, this is where to grab it from the EC2 Param store
LOGGING_LEVEL the logging level to use. Default is INFO
ENABLE_FORK_COPY_SEVERITY Enable the ability to fork some messages to another channel by severity. Value can be True or False. Default is False
FORK_COPY_SEVERITY_VALUE If ENABLE_FORK_COPY_SEVERITY is True, what severity level to fork by. Should be an integer between 0 and 100. Default is 90
ENABLE_FORK_ONLY_SEVERITY Enable the ability to fork some messages to only another channel by severity. Value can be True or False. Default is False
FORK_ONLY_SEVERITY_VALUE If ENABLE_FORK_ONLY_SEVERITY is True, what severity level to fork by. Should be an integer between 0 and 100. Default is 100

Forking

There are a few use cases for forking.

In general (all defaults) YAWPS will only send to the channel found in the tag or the SLACK_FALLBACK_CHANNEL (because it's required).

This is great until you have rules that you want a second team (lets say security) to also see and follow up with.

Using ENABLE_FORK_COPY_SEVERITY and FORK_COPY_SEVERITY_VALUE lets you also send to that second slack channel. Lets say you set FORK_COPY_SEVERITY_VALUE to 90. This means that anything rated 90 will send to both.

Another use-case exists: not sending team specific alerts. Lets say that an alert is not actionable by the configured team, but is purely for security visibility (like failed IAM logins etc). You can use ENABLE_FORK_ONLY_SEVERITY set to, say 100, in this scenario so that custom rules can set severity to 100 and send it only to security and bypass the primary team. This is good for noise filtration and helping to keep things actionable by a singular source.

Deploy

ServerLess

TODO

Terraform

  1. Download this repository (or a released artifact)
  2. Run make zip to produce a fully deployable s3 artifact
  3. Deploy something similar to this terraform

Testing

$ poetry install
$ poetry run tox
Owner
myoung34
Cloud security engineer, tinkerer, tomato farmer
myoung34
:truck: Agile Data Preparation Workflows made easy with dask, cudf, dask_cudf and pyspark

To launch a live notebook server to test optimus using binder or Colab, click on one of the following badges: Optimus is the missing framework to prof

Iron 1.3k Dec 30, 2022
Data and code accompanying the paper Politics and Virality in the Time of Twitter

Politics and Virality in the Time of Twitter Data and code accompanying the paper Politics and Virality in the Time of Twitter. In specific: the code

Cardiff NLP 3 Jul 02, 2022
VHub - An API that permits uploading of vulnerability datasets and return of the serialized data

VHub - An API that permits uploading of vulnerability datasets and return of the serialized data

André Rodrigues 2 Feb 14, 2022
The Spark Challenge Student Check-In/Out Tracking Script

The Spark Challenge Student Check-In/Out Tracking Script This Python Script uses the Student ID Database to match the entries with the ID Card Swipe a

1 Dec 09, 2021
Pandas and Spark DataFrame comparison for humans

DataComPy DataComPy is a package to compare two Pandas DataFrames. Originally started to be something of a replacement for SAS's PROC COMPARE for Pand

Capital One 259 Dec 24, 2022
BinTuner is a cost-efficient auto-tuning framework, which can deliver a near-optimal binary code that reveals much more differences than -Ox settings.

BinTuner is a cost-efficient auto-tuning framework, which can deliver a near-optimal binary code that reveals much more differences than -Ox settings. it also can assist the binary code analysis rese

BinTuner 42 Dec 16, 2022
Leverage Twitter API v2 to analyze tweet metrics such as impressions and profile clicks over time.

Tweetmetric Tweetmetric allows you to track various metrics on your most recent tweets, such as impressions, retweets and clicks on your profile. The

Mathis HAMMEL 29 Oct 18, 2022
Implementation in Python of the reliability measures such as Omega.

reliabiliPy Summary Simple implementation in Python of the [reliability](https://en.wikipedia.org/wiki/Reliability_(statistics) measures for surveys:

Rafael Valero Fernández 2 Apr 27, 2022
An easy-to-use feature store

A feature store is a data storage system for data science and machine-learning. It can store raw data and also transformed features, which can be fed straight into an ML model or training script.

ByteHub AI 48 Dec 09, 2022
Statistical Analysis 📈 focused on statistical analysis and exploration used on various data sets for personal and professional projects.

Statistical Analysis 📈 This repository focuses on statistical analysis and the exploration used on various data sets for personal and professional pr

Andy Pham 1 Sep 03, 2022
BioMASS - A Python Framework for Modeling and Analysis of Signaling Systems

Mathematical modeling is a powerful method for the analysis of complex biological systems. Although there are many researches devoted on produ

BioMASS 22 Dec 27, 2022
Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis. You write a high level configuration file specifying your in

Blue Collar Bioinformatics 917 Jan 03, 2023
Numerical Analysis toolkit centred around PDEs, for demonstration and understanding purposes not production

Numerics Numerical Analysis toolkit centred around PDEs, for demonstration and understanding purposes not production Use procedure: Initialise a new i

George Whittle 1 Nov 13, 2021
First and foremost, we want dbt documentation to retain a DRY principle. Every time we repeat ourselves, we waste our time. Second, we want to understand column level lineage and automate impact analysis.

dbt-osmosis First and foremost, we want dbt documentation to retain a DRY principle. Every time we repeat ourselves, we waste our time. Second, we wan

Alexander Butler 150 Jan 06, 2023
A Big Data ETL project in PySpark on the historical NYC Taxi Rides data

Processing NYC Taxi Data using PySpark ETL pipeline Description This is an project to extract, transform, and load large amount of data from NYC Taxi

Unnikrishnan 2 Dec 12, 2021
This is an analysis and prediction project for house prices in King County, USA based on certain features of the house

This is a project for analysis and estimation of House Prices in King County USA The .csv file contains the data of the house and the .ipynb file con

Amit Prakash 1 Jan 21, 2022
Sample code for Harry's Airflow online trainng course

Sample code for Harry's Airflow online trainng course You can find the videos on youtube or bilibili. I am working on adding below things: the slide p

102 Dec 30, 2022
Analyzing Covid-19 Outbreaks in Ontario

My group and I took Covid-19 outbreak statistics from ontario, and analyzed them to find different patterns and future predictions for the virus

Vishwaajeeth Kamalakkannan 0 Jan 20, 2022
Renato 214 Jan 02, 2023
Stock Analysis dashboard Using Streamlit and Python

StDashApp Stock Analysis Dashboard Using Streamlit and Python If you found the content useful and want to support my work, you can buy me a coffee! Th

StreamAlpha 27 Dec 09, 2022