ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

⚡ python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

The Dolby.io Developer Days Getting Started with Media APIs Workshop repo.

Dolby.io Developer Days Media APIs Getting Started Application About this Workshop and Application This example is designed to get participants workin

Dolby.io Samples 2 Nov 03, 2022
Dicha herramienta esta creada con una api... esta api permite enviar un SMS cada 12 horas dependiendo del pais... Hay algunos paises y operadoras no están soportados.

SMSFree pkg install python3 pip install requests git clone https://github.com/Hidden-parker/SMSFree cd SMSFree python sms.py DISFRUTA... Dicha herrami

piter 2 Nov 14, 2021
The most Advanced yet simple Multi Cloud tool to transfer Your Data from any cloud to any cloud remotely based on Rclone.⚡

Multi Cloud Transfer (Advanced!) 🔥 1.Setup and Start using Rclone on Google Colab and Create/Edit/View and delete your Rclone config file and keep th

Dr.Caduceus 162 Jan 08, 2023
Space Bot, a Discord bot built for HackerSpace Club of PES University

Space Bot Space Bot, a Discord bot built for HackerSpace Club of PES University What can Space Bot do? Space Bot allows you to lookup any mentor or to

HackerSpace @PESU 7 Oct 23, 2022
A script to forward mass number of media to another group/channel. Heroku deploy

Telegram Forward Script 😇 This is a Script to Forward Large Number of Files to Another Telegram Channel. Star එකක් දාල fork එකක් ගහපියව් 🥴 If You Tr

Anjana Madu 17 Oct 21, 2022
A script to find the people whom you follow, but they don't follow you back

insta-non-followers A script to find the people whom you follow, but they don't follow you back Dependencies: python3 libraries - instaloader, getpass

Ritvik 5 Jul 03, 2022
You can share your Chegg account for answers using this bot with your friends without getting your account blocked/flagged

Chegg-Answer-Bot You can share your Chegg account for answers using this bot with your friends without getting your account blocked/flagged Reuirement

Ammey Saini 27 Dec 24, 2022
OpenQuake's Engine for Seismic Hazard and Risk Analysis

OpenQuake Engine The OpenQuake Engine is an open source application that allows users to compute seismic hazard and seismic risk of earthquakes on a g

Global Earthquake Model 281 Dec 21, 2022
Projeto de estudantes do primeiro período do CIn - UFPE voltado para a criação de um sistema interativo no fechamento da disciplina IF669 - Introdução a Programação.

Projeto Game: Dona da Lua Alunos: Beatriz Férre Clara Kenderessy Matheus Silva Rafael Baltar Roseane Oliveira Samuel Marsaro Sinopse O Cebolinha apron

Maria Clara Kenderessy 5 Dec 20, 2021
Checks if Minecraft accounts are available, or taken.

MCNameChecker Checks validity of Minecraft IGN's. Using async to make it even faster. Has rate-limit detections and Proxy support Usage Q. How do I us

Dimitri Demarkus 5 Apr 22, 2022
AWSXenos will list all the trust relationships in all the IAM roles and S3 buckets

AWS External Account Scanner Xenos, is Greek for stranger. AWSXenos will list all the trust relationships in all the IAM roles, and S3 buckets, in an

AirWalk 57 Nov 07, 2022
Repository for the IPvSeeYou talk at Black Hat 2021

IPvSeeYou Geolocation Lookup Tool Overview IPvSeeYou.py is a tool to assist with geolocating EUI-64 IPv6 hosts. It takes as input an EUI-64-derived MA

57 Nov 08, 2022
Sakura: an powerfull Autofilter bot that can be used in your groups

Sakura AutoFilter This Bot May Look Like Mwk_AutofilterBot And Its Because I Like Its UI, That's All Sakura is an powerfull Autofilter bot that can be

PaulWalker 12 Oct 21, 2022
pylunasvg - Python bindings for lunasvg

pylunasvg - Python bindings for lunasvg Pylunasvg is a simple wrapper around lunasvg that uses pybind11 to create python bindings. All public API of t

Eren 6 Jan 05, 2023
A melhor maneira de atender seus clientes no Telegram!

Clientes.Chat Sobre o serviço Configuração Banco de Dados Variáveis de Ambiente Docker Python Heroku Contribuição Sobre o serviço A maneira mais organ

Gabriel R F 10 Oct 12, 2022
A wrapper for slurm especially on Taiwania2 (HPC CLI)A wrapper for slurm especially on Taiwania2 (HPC CLI)

TWCC-slurm-wrapper A wrapper for slurm especially on Taiwania2 (HPC CLI). For Taiwania2 (HPC CLI) usage, please refer to here. (中文) How to Install? gi

Chi-Liang, Liu 5 Oct 07, 2022
Discord spam bots with multiple account support and more

Discord spam bots with multiple account support and more. PLEASE READ EVERYTHING BEFORE WRITING AN ISSUE!! Server Messages Text Image Dm Messages Text

Mr. Nobody 6 Sep 14, 2022
Module to use some statistics from Spotify API

statify Module to use some statistics from Spotify API To use it you have to import the functions into your own project. You have also to authenticate

Miguel Cózar 2 Jun 02, 2022
Design and build a wrapper for the Open Weather API current weather data service

Design and build a wrapper for the Open Weather API current weather data service that returns a city's temperature, with caching, also allowing for the temperature of the latest queried cities that a

Duan Rafael Ribeiro 1 Jun 27, 2022
Twitter-bot - A Simple Twitter bot with python

twitterbot To use this bot, You will require API Key and Access Key. Signup at h

Bentil Shadrack 8 Nov 18, 2022