ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

⚡ python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

A python package for fetching informations from GitHub API

Py-GitHub A python package for fetching informations from GitHub API Made with Python3 (C) @FayasNoushad Copyright permission under MIT License Licens

Fayas Noushad 6 Nov 28, 2021
A Python Script to automate searching of available vaccination centers in the city and hence booking

Cowin Vaccine Availability Notifier Cowin Vaccine Availability Notifier takes your City or PIN code as an input and automatically notifies you via ema

Jayesh Padhiar 7 Sep 05, 2021
Rust UserBot, Telegram istifadəsini asanlaşdıran bir proyektdir.

RUST USERBOT 🇦🇿 Rust UserBot, Telegram istifadəsini asanlaşdıran bir proyektdir. Qurulum Heroku Serverə qurulum git clone https://github.com/rustres

1 Oct 25, 2021
A script that takes what you're listening too on Spotify and sets it as your Nertivia custom status.

nertivia-spotify-listening-status A script that takes what you're listening too on Spotify and sets it as your Nertivia custom status. setup Install r

Ben Tettmar 2 Feb 03, 2022
✖️ Unofficial API of 1337x.to

✖️ Unofficial Python API Wrapper of 1337x This is the unofficial API of 1337x. It supports all proxies of 1337x and almost all functions of 1337x. You

Hemanta Pokharel 71 Dec 26, 2022
📅 Calendar file generator for triathlonlive.tv upcoming events

Triathlon Live Calendar Calendar file generator for triathlonlive.tv upcoming events. Install Requires Python 3.9.4 and Poetry. $ poetry install Runni

Eduardo Cuducos 4 Sep 02, 2022
Python functions for opentargets.org API

What is opy_Targets? Opentargets.org uses GraphQL API to explore it's content via coding. This ensemble of functions aim is to make it easy to use the

1 Jan 10, 2022
Telegram Bot to store Posts and Documents and it can Access by Special Links.

File-sharing-Bot Telegram Bot to store Posts and Documents and it can Access by Special Links. I Guess This Will Be Usefull For Many People..... 😇 .

Code X Botz 1.2k Jan 08, 2023
A Discord Bot created using Pycord!

Hey, I am Slash Bot. A Bot which works with Slash Commands! Prerequisites Python 3+ Check out. the requirements.txt and install all the pakages. Insta

Saumya Patel 1 Nov 29, 2021
Match-making API for OpenSanctions

OpenSanctions Match-making API This directory contains code and a Docker image for running an API to match data against OpenSanctions. It is intended

OpenSanctions.org 26 Dec 15, 2022
Senditapp.com bot spammer, spam your friends

Sendit Spammer Python ⚠️ I am not responsible for how you use this tool. This tool is against "Sendit" ToS and shall not be used in a production envir

Glaukio 1 Dec 31, 2021
Python script to download WAX transactions

WAXtax Python script to download WAX transactions WAXtax uses the CoinGecko API and the WAX Blockchain History API to download csvs for each account y

SixPM Software 11 Oct 09, 2022
Gera um PDF, logo depois de você responder um questionário simples, e envia para o e-mail que você informar.

PDF generator and send it for your email Criador: Francisco Robson de O. Dutra Filho Repositório criado no dia 18/09/2021 Instagram: @robsondutra_ Sob

8 Nov 22, 2021
Rocks vc Userbot: A Telegram Bot Project That's Allow You To Play Audio And Video Music On Telegram Voice Chat Group

⭐️ Rocks VC Userbot ⭐️ Telegram Userbot To Play Audio And Video Song On VC Chat

Dr Asad Ali 10 Jul 18, 2022
A tool written in Python used to instalock agents in VALORANT using the local API.

Valorant Instalock Tool v2.1.0 by Mr. SOSA A tool written in Python used to instalock agents in VALORANT using the local API. This is NOT a hotkey pro

Mr. SOSA 3 Nov 18, 2021
Amanda-A next gen powerful telegram group manager bot for manage your groups and have fun with other cool modules.

Amanda-A next gen powerful telegram group manager bot for manage your groups and have fun with other cool modules.

Team Amanda 4 Oct 21, 2022
A python library for creating selfbots/automating your Nertivia account.

nertivia-selfbot (WIP) A python library for creating selfbots/automating your Nertivia account. how to use Download the nertivia_selfbot folder from t

Ben Tettmar 2 Feb 03, 2022
A Telegram mirror bot which can be deployed using Heroku.

Slam Mirror Bot This is a telegram bot writen in python for mirroring files on the internet to our beloved Google Drive. Getting Google OAuth API cred

Hafitz Setya 1.2k Jan 01, 2023
This repository contains modules that extend / modify parts of Odoo ERP

Odoo Custom Addons This repository contains addons that extend / modify parts of Odoo ERP. Addons list account_cancel_permission Only shows the button

Daniel Luque 3 Dec 28, 2022
Easily update resume to naukri with one click

NAUKRI RESUME AUTO UPDATER I am using poetry for dependencies. you can check or change in data.txt file for username and password Resume file must be

Rahul.p 1 May 02, 2022