ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

⚡ python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

Bot-moderator for Telegram group chats

Project title A little info about your project and/ or overview that explains what the project is about. 🌟 Hello everyone! This is the repository of

Maxim Zavalniuk 6 Nov 01, 2022
Karen is a Discord Bot that will check for a list of forbidden words/expressions, removing the message that contains them and replying with another message.

Karen is a Discord Bot that will check for a list of forbidden words/expressions, removing the message that contains them and replying with another message. Everything is highly customizable.

Rafael Almeida 1 Nov 03, 2021
A Telegram Bot That Provides Permanent Download Links For Sent Files.

FileStreamBot A Telegram bot to all media and documents files to web link . Report a Bug | Request Feature Demo Bot: 🍁 About This Bot : This bot will

Flux Inc. 1 Nov 02, 2021
Make your Pass (sanitaire) Fun Again

Make Your Pass (Sanitaire) Fun Again Customize your Pass Sanitaire (French name for EU digital Covide19 Certificate) with colors and images, thus make

101 Dec 01, 2022
Detects members having unicode names. Public bot: @scarletwitchprobot

✨ Scarletwitch bot ✨ Detects unicode names members in a tg chat & provides a option to take action on that user ! Public bot: @scarletwitchprobot Supp

ÁÑÑÍHÌLÅTØR SPÄRK 18 Nov 12, 2022
Documentation and Samples for the Official HN API

Hacker News API Overview In partnership with Firebase, we're making the public Hacker News data available in near real time. Firebase enables easy acc

Y Combinator Hacker News 9.6k Jan 03, 2023
Discord raid tool!

GANG Multi Tool Menu: -- YOUTUBE TUTORIAL! Features: Most Advanced Multi Tool! Spammer DM Spammer Friend Spammer Reaction Spam WebhookSpammer Typing

1 Feb 13, 2022
Template to create a telegram bot in python

Template for Telegram Bot Template to create a telegram bot in python. How to Run First add src to PYTHONPATH: export PYTHONPATH=${PWD} Then run: pyt

Ali Hejazizo 12 Dec 24, 2022
Bot playing "mathbattle" game from Telegram messenger

mathbattlebot Bot playing mathbattle game from Telegram messenger Installing: run in command line pip3 install -r requirements.txt Running: Example c

Egor 1 May 30, 2022
Ethereum transactions and wallet information for people you follow on Twitter.

ethFollowing Ethereum transactions and wallet information for people you follow on Twitter. Set up Setup python environment (requires python 3.8): vir

Brian Donohue 2 Dec 28, 2021
This software's intent is to automate all activities related to manage Axie Infinity Scholars. It is specially aimed to mangers with large scholar roasters.

Axie Scholars Utilities This software's intent is to automate all activities related to manage Scholars. It is specially aimed to mangers with large s

Ferran Marin 153 Nov 16, 2022
A simple python script for rclone. Use multiple Google Service Accounts and cycle through them.

About GSAclone GSAclone is a simple python script for rclone, written with the purpose of using multiple Google service accounts on Google Drive and "

Shiro39 6 Feb 25, 2022
Manage Proxmox KVM Virtual Machines via Slack bot.

proxmox-slack-bot Create KVM Virtual Machines on Proxmox, the easy way. Not much works works here yet... Setup dev environment Setup fully editable st

Plenus Pyramis 3 Mar 20, 2022
Tweet stream in OBS browser source

Tweetron TweetronはOBSブラウザーソースを使用してツイートを画面上に表示するツールソフトです Windowsのみ対応 (Windows10動作確認済) ダウンロード こちらから最新版をダウンロードしてください (現在ベータテスト版を配布しています) Download ver0.0.

Cube 0 Apr 05, 2022
Asca - Antiscam Discord Bot With Python

asca Antiscam Discord Bot Asca moderates scammers and deletes scam messages Opti

11 Nov 01, 2022
This Lambda will Pull propagated routes from TGW and update VPC route table

AWS-Transitgateway-Route-Propagation This Lambda will Pull propagated routes from TGW and update VPC route table. Tested on python 3.8 Lambda AWS INST

4 Jan 20, 2022
MashaRobot : New Generation Telegram Group Manager Bot (🔸Fast 🔸Python🔸Pyrogram 🔸Telethon 🔸Mongo db )

MashaRobot Me On Telegram ✨ MASHA ✨ This is just a demo bot.. Don't try to add to your group.. Create your own bot How To Host The easiest way to depl

Mr Dark Prince 40 Oct 09, 2022
Credit Card And SK Checker Written In Python

Credit Card And SK Checker Written In Python

Rimuru Tempest 57 Jan 08, 2023
A part of HyRiver software stack for accessing hydrology data through web services

Package Description Status PyNHD Navigate and subset NHDPlus (MR and HR) using web services Py3DEP Access topographic data through National Map's 3DEP

Taher Chegini 51 Dec 10, 2022
A pdisk uploader bot written in Python

Pdisk Uploader Bot 🔥 Upload on Pdisk by Url, File and also by direct forward post from other channel... Features Post to Post Conversion Url Upload D

Paritosh Kumar 33 Oct 21, 2022