ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

โšก python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No โˆž

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

๐Ÿ›’ Bot de lista de compras compartilhada para o Telegram

Lista de Compras Lista de compras de Cuducos e Flรกvia. Comandos do bot Comando Descriรงรฃo /add item Adiciona item ร  lista de compras /remove item

Eduardo Cuducos 4 Jan 15, 2022
Discord bot written in discord.py

Orion Discord bot written in discord.py Installation Installation of code is supported for macOS only currently First open the terminal. If incase you

Zeus 3 May 19, 2022
The best (and now open source) Discord selfbot.

React Selfbot Yes, for real Why am I making this open source? Because can't stop calling my product a rat, tokenlogger and what else not. But there is

30 Nov 13, 2022
๐Ÿ™ Share your Github stats for 2020 on Twitter

Year on Github ๐Ÿ™ Share your Github stats for 2020 on Twitter. This project contains a small web app that let's you share stats about your Github acti

Johannes Rieke 129 Dec 25, 2022
A python client for the Software-Challenge Germany.

sc-client-python A python client for the Software-Challenge Germany. Creating a new project (Optional) Install virtualenv virtualenv is a tool that cr

rpkak 3 Jan 22, 2022
GUI Pancakeswap2 and Uniswap3 trading client (and bot)

GUI Pancakeswap2 and Uniswap3 trading client (and bot) (MOST ADVANCE TRADING BOT SUPPORT WINDOWS LINUX MAC) (AUTO BUY TOKEN ON LUNCH AFTER ADD LIQUIDI

16 Dec 23, 2021
A simple telegram bot that takes a list of files sent by the user and returns them 7zipped

A simple telegram bot that takes a list of files sent by the user and returns them 7zipped

1 Oct 28, 2022
Translator based on Google API

Yakusu Toshiko Translator based on Google API. Instance of this bot is running as @yakusubot. Features Add a plus to a language's name to show an orig

Arisu W. 2 Sep 21, 2022
Freqtrade 3commas wrapper for python

Freqtrade 3commas wrapper The aim of this project is to provide an easy way to integrate freqtrade with 3commas. The main reason someone would want to

Alex Babescu 43 Dec 17, 2022
๐Ÿ”Ž Hunt down social media accounts by username across social networks

Hunt down social media accounts by username across social networks Installation | Usage | Docker Notes | Contributing Installation # clone the repo $

Sherlock 38.2k Jan 01, 2023
a Disqus alternative

Isso โ€“ a commenting server similar to Disqus Isso โ€“ Ich schrei sonst โ€“ is a lightweight commenting server written in Python and JavaScript. It aims to

Martin Zimmermann 4.7k Jan 02, 2023
An async-ready Python wrapper around FerrisChat's API.

FerrisWheel An async-ready Python wrapper around FerrisChat's API. Installation Instructions Linux: $ python3.9 -m pip install -U ferriswheel Python 3

FerrisChat 8 Feb 08, 2022
Telegram PHub Bot using ARQ Api and Pyrogram. This Bot can Download and Send PHub HQ videos in Telegram using ARQ API.

Tg_PHub_Bot Telegram PHub Bot using ARQ Api and Pyrogram. This Bot can Download and Send PHub HQ videos in Telegram using ARQ API. OS Support All linu

TheProgrammerCat 13 Oct 21, 2022
A Python script that wraps the gitleaks tool to enable scanning of multiple repositories in parallel

mpgitleaks A Python script that wraps the gitleaks tool to enable scanning of multiple repositories in parallel. The motivation behind writing this sc

Emilio Reyes 7 Dec 29, 2022
Generate direct m3u playlist for all the channels subscribed in the Tata Sky portal

Tata Sky IPTV Script generator A script to generate the m3u playlist containing direct streamable file (.mpd or MPEG-DASH or DASH) based on the channe

Gaurav Thakkar 250 Jan 01, 2023
This python cheat utilizes PyMeow, PyMem, and others to enhance your CS:GO experience ;).

CSGO-Python-Cheat This python cheat utilizes PyMeow, PyMem, and others to enhance your CS:GO experience ;). Features Esp Tracers Chams (More to come)

Addi 1 Nov 30, 2021
A python telegram bot to fetch the details of an ipadress with help of ip-api

ipfetcher A python(Pyrogram) oriented telegram bot to fetch the details of an ipadress developed by @riz4d with the API of https://ip-api.com Deployme

Mohamed Rizad 5 Mar 12, 2022
Building and deploying AWS Lambda Shared Layers

AWS Lambda Shared Layers This repository is hosting the code from the following blog post: AWS Lambda & Shared layers for Python. The goal of this rep

Flaneer 7 Dec 02, 2021
A python based Telegram Bot for Compressing Videos with negligible Quality change

๐•๐•š๐••๐•–๐•  โ„‚๐•†๐•„โ„™โ„๐”ผ๐•Š๐•Š๐•†โ„ ๐”น๐•†๐•‹ แดแดœสŸแด›ษชา“แดœษดแด„แด›ษชแดษด วซแดœแด€สŸษชแด›ส แด„แดแดแด˜ส€แด‡ssแดส€ A Telegram Video CompressorBot it compress videos with negligible Quality change.

Danish 154 Dec 04, 2022
Simple Discord Nuke Bot.

Discord-Nuke-Bot Simple Discord Nuke Bot. Simple Discord Nuke Bot Python 3.6 - 3.8 Features Delete Channels Ban All Members Delete Roles Create Channe

9X4N 6 Aug 16, 2022