Aiotor - a pool of proxies, shifting on each request

Overview

▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒ ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ░████████╗░██████╗░██████╗░░█████╗░██╗░██████╗░░ ░╚══██╔══╝██╔═══██╗██╔══██╗██╔══██╗██║██╔═══██╗░ ░░░░██║░░░██║░░░██║██████╔╝███████║██║██║░░░██║░ ░░░░██║░░░██║░░░██║██╔══██╗██╔══██║██║██║░░░██║░ ░░░░██║░░░╚██████╔╝██║░░██║██║░░██║██║╚██████╔╝░ ░░░░╚═╝░░░░╚═════╝░╚═╝░░╚═╝╚═╝░░╚═╝╚═╝░╚═════╝░░ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ ▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒

Aiotor - a pool of proxies, shifting on each request

What?

using aiotor you can easily create a small pool of tor proxies so you can take advantage of having multiple circuits at your disposal and multiple ip addresses, while running just 1 process. It also brings a ClientSession, which is the ClientSession from aiohttp with a small modification: it shifts proxies at every request.

Does it work for requests package? and urllib? and plain sockets ?

yes! aitor.Pool() is a context manager which, while in context, patches the low level socket library, so it's usage is easy and compatible for almost all setups/libraries.

Cleaning up

Pool will automatically cleanup the temp folder after your program ends. the folder is needed for tor to store circuit and consensus data. if you specify a folder explicitly, like Pool(user_data_dir="c:\some\folder\tordata"), it is assumed you want a dedicated folder. in that case cleanup will not happen. using a dedicated folder will greatly improve loading time on next runs.

It is blocking!

only once! aiotor should ideally be started on your mainthread. it will block to bootstrap the tor connections, and this only happens once during the full lifecycle of your program. while subprocesses are supported by asyncio, it would render useless as proxies can't be used anyway, until initialization is finished. It also brings a lot of other unexpected behavior.

aiotor.Pool is a true singleton. Every Pool() you create will refer to the same instance. If you need more proxies, just use the amount parameter (defaults to 10, which is more than sufficient, and actually too much).

proxies in the pool are refreshed at least once 15 minutes, getting new circuits from different countries, and new ip's.

Why another library around Onion/Tor?

for most people, setting up tor is already quite a task, implementing them as proxies in their programs requires much more work. the purpose of aiotor is to make it as easy as it can possibly get.

Show me how easy it is!

import logging
import asyncio
import aiotor

async def main():
    async with aiotor.ClientSession() as session:
        for _ in range(10):
            async with session.get('http://httpbin.org/ip') as response:
                print ( await response.json() )


logging.basicConfig(level=10)
asyncio.run(main())

this will output

DEBUG:asyncio:Using proactor: IocpProactor
INFO:pool:starting proxy pool 0x2a214dc9be0
INFO:pool:tor bootstrap 0.0 completed
INFO:pool:tor bootstrap 5.0 completed
INFO:pool:tor bootstrap 10.0 completed
INFO:pool:tor bootstrap 14.0 completed
INFO:pool:tor bootstrap 15.0 completed
INFO:pool:tor bootstrap 20.0 completed
INFO:pool:tor bootstrap 25.0 completed
INFO:pool:tor bootstrap 30.0 completed
INFO:pool:tor bootstrap 40.0 completed
INFO:pool:tor bootstrap 45.0 completed
INFO:pool:tor bootstrap 50.0 completed
INFO:pool:tor bootstrap 55.0 completed
INFO:pool:tor bootstrap 60.0 completed
INFO:pool:tor bootstrap 68.0 completed
INFO:pool:tor bootstrap 75.0 completed
INFO:pool:tor bootstrap 80.0 completed
INFO:pool:tor bootstrap 85.0 completed
INFO:pool:tor bootstrap 89.0 completed
INFO:pool:tor bootstrap 90.0 completed
INFO:pool:tor bootstrap 95.0 completed
INFO:pool:tor bootstrap 100.0 completed

DEBUG:aiotor._client_session:proxy switched
{'origin': '109.70.100.32'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '107.189.31.241'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '89.163.143.8'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '45.153.160.133'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '104.244.75.33'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '199.249.230.187'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '185.220.102.245'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '213.61.215.54'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '193.189.100.199'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '176.10.99.200'}

Lets run this again

asyncio.run(main())
# no initialization is done
# it returns immediately 

DEBUG:aiotor._client_session:proxy switched
{'origin': '109.70.100.32'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '107.189.31.241'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '89.163.143.8'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '45.153.160.133'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '104.244.75.33'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '199.249.230.187'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '185.220.102.245'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '213.61.215.54'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '193.189.100.199'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '176.10.99.200'}

Showing different ways to get a pool and showing how to use the pool as context manager

import aiotor

pool = aiotor.start()   # this gives you a running pool

pool2 = aiotor.Pool()   # this gives you the same pool (because singleton) if already bootstrapped

pool2.start()           # will start and bootstrap the pool, if not already running
                        # start() has the pool itself as return value

pool3 = aiotor.Pool.get_instance()  # same story

pool4 = aiotor.start()   # same story


import requests

# context manager

with pool:       
    # in this block, all network will be handled by the proxies and it will return 
    # different ip's on subsequent calls
    print(requests.get('http://httpbin.org/ip').text)

# not in context
# you will see your own ip
print(requests.get('http://httpbin.org/ip').text)

Owner
Leon
DevOps | Reverse-engineering (mobile) apps | Hacking (certified) | IDA, Frida, RetDec, Ghidra, Simplify | Python, Flutter/Dart, PHP, js , Powershell, html5
Leon
Docker container for demoing Wi-Fi calling stack.

VoWiFiLocalDemo - Docker container that runs StrongSwan and Kamailio to demonstrate how Wi-Fi calling works on smartphones.

18 Nov 12, 2022
Simple app that redirect fixed URL to changing URL, configurable via POST requests

This is a basic URL redirection service. It stores associations between apps and redirection URLs, for apps with changing URLs. You can then use GET r

Maxime Weyl 2 Jan 28, 2022
A repo with study material, exercises, examples, etc for Devnet SPAUTO

MPLS in the SDN Era -- DevNet SPAUTO All of the study notes have now been moved to use auto-generated documentation to build a static site with Githu

Hugo Tinoco 67 Nov 16, 2022
With the use of this tool, you can change your MAC address

Akshat0404/MAC_CHANGER This tool has to be used on linux kernel. Now o

1 Jan 25, 2022
Nautobot is a Network Source of Truth and Network Automation Platform.

Nautobot is a Network Source of Truth and Network Automation Platform. Nautobot was initially developed as a fork of NetBox (v2.10.4). Nautobot runs as a web application atop the Django Python framew

Nautobot 549 Dec 31, 2022
This is a simple python code to get the list of banned IP addresses from Fail2ban

Fail2ban Scripts Usage banned_list.py This script tries to get the banned list of IP addresses by Fail2ban for the service freeswitch. You can modify

Yehor Smoliakov 9 Dec 28, 2022
Building a Robust IOT device which is customizable, encrypted, secure and user friendly

Building a Robust IOT device which is customizable, encrypted, secure and user friendly, which uses a single GPIO pin to extract multiple sensor values

1 Jan 03, 2022
This tools just for education only - Layer-7 or HTTP FLOODER

Layer-7-Flooder This tools just for education only - Layer-7 or HTTP FLOODER Require Col1 Before You Run this tools How To Use Download This Source Ex

NumeX 7 Oct 30, 2022
A Python Tor template on Gitpod

A Python Tor template on Gitpod This is template configured for ephemeral development environments on Gitpod. prebuild Get Started With Your Own Proje

Ivan Yastrebov 1 Dec 17, 2021
Simple DNS resolver for asyncio

Simple DNS resolver for asyncio aiodns provides a simple way for doing asynchronous DNS resolutions using pycares. Example import asyncio import aiodn

Saúl Ibarra Corretgé 471 Dec 27, 2022
syncio: asyncio, without await

syncio: asyncio, without await asyncio can look very intimidating to newcomers, because of the async/await syntax. Even experienced programmers can ge

David Brochart 10 Nov 21, 2022
An automatic web reconnaissance tool written in python3.

WebRecon is an automatic web reconnaissance tool written in python3. Provides a command line interaction similar to msfconsole. The Exasmple.py file is provided, and you can write your own scripts yo

prophet 1 Feb 06, 2022
Ipscanner - A simple threaded IP-Scanner written in python3 that can monitor local IP's in your network

IPScanner 🔬 A simple threaded IP-Scanner written in python3 that can monitor lo

4 Dec 12, 2022
Huawei firewall automatically updates Chinese ip to target IP group.

Huawei firewall automatically updates Chinese ip to target IP group.

Lundaa 0 Jan 11, 2022
libsigrok stacked Protocol Decoder for TPM 2.0 transactions from an SPI bus. BitLocker Volume Master Key (VMK) are automatically extracted.

libsigrok stacked Protocol Decoder for TPM 2.0 transactions from an SPI bus. BitLocker Volume Master Key (VMK) are automatically extracted.

Jordan Ovrè 9 Dec 26, 2022
A Python tool used to automate the execution of the following tools : Nmap , Nikto and Dirsearch but also to automate the report generation during a Web Penetration Testing

📡 WebMap A Python tool used to automate the execution of the following tools : Nmap , Nikto and Dirsearch but also to automate the report generation

Iliass Alami Qammouri 274 Jan 01, 2023
Jogo da forca simples com conexão entre cliente e servidor utilizando TCP.

JogoDaForcaTCP Um jogo da forca simples com conexão entre cliente e servidor utilizando o protocólo TCP. Como jogar: Habilite a porta 20000, inicie o

Kelvin Santos 1 Dec 01, 2021
Nyx-Net: Network Fuzzing with Incremental Snapshots

Nyx-Net: Network Fuzzing with Incremental Snapshots Nyx-Net is fast full-VM snapshot fuzzer for complex network based targets. It's built upon kAFL, R

Chair for Sys­tems Se­cu­ri­ty 146 Dec 16, 2022
A pure python implementation of multicast DNS service discovery

python-zeroconf Documentation. This is fork of pyzeroconf, Multicast DNS Service Discovery for Python, originally by Paul Scott-Murphy (https://github

Jakub Stasiak 483 Dec 29, 2022
Monitoring plugin to check network interfaces with Icinga, Nagios and other compatible monitoring solutions

check_network_interface - Monitor network interfaces This is a monitoring plugin for Icinga, Nagios and other compatible monitoring solutions to check

DinoTools 3 Nov 15, 2022