Aiotor - a pool of proxies, shifting on each request

Overview

▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒ ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ░████████╗░██████╗░██████╗░░█████╗░██╗░██████╗░░ ░╚══██╔══╝██╔═══██╗██╔══██╗██╔══██╗██║██╔═══██╗░ ░░░░██║░░░██║░░░██║██████╔╝███████║██║██║░░░██║░ ░░░░██║░░░██║░░░██║██╔══██╗██╔══██║██║██║░░░██║░ ░░░░██║░░░╚██████╔╝██║░░██║██║░░██║██║╚██████╔╝░ ░░░░╚═╝░░░░╚═════╝░╚═╝░░╚═╝╚═╝░░╚═╝╚═╝░╚═════╝░░ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ ▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒▓▒

Aiotor - a pool of proxies, shifting on each request

What?

using aiotor you can easily create a small pool of tor proxies so you can take advantage of having multiple circuits at your disposal and multiple ip addresses, while running just 1 process. It also brings a ClientSession, which is the ClientSession from aiohttp with a small modification: it shifts proxies at every request.

Does it work for requests package? and urllib? and plain sockets ?

yes! aitor.Pool() is a context manager which, while in context, patches the low level socket library, so it's usage is easy and compatible for almost all setups/libraries.

Cleaning up

Pool will automatically cleanup the temp folder after your program ends. the folder is needed for tor to store circuit and consensus data. if you specify a folder explicitly, like Pool(user_data_dir="c:\some\folder\tordata"), it is assumed you want a dedicated folder. in that case cleanup will not happen. using a dedicated folder will greatly improve loading time on next runs.

It is blocking!

only once! aiotor should ideally be started on your mainthread. it will block to bootstrap the tor connections, and this only happens once during the full lifecycle of your program. while subprocesses are supported by asyncio, it would render useless as proxies can't be used anyway, until initialization is finished. It also brings a lot of other unexpected behavior.

aiotor.Pool is a true singleton. Every Pool() you create will refer to the same instance. If you need more proxies, just use the amount parameter (defaults to 10, which is more than sufficient, and actually too much).

proxies in the pool are refreshed at least once 15 minutes, getting new circuits from different countries, and new ip's.

Why another library around Onion/Tor?

for most people, setting up tor is already quite a task, implementing them as proxies in their programs requires much more work. the purpose of aiotor is to make it as easy as it can possibly get.

Show me how easy it is!

import logging
import asyncio
import aiotor

async def main():
    async with aiotor.ClientSession() as session:
        for _ in range(10):
            async with session.get('http://httpbin.org/ip') as response:
                print ( await response.json() )


logging.basicConfig(level=10)
asyncio.run(main())

this will output

DEBUG:asyncio:Using proactor: IocpProactor
INFO:pool:starting proxy pool 0x2a214dc9be0
INFO:pool:tor bootstrap 0.0 completed
INFO:pool:tor bootstrap 5.0 completed
INFO:pool:tor bootstrap 10.0 completed
INFO:pool:tor bootstrap 14.0 completed
INFO:pool:tor bootstrap 15.0 completed
INFO:pool:tor bootstrap 20.0 completed
INFO:pool:tor bootstrap 25.0 completed
INFO:pool:tor bootstrap 30.0 completed
INFO:pool:tor bootstrap 40.0 completed
INFO:pool:tor bootstrap 45.0 completed
INFO:pool:tor bootstrap 50.0 completed
INFO:pool:tor bootstrap 55.0 completed
INFO:pool:tor bootstrap 60.0 completed
INFO:pool:tor bootstrap 68.0 completed
INFO:pool:tor bootstrap 75.0 completed
INFO:pool:tor bootstrap 80.0 completed
INFO:pool:tor bootstrap 85.0 completed
INFO:pool:tor bootstrap 89.0 completed
INFO:pool:tor bootstrap 90.0 completed
INFO:pool:tor bootstrap 95.0 completed
INFO:pool:tor bootstrap 100.0 completed

DEBUG:aiotor._client_session:proxy switched
{'origin': '109.70.100.32'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '107.189.31.241'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '89.163.143.8'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '45.153.160.133'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '104.244.75.33'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '199.249.230.187'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '185.220.102.245'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '213.61.215.54'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '193.189.100.199'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '176.10.99.200'}

Lets run this again

asyncio.run(main())
# no initialization is done
# it returns immediately 

DEBUG:aiotor._client_session:proxy switched
{'origin': '109.70.100.32'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '107.189.31.241'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '89.163.143.8'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '45.153.160.133'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '104.244.75.33'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '199.249.230.187'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '185.220.102.245'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '213.61.215.54'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '193.189.100.199'}
DEBUG:aiotor._client_session:proxy switched
{'origin': '176.10.99.200'}

Showing different ways to get a pool and showing how to use the pool as context manager

import aiotor

pool = aiotor.start()   # this gives you a running pool

pool2 = aiotor.Pool()   # this gives you the same pool (because singleton) if already bootstrapped

pool2.start()           # will start and bootstrap the pool, if not already running
                        # start() has the pool itself as return value

pool3 = aiotor.Pool.get_instance()  # same story

pool4 = aiotor.start()   # same story


import requests

# context manager

with pool:       
    # in this block, all network will be handled by the proxies and it will return 
    # different ip's on subsequent calls
    print(requests.get('http://httpbin.org/ip').text)

# not in context
# you will see your own ip
print(requests.get('http://httpbin.org/ip').text)

Owner
Leon
DevOps | Reverse-engineering (mobile) apps | Hacking (certified) | IDA, Frida, RetDec, Ghidra, Simplify | Python, Flutter/Dart, PHP, js , Powershell, html5
Leon
Web-server with a parser, connection to DBMS, and the Hugging Face.

Final_Project Web-server with parser, connection to DBMS and the Hugging Face. Team: Aisha Bazylzhanova(SE-2004), Arysbay Dastan(SE-2004) Installation

Aisha Bazylzhanova 2 Nov 18, 2021
Simple client for the Sirah Matisse Commander TCP server.

Simple client for the Sirah Matisse Commander TCP server.

Nelson Darkwah Oppong 1 Nov 02, 2021
ExtDNS synchronizes labeled records in docker-compose with DNS providers.

ExtDNS for docker-compose ExtDNS synchronizes labeled records in docker-compose with DNS providers. Inspired by External DNS, ExtDNS makes resources d

DNTSK 6 Dec 24, 2022
Bittensor - an open, decentralized, peer-to-peer network that functions as a market system for the development of artificial intelligence

At Bittensor, we are creating an open, decentralized, peer-to-peer network that functions as a market system for the development of artificial intelligence.

Opentensor 169 Dec 30, 2022
TradingView Interactive Brokers Integration using Webhooks

TradingView Interactive Brokers Integration using Webhooks

84 Dec 19, 2022
Use Fast Redirect to easily redirect your domains.

Fast Redirect Use Fast Redirect to easily redirect your domains. Fast Redirects expects a JSON 'database'. This JSON 'database' contains the domains t

Cyberfusion 1 Dec 20, 2021
High capacity, high availability, well connected, fast lightning node.

LND ⚡ Routing High capacity, high availability, well connected, fast lightning node. We aim to become a top liquidity provider for the lightning netwo

18 Dec 16, 2022
The module that allows the collection of data sampling, which is transmitted with WebSocket via WIFI or serial port for CSV file.

The module that allows the collection of data sampling, which is transmitted with WebSocket via WIFI or serial port for CSV file.

Nelson Wenner 2 Apr 01, 2022
Easy to use gRPC-web client in python

pyease-grpc Easy to use gRPC-web client in python Tutorial This package provides a requests like interface to make calls to gRPC-Web servers.

Sudipto Chandra 4 Dec 03, 2022
Linkedin Connection Automation

Why spend an hour+ a week, connecting with the correct people on LinkedIn when you can go for lunch and let your computer do the hard work?

1 Nov 29, 2021
Whoisss is a website information gatharing Tool.

Whoisss Whoisss is a website information gatharing Tool. You can cse it to collect information about website. Usage apt-get update apt-get upgrade pkg

Md. Nur habib 2 Jan 23, 2022
Python Scrcpy Client - allows you to view and control android device in realtime

Python Scrcpy Client This package allows you to view and control android device in realtime. Note: This gif is compressed and experience lower quality

LengYue 126 Jan 02, 2023
PySocks lets you send traffic through SOCKS proxy servers.

PySocks lets you send traffic through SOCKS proxy servers. It is a modern fork of SocksiPy with bug fixes and extra features. Acts as a drop-i

1.1k Dec 07, 2022
This tool will scans your wi-fi/wlan and show you the connected clients

This tool will scans your wi-fi/wlan and show you the connected clients

VENKAT SAI SAGAR 3 Mar 24, 2022
A p2p chat app for zephyr

A p2p chat app for zephyr

L3gacy B3ta 4 Jun 02, 2021
It's a little project for change MAC address, for ethical hacking purposes

MACChangerPy It's a small project for MAC address change, for ethical hacking purposes, don't use it for bad purposes, any infringement will be your r

Erick Adriano Nunes da Silva 1 Mar 11, 2022
A Project to resolve hostname and receive IP

hostname-resolver A Project to resolve hostname and receive IP Installation git clone https://github.com/ihapiw/hostname-resolver.git Head into the ho

iHapiW 5 Sep 12, 2022
Interact remotely with the computer using Python and MQTT protocol 💻

Comandos_Remotos Interagir remotamento com o computador através do Python e protocolo MQTT. 💻 Status: em desenvolvimento 🚦 Objetivo: Interagir com o

Guilherme_Donizetti 6 May 10, 2022
InfraGenie is allows you to split out your infrastructure project into separate independent pieces, each with its own terraform state.

🧞 InfraGenie InfraGenie is allows you to split out your infrastructure project into separate independent pieces, each with its own terraform state. T

Digger 53 Nov 23, 2022
Build custom OSINT tools and APIs (Ping, Traceroute, Scans, Archives, DNS, Scrape, Whois, Metadata & built-in database for more info) with this python package

Build custom OSINT tools and APIs with this python package - It includes different OSINT modules (Ping, Traceroute, Scans, Archives, DNS, Scrape, Whoi

QeeqBox 52 Jan 06, 2023