Divide full port scan results and use it for targeted Nmap runs

Overview

DivideAndScan

Divide Et Impera And Scan (and also merge the scan results)

version python codacy pypi docker twitter


DivideAndScan is used to efficiently automate port scanning routine by splitting it into 3 phases:

  1. Discover open ports for a bunch of targets.
  2. Run Nmap individually for each target with version grabbing and NSE actions.
  3. Merge the results into a single Nmap report (different formats available).

For the 1st phase a fast port scanner is intended to be used (Masscan / RustScan), whose output is parsed and stored in a database (TinyDB). Next, during the 2nd phase individual Nmap scans are launched for each target with its set of open ports (multiprocessing is supported) according to the database data. Finally, in the 3rd phase separate Nmap outputs are merged into a single report in different formats (XML / HTML / Simple text / Grepable) with nMap_Merger.

Potential use cases:

  • Pentest engagements / red teaming with a large scope to enumerate.
  • Cybersecurity wargames / training CTF labs.
  • OSCP certification exam.

How It Works

how-it-works.png

How to Install

Prerequisites

To successfully divide and scan we need to get some good port scanning tools.

📑 Note: if you don't feel like messing with dependecies on your host OS, skip to the Docker part.

Nmap

sudo apt install nmap sudo xsltproc -y
sudo nmap --script-updatedb

Masscan

cd /tmp
git clone https://github.com/robertdavidgraham/masscan.git
cd masscan
make
sudo make install
cd && rm -rf /tmp/masscan

RustScan

cd /tmp

wget -qO- https://api.github.com/repos/RustScan/RustScan/releases/latest \
| grep "browser_download_url.*amd64.deb" \
| cut -d: -f2,3 \
| tr -d \" \
| wget -qO rustscan.deb -i-

sudo dpkg -i rustscan.deb
cd && rm /tmp/rustscan.deb

sudo wget https://gist.github.com/snovvcrash/c7f8223cc27154555496a9cbb4650681/raw/a76a2c658370d8b823a8a38a860e4d88051b417e/rustscan-ports-top1000.toml -O /root/.rustscan.toml

Installation

DivideAndScan is available on PyPI as divideandscan, though I recommend installing it from GitHub with pipx in order to always have the bleeding-edge version:

~$ pipx install -f "git+https://github.com/snovvcrash/DivideAndScan.git"
~$ das

For debbugging purposes you can set up a dev environment with poetry:

~$ git clone https://github.com/snovvcrash/DivideAndScan
~$ cd DivideAndScan
~$ poetry install
~$ poetry run das

📑 Note: DivideAndScan uses sudo to run all the port scanners, so it will ask for the password when scanning commands are invoked.

Using from Docker

You can run DivideAndScan in a Docker container as follows:

~$ docker run -it --rm --name das -v `pwd`:/app snovvcrash/divideandscan

Since the tool requires some input data and produces some output data, you should specify your current working directory as the mount point at /app within the container. You may want to set an alias to make the base command shorter:

~$ alias das='docker run -it --rm --name das -v `pwd`:/app snovvcrash/divideandscan'
~$ das

How to Use

how-to-use.png

More examples coming soon at snovvcrash.rocks...

0. Preparations

Make a new directory to start DivideAndScan from. The tool will create subdirectories to store the output, so I recommend launching it from a clean directory to stay organized:

~$ mkdir divideandscan
~$ cd divideandscan

1. Filling the DB

Provide the add module a command for either Masscan or RustScan to discover open ports in a desired range.

⚠️ Warning: please, make sure that you understand what you're doing, because nearly all port scanning tools can damage the system being tested if used improperly.

# Masscan example
~$ das add -db testdb masscan '--rate 1000 -iL hosts.txt --open -p1-65535'
# RustScan example
~$ das add -db testdb rustscan '-b 1000 -t 2000 -u 5000 -a hosts.txt -r 1-65535 -g --no-config'

When the module completes its work, a hidden directory .db is created in CWD containig the database file and raw scan results.

2. Targeted Scanning

Launch targeted Nmap scans with the scan module. You can adjust the scan surface with either -hosts or -ports option:

# Scan by hosts
~$ das scan -db testdb -hosts all -oA report1
~$ das scan -db testdb -hosts 192.168.1.0/24,10.10.13.37 -oA report1
~$ das scan -db testdb -hosts hosts.txt -oA report1
# Scan by ports
~$ das scan -db testdb -ports all -oA report2
~$ das scan -db testdb -ports 22,80,443,445 -oA report2
~$ das scan -db testdb -ports ports.txt -oA report2

To start Nmap simultaneously in multiple processes, specify the -parallel switch and set number of workers with the -proc option (if no value is provided, it will default to the number of processors on the machine):

~$ das scan -db testdb -hosts all -oA report -parallel [-proc 4]

The output format is selected with -oX, -oN, -oG and -oA options for XML+HTML formats, simple text format, grepable format and all formats respectively. When the module completes its work, a hidden directory .nmap is created in CWD containig Nmap raw scan reports.

Also, you can inspect the contents of the database with -show option before actually launching the scans:

~$ das scan -db testdb -hosts all -show

3 (Optional). Merging the Reports

In order to generate a report independently of the scan module, you should use the report module. It will search for Nmap raw scan reports in the .nmap directory and process and merge them based on either -hosts or -ports option:

# Merge outputs by hosts
~$ das report -db testdb -hosts all -oA report1
~$ das report -db testdb -hosts 192.168.1.0/24,10.10.13.37 -oA report1
~$ das report -db testdb -hosts hosts.txt -oA report1
# Merge outputs by ports
~$ das report -db testdb -ports all -oA report2
~$ das report -db testdb -ports 22,80,443,445 -oA report2
~$ das report -db testdb -ports ports.txt -oA report2

📑 Note: keep in mind that the report module does not search the DB when processing the -hosts or -ports options, but looks for Nmap raw reports directly in .nmap directory instead; it means that -hosts 127.0.0.1 argument value will be successfully resolved only if .nmap/127-0-0-1.* file exists, and -ports 80 argument value will be successfully resolved only if .nmap/port80.* file exists.

Help

usage: das [-h] {add,scan,report} ...

 -----------------------------------------------------------------------------------------------
|  ________  .__      .__    .___        _____              .____________                       |
|  \______ \ |__|__  _|__| __| _/____   /  _  \   ____    __| _/   _____/ ____ _____    ____    |
|   |    |  \|  \  \/ /  |/ __ |/ __ \ /  /_\  \ /    \  / __ |\_____  \_/ ___\\__  \  /    \   |
|   |    `   \  |\   /|  / /_/ \  ___//    |    \   |  \/ /_/ |/        \  \___ / __ \|   |  \  |
|  /_______  /__| \_/ |__\____ |\___  >____|__  /___|  /\____ /_______  /\___  >____  /___|  /  |
|          \/                 \/    \/        \/     \/      \/       \/     \/     \/     \/   |
|  {@snovvcrash}            {https://github.com/snovvcrash/DivideAndScan}             {vX.Y.Z}  |
 -----------------------------------------------------------------------------------------------

positional arguments:
  {add,scan,report}
    add              run a full port scan {masscan,rustscan} and add the output to DB
    scan             run targeted Nmap scans against hosts and ports from DB
    report           merge separate Nmap outputs into a single report in different formats

optional arguments:
  -h, --help         show this help message and exit

Psst, hey buddy... Wanna do some organized p0r7 5c4nn1n6?

ToDo

Support

If this tool has been useful for you, feel free to buy me a beer coffee!

beer.png

You might also like...
It helps to use fixtures in pytest.mark.parametrize

pytest-lazy-fixture Use your fixtures in @pytest.mark.parametrize. Installation pip install pytest-lazy-fixture Usage import pytest @pytest.fixture(p

An AWS Pentesting tool that lets you use one-liner commands to backdoor an AWS account's resources with a rogue AWS account - or share the resources with the entire internet 😈
An AWS Pentesting tool that lets you use one-liner commands to backdoor an AWS account's resources with a rogue AWS account - or share the resources with the entire internet 😈

An AWS Pentesting tool that lets you use one-liner commands to backdoor an AWS account's resources with a rogue AWS account - or share the resources with the entire internet 😈

This file will contain a series of Python functions that use the Selenium library to search for elements in a web page while logging everything into a file

element_search with Selenium (Now With docstrings 😎 ) Just to mention, I'm a beginner to all this, so it it's very possible to make some mistakes The

WIP SAT benchmarking tooling, written with only my personal use in mind.

SAT Benchmarking Some early work in progress tooling for running benchmarks and keeping track of the results when working on SAT solvers and related t

A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax

1.0 Release See here for details about breaking changes with the upcoming 1.0 release: https://github.com/taverntesting/tavern/issues/495 Easier API t

Turn any OpenAPI2/3 and Postman Collection file into an API server with mocking, transformations and validations.

Prism is a set of packages for API mocking and contract testing with OpenAPI v2 (formerly known as Swagger) and OpenAPI v3.x. Mock Servers: Life-like

To automate the generation and validation tests of COSE/CBOR Codes and it's base45/2D Code representations

To automate the generation and validation tests of COSE/CBOR Codes and it's base45/2D Code representations, a lot of data has to be collected to ensure the variance of the tests. This respository was established to collect a lot of different test data and related test cases of different member states in a standardized manner. Each member state can generate a folder in this section.

Test django schema and data migrations, including migrations' order and best practices.

django-test-migrations Features Allows to test django schema and data migrations Allows to test both forward and rollback migrations Allows to test th

An Instagram bot that can mass text users, receive and read a text, and store it somewhere with user details.
An Instagram bot that can mass text users, receive and read a text, and store it somewhere with user details.

Instagram Bot 🤖 July 14, 2021 Overview 👍 A multifunctionality automated instagram bot that can mass text users, receive and read a message and store

Comments
  • Runtime error FileNotFoundError: [Errno 2] No such file or directory: '/root/.das/db/main.json'

    Runtime error FileNotFoundError: [Errno 2] No such file or directory: '/root/.das/db/main.json'

    I just cloned the repo and run the following commands :

    /tmp/DivideAndScan# alias das='docker run -it --rm --name das -v ~/.das:/root/.das -v pwd:/app divideandscan:latest'

    /tmp/DivideAndScan# das scan -hosts 185.165.116.100-185.165.116.200 -oA report1

    and here is the error that I faced :

    Traceback (most recent call last): File "/usr/local/bin/das", line 8, in sys.exit(main()) File "/usr/local/lib/python3.9/dist-packages/das/divideandscan.py", line 158, in main sr = ScanRun(str(P), args.hosts, args.ports) File "/usr/local/lib/python3.9/dist-packages/das/scan.py", line 35, in init self.db = TinyDB(db_path) File "/usr/local/lib/python3.9/dist-packages/tinydb/database.py", line 94, in init self._storage: Storage = storage(*args, **kwargs) File "/usr/local/lib/python3.9/dist-packages/tinydb/storages.py", line 102, in init touch(path, create_dirs=create_dirs) File "/usr/local/lib/python3.9/dist-packages/tinydb/storages.py", line 31, in touch with open(path, 'a'): FileNotFoundError: [Errno 2] No such file or directory: '/root/.das/db/main.json'

    opened by neginsadeghi 1
  • Bump cryptography from 38.0.2 to 38.0.3

    Bump cryptography from 38.0.2 to 38.0.3

    Bumps cryptography from 38.0.2 to 38.0.3.

    Changelog

    Sourced from cryptography's changelog.

    38.0.3 - 2022-11-01

    
    * Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.0.7,
      which resolves *CVE-2022-3602* and *CVE-2022-3786*.
    

    .. _v38-0-2:

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump urllib3 from 1.26.4 to 1.26.5

    Bump urllib3 from 1.26.4 to 1.26.5

    Bumps urllib3 from 1.26.4 to 1.26.5.

    Release notes

    Sourced from urllib3's releases.

    1.26.5

    :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

    • Fixed deprecation warnings emitted in Python 3.10.
    • Updated vendored six library to 1.16.0.
    • Improved performance of URL parser when splitting the authority component.

    If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

    Changelog

    Sourced from urllib3's changelog.

    1.26.5 (2021-05-26)

    • Fixed deprecation warnings emitted in Python 3.10.
    • Updated vendored six library to 1.16.0.
    • Improved performance of URL parser when splitting the authority component.
    Commits
    • d161647 Release 1.26.5
    • 2d4a3fe Improve performance of sub-authority splitting in URL
    • 2698537 Update vendored six to 1.16.0
    • 07bed79 Fix deprecation warnings for Python 3.10 ssl module
    • d725a9b Add Python 3.10 to GitHub Actions
    • 339ad34 Use pytest==6.2.4 on Python 3.10+
    • f271c9c Apply latest Black formatting
    • 1884878 [1.26] Properly proxy EOF on the SSLTransport test suite
    • See full diff in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Releases(v0.2.0)
Owner
snovvcrash
Penetration tester
snovvcrash
A wrapper for webdriver that is a jumping off point for web automation.

Webdriver Automation Plus ===================================== Description: Tests the user can save messages then find them in search and Saved items

1 Nov 08, 2021
Enabling easy statistical significance testing for deep neural networks.

deep-significance: Easy and Better Significance Testing for Deep Neural Networks Contents ⁉️ Why 📥 Installation 🔖 Examples Intermezzo: Almost Stocha

Dennis Ulmer 270 Dec 20, 2022
Useful additions to Django's default TestCase

django-test-plus Useful additions to Django's default TestCase from REVSYS Rationale Let's face it, writing tests isn't always fun. Part of the reason

REVSYS 546 Dec 22, 2022
Repository for JIDA SNP Browser Web Application: Local Deployment

JIDA JIDA is a web application that retrieves SNP information for a genomic region of interest in Homo sapiens and calculates specific summary statist

3 Mar 03, 2022
buX Course Enrollment Automation

buX automation BRACU - buX course enrollment automation Features: Automatically enroll into multiple courses at a time. Find courses just entering cou

Mohammad Shakib 1 Oct 06, 2022
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

Leon 3.5k Dec 30, 2022
Given some test cases, this program automatically queries the oracle and tests your Cshanty compiler!

The Diviner A complement to The Oracle for compilers class. Given some test cases, this program automatically queries the oracle and tests your compil

Grant Holmes 2 Jan 29, 2022
A set of pytest fixtures to test Flask applications

pytest-flask An extension of pytest test runner which provides a set of useful tools to simplify testing and development of the Flask extensions and a

pytest-dev 433 Dec 23, 2022
A framework-agnostic library for testing ASGI web applications

async-asgi-testclient Async ASGI TestClient is a library for testing web applications that implements ASGI specification (version 2 and 3). The motiva

122 Nov 22, 2022
Based on the selenium automatic test framework of python, the program crawls the score information of the educational administration system of a unive

whpu_spider 该程序基于python的selenium自动化测试框架,对某高校的教务系统的成绩信息实时爬取,在检测到成绩更新之后,会通过电子邮件的方式,将更新的成绩以文本的方式发送给用户,可以使得用户在不必手动登录教务系统网站时,实时获取成绩更新的信息。 该程序仅供学习交流,不可用于恶意攻

1 Dec 30, 2021
Django-google-optimize is a Django application designed to make running server side Google Optimize A/B tests easy.

Django-google-optimize Django-google-optimize is a Django application designed to make running Google Optimize A/B tests easy. Here is a tutorial on t

Adin Hodovic 39 Oct 25, 2022
RAT-el is an open source penetration test tool that allows you to take control of a windows machine.

To prevent RATel from being detected by antivirus, please do not upload the payload to TOTAL VIRUS. Each month I will test myself if the payload gets detected by antivirus. So you’ll have a photo eve

218 Dec 16, 2022
pytest splinter and selenium integration for anyone interested in browser interaction in tests

Splinter plugin for the pytest runner Install pytest-splinter pip install pytest-splinter Features The plugin provides a set of fixtures to use splin

pytest-dev 238 Nov 14, 2022
A simple asynchronous TCP/IP Connect Port Scanner in Python 3

Python 3 Asynchronous TCP/IP Connect Port Scanner A simple pure-Python TCP Connect port scanner. This application leverages the use of Python's Standa

70 Jan 03, 2023
A library to make concurrent selenium tests that automatically download and setup webdrivers

AutoParaSelenium A library to make parallel selenium tests that automatically download and setup webdrivers Usage Installation pip install autoparasel

Ronak Badhe 8 Mar 13, 2022
automate the procedure of 403 response code bypass

403bypasser automate the procedure of 403 response code bypass Description i notice a lot of #bugbountytips describe how to bypass 403 response code s

smackerdodi2 40 Dec 16, 2022
Generate random test credit card numbers for testing, validation and/or verification purposes.

Generate random test credit card numbers for testing, validation and/or verification purposes.

Dark Hunter 141 5 Nov 14, 2022
Generates realistic traffic for load testing tile servers

Generates realistic traffic for load testing tile servers. Useful for: Measuring throughput, latency and concurrency of your tile serving stack. Ident

Brandon Liu 23 Dec 05, 2022
This package is a python library with tools for the Molecular Simulation - Software Gromos.

This package is a python library with tools for the Molecular Simulation - Software Gromos. It allows you to easily set up, manage and analyze simulations in python.

14 Sep 28, 2022
Pymox - open source mock object framework for Python

Pymox is an open source mock object framework for Python. First Steps Installation Tutorial Documentation http://pymox.readthedocs.io/en/latest/index.

Ivan Rocha 7 Feb 02, 2022