The official wrapper for spyse.com API, written in Python, aimed to help developers build their integrations with Spyse.

Overview

Python wrapper for Spyse API

The official wrapper for spyse.com API, written in Python, aimed to help developers build their integrations with Spyse.

Spyse is the most complete Internet assets search engine for every cybersecurity professional.

Examples of data Spyse delivers:

  • List of 300+ most popular open ports found on 3.5 Billion publicly accessible IPv4 hosts.
  • Technologies used on 300+ most popular open ports and IP addresses and domains using a particular technology.
  • Security score for each IP host and website, calculated based on the found vulnerabilities.
  • List of websites hosted on each IPv4 host.
  • DNS and WHOIS records of the domain names.
  • SSL certificates provided by the website hosts.
  • Structured content of the website homepages.
  • Abuse reports associated with IPv4 hosts.
  • Organizations and industries associated with the domain names.
  • Email addresses found during the Internet scanning, associated with a domain name.

More information about the data Spyse collects is available on the Our data page.

Spyse provides an API accessible via token-based authentication. API tokens are available only for registered users on their account page.

For more information about the API, please check the API Reference.

Installation

pip3 install spyse-python

Updating

pip3 install --no-cache-dir spyse-python

Quick start

from spyse import Client

client = Client("your-api-token-here")

d = client.get_domain_details('tesla.com')

print(f"Domain details:")
print(f"Website title: {d.http_extract.title}")
print(f"Alexa rank: {d.alexa.rank}")
print(f"Certificate subject org: {d.cert_summary.subject.organization}")
print(f"Certificate issuer org: {d.cert_summary.issuer.organization}")
print(f"Updated at: {d.updated_at}")
print(f"DNS Records: {d.dns_records}")
print(f"Technologies: {d.technologies}")
print(f"Vulnerabilities: {d.cve_list}")
print(f"Trackers: {d.trackers}")
# ...

Examples

Note: You need to export access_token to run any example:

export SPYSE_API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

How to search

Using Spyse you can search for any Internet assets by their digital fingerprints. To do that, you need to form a specific search query and pass it to 'search', 'scroll', or 'count' methods.

Each search query can contain multiple search params. Each search param consists of name, operator, and value.

Check API docs to find out all existing combinations. Here is an example for domains search: https://spyse-dev.readme.io/reference/domains#domain_search You may also be interested in our GUI for building and testing queries before jumping to code: https://spyse.com/advanced-search/domain

Example search request to find subdomains of att.com:

from spyse import Client, SearchQuery, QueryParam, DomainSearchParams, Operators

# Prepare query
q = SearchQuery()
domain = "att.com"

# Add param to search for att.com subdomains
q.append_param(QueryParam(DomainSearchParams.name, Operators.ends_with, '.' + domain))

# Add param to search only for alive subdomains
q.append_param(QueryParam(DomainSearchParams.http_extract_status_code, Operators.equals, "200"))

# Add param to remove subdomains seen as PTR records
q.append_param(QueryParam(DomainSearchParams.is_ptr, Operators.equals, "False"))

# Next, you can use the query to run search, count or scroll methods
c = Client("your-api-token-here")
total_count = c.count_domains(q)
search_results = c.search_domains(q).results
scroll_results = c.scroll_domains(q).results

Example search request to find any alive IPv4 hosts in US, with open port 22 and running nginx:

from spyse import Client, SearchQuery, QueryParam, IPSearchParams, Operators

# Prepare query
q = SearchQuery()

# Add param to search for IPv4 hosts located in US
q.append_param(QueryParam(IPSearchParams.geo_country_iso_code, Operators.equals, 'US'))

# Add param to search only for hosts with open 22 port
q.append_param(QueryParam(IPSearchParams.open_port, Operators.equals, "22"))

# Add param to search only for hosts with nginx
q.append_param(QueryParam(IPSearchParams.port_technology_name, Operators.contains, "nginx"))

# Next, you can use the query to run search, count or scroll methods
c = Client("your-api-token-here")
total_count = c.count_ip(q)
search_results = c.search_ip(q).results
scroll_results = c.scroll_ip(q).results

Scroll vs Search

While a 'search' request allows to paginate over the first 10'000 results, the 'scroll search' can be used for deep pagination over a larger number of results (or even all results) in much the same way as you would use a cursor on a traditional database.

In order to use scrolling, the initial search response will return a 'search_id' data field which should be specified in the subsequent requests in order to iterate over the rest of results.

Limitations

The scroll is available only for customers with 'Pro' subscription.

Example code to check if the scroll is available for your account

from spyse import Client
c = Client("your-api-token-here")

if c.get_quotas().is_scroll_search_enabled:
    print("Scroll is available")
else:
    print("Scroll is NOT available")

Development

Installation

git clone https://github.com/spyse-com/spyse-python
pip install -e .

Run tests:

cd tests
python client_test.py

License

Distributed under the MIT License. See LICENSE for more information.

Troubleshooting and contacts

For any proposals and questions, please write at:

Comments
  • Update deps and add a dependabot.yml file

    Update deps and add a dependabot.yml file

    This PR is to fix issues with spyse using out of date deps and allow projects that use this SDK to be able to install new versions of deps that they use by fixing sypse SDK to use the latest of its deps. If this PR gets accepted can you then please tag a new release to pypi with these changes, thanks

    opened by L1ghtn1ng 0
  • Domain Lookup doesn't return result

    Domain Lookup doesn't return result

    As the issue mentioned, i'm trying to experiment with spyse-py to automate my search queries. Using the script template (Already input the token-key and some domain name example) it wont show the result. Screenshot_2022-04-30-15-09-35-27

    opened by MC189 0
  • Update license entry

    Update license entry

    It should be the license identifier and not the license text.

    At the moment PyPI shows the content of the LICENSE.md file.

    This would also allow third-party tools (e. g., pyp2rpm) to get the license details in a simple way.

    opened by fabaff 0
  • spyse.response.RateLimitError: too many requests

    spyse.response.RateLimitError: too many requests

    from spyse import Client
    
    
    def main():
        client = Client("token")
        q = client.get_quotas()
    
    
    if __name__ == '__main__':
        main()
    
    

    Traceback:

    Traceback (most recent call last):
      File "/home/name/py-trash/privatbank/main.py", line 11, in <module>
        main()
      File "/home/name/py-trash/privatbank/main.py", line 7, in main
        q = client.get_quotas()
      File "/home/name/py-trash/venv/lib/python3.9/site-packages/spyse/client.py", line 106, in get_quotas
        response.check_errors()
      File "/home/name/py-trash/venv/lib/python3.9/site-packages/spyse/response.py", line 117, in check_errors
        raise RateLimitError(m)
    spyse.response.RateLimitError: too many requests
    
    

    Python 3.9.9

    Package            Version
    ------------------ ---------
    certifi            2021.10.8
    charset-normalizer 2.0.11
    dataclasses        0.6
    dataclasses-json   0.5.6
    idna               3.3
    limiter            0.1.2
    marshmallow        3.14.1
    marshmallow-enum   1.5.1
    mypy-extensions    0.4.3
    pip                21.2.4
    requests           2.26.0
    responses          0.13.4
    setuptools         58.0.4
    six                1.16.0
    spyse-python       2.2.3
    token-bucket       0.2.0
    typing_extensions  4.0.1
    typing-inspect     0.7.1
    urllib3            1.26.8
    wheel              0.37.0
    
    opened by fabelx 2
  • make dataclasses optional

    make dataclasses optional

    Hi, can you make dataclasses conditional, since it is a backport for Python 3.6 and included in newer Python versions?

    https://github.com/spyse-com/spyse-python/blob/main/requirements.txt#L2 https://github.com/spyse-com/spyse-python/blob/main/setup.py#L21

    Thanks

    opened by blshkv 0
Releases(v2.2.3)
Owner
Spyse
Internet assets search engine
Spyse
Code release for Transferable Curriculum for Weakly-Supervised Domain Adaptation (AAAI2019)

TCL Code release for Transferable Curriculum for Weakly-Supervised Domain Adaptation (AAAI2019) Dataset Office-31 dataset, with 0.4 label noise Requir

THUML @ Tsinghua University 17 Jul 07, 2022
BaiduPCS API & App 百度网盘客户端

BaiduPCS-Py A BaiduPCS API and An App BaiduPCS-Py 是百度网盘 pcs 的非官方 api 和一个命令行运用程序。

Peter Ding 450 Jan 05, 2023
Weather_besac is a French twitter bot that tweet the weather of the city of Besançon in Franche-Comté in France every day at 8am and 4pm.

Weather Bot Besac Weather_besac is a French twitter bot that tweet the weather of the city of Besançon in Franche-Comté in France every day at 8am and

Rgld_ 1 Nov 15, 2021
通过GitHub的actions 自动采集节点 生成订阅信息

VmessActions 通过GitHub的actions 自动采集节点 自动生成订阅信息 订阅内容自动更新再仓库的 clash.yml 和 v2ray.txt 中 然后PC端/手机端根据自己的软件支持的格式,订阅对应的链接即可

skywolf627 372 Jan 04, 2023
Example of Telegram local API and aiogram 3.x

Telegram Local Full example of Telegram local application. Contains Telegram Bot API Local Telegram Bot API server based on aiogram Bot API Server ima

Oleg A. 9 Sep 16, 2022
Send Informative, Concise Slack Notifications With Minimal Effort

slack-templates Send Informative, Concise Slack Notifications With Minimal Effort slack-templates Slack Integration Available Templates Usage Report t

9 Nov 03, 2022
Simple Instagram Login Link Generator

instagram-account-login Simple Instagram Login Link Generator Info Program generates instagram login links and you may get into someone´s thought the

Kevin 5 Dec 03, 2022
Protect Discord server invite link

DiscordOauth2Join Protect discord server invite links! Setup I will not help setting up the discord application, but just python. First, install the r

ZEEE 4 Aug 12, 2021
Pybt: a BaoTa panel python sdk

About Pybt is a BaoTa panel python sdk. Pybt 是一个宝塔面板API的Python版本sdk封装库。 公司很多服务器都装了宝塔面板,通过宝塔来部署、安装、维护一些服务,服务器的数量上以后,导致了维护的不方便,这个时候就想使用宝塔提供的API来开发一个运维平台

Adam Zhang 9 Dec 05, 2022
An Advanced Telegram Bot to Play Radio & Music in Voice Chat. This is Also The Source Code of The Bot Which is Being Used For Playing Radio in @AsmSafone Channel ❤️

Telegram Radio Player V3 An Advanced Telegram Bot to Play Nonstop Radio/Music/YouTube Live in Channel or Group Voice Chats. This is also the source co

SAF ONE 421 Jan 05, 2023
Script Crack Facebook, and Instagram 🚶‍♂

in-mbf Script Crack Facebook, and Instagram 🚶‍♂ Bukti Install Script $ pkg update && pkg upgrade $ pkg install git $ pkg install python2 $ pip2 insta

Yumasaa 5 Dec 27, 2021
A discord self-bot to automate shitposting for your everyday needs.

Shitpost Selfbot A discord self-bot to automate shitposting for your everyday needs. Caution: May be a little racist. I have no clue where we are taki

stormy 1 Mar 31, 2022
A Python API to retrieve and read MLB GameDay data

mlbgame mlbgame is a Python API to retrieve and read MLB GameDay data. mlbgame works with real time data, getting information as games are being playe

Zach Panzarino 493 Dec 13, 2022
This bot can stream audio or video files and urls in telegram voice chats :)

Voice Chat Streamer This bot can stream audio or video files and urls in telegram voice chats :) 🎯 Follow me and star this repo for more telegram bot

Anjana Madu 63 Dec 25, 2022
汪汪Bot是一个Telegram Bot,用于帮助你管理一台服务器上的Docker

WangWangBot 汪汪Bot是一个Telegram Bot,用于帮助你管理一台服务器上的Docker运行的Bot。这是使用视频: 部署说明 安装运行 准备 local.env 普通版本 BOT_TOKEN=你的BOT_TOKEN ADMINS=使用,分隔的管理员ID列表 如果你使用doppl

老房东的代码练习册 56 Aug 23, 2022
A napari plugin for visualising and interacting with electron cryotomograms

napari-subboxer A napari plugin for visualising and interacting with electron cryotomograms. Installation You can install napari-subboxer via pip: pip

3 Nov 25, 2021
微信支付接口V3版python库

wechatpayv3 介绍 微信支付接口V3版python库。 适用对象 wechatpayv3支持微信支付直连商户,接口说明详见 官网。 特性 平台证书自动更新,无需开发者关注平台证书有效性,无需手动下载更新; 支持本地缓存平台证书,初始化时指定平台证书保存目录即可。 适配进度 微信支付V3版A

chen gang 258 Jan 06, 2023
⚡ A really fast and powerful Discord Token Checker

discord-token-checker ⚡ A really fast and powerful Discord Token Checker How To Use? Do pip install -r requirements.txt in your command prompt Make to

vida 25 Feb 26, 2022
Bin Checker with Aiogram, Telegram

Bin Checker with Aiogram, Telegram

JEFF 1 Aug 16, 2022
Telegram bot to stream videos in telegram voicechat for both groups and channels. Supports live strams, YouTube videos and telegram media.

Telegram bot to stream videos in telegram voicechat for both groups and channels. Supports live strams, YouTube videos and telegram media.

SUBIN 449 Dec 27, 2022