Python script for crawling ResearchGate.net papers
This code start crawling process by urls in start.txt and give paper details in crawled.json.
First install Python. Then install these libraries:
pip install selenium
pip install webdriver-manager
MAX_FETCH_COUNT: How many pages you want to crawl?
MAX_CACHED_NUM: We renew crawled.json after crawling each MAX_CACHED_NUM papers.
covid-tracker-script A simple python script to fetch the latest covid info How it works First, get the current date in MM-DD-YYYY format. Check if the
🐛 MASS DBLTEK EXPLOIT CHECKER USING SHODAN 🕸 Python script who crawl first shodan page and check DBLTEK vulnerability
quora-plus-bypass A python script to extract answers to any question on Quora (Quora+ included) Requirements Python 3.x
poolbooru_gelscraper a simple python script for scraping images off gelbooru pools. modules required:requests_html, and os by default saves files with
Binance harvester - A Python 3 script to harvest data from the Binance socket stream and calculate popular TA indicators and produce lists of top trending coins
Aliexpress to telegram post Python script that reads Aliexpress offers urls from a Excel filename (.csv) and post then in a Telegram channel using a b
This script is useful for downloading stock market data for a wide range of companies specified by their respective tickers. The script reads in the d
Auto Invite To The Organization By Star A GitHub Action script to automatically invite everyone to your organization that stars your repository. What
AutoReserve A webdriver-based script for reserving badminton courts. 使用说明 下载 chromedriver 选择当前Chrome对应版本 安装 selenium pip install selenium 更改场次、金额信息dat
Introduction This is a project I built with the sole intent to learn more about
Twitter Eye is a Twitter Information Gathering Tool With Twitter Eye, you can search with various keywords and usernames on Twitter.
腾讯课堂脚本 要学一些东西，但腾讯课堂不支持自定义变速，播放时有水印，且有些老师的课一遍不够看，于是这个脚本诞生了。 时间比较紧张，只会不定时修复重大bug。多线程下载之类的功能更新短期内不会有，如果你想一起完善这个脚本，欢迎pr 2020.5.22测试可用 使用方法 很简单，三部完成 下载代码,
Pixiv-spider A Pixiv spider module WARNING It's an unfinished work, browsing the code carefully before using it. Features 0004 - Readme.md updated, co
Stealthy Scraping Tools Do not use puppeteer and playwright for scraping. Explanation. We only use the CDP to obtain the page source and to get the ab
ScrapInsta A script to scraping data from Instagram Install First of all you can run: pip install scrapinsta After that you need to install these requ
BiliComment A spider for BiliBili comment. Spider Usage Put config.json into config directory, and then python . ./config/config.json. A example confi
PyMDL An API for MyDramaList(MDL) based on webscraping for python. Description An API for MDL to make your life easier in retriving and working on dat
WEB-INFO- This tool can be used to extract information from any website Install Termux and run the command --- $ apt-get update $ apt-get upgrade $ pk
Soup Sieve Overview Soup Sieve is a CSS selector library designed to be used with Beautiful Soup 4. It aims to provide selecting, matching, and filter
Deals of the Day This is a web scraper, using the Python framework Scrapy, built to extract data such as price and product name from the Deals of the
Searches YouTube, queries recommended videos and watches them. All fully automated and anonymised through the Tor network. The project consists of two independently usable components, the YouTube aut
Home page https://mechanicalsoup.readthedocs.io/ Overview A Python library for automating interaction with websites. MechanicalSoup automatically stor
Iceberg Locations Antarctic large iceberg positions derived from ASCAT and OSCAT-2. All data collected here are from the NASA SCP website Overview Thi
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
torrent-scrapper Python scrapper scrapping torrent website and download new movies Automatically. If you like it Put a ⭐ on this repo 😇 Run this git
Phone Crawler: Phone Crawler with multi-thread functionality Disclaimer: I'm not responsible for any illegal/misuse actions, this program was made for
What is Moon? Moon is a lightweight and fast proxy scraper made by using ProxyScrape's api. What can i do with this? You can use proxies for varietys
LeasePlan - Scraper This app will let you continuously scrape certain parts of LeasePlan and extract data of cars becoming available for lease. It has
Zillow-Scraper Instructions All terminal commands are highlighted. Make sure you first have python 3 installed. You can check this by running "python
0 Dec 15, 2021
4 Jan 9, 2022
10 Aug 18, 2022
1 Jan 2, 2022
68 Oct 8, 2022
6 Dec 6, 2022
71 Oct 4, 2022
11 Dec 11, 2022
4 Nov 9, 2021
1 Jan 14, 2022
19 Dec 12, 2022
163 Dec 30, 2022
1 Nov 14, 2021
88 Jan 04, 2023
1 Sep 05, 2021
3 Jul 05, 2021
6 Dec 10, 2022
1 Oct 24, 2021
151 Dec 23, 2022
1 Jan 12, 2022
44 Oct 18, 2022
4.3k Jan 07, 2023
5 Jul 27, 2022
1.1k Jan 06, 2023
1 Jan 08, 2022
3 Feb 10, 2022
1 Jul 04, 2022
4 Nov 18, 2022
1 Nov 23, 2021