This project is a simple web scraper that scraps pages of jobinja.ir concurrently and writes and update (if file gets run more than once) dataset to .csv and .xlsx format.
This project is written in using Python3.9 but Python3.8 will work fine too.
Linux users
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python main.py
Scrapy Cluster This Scrapy project uses Redis and Kafka to create a distributed
GithubLicenseCrawler This script is intended to crawl license information of repositories through the GitHub API. Taking a csv file with requirements.
Unja Fetch Known Urls What's Unja? Unja is a fast & light tool for fetching known URLs from Wayback Machine, Common Crawl, Virus Total & AlienVault's
#Amazon-web-scraper This is a python program, which use scrapy python framework to crawl all pages of the product and scrap products data. This progra
Python based Web Scraper which can discover javascript files and parse them for juicy information (API keys, IP's, Hidden Paths etc).
This repo contains the source code for crawler and crawled data of cars specifications from autodata. The data has roughly 45k cars
WEB-INFO- This tool can be used to extract information from any website Install Termux and run the command --- $ apt-get update $ apt-get upgrade $ pk
A leetcode scraper to compile all questions in leetcode free tier to text file, pdf also available. if new questions get added, run again to get new questions.
AutoScraper: A Smart, Automatic, Fast and Lightweight Web Scraper for Python This project is made for automatic web scraping to make scraping easy. It
Phone Crawler: Phone Crawler with multi-thread functionality Disclaimer: I'm not responsible for any illegal/misuse actions, this program was made for
pytrends Introduction Unofficial API for Google Trends Allows simple interface for automating downloading of reports from Google Trends. Only good unt
Google Scholar Web Scraping This is a python script that asks for a user to input the url for a google scholar profile, and then it writes publication
Agroforestry Species Switchboard 2.0 Scraper Scrape plants scientific name information from Species Switchboard 2.0. Requirements python = 3.10 (you
Media-WebScraper This program scrapes information and images for movies and TV shows. Summary For more information on the program, read the WebScrape_
Scrapping the data from each page of biocides listed on the BAUA website into a csv file
Parsel Parsel is a BSD-licensed Python library to extract and remove data from HTML and XML using XPath and CSS selectors, optionally combined with re
Lovely Scrapper
Audio media crawler for lbry. Requirements Python 3.8 Poetry 1.1.7 Elasticsearch 7.14.0 Lbry-sdk 0.99.0 Development This project uses poetry as a depe
ScrapInsta A script to scraping data from Instagram Install First of all you can run: pip install scrapinsta After that you need to install these requ
🤖 Scrape data from HTML websites automatically with Machine Learning
0 Jan 06, 2022
4 Oct 25, 2022
10 Aug 07, 2022
1 Dec 26, 2021
6 Aug 26, 2022
5 Nov 22, 2022
1 Oct 24, 2021
3 Dec 07, 2021
4.8k Jan 04, 2023
3 Feb 10, 2022
2.6k Dec 28, 2022
1 Dec 12, 2021
2 Dec 23, 2021
1 Dec 05, 2021
1 Nov 30, 2021
859 Dec 29, 2022
2 Jan 01, 2022
4 Dec 03, 2022
1 Sep 05, 2021
798 Dec 29, 2022