Iptvcrawl - A scrapy project for crawl IPTV playlist

Related tags

Web Crawlingiptvcrawl
Overview

iptvcrawl

a scrapy project for crawl IPTV playlist.

Dependency

  • Python3
  • pip install scrapy

Usage

scrapy crawl ejatv

Output

Output playlist file is playlist.m3u. You should note that this file will be overwritten every time when you run spider.

Customize

You can customer the filter condition. Just edit the start_urls in ejatv.py

Example:

this url

https://eja.tv/?limit=0&country=js&language=Chinese&category=&level=0&search=

means channel from Japan, language is Chinese, and any category

Avaliable parameters value are follow:

Category

Animation => Animation
Auto => Auto
Business => Business
Classic => Classic
Comedy => Comedy
Cooking => Cooking
Culture => Culture
Documentary => Documentary
Education => Education
Entertainment => Entertainment
Family => Family
Fashion => Fashion
General => General
Kids => Kids
Legislative => Legislative
Lifestyle => Lifestyle
Local => Local
Movies => Movies
Music => Music
News => News
Outdoor => Outdoor
Relax => Relax
Religious => Religious
Science => Science
Series => Series
Shop => Shop
Sport => Sport
Sports => Sports
Travel => Travel
Weather => Weather
XXX => XXX
Youtube => Youtube
VOD => VOD

Language

Akan => Akan
Albanian => Albanian
Amharic => Amharic
Arabic => Arabic
Armenian => Armenian
Azerbaijani => Azerbaijani
Bosnian => Bosnian
Bulgarian => Bulgarian
Catalan => Catalan
Chinese => Chinese
Croatian => Croatian
Czech => Czech
Danish => Danish
Divehi => Divehi
Dutch => Dutch
English => English
Estonian => Estonian
Faroese => Faroese
Finnish => Finnish
French => French
Galician => Galician
Georgian => Georgian
German => German
Greek => Greek
Hebrew => Hebrew
Hindi => Hindi
Hungarian => Hungarian
Icelandic => Icelandic
Ignota => Ignota
Indonesian => Indonesian
Italian => Italian
Japanese => Japanese
Javanese => Javanese
Kannada => Kannada
Kazakh => Kazakh
Khmer => Khmer
Kinyarwanda => Kinyarwanda
Korean => Korean
Kurdish => Kurdish
Lao => Lao
Latvian => Latvian
Lithuanian => Lithuanian
Luxembourgish => Luxembourgish
Macedonian => Macedonian
Malay => Malay
Malay => Malay 
Malayalam => Malayalam
Maltese => Maltese
Mandarin%20Chinese => Mandarin Chinese
Min%20Nan%20Chinese => Min Nan Chinese
Modern%20Greek => Modern Greek 
Montenegrin => Montenegrin
Music => Music
Māori => Māori
Norwegian%20Bokmål => Norwegian Bokmål
Persian => Persian
Polish => Polish
Portuguese => Portuguese
Punjabi => Punjabi
Pushto => Pushto
Romanian => Romanian
Russian => Russian
Serbian => Serbian
Sinhala => Sinhala
Slovak => Slovak
Slovenian => Slovenian
Somali => Somali
Spanish => Spanish
Sundanese => Sundanese
Swahili => Swahili
Swedish => Swedish
Tagalog => Tagalog
Tamil => Tamil
Telugu => Telugu
Thai => Thai
Turkish => Turkish
Ukrainian => Ukrainian
Urdu => Urdu
Uzbek => Uzbek
Vietnamese => Vietnamese
Western%20Frisian => Western Frisian
Yue%20Chinese => Yue Chinese

Country

af => Afghanistan
al => Albania
dz => Algeria
ar => Argentina
am => Armenia
aw => Aruba
au => Australia
at => Austria
az => Azerbaijan
bh => Bahrain
bb => Barbados
by => Belarus
be => Belgium
bo => Bolivia
ba => Bosnia and Herzegovina
br => Brazil
bg => Bulgaria
bf => Burkina Faso
kh => Cambodia
cm => Cameroon
ca => Canada
cl => Chile
cn => China
co => Colombia
cr => Costa Rica
hr => Croatia
cw => Curacao
cy => Cyprus
cz => Czechia
cd => Democratic Republic of the Congo
dk => Denmark
do => Dominican Republic
ec => Ecuador
eg => Egypt
sv => El Salvador
gq => Equatorial Guinea
ee => Estonia
et => Ethiopia
fo => Faroe Islands
fj => Fiji
fi => Finland
fr => France
ge => Georgia
de => Germany
gh => Ghana
gr => Greece
gt => Guatemala
gy => Guyana
ht => Haiti
hn => Honduras
hk => Hong Kong
hu => Hungary
is => Iceland
in => India
id => Indonesia
int => International
ir => Iran
iq => Iraq
ie => Ireland
il => Israel
it => Italy
jm => Jamaica
jp => Japan
jo => Jordan
kz => Kazakhstan
ke => Kenya
xk => Kosovo
kw => Kuwait
kg => Kyrgyzstan
la => Laos
lv => Latvia
lb => Lebanon
ly => Libya
li => Liechtenstein
lt => Lithuania
lu => Luxembourg
mo => Macao
my => Malaysia
mv => Maldives
mt => Malta
mx => Mexico
md => Moldova
mc => Monaco
me => Montenegro
ma => Morocco
mz => Mozambique
nl => Netherlands
an => Netherlands Antilles
nz => New Zealand
ni => Nicaragua
ng => Nigeria
mk => North Macedonia
no => Norway
om => Oman
pk => Pakistan
ps => Palestinian Territory
pa => Panama
py => Paraguay
pe => Peru
ph => Philippines
pl => Poland
pt => Portugal
pr => Puerto Rico
qa => Qatar
cg => Republic of the Congo
ro => Romania
ru => Russia
rw => Rwanda
sm => San Marino
sa => Saudi Arabia
sn => Senegal
rs => Serbia
sl => Sierra Leone
sg => Singapore
sk => Slovakia
si => Slovenia
so => Somalia
kr => South Korea
es => Spain
lk => Sri Lanka
sd => Sudan
se => Sweden
ch => Switzerland
sy => Syria
tw => Taiwan
tj => Tajikistan
tz => Tanzania
th => Thailand
tn => Tunisia
tr => Turkey
vi => U.S. Virgin Islands
ug => Uganda
ua => Ukraine
ae => United Arab Emirates
gb => United Kingdom
us => United States
uy => Uruguay
uz => Uzbekistan
va => Vatican
ve => Venezuela
vn => Vietnam
ye => Yemen
zm => Zambia
Owner
Zhijun
Zhijun
Web crawling framework based on asyncio.

Web crawling framework for everyone. Written with asyncio, uvloop and aiohttp. Requirements Python3.5+ Installation pip install gain pip install uvloo

Jiuli Gao 2k Jan 05, 2023
Unja is a fast & light tool for fetching known URLs from Wayback Machine

Unja Fetch Known Urls What's Unja? Unja is a fast & light tool for fetching known URLs from Wayback Machine, Common Crawl, Virus Total & AlienVault's

Sheryar 10 Aug 07, 2022
A web scraper that exports your entire WhatsApp chat history.

WhatSoup 🍲 A web scraper that exports your entire WhatsApp chat history. Table of Contents Overview Demo Prerequisites Instructions Frequen

Eddy Harrington 87 Jan 06, 2023
A simple Discord scraper for discord bots

A simple Discord scraper for discord bots. That includes sending an guild members ids to an file, Mass inviter for joining servers your bot is in and Fetching all the servers of the bot (w/MemberCoun

3zg 1 Jan 06, 2022
Demonstration on how to use async python to control multiple playwright browsers for web-scraping

Playwright Browser Pool This example illustrates how it's possible to use a pool of browsers to retrieve page urls in a single asynchronous process. i

Bernardas Ališauskas 8 Oct 27, 2022
A tool can scrape product in aliexpress: Title, Price, and URL Product.

Scrape-Product-Aliexpress A tool can scrape product in aliexpress: Title, Price, and URL Product. Usage: 1. Install Python 3.8 3.9 padahal halaman ins

Rahul Joshua Damanik 1 Dec 30, 2021
🐞 Douban Movie / Douban Book Scarpy

Python3-based Douban Movie/Douban Book Scarpy crawler for cover downloading + data crawling + review entry.

Xingbo Jia 1 Dec 03, 2022
A Python Covid-19 cases tracker that scrapes data off the web and presents the number of Cases, Recovered Cases, and Deaths that occurred because of the pandemic.

A Python Covid-19 cases tracker that scrapes data off the web and presents the number of Cases, Recovered Cases, and Deaths that occurred because of the pandemic.

Alex Papadopoulos 1 Nov 13, 2021
Scrapy-soccer-games - Scraping information about soccer games from a few websites

scrapy-soccer-games Esse projeto tem por finalidade pegar informação de tabela d

Caio Alves 2 Jul 20, 2022
A command-line program to download media, like and unlike posts, and more from creators on OnlyFans.

onlyfans-scraper A command-line program to download media, like and unlike posts, and more from creators on OnlyFans. Installation You can install thi

185 Jul 23, 2022
A repository with scraping code and soccer dataset from understat.com.

UNDERSTAT - SHOTS DATASET As many people interested in soccer analytics know, Understat is an amazing source of information. They provide Expected Goa

douglasbc 48 Jan 03, 2023
Scrapy, a fast high-level web crawling & scraping framework for Python.

Scrapy Overview Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pag

Scrapy project 45.5k Jan 07, 2023
script to scrape direct download links (ddls) from google drive index.

bhadoo Google Personal/Shared Drive Index scraper. A small script to scrape direct download links (ddls) of downloadable files from bhadoo google driv

sαɴᴊɪᴛ sɪɴʜα 53 Dec 16, 2022
An IpVanish Proxies Scraper

EzProxies Tired of searching for good proxies for hours? Just get an IpVanish account and get thousands of good proxies in few seconds! Showcase Watch

11 Nov 13, 2022
Crawl BookCorpus

These are scripts to reproduce BookCorpus by yourself.

Sosuke Kobayashi 590 Jan 03, 2023
crypto currency scraping

SCRYPTO What ? Crypto currencies scraping (At the moment, only bitcoin and ethereum crypto currencies are supported) How ? A python script is running

15 Sep 01, 2022
SkyScrapers: A collection of variety of Scraping Apps

SkyScrapers Collection of variety of Web Scraping Apps The web-scrapers involved

Biplov Pokhrel 3 Feb 17, 2022
Dex-scrapper - Hobby project for scrapping dex data on VeChain

Folders /zumo_abis # abi extracted from zumo repo /zumo_pools # runtime e

3 Jan 20, 2022
A dead simple crawler to get books information from Douban.

Introduction A dead simple crawler to get books information from Douban. Pre-requesites Python 3 Install dependencies from requirements.txt (Optional)

Yun Wang 1 Jan 10, 2022
LSpider 一个为被动扫描器定制的前端爬虫

LSpider LSpider - 一个为被动扫描器定制的前端爬虫 什么是LSpider? 一款为被动扫描器而生的前端爬虫~ 由Chrome Headless、LSpider主控、Mysql数据库、RabbitMQ、被动扫描器5部分组合而成。

Knownsec, Inc. 321 Dec 12, 2022