for those who dont want to pay $10/month for high school game footage with ads

Overview

nfhs-scraper

Disclaimer: I am in no way responsible for what you choose to do with this script and guide. I do not endorse avoiding paywalls or any illegal activity relating to this matter, I am simply providing a Python script to those who are interested.

NFHS Network is "the leader in streaming Live and On Demand high school sports."

In short, you need to pay $10 a month for a subscription to watch these games. As an athlete, I didn't want to spend $10 a month to watch my own games, with ads in it, so I made this.

Usage

Download the provided main.py Python file, so you can run it yourself. Remember, whatever you do is your choice and your responsibility.

Navigate to https://www.nfhsnetwork.com/, and find your school and sport, and select the game video you'd like to download.

In the last portion of the url, you will find the game ID.

e.g. https://www.nfhsnetwork.com/events/cool-high-school-cool-town/gam4576a0f402 -> game ID is gam4576a0f402


In the main.py file, do 2 things:

  • Change the game_id variable to your game ID.
  • Change the scrub_count variable to however much of the game you'd like to download. The game footage sometimes goes until 1-2 hours after the game ends, so you can usually omit this by lowering the count.
    • Scrubs are 10 seconds long each, you figure out how many of them you want to get your desired video length.

Run the Python file, and let the magic of computers do it's thing. It can take a while, but the video will be saved to output.mp4 in the same directory as the project, by default.

How it works

With only a bit of reverse engineering, it isn't too hard to understand how NFHS streams video to the user, and why this script works.

NFHS requires a subscription to watch the videos, and with this subscription comes an API key used to fetch the stream. In this case, you need a valid API key to fetch the stream, which is a .m3u8 file that looks a little something like this:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGET-DURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXT-INF:10.000000,
gamed408a95df_000000.ts
#EXT-INF:10.000000,
gamed408a95df_000001.ts
#EXT-INF:10.000000,
gamed408a95df_000002.ts

...and so on

Here, we notice a few things.

  • a) each media file is 10 seconds long, as specified by EXT-INF and EXT-X-TARGET-DURATION
  • b) the media files are incremental, meaning we don't need the .m3u8 stream at all to construct one ourselves

In the network tab, while watching the game, I could see my browser making a request to these files, which are hosted at https://cfscrubbed.nfhsnetwork.com/. I tried downloading one of these files myself, and was able to do so successfully with no authentication needed. So, hypothetically, I could download every file and then patch them together into one big video.

Hence, nfhs-scraper. :)


Feel free to star this repo
Owner
Conrad Crawford
full-stack typescript engineer • i write code sometimes (i think)
Conrad Crawford
Library to scrape and clean web pages to create massive datasets.

lazynlp A straightforward library that allows you to crawl, clean up, and deduplicate webpages to create massive monolingual datasets. Using this libr

Chip Huyen 2.1k Jan 06, 2023
Scrape and display grades onto the console

WebScrapeGrades About The Project This Project is a personal project where I learned how to webscrape using python requests. Being able to get request

Cyrus Baybay 1 Oct 23, 2021
Scraping news from Ucsal portal with Scrapy.

NewsScraping Esse é um projeto de raspagem das últimas noticias, de 2021, do portal da universidade Ucsal http://noosfero.ucsal.br/institucional Tecno

Crissiano Pires 0 Sep 30, 2021
Amazon scraper using scrapy, a python framework for crawling websites.

#Amazon-web-scraper This is a python program, which use scrapy python framework to crawl all pages of the product and scrap products data. This progra

Akash Das 1 Dec 26, 2021
Automated Linkedin bot that will improve your visibility and increase your network.

LinkedinSpider LinkedinSpider is a small project using browser automating to increase your visibility and network of connections on Linkedin. DISCLAIM

Frederik 2 Nov 26, 2021
Visual scraping for Scrapy

Portia Portia is a tool that allows you to visually scrape websites without any programming knowledge required. With Portia you can annotate a web pag

Scrapinghub 8.7k Jan 05, 2023
The first public repository that provides free BUBT website scraping API script on Github.

BUBT WEBSITE SCRAPPING SCRIPT I think this is the first public repository that provides free BUBT website scraping API script on github. When I was do

Md Imam Hossain 3 Feb 10, 2022
Dude is a very simple framework for writing web scrapers using Python decorators

Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-lea

Ronie Martinez 326 Dec 15, 2022
京东茅台抢购最新优化版本,京东秒杀,添加误差时间调整,优化了茅台抢购进程队列

京东茅台抢购最新优化版本,京东秒杀,添加误差时间调整,优化了茅台抢购进程队列

776 Jul 28, 2021
An application that on a given url, crowls a web page and gets all words, sorts and counts them.

Web-Scrapping-1 An application that on a given url, crowls a web page and gets all words, sorts and counts them. Installation Using the package manage

adriano atambo 1 Jan 16, 2022
A low-code tool that generates python crawler code based on curl or url

KKBA Intruoduction A low-code tool that generates python crawler code based on curl or url Requirement Python = 3.6 Install pip install kkba Usage Co

8 Sep 20, 2021
Goblyn is a Python tool focused to enumeration and capture of website files metadata.

Goblyn Metadata Enumeration What's Goblyn? Goblyn is a tool focused to enumeration and capture of website files metadata. How it works? Goblyn will se

Gustavo 46 Nov 22, 2022
腾讯课堂,模拟登陆,获取课程信息,视频下载,视频解密。

腾讯课堂脚本 要学一些东西,但腾讯课堂不支持自定义变速,播放时有水印,且有些老师的课一遍不够看,于是这个脚本诞生了。 时间比较紧张,只会不定时修复重大bug。多线程下载之类的功能更新短期内不会有,如果你想一起完善这个脚本,欢迎pr 2020.5.22测试可用 使用方法 很简单,三部完成 下载代码,

163 Dec 30, 2022
An arxiv spider

An Arxiv Spider 做为一个cser,杰出男孩深知内核对连接到计算机上的硬件设备进行管理的高效方式是中断而不是轮询。每当小伙伴发来一篇刚挂在arxiv上的”热乎“好文章时,杰出男孩都会感叹道:”师兄这是每天都挂在arxiv上呀,跑的好快~“。于是杰出男孩找了找 github,借鉴了一下其

Jie Liu 11 Sep 09, 2022
Creating Scrapy scrapers via the Django admin interface

django-dynamic-scraper Django Dynamic Scraper (DDS) is an app for Django which builds on top of the scraping framework Scrapy and lets you create and

Holger Drewes 1.1k Dec 17, 2022
Web-Scrapper using Python and Flask

Web-Scrapper "[초급]Python으로 웹 스크래퍼 만들기" 코스 -NomadCoders 기초적인 Python 문법강의부터 시작하여 웹사이트의 html파일에서 원하는 내용을 Scrapping해서 출력, csv 파일로 저장, flask를 이용한 간단한 웹페이지

윤성도 1 Nov 10, 2021
Subscrape - A Python scraper for substrate chains

subscrape A Python scraper for substrate chains that uses Subscan. Usage copy co

ChaosDAO 14 Dec 15, 2022
A Powerful Spider(Web Crawler) System in Python.

pyspider A Powerful Spider(Web Crawler) System in Python. Write script in Python Powerful WebUI with script editor, task monitor, project manager and

Roy Binux 15.7k Jan 04, 2023
Get-web-images - A python code that get images from any site

image retrieval This is a python code to retrieve an image from the internet, a

CODE 1 Dec 30, 2021
A package designed to scrape data from Yahoo Finance.

yahoostock A package designed to scrape data from Yahoo Finance. Installation The most simple installation method is through PIP. pip install yahoosto

Rohan Singh 2 May 28, 2022