Bulk download tool for the MyMedia platform

Overview

MyMedia Bulk Content Downloader

This is a bulk download tool for the MyMedia platform.

USE ONLY WHERE ALLOWED BY THE COPYRIGHT OWNER. NOT AFFILIATED WITH ANYTHING/ANYONE/ANY ORGANISATION.

Usage

0. Prerequisites

To use this tool, you will have to have all of the following installed.

As long as they are installed correctly, you can put them wherever you want, I have not hardcoded any default locations.

a. Python 3

This tool is written in Python and you need to have Python installed for it to work.

You can get Python from its official website. Download the installer, then finish the install before continuing.

b. The Selenium Python package

This is what I use to interact with Google Chrome. It comes with its own Chrome installation, so no need to install it seperately, and it shouldn't change anything on your regular Chrome.

Open your favorite terminal emulator (see the FAQ for how to do this) and type the following commands

py -m ensurepip --upgrade

pip3 install selenium

c. ffmpeg

MyMedia stores videos in a way that makes them difficult to download and use. ffmpeg is a tool that is used (among other things) to convert video formats. This tool uses it to convert that format to mp4 files.

You can download and install ffmpeg from its official website. Please make sure it is in your "path". See the FAQ if you don't know what that means, or if you don't know how to put it in your path.

1. Download the tool

Secondary (right) click here then select "Save Link As" to download it. You can put it wherever you want, but if you put it in your Downloads folder, you can just go off the examples in this guide.

If you're using Mac and don't know what a secondary click is, just click the link normally, then save the resulting page with ⌘+S

2. Get a list of all the videos you want to download

Create a text file (not a Word document) of all the MyMedia URLs (the thingy in the address bar when you're watching the video) you want to get the videos for, one URL per line. Make a note of where you saved this file

3. Create a folder where all your downloaded videos will be put in

Again, make a note of where this folder is

4. Open a terminal emulator and navigate to where you downloaded the tool

Please see the FAQs if you don't know how to do this

5. Run the tool

On Windows (Using cmd)

Type .\mymedia-downloader.py (path to the list) (path to the folder to put all the videos with the trailing \)

Replace the bits in the brackets with the paths. You can drag-and-drop the file and the folder to the terminal if you don't know how to find the path to a file, you will just need to add a \ after the second one. (This not a regular slash, it is a backslah. You type it by pressing the key to the left of the shift key on most English keyboards.)

On Mac or Linux

Type ./mymedia-downloader.py (path to the list) (path to the folder to put all the videos with the trailing /)

Replace the bits in the brackets with the paths. You can probably drag-and-drop the file and the folder to the terminal if you don't know how to find the path to a file, you might need to add a / after the second one.

The reason I'm saying probably is because Linux comes in a variety of flavours, and yours might not support this. I assume if you're using Linux, you know what a path is and how to find one. Otherwise google "find absolute path in (name of your Linux distro)".

6. Log in with your UTORid and password

Wait for the Chrome window to pop up, then click Log In and authenticate. I do not ever see your UTORid and password, all the communication happens directly between you and the university. Please see the source code if you don't believe me.

7. Wait until the screen says "Done."

After that, you should see all the videos you wanted to download in the folder you provided. Unfortunately I have not implemented naming these files correctly, so they are just numbered from 0 onwards. I might come back at this and fix that at some point.

Frequently Asked Querstions

Why should I trust you to enter my UTORid and password?

Great question. In theory, you shouldn't. However, I have provided the source code and you can see if you speak Python that the login process is identical to what would happen if you used Google Chrome yourself. After you log in, I hide the Chrome window to make things more efficient and create less of an eye sore, and download all the videos. If you don't speak Python, please feel free to show this to a friend who does. (Hint: Anyone who studies anything adjacent to computer science, data science or engineering should know Python after their first semester.)

Are you affiliated with (inster organisation here)?

No. I am not. I created this tool on my own, all errors are mine only and I do not claim to represent anyone. This tool is not "official" in any way, shape or form.

I am affiliated with (insert organisation here) and want to yell at you for creating this.

I have made my absolute best to make sure I'm not violating any EULA's or trademarks etc. It does not exploit any vulnerabilities, it simply automates a task that would be very tedious to do by hand. If you disagree and think that I'm either breaking some law/agreement/etc. or if you think this tool will create harm, please contact me. I'm sure you can find my UofT email, email me there. If you can't, please create an issue here.

What is a terminal emulator and how do I open/use one?

Using a terminal is like having a text message conversation with your computer. You tell it (in a very specific format) to do things, press enter, and it does them for you, and replies with the result.

On Windows (cmd)

Open the start menu, then type cmd.exe and press enter. This will open "Command Prompt". You can type cd then where you want to go to, to navigate to a folder. Again, cou can drag-and-drop folders to the command prompt to place avoid having to type their "paths" out. For example, to go to your Downloads folder, you can type cd Downloads right after launching command prompt. You will notice the bit before where you type your command change from C:\Users\yourusername to C:\Users\yourusername\Downloads. If you have changed the location of your Downloads folder, this will not work. I trust if you know how to do that, you know how to use command prompt and can figure this out on your own (or just drag-and-drop the folder.)

On Mac

Read this. You can use the cd gommand to move around. For example, you can type cd ~/Downloads to go to your downloads folder

On Linux

If you run Linux, you probably know how to do this. If you don't, google "(your distribution name) how to use terminal".

How do I put something on my path?

Your path is where your computer looks for things to run.

On Mac/Linux

Run export (path to the thing you want to add to your path) in your terminal each time

On Windows

Read this and do it once. You can remove what you added when you're done if you'd like.

Owner
Ege Feyzioglu
Ege Feyzioglu
This is a sport analytics project that combines the knowledge of OOP and Webscraping

This is a sport analytics project that combines the knowledge of Object Oriented Programming (OOP) and Webscraping, the weekly scraping of the English Premier league table is carried out to assess th

Dolamu Oludare 1 Nov 26, 2021
Divar.ir Ads scrapper

Divar.ir Ads Scrapper Introduction This project first asynchronously grab Divar.ir Ads and then save to .csv and .xlsx files named data.csv and data.x

Iman Kermani 4 Aug 29, 2022
A spider for Universal Online Judge(UOJ) system, converting problem pages to PDFs.

Universal Online Judge Spider Introduction This is a spider for Universal Online Judge (UOJ) system (https://uoj.ac/). It also works for all other Onl

TriNitroTofu 1 Dec 07, 2021
Download images from forum threads

Forum Image Scraper Downloads images from forum threads Only works with forums which doesn't require a login to view and have an incremental paginatio

9 Nov 16, 2022
Scraping web pages to get data

Scraping Data Get public data and save in database This is project use Python How to run a project 1 - Clone the repository 2 - Install beautifulsoup4

Soccer Project 2 Nov 01, 2021
This is a web crawler that works on employ email data by gmane.org and visualizes it in different ways.

crawler_to_visual_gmane Analyzing an EMAIL Archive from gmane and vizualizing the data using the D3 JavaScript library. This is a set of tools that al

Saim Zafar 1 Dec 20, 2021
A web Scraper for CSrankings.com that scrapes University and Faculty list for a particular country

A look into what we're building Demo.mp4 Prerequisites Python 3 Node v16+ Steps to run Create a virtual environment. Activate the virtual environment.

2 Jun 06, 2022
Visual scraping for Scrapy

Portia Portia is a tool that allows you to visually scrape websites without any programming knowledge required. With Portia you can annotate a web pag

Scrapinghub 8.7k Jan 05, 2023
Crawl the information of a given keyword on Google search engine

Crawl the information of a given keyword on Google search engine

4 Nov 09, 2022
Scrape Twitter for Tweets

Backers Thank you to all our backers! 🙏 [Become a backer] Sponsors Support this project by becoming a sponsor. Your logo will show up here with a lin

Ahmet Taspinar 2.2k Jan 05, 2023
A training task for web scraping using python multithreading and a real-time-updated list of available proxy servers.

Parallel web scraping The project is a training task for web scraping using python multithreading and a real-time-updated list of available proxy serv

Kushal Shingote 1 Feb 10, 2022
Web scrapping tool written in python3, using regex, to get CVEs, Source and URLs.

searchcve Web scrapping tool written in python3, using regex, to get CVEs, Source and URLs. Generates a CSV file in the current directory. Uses the NI

32 Oct 10, 2022
Scrape and display grades onto the console

WebScrapeGrades About The Project This Project is a personal project where I learned how to webscrape using python requests. Being able to get request

Cyrus Baybay 1 Oct 23, 2021
Pseudo API for Google Trends

pytrends Introduction Unofficial API for Google Trends Allows simple interface for automating downloading of reports from Google Trends. Only good unt

General Mills 2.6k Dec 28, 2022
a high-performance, lightweight and human friendly serving engine for scrapy

a high-performance, lightweight and human friendly serving engine for scrapy

Speakol Ads 30 Mar 01, 2022
Semplice scraper realizzato in Python tramite la libreria BeautifulSoup

Semplice scraper realizzato in Python tramite la libreria BeautifulSoup

2 Nov 22, 2021
Snowflake database loading utility with Scrapy integration

Snowflake Stage Exporter Snowflake database loading utility with Scrapy integration. Meant for streaming ingestion of JSON serializable objects into S

Oleg T. 0 Dec 06, 2021
This scrapper scrapes the mail ids of faculty members from a given linl/page and stores it in a csv file

This scrapper scrapes the mail ids of faculty members from a given linl/page and stores it in a csv file

Devansh Singh 1 Feb 10, 2022
让中国用户使用git从github下载的速度提高1000倍!

序言 github上有很多好项目,但是国内用户连github却非常的慢.每次都要用插件或者其他工具来解决. 这次自己做一个小工具,输入github原地址后,就可以自动替换为代理地址,方便大家更快速的下载. 安装 pip install cit 主要功能与用法 主要功能 change 将目标地址转换为

35 Aug 29, 2022
jd_maotai rpa 基于selenium驱动的jd抢购rpa机器人

jd_maotai rpa 基于selenium驱动的jd抢购rpa机器人, 照顾我们这样的马大哈, 不会忘记抢购了, 祝大家过年都能喝上茅台. 特别声明: 本仓库发布的jd_maotai_rpa项目定义为自动化rpa项目, 是用于防止忘记参与jd茅台的活动(由于本人时常忘记), 而不是为了秒杀和抢

35 Nov 18, 2022