find all the URL of a site with a specific Regex

Overview

href

this program will find all the link with a spesfic Regex pattern from a site.

what it will do

in any site there are a lots of url that may you need the file behind them, this program will find all the tag, then list the href of the tags. you can use Regex to find the special link(s)

all the finded url's have some special charater's, so the Regex pattern will try to match with all finded url, if match, the url will return. if not match, try for next url in the list.

if you do not write any pattern the program will print all link of site, defualt pattern is: .*

and the last thing is that the program is case-insensitive

how to use

usage:

    --url 'the url of site'

    --pattern 'Regex pattern'

    --load-headers` path header file

    href.py --url 'URL' --pattern 'RegegPattern' --load-headers ./headers

example:

    href.py --url 'https://guitarmusic.ir/hayedeh-songs/' --pattern '.*mp3.*'

example:

href

note:

  1. all the switch have a small way to use

--help: -h

--url: -u

--pattern: -p

--load-headers: -l

  1. use pipe

to use the program some time you need to pipe or redirect the result

some site repeated their link to preview a video or music before download them, so you can pipe the result to uniq command for prevent link duplicate.

and for having the link in a text file, you should redirect the result to a file. href.py -u "URL" -p "patternt" > links.txt

  1. run easy

to run the program witch oud cd to the source dir or wite the full path every time, you can link it to your ~//.local/bin/href do it by this command: ln -s href.py ~/.local/bin/href

and do not forget to make it executable

  1. headers

if you got any http status code try to use a the header of site, or use the default header in the directory by this switch --load-header [file]

Owner
Arya Shabane
The Only Currency Left In This World Is POWER | Knowledge is POWER
Arya Shabane
A URL builder for genius :D

genius-url A URL builder for genius :D Usage from gurl import genius_url

ꌗᖘ꒒ꀤ꓄꒒ꀤꈤꍟ 12 Aug 14, 2021
Use this module to detect if a URL is on discord's phishing list.

PhishDetector This module was made so you can check a URL and see if it's in discord's official list of phishing and suspicious URLs. Installation pip

Elijah 4 Mar 25, 2022
find all the URL of a site with a specific Regex

href this program will find all the link with a spesfic Regex pattern from a site. what it will do in any site there are a lots of url that may you ne

Arya Shabane 12 Dec 05, 2022
This is a no-bullshit file hosting and URL shortening service that also runs 0x0.st. Use with uWSGI.

This is a no-bullshit file hosting and URL shortening service that also runs 0x0.st. Use with uWSGI.

mia 1.6k Dec 31, 2022
🔗 FusiShort is a URL shortener built with Python, Redis, Docker and Kubernetes

This is a playground application created with goal of applying full cycle software development using popular technologies like Python, Redis, Docker and Kubernetes.

Lucas Fusinato Zanis 7 Nov 10, 2022
URL Shortener in Flask - Web service using Flask framework for Shortener URLs

URL Shortener in Flask Web service using Flask framework for Shortener URLs Install Create Virtual env $ python3 -m venv env Install requirements.txt

Rafnix Guzman 1 Sep 21, 2021
Ukiyo - A simple, minimalist and efficient discord vanity URL sniper

Ukiyo - a simple, minimalist and efficient discord vanity URL sniper. Ukiyo is easy to use, has a very visually pleasing interface, and has great spee

13 Apr 14, 2022
A python code for url redirect check

A python code for url redirect check

Fayas Noushad 1 Oct 24, 2021
Fast pattern fetcher, Takes a URLs list and outputs the URLs which contains the parameters according to the specified pattern.

Fast Pattern Fetcher (fpf) Coded with 3 by HS Devansh Raghav Fast Pattern Fetcher, Takes a URLs list and outputs the URLs which contains the paramete

whoami security 5 Feb 20, 2022
coURLan: Clean, filter, normalize, and sample URLs

coURLan: Clean, filter, normalize, and sample URLs Why coURLan? “Given that the bandwidth for conducting crawls is neither infinite nor free, it is be

Adrien Barbaresi 20 Dec 14, 2022
UDdup - URLs Deduplication Tool

UDdup - URLs Deduplication Tool The tool gets a list of URLs, and removes "duplicate" pages in the sense of URL patterns that are probably repetitive

Rotem Reiss 128 Dec 21, 2022
A tool programmed to shorten links/mask links

A tool programmed to shorten links/mask links

Anontemitayo 6 Dec 02, 2022
Temporary-shortner - A webapp that shortner URLs but for limited time

temporary-shortner A webapp that shortens URLs but for a limited time Demo site

Vitor 2 Jan 07, 2022
A url shortner written in Flask.

url-shortener-elitmus This is a simple flask app which takes an URL and shortens it. This shortened verion of the URL redirects to the user to the lon

2 Nov 23, 2021
A tool to manage the base URL of the Python package index.

chpip A tool to manage the base URL of the Python package index. Installation $ pip install chpip Usage Set pip index URL Set the base URL of the Pyth

Prodesire 4 Dec 20, 2022
Customizable URL shortener written in Python3 for sniffing and spoofing

Customizable URL shortener written in Python3 for sniffing and spoofing

3 Nov 22, 2022
A simple URL shortener built with Flask

A simple URL shortener built with Flask and MongoDB.

Mike Lowe 2 Feb 05, 2022
a little project to make custom discord invites over a url

custom-dc-invite a little project to make custom discord invites over a url how it works you create a account for

baum1810 2 Oct 03, 2022
Shorten-Link - Make shorten URL with Cuttly API

Shorten-Link This Script make shorten URL with custom slashtag The script take f

Ahmed Hossam 3 Feb 13, 2022
A simple URL shortener app using Python AWS Chalice, AWS Lambda and AWS Dynamodb.

url-shortener-chalice A simple URL shortener app using AWS Chalice. Please make sure you configure your AWS credentials using AWS CLI before starting

Ranadeep Ghosh 2 Dec 09, 2022