A simple, immutable URL class with a clean API for interrogation and manipulation.

Related tags

URL Manipulationpurl
Overview

purl - A simple Python URL class

A simple, immutable URL class with a clean API for interrogation and manipulation. Supports Pythons 2.7, 3.3, 3.4, 3.5, 3.6 and pypy.

Also supports template URLs as per RFC 6570

Contents:

https://secure.travis-ci.org/codeinthehole/purl.png

Docs

http://purl.readthedocs.org/en/latest/

Install

From PyPI (stable):

$ pip install purl

From Github (unstable):

$ pip install git+git://github.com/codeinthehole/purl.git#egg=purl

Use

Construct:

>>> from purl import URL

# String constructor
>>> from_str = URL('https://www.google.com/search?q=testing')

# Keyword constructor
>>> from_kwargs = URL(scheme='https', host='www.google.com', path='/search', query='q=testing')

# Combine
>>> from_combo = URL('https://www.google.com').path('search').query_param('q', 'testing')

URL objects are immutable - all mutator methods return a new instance.

Interrogate:

>>> u = URL('https://www.google.com/search?q=testing')
>>> u.scheme()
'https'
>>> u.host()
'www.google.com'
>>> u.domain()
'www.google.com'
>>> u.username()
>>> u.password()
>>> u.netloc()
'www.google.com'
>>> u.port()
>>> u.path()
'/search'
>>> u.query()
'q=testing'
>>> u.fragment()
''
>>> u.path_segment(0)
'search'
>>> u.path_segments()
('search',)
>>> u.query_param('q')
'testing'
>>> u.query_param('q', as_list=True)
['testing']
>>> u.query_param('lang', default='GB')
'GB'
>>> u.query_params()
{'q': ['testing']}
>>> u.has_query_param('q')
True
>>> u.has_query_params(('q', 'r'))
False
>>> u.subdomains()
['www', 'google', 'com']
>>> u.subdomain(0)
'www'

Note that each accessor method is overloaded to be a mutator method too, similar to the jQuery API. Eg:

>>> u = URL.from_string('https://github.com/codeinthehole')

# Access
>>> u.path_segment(0)
'codeinthehole'

# Mutate (creates a new instance)
>>> new_url = u.path_segment(0, 'tangentlabs')
>>> new_url is u
False
>>> new_url.path_segment(0)
'tangentlabs'

Hence, you can build a URL up in steps:

>>> u = URL().scheme('http').domain('www.example.com').path('/some/path').query_param('q', 'search term')
>>> u.as_string()
'http://www.example.com/some/path?q=search+term'

Along with the above overloaded methods, there is also a add_path_segment method for adding a segment at the end of the current path:

>>> new_url = u.add_path_segment('here')
>>> new_url.as_string()
'http://www.example.com/some/path/here?q=search+term'

Couple of other things:

  • Since the URL class is immutable it can be used as a key in a dictionary
  • It can be pickled and restored
  • It supports equality operations
  • It supports equality operations

URL templates can be used either via a Template class:

>>> from purl import Template
>>> tpl = Template("http://example.com{/list*}")
>>> url = tpl.expand({'list': ['red', 'green', 'blue']})
>>> url.as_string()
'http://example.com/red/green/blue'

or the expand function:

>>> from purl import expand
>>> expand(u"{/list*}", {'list': ['red', 'green', 'blue']})
'/red/green/blue'

A wide variety of expansions are possible - refer to the RFC for more details.

Changelog

v1.5 - 2019-03-10

  • Allow @ in passwords.

v1.4 - 2018-03-11

  • Allow usernames and passwords to be removed from URLs.

v1.3.1

  • Ensure paths always have a leading slash.

v1.3

  • Allow absolute URLs to be converted into relative.

v1.2

  • Support password-less URLs.
  • Allow slashes to be passed as path segments.

v1.1

  • Support setting username and password via mutator methods

v1.0.3

  • Handle some unicode compatibility edge-cases

v1.0.2

  • Fix template expansion bug with no matching variables being passed in. This ensures purl.Template works correctly with the URLs returned from the Github API.

v1.0.1

  • Fix bug with special characters in paths not being escaped.

v1.0

  • Slight tidy up. Document support for PyPy and Python 3.4.

v0.8

  • Support for RFC 6570 URI templates

v0.7

  • All internal strings are unicode.
  • Support for unicode chars in path, fragment, query, auth added.

v0.6

  • Added append_query_param method
  • Added remove_query_param method

v0.5

  • Added support for Python 3.2/3.3 (thanks @pmcnr and @mitchellrj)

v0.4.1

  • Added API docs
  • Added to readthedocs.org

v0.4

  • Modified constructor to accept full URL string as first arg
  • Added add_path_segment method

v0.3.2

  • Fixed bug port number in string when using from_string constructor

v0.3.1

  • Fixed bug with passing lists to query param setter methods

v0.3

  • Added support for comparison and equality
  • Added support for pickling
  • Added __slots__ so instances can be used as keys within dictionaries

Contribute

Clone, create a virtualenv then install purl and the packages required for testing:

$ git clone [email protected]:codeinthehole/purl.git
$ cd purl
$ mkvirtualenv purl  # requires virtualenvwrapper
(purl) $ make

Ensure tests pass using:

(purl) $ ./runtests.sh

or:

$ tox
Owner
David Winterbottom
Head of Software Engineering at Octopus Energy
David Winterbottom
UDdup - URLs Deduplication Tool

UDdup - URLs Deduplication Tool The tool gets a list of URLs, and removes "duplicate" pages in the sense of URL patterns that are probably repetitive

Rotem Reiss 128 Dec 21, 2022
python3 flask based python-url-shortener microservice.

python-url-shortener This repository is for managing all public/private entity specific api endpoints for an organisation. In this case we have entity

Asutosh Parida 1 Oct 18, 2021
A simple URL shortener app using Python AWS Chalice, AWS Lambda and AWS Dynamodb.

url-shortener-chalice A simple URL shortener app using AWS Chalice. Please make sure you configure your AWS credentials using AWS CLI before starting

Ranadeep Ghosh 2 Dec 09, 2022
A tool programmed to shorten links/mask links

A tool programmed to shorten links/mask links

Anontemitayo 6 Dec 02, 2022
🌐 URL parsing and manipulation made easy.

furl is a small Python library that makes parsing and manipulating URLs easy. Python's standard urllib and urlparse modules provide a number of URL re

Ansgar Grunseid 2.4k Jan 04, 2023
Fast pattern fetcher, Takes a URLs list and outputs the URLs which contains the parameters according to the specified pattern.

Fast Pattern Fetcher (fpf) Coded with 3 by HS Devansh Raghav Fast Pattern Fetcher, Takes a URLs list and outputs the URLs which contains the paramete

whoami security 5 Feb 20, 2022
:electric_plug: Generating short urls with python has never been easier

pyshorteners A simple URL shortening API wrapper Python library. Installing pip install pyshorteners Documentation https://pyshorteners.readthedocs.i

Ellison 350 Dec 24, 2022
Customizable URL shortener written in Python3 for sniffing and spoofing

Customizable URL shortener written in Python3 for sniffing and spoofing

3 Nov 22, 2022
A URL builder for genius :D

genius-url A URL builder for genius :D Usage from gurl import genius_url

ꌗᖘ꒒ꀤ꓄꒒ꀤꈤꍟ 12 Aug 14, 2021
🔗 FusiShort is a URL shortener built with Python, Redis, Docker and Kubernetes

This is a playground application created with goal of applying full cycle software development using popular technologies like Python, Redis, Docker and Kubernetes.

Lucas Fusinato Zanis 7 Nov 10, 2022
Simple Version of ouo.io. shorten any link on the web easily

OUO.IO LINK SHORTENER This is a simple python script that made to short links. currently ouo.io doesn't have Application Programming Interface so i de

Danushka-Madushan 1 Dec 11, 2021
Qysqa - URL shortener website with python

Qysqa - shorten your URL. ~ A simple URL-shortening website. how do you pronounc

Dastan Ozgeldi 0 Nov 18, 2022
A url shortner written in Flask.

url-shortener-elitmus This is a simple flask app which takes an URL and shortens it. This shortened verion of the URL redirects to the user to the lon

2 Nov 23, 2021
A teeny Tiny module to check URLs against discord's list of phishing domains

A teeny Tiny module to check URLs against discord's list of phishing domains

kaj 1 Aug 29, 2022
A tool to manage the base URL of the Python package index.

chpip A tool to manage the base URL of the Python package index. Installation $ pip install chpip Usage Set pip index URL Set the base URL of the Pyth

Prodesire 4 Dec 20, 2022
Extract countries, regions and cities from a URL or text

This project is no longer being maintained and has been archived. Please check the Forks list for newer versions. Forks We are aware of two 3rd party

Ushahidi 216 Nov 18, 2022
a url shortener project from semicolonworld

Url Shortener With Django Written by Semicolon World

3 Aug 24, 2021
C++ library for urlencode.

liburlencode C library for urlencode.

Khaidi Chu 6 Oct 31, 2022
Simple python library to deal with URI Templates.

uritemplate Documentation -- GitHub -- Travis-CI Simple python library to deal with URI Templates. The API looks like from uritemplate import URITempl

Hyper 210 Dec 19, 2022
Use this module to detect if a URL is on discord's phishing list.

PhishDetector This module was made so you can check a URL and see if it's in discord's official list of phishing and suspicious URLs. Installation pip

Elijah 4 Mar 25, 2022