A generator library for concise, unambiguous and URL-safe UUIDs.

Overview

Description

shortuuid is a simple python library that generates concise, unambiguous, URL-safe UUIDs.

Often, one needs to use non-sequential IDs in places where users will see them, but the IDs must be as concise and easy to use as possible. shortuuid solves this problem by generating uuids using Python's built-in uuid module and then translating them to base57 using lowercase and uppercase letters and digits, and removing similar-looking characters such as l, 1, I, O and 0.

https://travis-ci.org/skorokithakis/shortuuid.svg?branch=master

Installation

To install shortuuid you need:

  • Python 2.5 or later in the 2.x line (earlier than 2.6 not tested), or any 3.x.

If you have the dependencies, you have multiple options of installation:

Usage

To use shortuuid, just import it in your project like so:

>>> import shortuuid

You can then generate a short UUID:

>>> shortuuid.uuid()
'vytxeTZskVKR7C7WgdSP3d'

If you prefer a version 5 UUID, you can pass a name (DNS or URL) to the call and it will be used as a namespace (uuid.NAMESPACE_DNS or uuid.NAMESPACE_URL) for the resulting UUID:

>>> shortuuid.uuid(name="example.com")
'wpsWLdLt9nscn2jbTD3uxe'
>>> shortuuid.uuid(name="http://example.com")
'c8sh5y9hdSMS6zVnrvf53T'

You can also generate a cryptographically secure random string (using os.urandom(), internally) with:

>>> shortuuid.ShortUUID().random(length=22)
'RaF56o2r58hTKT7AYS9doj'

To see the alphabet that is being used to generate new UUIDs:

>>> shortuuid.get_alphabet()
'23456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'

If you want to use your own alphabet to generate UUIDs, use set_alphabet():

>>> shortuuid.set_alphabet("aaaaabcdefgh1230123")
>>> shortuuid.uuid()
'0agee20aa1hehebcagddhedddc0d2chhab3b'

shortuuid will automatically sort and remove duplicates from your alphabet to ensure consistency:

>>> shortuuid.get_alphabet()
'0123abcdefgh'

If the default 22 digits are too long for you, you can get shorter IDs by just truncating the string to the desired length. The IDs won't be universally unique any longer, but the probability of a collision will still be very low.

To serialize existing UUIDs, use encode() and decode():

>>> import uuid ; u = uuid.uuid4() ; u
UUID('6ca4f0f8-2508-4bac-b8f1-5d1e3da2247a')
>>> s = shortuuid.encode(u) ; s
'cu8Eo9RyrUsV4MXEiDZpLM'
>>> shortuuid.decode(s) == u
True
>>> short = s[:7] ; short
'cu8Eo9R'
>>> h = shortuuid.decode(short)
UUID('00000000-0000-0000-0000-00b8c0b9f952')
>>> shortuuid.decode(shortuuid.encode(h)) == h
True

Class-based usage

If you need to have various alphabets per-thread, you can use the ShortUUID class, like so:

>>> su = shortuuid.ShortUUID(alphabet="01345678")
>>> su.uuid()
'034636353306816784480643806546503818874456'
>>> su.get_alphabet()
'01345678'
>>> su.set_alphabet("21345687654123456")
>>> su.get_alphabet()
'12345678'

Command-line usage

shortuuid provides a simple way to generate a short UUID in a terminal:

$ python3 -m shortuuid
fZpeF6gcskHbSpTgpQCkcJ

(Replace python3 with py if you are using Windows)

Compatibility note

Versions of ShortUUID prior to 1.0.0 generated UUIDs with their MSB last, i.e. reversed. This was later fixed, but if you have some UUIDs stored as a string with the old method, you need to pass legacy=True to decode() when converting your strings back to UUIDs.

That option will go away in the future, so you will want to convert your UUIDs to strings using the new method. This can be done like so:

>>> new_uuid_str = encode(decode(old_uuid_str, legacy=True))

License

shortuuid is distributed under the BSD license.

Comments
  • LSB-first encoding makes lexicographic ordering hard

    LSB-first encoding makes lexicographic ordering hard

    Let's say I want to group a number of shortuuids into several non-overlapping partitions and use a string-based comparison that defines each partition, e.g. "aaa" <= shortuuid < "bbb". Since shortuuids are LSB-first, this isn't possible, since 0 corresponds to '2222222222222222222222' and 1 corresponds to '3222222222222222222222'. With UUIDs generated with str(uuid.uuid4()) this would be '00000000-0000-0000-0000-000000000000' for 0 and '00000000-0000-0000-0000-000000000001' for 1, which is useable for lexicographic ordering based on their integer representation. Is there a specific reason why shortuuids are LSB-first? In my case it complicates things, but are there advantages I'm ignoring?

    opened by letmaik 22
  • ShortUUID should not sort the alphabet

    ShortUUID should not sort the alphabet

    I am attempting to convert a shortuuid generated using node.js and the Flicker base 58 alphabet:

    shortid = 'apECAPA6RdHB6HB51FwsdN'
    FLICKR_BASE58 = '123456789abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ'
    uuid = shortuuid.ShortUUID(alphabet=FLICKR_BASE58).decode(shortid)
    
    

    Unfortunately this doesn't work:

    Traceback (most recent call last):
      File "<ipython-input-54-88d3ac3dd885>", line 3, in <module>
        uuid = shortuuid.ShortUUID(alphabet=FLICKR_BASE58).decode(shortid)
      File "/Users/scotts/.pyenv/versions/python38/lib/python3.8/site-packages/shortuuid/main.py", line 79, in decode
        return _uu.UUID(int=string_to_int(string, self._alphabet))
      File "/Users/scotts/.pyenv/versions/3.8.7/lib/python3.8/uuid.py", line 205, in __init__
        raise ValueError('int is out of range (need a 128-bit value)')
    ValueError: int is out of range (need a 128-bit value)
    

    It turns out that ShortUUID is sorting the alphabet before using it. This is incorrect and leads to this failure. It is possible to correctly decode the id like this:

    import uuid as _uu
    uuid = _uu.UUID(int=string_to_int(shortid, FLICKR_BASE58))
    

    This is a backwards compatibility issue, so switching to an unsorted implementation will probably require a compatibility flag similar to legacy.

    opened by snstanton 17
  • Generating invalid ShortUUIDs

    Generating invalid ShortUUIDs

    I'm calling shortuuid.uuid() in our unit test which run regularly and I'm seeing it generate some invalid UUIDs

    Such as 5vHqLc9etTWW3AVrEJvBez.

    The library seems to think this is a valid UUID

    >>> shortuuid.decode("5vHqLc9etTWW3AVrEJvBez")
    UUID('160427e7-f28e-40ee-9f7a-d473721488c8')
    

    But your website does not: https://shortuuid.com/

    I'm also validating this with some Elixir libraries and they are reporting that it's invalid as well.

    Any help would be greatly appreciated.

    opened by kevinkirkup 14
  • Added PaddedShortUUID for constant length short uuid strings

    Added PaddedShortUUID for constant length short uuid strings

    Small enough UUID integers can create shorter uuid strings (21 character strings are generated ~1-2% of the time with the default alphabet). When you want a fixed length ID generated, use this PaddedShortUUID class instead.

    opened by kevinastone 13
  • Add MSB-first to https://shortuuid.com/

    Add MSB-first to https://shortuuid.com/

    I'm not sure how is the owner of https://shortuuid.com/, but I think this is the right place to post the request.

    I'm heavly using shortuuid in the system, so I'm using https://shortuuid.com/ to convert UUID<->shortuuid. However, I'm using shortuuid v1.0.9 which has MSB-first decoding while the site use legacy decoding, so I'm using string reverse tool to get the MSB shortuuid.

    It would be great if you can add a new field for MSB or a radio button to switch to MSB decoding to https://shortuuid.com/

    Thanks!

    opened by MotasemAghbar 12
  • shortuuid performance and sorting

    shortuuid performance and sorting

    I was playing with our Java UUID shortner I wrote several years ago and I decided to see what other folks are doing and came across library.

    Anyway I have few critiques.

    The algorithm as I understand turns the UUID into one giant 128 bit integer and divmods appending the remainder to a string. The most significant part of the UUID is appended first but the contents of the most significant has the most significant right most. For example base10 a "100" would be 001 or in base57 m3.

    There are two problems to this approach:

    • Sorting is no longer the same. That is if you have a list of UUIDs and list of corresponding shortuuids the sort order will not be same. This is because it follows the bit order which makes since for a generic encoding algorithm but I don't think its good for a UUID shortner.
    • The algorithm is very slow on Java. Or at least this implementation which appears to be a clone of the golang https://github.com/hsingh/java-shortuuid and mostly follows the python version.

    An easy replacement for the Java one is to use Bitcoin's Base58 and just change the alphabet which is originally what I used to use for UUID shortening. However Base58 divmods byte by byte and does not construct a full 128 bit integer.

    Still Base58 is actually still not that fast compared to say Base64 and that is obviously because of power of 2 division.

    My algorithm is almost as fast as base 64. My algorithm works by divmoding longs (64 bit numbers in java) one at a time. This is fine because a UUID is two 64 bit numbers. Secondly there are tricks divmoding 64 bit numbers super fast even if they are not powers of 2 divisors. Secondly it appends the most significant left to write to preserve lexicographical sort order. Padding is required but based on my tests only like ~2% of UUIDs have less than 22 characters (I was too lazy to do the math).

    Anyway before I go set an opensource repository and try to port it to python I wanted to get some feedback. Is this a bad idea?

    opened by agentgt 12
  • Using pkg_resources makes package load really slow

    Using pkg_resources makes package load really slow

    Heya, this change really hurts package loading times 😬 https://github.com/skorokithakis/shortuuid/blob/bcd01ceaf2419d874b8543e6d7c28e2c56da7a91/shortuuid/init.py#L13

    For example, with our CLI this change has meant an increase in load time from ~0.3 seconds to ~0.8 seconds!

    pkg_resources is essentially deprecated, in part, for this very reason; replaced by https://docs.python.org/3/library/importlib.metadata.html and the https://github.com/python/importlib_metadata backport

    opened by chrisjsewell 8
  • Duplicated shord uuid

    Duplicated shord uuid

    I'm using shortuuid.ShortUUID().random(length=22) for generating short-uuids in a django application on Heroku.

    In already 2 cases I found a duplicated uuid. I haven't been able to reproduce whenever I want the error yet.

    opened by leomrocha 8
  • add py.typed to MANIFEST.in, re-export API

    add py.typed to MANIFEST.in, re-export API

    Thanks for shortuuid!

    This PR:

    • [x] updates MANIFEST.in so that the py.typed file is included in the sdist on PyPI, useful for downstream re-builders.
    • [x] populates __all__ in __init__.py to avoid some mypy warnings
      • [x] allows removing the noqa comments
    opened by bollwyvl 7
  • Unclear how to use UUID5, namespace

    Unclear how to use UUID5, namespace

    My understanding of UUID v5 is you can provide a namespace and a value to get a deterministic hash for that namespace + value.

    I see you can provide shortuuid.uuid a name argument, and I see in the code how that maps to uuid5 in the code... but it's unclear to me what to do from there.

    I see that providing the same name argument produces the same output (as expected!). How do you now combine a namespace and value to get a deterministic output, such that func(namespace, value) produces the same output every time (where namespace remains the same and value changes)?

    Perhaps I'm misunderstanding something. Thanks.

    opened by connerxyz 7
  • New release has broken existing URLs

    New release has broken existing URLs

    I store UUIDs in my database and render to short UUIDs in the UI, especially in URLs. Last week's new releases have broken all my existing URLs because of the MSB issue.

    While the legacy=True flag is available, the promise that it will go away in future is not comforting. There is absolutely nothing I can do to withdraw the thousands of existing URLs I have in circulation (as will be the case for anyone who has used ShortUUID in production).

    I'd like the option to continue generating legacy short UUIDs forever. The MSB order is irrelevant anyway in a rendered string.

    opened by jace 7
Releases(v1.0.0)
  • v1.0.0(Mar 6, 2020)

    NOTE: THIS IS A BREAKING RELEASE. See the compatibility note in the README before upgrading.

    v1.0.0 (2020-03-06)

    Features

    • Drop support for Python before 3.5. [Stavros Korokithakis]

    • Add simple command-line interface (#43) [Éric Araujo]

    Fixes

    • Make encode and decode MSB-first (#36) [Keane Nguyen]

    • Make the URL check more robust (fixes #32) [Stavros Korokithakis]

    v0.5.0 (2017-02-19)

    Features

    • Make int_to_string and string_to_int available globally. [Stavros Korokithakis]
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Feb 20, 2017)

  • v0.4.3(Jan 13, 2016)

Owner
Stavros Korokithakis
I love writing code, making stupid devices and writing about writing code and making stupid devices.
Stavros Korokithakis
LazyText is inspired b the idea of lazypredict, a library which helps build a lot of basic models without much code.

LazyText is inspired b the idea of lazypredict, a library which helps build a lot of basic models without much code. LazyText is for text what lazypredict is for numeric data.

Jay Vala 13 Nov 04, 2022
StealBit1.1 and earlier strings and config extraction scripts

StealBit1.1 and earlier scripts Use strings_decryptor.py to extract RC4 encrypted strings from a StealBit1.1 sample(s). Use config_extractor.py to ext

Soolidsnake 5 Dec 29, 2022
Deasciify-highlighted - A Python script for deasciifying text to Turkish and copying clipboard

deasciify-highlighted is a Python script for deasciifying text to Turkish and copying clipboard.

Ümit Altıntaş 3 Mar 18, 2022
Wikipedia Reader for the GNOME Desktop

Wike Wike is a Wikipedia reader for the GNOME Desktop. Provides access to all the content of this online encyclopedia in a native application, with a

Hugo Olabera 126 Dec 24, 2022
Making simplex testing clean and simple

Making Simplex Project Testing - Clean and Simple What does this repo do? It organizes the python stack for the coding project What do I need to do in

Mohit Mahajan 1 Jan 30, 2022
pydantic-i18n is an extension to support an i18n for the pydantic error messages.

pydantic-i18n is an extension to support an i18n for the pydantic error messages

Boardpack 48 Dec 21, 2022
A python tool one can extract the "hash" from a WINDOWS HELLO PIN

WINHELLO2hashcat About With this tool one can extract the "hash" from a WINDOWS HELLO PIN. This hash can be cracked with Hashcat, more precisely with

33 Dec 05, 2022
Redlines produces a Markdown text showing the differences between two strings/text

Redlines Redlines produces a Markdown text showing the differences between two strings/text. The changes are represented with strike-throughs and unde

Houfu Ang 2 Apr 08, 2022
box is a text-based visual programming language inspired by Unreal Engine Blueprint function graphs.

Box is a text-based visual programming language inspired by Unreal Engine blueprint function graphs. $ cat factorial.box ┌─ƒ(Factorial)───┐

Pranav 104 Dec 24, 2022
A query extract python package

A query extract python package

Fayas Noushad 4 Nov 28, 2021
Vastasanuli - Vastasanuli pelaa Sanuli-peliä.

Vastasanuli Vastasanuli pelaa SANULI -peliä. Se ei aina voita. Käyttö Tarttet Pythonin (3.6+). Aja make (tai lataa words.txt muualta) Asentele vaaditt

Aarni Koskela 1 Jan 06, 2022
JSON and CSV data for Swahili dictionary with over 16600+ words

kamusi JSON and CSV data for swahili dictionary with over 16600+ words. This repo consists of data from swahili dictionary with about 16683 words toge

Jordan Kalebu 8 Jan 13, 2022
A python Tk GUI that creates, writes text and attaches images into a custom spreadsheet file

A python Tk GUI that creates, writes text and attaches images into a custom spreadsheet file

Mirko Simunovic 13 Dec 09, 2022
This script has been created in order to find what are the most common demanded technologies in Data Engineering field.

This is a Python script that given a whole corpus of job descriptions and a file with keywords it extracts the number of number of ocurrences of these keywords and write it to a file. This script it

Antonio Bri Pérez 0 Jul 17, 2022
The bot creates hashtags for user's texts in Russian and English.

telegram_bot_hashtags The bot creates hashtags for user's texts in Russian and English. It is a simple bot for creating hashtags. NOTE file config.py

Yana Davydovich 2 Feb 12, 2022
A simple text editor for linux

wolf-editor A simple text editor for linux Installing using Deb Package Download newest package from releases CD into folder where the downloaded acka

Focal Fossa 5 Nov 30, 2021
a python package that lets you add custom colors and text formatting to your scripts in a very easy way!

colormate Python script text formatting package What is colormate? colormate is a python library that lets you add text formatting to your scripts, it

Rodrigo 2 Dec 14, 2022
一款高性能敏感词(非法词/脏字)检测过滤组件,附带繁体简体互换,支持全角半角互换,汉字转拼音,模糊搜索等功能。

一款高性能非法词(敏感词)检测组件,附带繁体简体互换,支持全角半角互换,获取拼音首字母,获取拼音字母,拼音模糊搜索等功能。

ToolGood 3.6k Jan 07, 2023
Compute distance between sequences. 30+ algorithms, pure python implementation, common interface, optional external libs usage.

TextDistance TextDistance -- python library for comparing distance between two or more sequences by many algorithms. Features: 30+ algorithms Pure pyt

Life4 3k Jan 02, 2023
Hotpotato is a recipe portfolio App that assists users to discover and comment new recipes.

Hotpotato Hotpotato is a recipe portfolio App that assists users to discover and comment new recipes. It is a fullstack React App made with a Redux st

Nico G Pierson 13 Nov 05, 2021