A generator library for concise, unambiguous and URL-safe UUIDs.

Overview

Description

shortuuid is a simple python library that generates concise, unambiguous, URL-safe UUIDs.

Often, one needs to use non-sequential IDs in places where users will see them, but the IDs must be as concise and easy to use as possible. shortuuid solves this problem by generating uuids using Python's built-in uuid module and then translating them to base57 using lowercase and uppercase letters and digits, and removing similar-looking characters such as l, 1, I, O and 0.

https://travis-ci.org/skorokithakis/shortuuid.svg?branch=master

Installation

To install shortuuid you need:

  • Python 2.5 or later in the 2.x line (earlier than 2.6 not tested), or any 3.x.

If you have the dependencies, you have multiple options of installation:

Usage

To use shortuuid, just import it in your project like so:

>>> import shortuuid

You can then generate a short UUID:

>>> shortuuid.uuid()
'vytxeTZskVKR7C7WgdSP3d'

If you prefer a version 5 UUID, you can pass a name (DNS or URL) to the call and it will be used as a namespace (uuid.NAMESPACE_DNS or uuid.NAMESPACE_URL) for the resulting UUID:

>>> shortuuid.uuid(name="example.com")
'wpsWLdLt9nscn2jbTD3uxe'
>>> shortuuid.uuid(name="http://example.com")
'c8sh5y9hdSMS6zVnrvf53T'

You can also generate a cryptographically secure random string (using os.urandom(), internally) with:

>>> shortuuid.ShortUUID().random(length=22)
'RaF56o2r58hTKT7AYS9doj'

To see the alphabet that is being used to generate new UUIDs:

>>> shortuuid.get_alphabet()
'23456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'

If you want to use your own alphabet to generate UUIDs, use set_alphabet():

>>> shortuuid.set_alphabet("aaaaabcdefgh1230123")
>>> shortuuid.uuid()
'0agee20aa1hehebcagddhedddc0d2chhab3b'

shortuuid will automatically sort and remove duplicates from your alphabet to ensure consistency:

>>> shortuuid.get_alphabet()
'0123abcdefgh'

If the default 22 digits are too long for you, you can get shorter IDs by just truncating the string to the desired length. The IDs won't be universally unique any longer, but the probability of a collision will still be very low.

To serialize existing UUIDs, use encode() and decode():

>>> import uuid ; u = uuid.uuid4() ; u
UUID('6ca4f0f8-2508-4bac-b8f1-5d1e3da2247a')
>>> s = shortuuid.encode(u) ; s
'cu8Eo9RyrUsV4MXEiDZpLM'
>>> shortuuid.decode(s) == u
True
>>> short = s[:7] ; short
'cu8Eo9R'
>>> h = shortuuid.decode(short)
UUID('00000000-0000-0000-0000-00b8c0b9f952')
>>> shortuuid.decode(shortuuid.encode(h)) == h
True

Class-based usage

If you need to have various alphabets per-thread, you can use the ShortUUID class, like so:

>>> su = shortuuid.ShortUUID(alphabet="01345678")
>>> su.uuid()
'034636353306816784480643806546503818874456'
>>> su.get_alphabet()
'01345678'
>>> su.set_alphabet("21345687654123456")
>>> su.get_alphabet()
'12345678'

Command-line usage

shortuuid provides a simple way to generate a short UUID in a terminal:

$ python3 -m shortuuid
fZpeF6gcskHbSpTgpQCkcJ

(Replace python3 with py if you are using Windows)

Compatibility note

Versions of ShortUUID prior to 1.0.0 generated UUIDs with their MSB last, i.e. reversed. This was later fixed, but if you have some UUIDs stored as a string with the old method, you need to pass legacy=True to decode() when converting your strings back to UUIDs.

That option will go away in the future, so you will want to convert your UUIDs to strings using the new method. This can be done like so:

>>> new_uuid_str = encode(decode(old_uuid_str, legacy=True))

License

shortuuid is distributed under the BSD license.

Comments
  • LSB-first encoding makes lexicographic ordering hard

    LSB-first encoding makes lexicographic ordering hard

    Let's say I want to group a number of shortuuids into several non-overlapping partitions and use a string-based comparison that defines each partition, e.g. "aaa" <= shortuuid < "bbb". Since shortuuids are LSB-first, this isn't possible, since 0 corresponds to '2222222222222222222222' and 1 corresponds to '3222222222222222222222'. With UUIDs generated with str(uuid.uuid4()) this would be '00000000-0000-0000-0000-000000000000' for 0 and '00000000-0000-0000-0000-000000000001' for 1, which is useable for lexicographic ordering based on their integer representation. Is there a specific reason why shortuuids are LSB-first? In my case it complicates things, but are there advantages I'm ignoring?

    opened by letmaik 22
  • ShortUUID should not sort the alphabet

    ShortUUID should not sort the alphabet

    I am attempting to convert a shortuuid generated using node.js and the Flicker base 58 alphabet:

    shortid = 'apECAPA6RdHB6HB51FwsdN'
    FLICKR_BASE58 = '123456789abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ'
    uuid = shortuuid.ShortUUID(alphabet=FLICKR_BASE58).decode(shortid)
    
    

    Unfortunately this doesn't work:

    Traceback (most recent call last):
      File "<ipython-input-54-88d3ac3dd885>", line 3, in <module>
        uuid = shortuuid.ShortUUID(alphabet=FLICKR_BASE58).decode(shortid)
      File "/Users/scotts/.pyenv/versions/python38/lib/python3.8/site-packages/shortuuid/main.py", line 79, in decode
        return _uu.UUID(int=string_to_int(string, self._alphabet))
      File "/Users/scotts/.pyenv/versions/3.8.7/lib/python3.8/uuid.py", line 205, in __init__
        raise ValueError('int is out of range (need a 128-bit value)')
    ValueError: int is out of range (need a 128-bit value)
    

    It turns out that ShortUUID is sorting the alphabet before using it. This is incorrect and leads to this failure. It is possible to correctly decode the id like this:

    import uuid as _uu
    uuid = _uu.UUID(int=string_to_int(shortid, FLICKR_BASE58))
    

    This is a backwards compatibility issue, so switching to an unsorted implementation will probably require a compatibility flag similar to legacy.

    opened by snstanton 17
  • Generating invalid ShortUUIDs

    Generating invalid ShortUUIDs

    I'm calling shortuuid.uuid() in our unit test which run regularly and I'm seeing it generate some invalid UUIDs

    Such as 5vHqLc9etTWW3AVrEJvBez.

    The library seems to think this is a valid UUID

    >>> shortuuid.decode("5vHqLc9etTWW3AVrEJvBez")
    UUID('160427e7-f28e-40ee-9f7a-d473721488c8')
    

    But your website does not: https://shortuuid.com/

    I'm also validating this with some Elixir libraries and they are reporting that it's invalid as well.

    Any help would be greatly appreciated.

    opened by kevinkirkup 14
  • Added PaddedShortUUID for constant length short uuid strings

    Added PaddedShortUUID for constant length short uuid strings

    Small enough UUID integers can create shorter uuid strings (21 character strings are generated ~1-2% of the time with the default alphabet). When you want a fixed length ID generated, use this PaddedShortUUID class instead.

    opened by kevinastone 13
  • Add MSB-first to https://shortuuid.com/

    Add MSB-first to https://shortuuid.com/

    I'm not sure how is the owner of https://shortuuid.com/, but I think this is the right place to post the request.

    I'm heavly using shortuuid in the system, so I'm using https://shortuuid.com/ to convert UUID<->shortuuid. However, I'm using shortuuid v1.0.9 which has MSB-first decoding while the site use legacy decoding, so I'm using string reverse tool to get the MSB shortuuid.

    It would be great if you can add a new field for MSB or a radio button to switch to MSB decoding to https://shortuuid.com/

    Thanks!

    opened by MotasemAghbar 12
  • shortuuid performance and sorting

    shortuuid performance and sorting

    I was playing with our Java UUID shortner I wrote several years ago and I decided to see what other folks are doing and came across library.

    Anyway I have few critiques.

    The algorithm as I understand turns the UUID into one giant 128 bit integer and divmods appending the remainder to a string. The most significant part of the UUID is appended first but the contents of the most significant has the most significant right most. For example base10 a "100" would be 001 or in base57 m3.

    There are two problems to this approach:

    • Sorting is no longer the same. That is if you have a list of UUIDs and list of corresponding shortuuids the sort order will not be same. This is because it follows the bit order which makes since for a generic encoding algorithm but I don't think its good for a UUID shortner.
    • The algorithm is very slow on Java. Or at least this implementation which appears to be a clone of the golang https://github.com/hsingh/java-shortuuid and mostly follows the python version.

    An easy replacement for the Java one is to use Bitcoin's Base58 and just change the alphabet which is originally what I used to use for UUID shortening. However Base58 divmods byte by byte and does not construct a full 128 bit integer.

    Still Base58 is actually still not that fast compared to say Base64 and that is obviously because of power of 2 division.

    My algorithm is almost as fast as base 64. My algorithm works by divmoding longs (64 bit numbers in java) one at a time. This is fine because a UUID is two 64 bit numbers. Secondly there are tricks divmoding 64 bit numbers super fast even if they are not powers of 2 divisors. Secondly it appends the most significant left to write to preserve lexicographical sort order. Padding is required but based on my tests only like ~2% of UUIDs have less than 22 characters (I was too lazy to do the math).

    Anyway before I go set an opensource repository and try to port it to python I wanted to get some feedback. Is this a bad idea?

    opened by agentgt 12
  • Using pkg_resources makes package load really slow

    Using pkg_resources makes package load really slow

    Heya, this change really hurts package loading times 😬 https://github.com/skorokithakis/shortuuid/blob/bcd01ceaf2419d874b8543e6d7c28e2c56da7a91/shortuuid/init.py#L13

    For example, with our CLI this change has meant an increase in load time from ~0.3 seconds to ~0.8 seconds!

    pkg_resources is essentially deprecated, in part, for this very reason; replaced by https://docs.python.org/3/library/importlib.metadata.html and the https://github.com/python/importlib_metadata backport

    opened by chrisjsewell 8
  • Duplicated shord uuid

    Duplicated shord uuid

    I'm using shortuuid.ShortUUID().random(length=22) for generating short-uuids in a django application on Heroku.

    In already 2 cases I found a duplicated uuid. I haven't been able to reproduce whenever I want the error yet.

    opened by leomrocha 8
  • add py.typed to MANIFEST.in, re-export API

    add py.typed to MANIFEST.in, re-export API

    Thanks for shortuuid!

    This PR:

    • [x] updates MANIFEST.in so that the py.typed file is included in the sdist on PyPI, useful for downstream re-builders.
    • [x] populates __all__ in __init__.py to avoid some mypy warnings
      • [x] allows removing the noqa comments
    opened by bollwyvl 7
  • Unclear how to use UUID5, namespace

    Unclear how to use UUID5, namespace

    My understanding of UUID v5 is you can provide a namespace and a value to get a deterministic hash for that namespace + value.

    I see you can provide shortuuid.uuid a name argument, and I see in the code how that maps to uuid5 in the code... but it's unclear to me what to do from there.

    I see that providing the same name argument produces the same output (as expected!). How do you now combine a namespace and value to get a deterministic output, such that func(namespace, value) produces the same output every time (where namespace remains the same and value changes)?

    Perhaps I'm misunderstanding something. Thanks.

    opened by connerxyz 7
  • New release has broken existing URLs

    New release has broken existing URLs

    I store UUIDs in my database and render to short UUIDs in the UI, especially in URLs. Last week's new releases have broken all my existing URLs because of the MSB issue.

    While the legacy=True flag is available, the promise that it will go away in future is not comforting. There is absolutely nothing I can do to withdraw the thousands of existing URLs I have in circulation (as will be the case for anyone who has used ShortUUID in production).

    I'd like the option to continue generating legacy short UUIDs forever. The MSB order is irrelevant anyway in a rendered string.

    opened by jace 7
Releases(v1.0.0)
  • v1.0.0(Mar 6, 2020)

    NOTE: THIS IS A BREAKING RELEASE. See the compatibility note in the README before upgrading.

    v1.0.0 (2020-03-06)

    Features

    • Drop support for Python before 3.5. [Stavros Korokithakis]

    • Add simple command-line interface (#43) [Éric Araujo]

    Fixes

    • Make encode and decode MSB-first (#36) [Keane Nguyen]

    • Make the URL check more robust (fixes #32) [Stavros Korokithakis]

    v0.5.0 (2017-02-19)

    Features

    • Make int_to_string and string_to_int available globally. [Stavros Korokithakis]
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Feb 20, 2017)

  • v0.4.3(Jan 13, 2016)

Owner
Stavros Korokithakis
I love writing code, making stupid devices and writing about writing code and making stupid devices.
Stavros Korokithakis
TextStatistics - Get a text file wich contains English text

TextStatistics This program get a text file wich contains English text. The program analyses the text, and print some information. For this program I

2 Nov 15, 2021
Production First and Production Ready End-to-End Keyword Spotting Toolkit

WeKws Production First and Production Ready End-to-End Keyword Spotting Toolkit. The goal of this toolkit it to... Small footprint keyword spotting (K

222 Dec 30, 2022
Etranslate is a free and unlimited python library for transiting your texts

Etranslate is a free and unlimited python library for transiting your texts

Abolfazl Khalili 16 Sep 13, 2022
Making simplex testing clean and simple

Making Simplex Project Testing - Clean and Simple What does this repo do? It organizes the python stack for the coding project What do I need to do in

Mohit Mahajan 1 Jan 30, 2022
Open-source linguistic ethnography tool for framing public opinion in mediatized groups.

Open-source linguistic ethnography tool for framing public opinion in mediatized groups. Table of Contents Installing Quickstart Links Installing Pyth

Qualichat 7 Jun 02, 2022
Python Q&A for Network Engineers

Q & A I am often asked questions about how to solve this or that problem, and I decided to post these questions and solutions here, in case it is also

Natasha Samoylenko 30 Nov 15, 2022
Map Reduce Wordcount in Python using gRPC

This project is implemented in Python using gRPC. The input files are given in .txt format and the word count operation is performed.

Divija 4 Dec 05, 2022
Extract price amount and currency symbol from a raw text string

price-parser is a small library for extracting price and currency from raw text strings.

Scrapinghub 252 Dec 31, 2022
Translate .sbv subtitle files

deepl4subtitle Deeplを使って字幕ファイル(.sbv)を翻訳します。タイムスタンプも含めて出力しますが、翻訳時はタイムスタンプは文の一部とは切り離されるので、.sbvファイルをそのまま翻訳機に突っ込むよりも高精度な翻訳ができるはずです。 つかいかた 入力する.sbvファイルの前処理

Yasunori Toshimitsu 1 Oct 20, 2021
Meeting, rendezvous, confluence (Finnish kohtaaminen) mark up, down, and up again.

kohtaaminen Meeting, rendezvous, confluence (Finnish kohtaaminen) mark up, down, and up again. Given a zip file containing a tree of html and media fi

Stefan Hagen 2 Dec 14, 2022
WorldCloud Orçamento de Estado 2022

World Cloud Orçamento de Estado 2022 What it does This script creates a worldcloud, masked on a image, from a txt file How to run it? Install all libr

Jorge Gomes 2 Oct 12, 2021
从flomo导出的笔记中生成词云

flomo-word-cloud 从flomo导出的笔记中生成词云 如何使用? 将本项目克隆到你的电脑上,使用如下的命令,安装所需python库 pip install -r requirements.txt 在项目里新建一个file文件夹,把所有从flomo导出的html文件放入其中 运行main

Hannnk 9 Dec 30, 2022
Python character encoding detector

Chardet: The Universal Character Encoding Detector Detects ASCII, UTF-8, UTF-16 (2 variants), UTF-32 (4 variants) Big5, GB2312, EUC-TW, HZ-GB-2312, IS

Character Encoding Detector 1.8k Jan 08, 2023
Deasciify-highlighted - A Python script for deasciifying text to Turkish and copying clipboard

deasciify-highlighted is a Python script for deasciifying text to Turkish and copying clipboard.

Ümit Altıntaş 3 Mar 18, 2022
Format Covid values to ASCII-Table (Only for Germany and Austria)

Covid-19-Formatter (Only for Germany and Austria) Dieses Script speichert die gemeldeten Daten des RKIs / BMSGPK und formatiert diese zu einer Asci Ta

56 Jan 22, 2022
BaseCrack is a tool written in Python that can decode all alphanumeric base encoding schemes.

BaseCrack Decoder For Base Encoding Schemes BaseCrack is a tool written in Python that can decode all alphanumeric base encoding schemes. This tool ca

Mufeed VH 383 Dec 27, 2022
Make writing easier!

Handwriter Make writing easier! How to Download and install a handwriting font, or create a font from your handwriting. Use a word processor like Micr

64 Dec 25, 2022
Free & simple way to encipher text

VenSipher VenSipher is a free medium through which text can be enciphered. It can convert any text into an unrecognizable secret text that can only be

3 Jan 28, 2022
Convert text to morse code and play morse code sound.

Convert text(english) to morse codes and play morse sound!

Mohammad Dori 5 Jul 15, 2022
This is a text summarizing tool written in Python

Summarize Written by: Ling Li Ya This is a text summarizing tool written in Python. User Guide Some things to note: The application is accessible here

Marcus Lee 2 Feb 18, 2022