pgvector support for Python

Overview

pgvector-python

pgvector support for Python

Great for online recommendations 🎉

Supports Django, SQLAlchemy, Psycopg 2, Psycopg 3, and asyncpg

Build Status

Installation

Run:

pip install pgvector

And follow the instructions for your database library:

Or check out some examples:

Django

Create the extension

from pgvector.django import VectorExtension

class Migration(migrations.Migration):
    operations = [
        VectorExtension()
    ]

Add a vector field

from pgvector.django import VectorField

class Item(models.Model):
    factors = VectorField(dimensions=3)

Insert a vector

item = Item(factors=[1, 2, 3])
item.save()

Get the nearest neighbors to a vector

from pgvector.django import L2Distance

Item.objects.order_by(L2Distance('factors', [3, 1, 2]))[:5]

Also supports MaxInnerProduct and CosineDistance

Add an approximate index

from pgvector.django import IvfflatIndex

class Item(models.Model):
    class Meta:
        indexes = [
            IvfflatIndex(
                name='my_index',
                fields=['factors'],
                lists=100,
                opclasses=['vector_l2_ops']
            )
        ]

Use vector_ip_ops for inner product and vector_cosine_ops for cosine distance

SQLAlchemy

Add a vector column

from pgvector.sqlalchemy import Vector

class Item(Base):
    factors = Column(Vector(3))

Insert a vector

item = Item(factors=[1, 2, 3])
session.add(item)
session.commit()

Get the nearest neighbors to a vector

session.query(Item).order_by(Item.factors.l2_distance([3, 1, 2])).limit(5).all()

Also supports max_inner_product and cosine_distance

Add an approximate index

index = Index('my_index', Item.factors,
    postgresql_using='ivfflat',
    postgresql_with={'lists': 100},
    postgresql_ops={'factors': 'vector_l2_ops'}
)
index.create(engine)

Use vector_ip_ops for inner product and vector_cosine_ops for cosine distance

Psycopg 2

Register the vector type with your connection or cursor

from pgvector.psycopg2 import register_vector

register_vector(conn)

Insert a vector

factors = np.array([1, 2, 3])
cur.execute('INSERT INTO item (factors) VALUES (%s)', (factors,))

Get the nearest neighbors to a vector

cur.execute('SELECT * FROM item ORDER BY factors <-> %s LIMIT 5', (factors,))
cur.fetchall()

Psycopg 3

Register the vector type with your connection or cursor

from pgvector.psycopg3 import register_vector

register_vector(conn)

Insert a vector

factors = np.array([1, 2, 3])
cur.execute('INSERT INTO item (factors) VALUES (%s)', (factors,))

Get the nearest neighbors to a vector

cur.execute('SELECT * FROM item ORDER BY factors <-> %s LIMIT 5', (factors,)).fetchall()

asyncpg

Register the vector type with your connection

from pgvector.asyncpg import register_vector

await register_vector(conn)

Insert a vector

factors = np.array([1, 2, 3])
await conn.execute('INSERT INTO item (factors) VALUES ($1)', factors)

Get the nearest neighbors to a vector

await conn.fetch('SELECT * FROM item ORDER BY factors <-> $1 LIMIT 5', factors)

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

To get started with development:

git clone https://github.com/ankane/pgvector-python.git
cd pgvector-python
pip install -r requirements.txt
pytest
Owner
Andrew Kane
Andrew Kane
Simple Crud Python vs MySQL

Simple Crud Python vs MySQL The idea came when I was studying MySQ... A desire to create a python program that can give access to a "localhost" databa

Lucas 1 Jan 21, 2022
Code needed for hybrid land cover change analysis for NASA IDS project

Documentation for the NASA IDS change analysis Poley 10/21/2021 Required python packages: whitebox numpy rasterio rasterio.mask os glob math itertools

Andrew Poley 2 Nov 12, 2021
A very terrible python-based programming language that uses folders instead of text files

PYFolders by Lewis L. Foster PYFolders is a very terrible python-based programming language that uses folders instead of regular text files. In this r

Lewis L. Foster 5 Jan 08, 2022
Comics/doujinshi reader application. Web-based, will work on desktop and tablet devices with swipe interface.

Yomiko Comics/doujinshi reader application. Web-based, will work on desktop and tablet devices with swipe interface. Scans one or more directories of

Kyubi Systems 26 Aug 10, 2022
Simple package to make requests throughout Tor with circuit renewal.

AutoTor Table of Contents About the Project Contents Dependencies Getting Started Installation Coding Contributing About the Project Simple package to

Salvador Belenguer 6 Jan 01, 2023
AIST++ API This repo contains starter code for using the AIST++ dataset.

Explainability for Vision Transformers (in PyTorch) This repository implements methods for explainability in Vision Transformers

Google 260 Dec 30, 2022
PressurePlate is a multi-agent environment that requires agents to cooperate during the traversal of a gridworld.

PressurePlate is a multi-agent environment that requires agents to cooperate during the traversal of a gridworld. The grid is partitioned into several rooms, and each room contains a plate and a clos

Autonomous Agents Research Group (University of Edinburgh) 6 Dec 03, 2022
A simple program to run through inputs for a 3n+1 problem

Author Tyler Windemuth Collatz_Conjecture A simple program to run through inputs for a 3n+1 problem Purpose: doesn't really have a purpose, did this t

0 Apr 22, 2022
🙌Kart of 210+ projects based on machine learning, deep learning, computer vision, natural language processing and all. Show your support by ✨ this repository.

ML-ProjectKart 📌 Repository This kart showcases the finest collection of all projects based on machine learning, deep learning, computer vision, natu

Prathima Kadari 203 Dec 28, 2022
Python code for YouTube videos.

#This is a open source project. Python 3 These files are mainly intended to accompany my series of YouTube tutorial videos here, https://www.youtube.c

Joe James 1.3k Dec 26, 2022
Data Orchestration Platform

Table of contents What is DOP Design Concept A Typical DOP Orchestration Flow Prerequisites - Run in Docker For DOP Native Features For DBT Instructio

Datatonic 61 Mar 04, 2022
通过简单的卷积神经网络直接预测出验证码图片中滑块的位置

使用说明 1. 在本地测试 运行python3 prdict_one.py即可,默认需要预测的图片路径位于testImg文件夹下的test1.png 运行python3 predict_folder.py预测testImg下的所有图片 2. 部署到服务器 运行python3 run_a_server

12 Mar 08, 2022
奇遇淘客服务器端

奇遇淘客 APP 服务器端 警告 正在使用 v0.2.0 版本的用户,请尽快升级到 v0.2.1。 v0.2.0 版本的 Docker 镜像中包含了有问题的 aiohttp。 奇遇淘客代码库 奇遇淘客 iOS APP 奇遇淘客 Android APP 奇遇淘客文档 服务器端文档 Docker 使用

奇遇科技 92 Nov 09, 2022
Lightweight library for accessing data and configuration

accsr This lightweight library contains utilities for managing, loading, uploading, opening and generally wrangling data and configurations. It was ba

appliedAI Initiative 7 Mar 09, 2022
1000+ ready code templates to kickstart your next AI experiment

AI Seed Projects Start with ready code for your next AI experiment. Choose from 1000+ code templates, across a wide variety of use cases. All examples

BlobCity, Inc 98 Jan 03, 2023
Identify unused production dependencies and avoid a bloated virtual environment.

creosote Identify unused production dependencies and avoid a bloated virtual environment. Quickstart # Install creosote in separate virtual environmen

Fredrik Averpil 7 Dec 29, 2022
A python script based on OpenCV-Python, you can automatically hang up the Destiny 2 Throne to get the Dawning Essence.

A python script based on OpenCV-Python, you can automatically hang up the Destiny 2 Throne to get the Dawning Essence.

1 Dec 19, 2021
Shopping-card - Shopping Card Project With Python

Shopping Card Project this application was built to handle problems with saving

moein98 1 May 06, 2022
Ssma is a tool that helps you collect your badges in a satr platform

satr-statistics-maker ssma is a tool that helps you collect your badges in a satr platform 🎖️ Requirements python = 3.7 Installation first clone the

TheAwiteb 3 Jan 04, 2022
Criando um jogo de naves espaciais com Pygame. Para iniciantes em Python

Curso de Programação de Jogos com Pygame Criando um jogo de naves espaciais com Pygame. Para iniciantes em Python Pré-requisitos Antes de começar este

Flávio Codeço Coelho 33 Dec 02, 2022