PyMongo - the Python driver for MongoDB

Overview

PyMongo

Info: See the mongo site for more information. See GitHub for the latest source.
Documentation: Available at pymongo.readthedocs.io
Author: Mike Dirolf
Maintainer: Bernie Hackett <[email protected]>

About

The PyMongo distribution contains tools for interacting with MongoDB database from Python. The bson package is an implementation of the BSON format for Python. The pymongo package is a native Python driver for MongoDB. The gridfs package is a gridfs implementation on top of pymongo.

PyMongo supports MongoDB 2.6, 3.0, 3.2, 3.4, 3.6, 4.0, 4.2, and 4.4.

Support / Feedback

For issues with, questions about, or feedback for PyMongo, please look into our support channels. Please do not email any of the PyMongo developers directly with issues or questions - you're more likely to get an answer on the MongoDB Community Forums.

Bugs / Feature Requests

Think you’ve found a bug? Want to see a new feature in PyMongo? Please open a case in our issue management tool, JIRA:

Bug reports in JIRA for all driver projects (i.e. PYTHON, CSHARP, JAVA) and the Core Server (i.e. SERVER) project are public.

How To Ask For Help

Please include all of the following information when opening an issue:

  • Detailed steps to reproduce the problem, including full traceback, if possible.

  • The exact python version used, with patch level:

    $ python -c "import sys; print(sys.version)"
    
  • The exact version of PyMongo used, with patch level:

    $ python -c "import pymongo; print(pymongo.version); print(pymongo.has_c())"
    
  • The operating system and version (e.g. Windows 7, OSX 10.8, ...)

  • Web framework or asynchronous network library used, if any, with version (e.g. Django 1.7, mod_wsgi 4.3.0, gevent 1.0.1, Tornado 4.0.2, ...)

Security Vulnerabilities

If you’ve identified a security vulnerability in a driver or any other MongoDB project, please report it according to the instructions here.

Installation

PyMongo can be installed with pip:

$ python -m pip install pymongo

Or easy_install from setuptools:

$ python -m easy_install pymongo

You can also download the project source and do:

$ python setup.py install

Do not install the "bson" package from pypi. PyMongo comes with its own bson package; doing "easy_install bson" installs a third-party package that is incompatible with PyMongo.

Dependencies

PyMongo supports CPython 3.6+ and PyPy3.6+.

Optional dependencies:

GSSAPI authentication requires pykerberos on Unix or WinKerberos on Windows. The correct dependency can be installed automatically along with PyMongo:

$ python -m pip install pymongo[gssapi]

MONGODB-AWS authentication requires pymongo-auth-aws:

$ python -m pip install pymongo[aws]

Support for mongodb+srv:// URIs requires dnspython:

$ python -m pip install pymongo[srv]

OCSP (Online Certificate Status Protocol) requires PyOpenSSL, requests, service_identity and may require certifi:

$ python -m pip install pymongo[ocsp]

Wire protocol compression with snappy requires python-snappy:

$ python -m pip install pymongo[snappy]

Wire protocol compression with zstandard requires zstandard:

$ python -m pip install pymongo[zstd]

Client-Side Field Level Encryption requires pymongocrypt:

$ python -m pip install pymongo[encryption]

You can install all dependencies automatically with the following command:

$ python -m pip install pymongo[gssapi,aws,ocsp,snappy,srv,tls,zstd,encryption]

Additional dependencies are:

  • (to generate documentation) sphinx

Examples

Here's a basic example (for more see the examples section of the docs):

>>> import pymongo
>>> client = pymongo.MongoClient("localhost", 27017)
>>> db = client.test
>>> db.name
'test'
>>> db.my_collection
Collection(Database(MongoClient('localhost', 27017), 'test'), 'my_collection')
>>> db.my_collection.insert_one({"x": 10}).inserted_id
ObjectId('4aba15ebe23f6b53b0000000')
>>> db.my_collection.insert_one({"x": 8}).inserted_id
ObjectId('4aba160ee23f6b543e000000')
>>> db.my_collection.insert_one({"x": 11}).inserted_id
ObjectId('4aba160ee23f6b543e000002')
>>> db.my_collection.find_one()
{'x': 10, '_id': ObjectId('4aba15ebe23f6b53b0000000')}
>>> for item in db.my_collection.find():
...     print(item["x"])
...
10
8
11
>>> db.my_collection.create_index("x")
'x_1'
>>> for item in db.my_collection.find().sort("x", pymongo.ASCENDING):
...     print(item["x"])
...
8
10
11
>>> [item["x"] for item in db.my_collection.find().limit(2).skip(1)]
[8, 11]

Documentation

Documentation is available at pymongo.readthedocs.io.

To build the documentation, you will need to install sphinx. Documentation can be generated by running python setup.py doc. Generated documentation can be found in the doc/build/html/ directory.

Testing

The easiest way to run the tests is to run python setup.py test in the root of the distribution.

To verify that PyMongo works with Gevent's monkey-patching:

$ python green_framework_test.py gevent

Or with Eventlet's:

$ python green_framework_test.py eventlet
Comments
  • PYTHON-436 Add support for a maximum number of open connections

    PYTHON-436 Add support for a maximum number of open connections

    This patch adds a semaphore to keep track of sockets opened and released from the pool in order to be able to enforce a maximum number of open connections. I've also fixed a few places where sockets were being leaked from the pool (never returned) and an extra socket return in a test.

    Timeout support for the semaphore is added in this patch to support connection timeouts properly.

    Changes have been made to make sure that internal functions can always get a socket from the pool. even if it's at max, so it's possible for more than the configured max to be opened, but these "forced" connections are still tracked and should always be returned.

    https://jira.mongodb.org/browse/PYTHON-436

    opened by reversefold 26
  • PYTHON-436 Change max_pool_size to limit the maximum concurrent connections

    PYTHON-436 Change max_pool_size to limit the maximum concurrent connections

    PYTHON-436 Change max_pool_size to limit the maximum concurrent connections rather than just the idle connections in the pool. Also add support for waitQueueTimeoutMS and waitQueueMultiple.

    https://jira.mongodb.org/browse/PYTHON-436

    See previous pull request for history: https://github.com/mongodb/mongo-python-driver/pull/163

    opened by reversefold 15
  • Change exception wrapping to use repr instead of str

    Change exception wrapping to use repr instead of str

    I have an app that makes heavy use of pymongo. However once in a while I get a "InvalidBSON('',)" exception. I haven't been able to track the original exception because pymongo discards useful information when wrapping exceptions.

    tracked-in-jira 
    opened by ionelmc 14
  • MemoryError while retrieving large cursors

    MemoryError while retrieving large cursors

    This is the issue we experience on windows machines: Python version v2.7.2 Mongo v2.0.7 (running on ubuntu)

    >>> ================================ RESTART ================================
    >>> from pymongo import Connection
    >>> c = Connection("mongodb://user:[email protected]/admin")
    >>> c_users  = c["data"]["user"]
    >>> users = c_users.find({}, ["_id"])
    >>> users.count()
    193845
    >>> for u in users:
        s = u["_id"]
    
    
    
    Traceback (most recent call last):
      File "<pyshell#24>", line 1, in <module>
        for u in users:
      File "build\bdist.win32\egg\pymongo\cursor.py", line 778, in next
        if len(self.__data) or self._refresh():
      File "build\bdist.win32\egg\pymongo\cursor.py", line 742, in _refresh
        limit, self.__id))
      File "build\bdist.win32\egg\pymongo\cursor.py", line 666, in __send_message
        **kwargs)
      File "build\bdist.win32\egg\pymongo\connection.py", line 907, in _send_message_with_response
        return self.__send_and_receive(message, sock_info)
      File "build\bdist.win32\egg\pymongo\connection.py", line 885, in __send_and_receive
        return self.__receive_message_on_socket(1, request_id, sock_info)
      File "build\bdist.win32\egg\pymongo\connection.py", line 877, in __receive_message_on_socket
        return self.__receive_data_on_socket(length - 16, sock_info)
      File "build\bdist.win32\egg\pymongo\connection.py", line 858, in __receive_data_on_socket
        chunk = sock_info.sock.recv(length)
    MemoryError
    >>> 
    

    At this point I am not even convinced that this is pymongo's issue, and not python's, but the change seems to fix the this without any other bad side affects. Please do not hesitate to request for more info, I'm quite eager to have it resolved in driver's mainstream code.

    opened by TomasB 14
  • PYTHON-2467 Fix hex only representation of UUID by allowing for canonical represe…

    PYTHON-2467 Fix hex only representation of UUID by allowing for canonical represe…

    …ntation (Shouldn't be too confusing.. doesn't stray from current behaviour)

    Added in canonical_uuid option to json_options. This was the only way I could think to not break current API globally but also allow for this option to exist.

    >>> test_uuid = uuid.uuid1()
    >>> test_uuid
    UUID('e387e6cc-3b45-11eb-9ec8-00163e0987ed')
    >>> JSON_OPTIONS = bson.json_util.DEFAULT_JSON_OPTIONS.with_options(canonical_uuid=True)
    >>> payload = bson.json_util.dumps({'uuid': test_uuid}, json_options=JSON_OPTIONS)
    >>> payload
    '{"uuid": {"$uuid": "e387e6cc-3b45-11eb-9ec8-00163e0987ed"}}'
    >>> bson.json_util.loads(payload)
    {'uuid': UUID('e387e6cc-3b45-11eb-9ec8-00163e0987ed')}
    >>> 
    
    tracked-in-jira 
    opened by whardier 11
  • Fix up shutdown for periodic executors

    Fix up shutdown for periodic executors

    Hello.

    It's from a robot test that uses loop operator. FOR ${lengthOfSCHMCDXXXX} IN 35 15 36

    Just temporary debug messages:

    _shutdown_executors set([]) _shutdown_executors None _shutdown_executors None

    We get this error when we launch tests.

    unexpected error: Error in atexit._run_exitfuncs:
    Traceback (most recent call last):
      File "C:\Python27\lib\atexit.py", line 24, in _run_exitfuncs
        func(*targs, **kargs)
      File "C:\Python27\lib\site-packages\pymongo\periodic_executor.py", line 133, in _shutdown_executors
        executors = list(_EXECUTORS)
    TypeError: 'NoneType' object is not iterable
    Error in atexit._run_exitfuncs:
    Traceback (most recent call last):
      File "C:\Python27\lib\atexit.py", line 24, in _run_exitfuncs
        func(*targs, **kargs)
      File "C:\Python27\lib\site-packages\pymongo\periodic_executor.py", line 133, in _shutdown_executors
        executors = list(_EXECUTORS)
    TypeError: 'NoneType' object is not iterable
    Error in sys.exitfunc:
    
    [2016.04.26-15:25:09] : accessed by C:\Python27\lib\site-packages\robot\run.py
    Traceback (most recent call last):
      File "C:\Python27\lib\atexit.py", line 24, in _run_exitfuncs
        func(*targs, **kargs)
      File "C:\Python27\lib\site-packages\pymongo\periodic_executor.py", line 133, in _shutdown_executors
        executors = list(_EXECUTORS)
    TypeError: 'NoneType' object is not iterable
    
    test finished 20160426 15:25:09
    
    opened by ghost 11
  • bson.decode_file_iter: allow to bypass read/decode errors

    bson.decode_file_iter: allow to bypass read/decode errors

    If a file is corrupted or cannot be processed by the python driver, the iterator used to stop the processing of the file on the first error. With the new yield_errors optional argument the user can bypass errors.

    Signed-off-by: Nicolas Sebrecht [email protected]

    opened by nicolas33 10
  • Avoid MemoryError on large queries

    Avoid MemoryError on large queries

    When receiving data from large queries we were running into a MemoryError. From investigating sock_info.sock.recv returns a buffer of length size, which is then inserted into the chunks list. Unfortunately we were only receiving a small amount of bytes per iteration so chunks was filling up with items of size (approximately) length and quickly running out of memory. In total our query looked like it would try to allocate about 4Tb worth of memory.

    I've rewritten the function to behave more like the a previous version which fixes the memory issue as the chunk memory is freed once the data has been concatenated to the message.

    opened by tompko 10
  • add hostname validation option during CA validation

    add hostname validation option during CA validation

    There are circumstances where you still require a CA to validate that your key and cert are valid, but forgo verifying that the hostname matches the CN.

    Link to JIRA: https://jira.mongodb.org/browse/PYTHON-834

    This patch adds an extra parameter: ssl_validate_hostname which defaults to True.

    I'm hoping that this will make it into the 2.8 and future branches.

    This is already done in TxMongo which uses Twisted's SSL Context Factory to achieve the same thing.

    opened by psi29a 9
  • AutoReconnect error should be raised instead of TypeError when ReadPrefrence.SECONDARY is used

    AutoReconnect error should be raised instead of TypeError when ReadPrefrence.SECONDARY is used

    This fix is for a bug where non-strings are joined using .join(...) in case the user has ReadPreference.SECONDARY. This causes a TypeError to be thrown instead of the expected AutoReconnect error.

    opened by eKIK 9
  • Automatic per-socket authentication in Connection class

    Automatic per-socket authentication in Connection class

    I have modified the pymongo's Connection class to perform per-socket authentication for databases when credentials are provided in advance. The connection caches information about already-authenticated socket+database combinations, so authentication is performed only once for each new socket.

    Please consider pulling this feature because it will make it much easier to efficiently use MongoDB's auth within multi-threaded web frameworks.

    The changeset includes unit tests that exercise the authentication in single- and multi-threaded scenarios -- these tests are skipped if the local server does not have auth enabled. There is no API documentation yet for the new methods on pymongo.Connection, but I would be happy to add some if it's likely my changes will be accepted.

    Cheers, James

    opened by jmurty 9
Releases(4.3.3)
  • 4.3.3(Nov 17, 2022)

  • 3.13.0(Nov 1, 2022)

  • 4.3.2(Oct 18, 2022)

  • 4.2.0(Jul 20, 2022)

  • 4.2.0b0(Jun 8, 2022)

  • 4.1.1(Apr 13, 2022)

  • 4.1.0(Apr 4, 2022)

  • 4.0.2(Mar 3, 2022)

  • 4.0.1(Mar 3, 2022)

  • 3.12.3(Mar 3, 2022)

  • 4.0(Mar 3, 2022)

  • 3.12.0(Jul 13, 2021)

  • 3.12.0b1(May 28, 2021)

  • 3.11.4(May 28, 2021)

  • 3.12.0b0(Mar 31, 2021)

  • 3.11.2(Dec 2, 2020)

  • 3.11.1(Nov 17, 2020)

  • 3.11.0(Jul 30, 2020)

    PyMongo 3.11.0 - Add support for MongoDB 4.4

    Release notes: https://developer.mongodb.com/community/forums/t/pymongo-3-11-0-released/7371

    Source code(tar.gz)
    Source code(zip)
  • 3.11.0b1(Jun 9, 2020)

    PyMongo 3.11.0b1 - Beta support for MongoDB 4.4

    Release notes: https://developer.mongodb.com/community/forums/t/pymongo-3-11-0b1-released/5156

    Source code(tar.gz)
    Source code(zip)
  • 3.11.0b0(Apr 10, 2020)

  • 3.10.1(Feb 3, 2020)

    Version 3.10.1 fixes the following issues discovered since the release of 3.10.0:

    • Fix a TypeError logged to stderr that could be triggered during server maintenance or during pymongo.mongo_client.MongoClient.close().
    • Avoid creating new connections during pymongo.mongo_client.MongoClient.close().

    Documentation - https://pymongo.readthedocs.io/en/3.10.1/ Changelog - https://pymongo.readthedocs.io/en/3.10.1/changelog.html Installation - https://pymongo.readthedocs.io/en/3.10.1/installation.html

    Source code(tar.gz)
    Source code(zip)
  • 3.10.0(Feb 3, 2020)

    Support for Client-Side Field Level Encryption with MongoDB 4.2 and support for Python 3.8.

    Documentation - https://pymongo.readthedocs.io/en/3.10.0/ Changelog - https://pymongo.readthedocs.io/en/3.10.0/changelog.html Installation - https://pymongo.readthedocs.io/en/3.10.0/installation.html

    Source code(tar.gz)
    Source code(zip)
  • 3.9.0(Feb 3, 2020)

    MongoDB 4.2 support.

    Documentation - https://pymongo.readthedocs.io/en/3.9.0/ Changelog - https://pymongo.readthedocs.io/en/3.9.0/changelog.html Installation - https://pymongo.readthedocs.io/en/3.9.0/installation.html

    Source code(tar.gz)
    Source code(zip)
  • 3.9.0b1(Jun 16, 2019)

    Beta release for MongoDB 4.2 support.

    Documentation - https://api.mongodb.com/python/3.9.0b1/ Changelog - https://api.mongodb.com/python/3.9.0b1/changelog.html#changes-in-version-3-9-0b1 Installation - https://api.mongodb.com/python/3.9.0b1/installation.html#installing-a-beta-or-release-candidate

    Source code(tar.gz)
    Source code(zip)
DBMS Mini-project: Recruitment Management System

# Hire-ME DBMS Mini-project: Recruitment Management System. 💫 ✨ Features Python + MYSQL using mysql.connector library Recruiter and Client Panel Beau

Karan Gandhi 35 Dec 23, 2022
Anomaly detection on SQL data warehouses and databases

With CueObserve, you can run anomaly detection on data in your SQL data warehouses and databases. Getting Started Install via Docker docker run -p 300

Cuebook 171 Dec 18, 2022
A library for python made by me,to make the use of MySQL easier and more pythonic

my_ezql A library for python made by me,to make the use of MySQL easier and more pythonic This library was made by Tony Hasson , a 25 year old student

3 Nov 19, 2021
An asyncio compatible Redis driver, written purely in Python. This is really just a pet-project for me.

asyncredis An asyncio compatible Redis driver. Just a pet-project. Information asyncredis is, like I've said above, just a pet-project for me. I reall

Vish M 1 Dec 25, 2021
google-cloud-bigtable Apache-2google-cloud-bigtable (🥈31 · ⭐ 3.5K) - Google Cloud Bigtable API client library. Apache-2

Python Client for Google Cloud Bigtable Google Cloud Bigtable is Google's NoSQL Big Data database service. It's the same database that powers many cor

Google APIs 39 Dec 03, 2022
A framework based on tornado for easier development, scaling up and maintenance

turbo 中文文档 Turbo is a framework for fast building web site and RESTFul api, based on tornado. Easily scale up and maintain Rapid development for RESTF

133 Dec 06, 2022
Example Python codes that works with MySQL and Excel files (.xlsx)

Python x MySQL x Excel by Zinglecode Example Python codes that do the processes between MySQL database and Excel spreadsheet files. YouTube videos MyS

Potchara Puttawanchai 1 Feb 07, 2022
Pystackql - Python wrapper for StackQL

pystackql - Python Library for StackQL Python wrapper for StackQL Usage from pys

StackQL Studios 6 Jul 01, 2022
MinIO Client SDK for Python

MinIO Python SDK for Amazon S3 Compatible Cloud Storage MinIO Python SDK is Simple Storage Service (aka S3) client to perform bucket and object operat

High Performance, Kubernetes Native Object Storage 582 Dec 28, 2022
A pandas-like deferred expression system, with first-class SQL support

Ibis: Python data analysis framework for Hadoop and SQL engines Service Status Documentation Conda packages PyPI Azure Coverage Ibis is a toolbox to b

Ibis Project 2.3k Jan 06, 2023
Pure Python MySQL Client

PyMySQL Table of Contents Requirements Installation Documentation Example Resources License This package contains a pure-Python MySQL client library,

PyMySQL 7.2k Jan 09, 2023
New generation PostgreSQL database adapter for the Python programming language

Psycopg 3 -- PostgreSQL database adapter for Python Psycopg 3 is a modern implementation of a PostgreSQL adapter for Python. Installation Quick versio

The Psycopg Team 880 Jan 08, 2023
asyncio (PEP 3156) Redis support

aioredis asyncio (PEP 3156) Redis client library. Features hiredis parser Yes Pure-python parser Yes Low-level & High-level APIs Yes Connections Pool

aio-libs 2.2k Jan 04, 2023
PostgreSQL database access simplified

Queries: PostgreSQL Simplified Queries is a BSD licensed opinionated wrapper of the psycopg2 library for interacting with PostgreSQL. The popular psyc

Gavin M. Roy 251 Oct 25, 2022
A Telegram Bot to manage Redis Database.

A Telegram Bot to manage Redis database. Direct deploy on heroku Manual Deployment python3, git is required Clone repo git clone https://github.com/bu

Amit Sharma 4 Oct 21, 2022
Logica is a logic programming language that compiles to StandardSQL and runs on Google BigQuery.

Logica: language of Big Data Logica is an open source declarative logic programming language for data manipulation. Logica is a successor to Yedalog,

Evgeny Skvortsov 1.5k Dec 30, 2022
A fast MySQL driver written in pure C/C++ for Python. Compatible with gevent through monkey patching.

:: Description :: A fast MySQL driver written in pure C/C++ for Python. Compatible with gevent through monkey patching :: Requirements :: Requires P

ESN Social Software 549 Nov 18, 2022
Motor - the async Python driver for MongoDB and Tornado or asyncio

Motor Info: Motor is a full-featured, non-blocking MongoDB driver for Python Tornado and asyncio applications. Documentation: Available at motor.readt

mongodb 2.1k Dec 26, 2022
Redis client for Python asyncio (PEP 3156)

Redis client for Python asyncio. Redis client for the PEP 3156 Python event loop. This Redis library is a completely asynchronous, non-blocking client

Jonathan Slenders 554 Dec 04, 2022
A Python DB-API and SQLAlchemy dialect to Google Spreasheets

Note: shillelagh is a drop-in replacement for gsheets-db-api, with many additional features. You should use it instead. If you're using SQLAlchemy all

Beto Dealmeida 185 Jan 01, 2023