A very simple asynchronous wrapper that allows you to get access to the Oracle database in asyncio programs.

Overview

cx_Oracle_async

fury licence pyversions Publish Build Docs

A very simple asynchronous wrapper that allows you to get access to the Oracle database in asyncio programs.

Easy to use , buy may not the best practice for efficiency concern.

Requirements

  • cx_Oracle >= 8.1.0 (Take into consideration that author of cx_Oracle said he's trying to implement asyncio support , APIs maybe change in future version. Switch to 8.1.0 if there's something wrong makes it not gonna work.)
  • ThreadPoolExecutorPlus >= 0.2.0

Install

pip install cx_Oracle_async

Feature

  • Nearly all the same as aiomysql in asynchronous operational approach , with limited cx_Oracle feature support.
  • No automaticly date format transition built-in.
  • AQ feature added , check docs here for further information.
  • You can modify some of the connection properties simply like you're using cx_Oracle.
  • You can do basic insert / select / delete etc.
  • If you're connecting to database which is on a different machine from python process , you need to install oracle client module in order to use this library. Check cx-Oracle's installation guide for further information.

Documentation

https://cx_oracle_async.readthedocs.io

Performance

query type asynchronous multithreading synchronous multithreading synchronous single thread
fast single line query 6259.80 q/s 28906.93 q/s 14805.61 q/s
single line insertion 1341.88 q/s 1898 q/s 1685.17 q/s

/* Test platform: */
AMD Ryzen 3700x
Windows 10 LTSC
Oracle 19c
You can find performance test codes here.

Examples

Before running examples , make sure you've already installed a Oracle Client on your machine.

# basic_usages.py
import asyncio
import cx_Oracle_async

async def main():
    oracle_pool = await cx_Oracle_async.create_pool(
        host='localhost', 
        port='1521',
        user='user', 
        password='password',
        service_name='orcl', 
        min = 2,
        max = 4,
    )

    async with oracle_pool.acquire() as connection:
        async with connection.cursor() as cursor:
            await cursor.execute("SELECT * FROM V$SESSION")
            print(await cursor.fetchall())

    await oracle_pool.close()

if __name__ == '__main__':
    asyncio.run(main())
Comments
  • Default cx_Oracle parameters

    Default cx_Oracle parameters

    Hello,

    1. Sometimes I have got cx_Oracle.DatabaseError: ORA-24422 I want to use FORCE parameter of SessionPool.close(force=False) for to make it true. https://cx-oracle.readthedocs.io/en/latest/api_manual/session_pool.html Could you please add this option.

    2. cx_Oracle.SessionPool defaults is encoding=None, nencoding=None Could you please make it as original values.

    bug 
    opened by sinys15 16
  • Support connection with dsn

    Support connection with dsn

    Could you please add create_pool function with dsn parameter. And it looks better to use cx_Oracle.makedsn to build dsn string instead of your variant "f"{host}:{port}/{db}"". Example from documentation: dsn = cx_Oracle.makedsn("dbhost.example.com", 1521, service_name="orclpdb1") connection = cx_Oracle.connect("hr", userpwd, dsn, encoding="UTF-8")

    opened by sinys15 11
  • Support Rollback and Ping

    Support Rollback and Ping

    Could you please support https://cx-oracle.readthedocs.io/en/latest/api_manual/connection.html Connection.ping() and Connection.rollback() methods. This is must have.

    Connection.module Connection.action Connection.client_identifier Connection.clientinfo Would be very nice too.

    Thank you.

    enhancement 
    opened by sinys15 8
  • Async Close procedure for Oracle pool

    Async Close procedure for Oracle pool

    Seems like the Close method of Oracle pool is missed for async. I'm not shure is it just

        async def close_connection_async(self) -> None:
            if self._connection:
                await self._connection.close()
    

    or should be like

        async def close_connection_async(self) -> None:
            if self._connection:
                self._connection.close()
                await self._connection.wait_closed()
    
    opened by sinys15 4
  • Poetry SolverProblemError

    Poetry SolverProblemError

    When I add cx-Oracle-async by Poetry and call "poetry show" command I got the next error: mydir>poetry show

    SolverProblemError

    Because myproject depends on cx-Oracle-async (^0.1.3) which doesn't match any versions, version solving failed.

    at c:\users\myuser\appdata\local\programs\python\python39-32\lib\site-packages\poetry\puzzle\solver.py:241 in _solve 237│ packages = result.packages 238│ except OverrideNeeded as e: 239│ return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest) 240│ except SolveFailure as e: → 241│ raise SolverProblemError(e) 242│ 243│ results = dict( 244│ depth_first_search( 245│ PackageNode(self._package, packages), aggregate_package_nodes

    opened by sinys15 1
  • AsyncCursorWarper add var

    AsyncCursorWarper add var

    Please add class AsyncCursorWarper: ... async def var(self, args): return await self._loop.run_in_executor(self._thread_pool , self._cursor.var, args)

    And why "Warper", may be "Wrapper"?

    enhancement 
    opened by sinys15 0
  • Consider using the new python-oracledb driver

    Consider using the new python-oracledb driver

    Hi, while I was wrangling with cx_Oracle and my M1, I found out that Oracle released a new way to connect to their database systems: https://oracle.github.io/python-oracledb/

    Maybe that will make it easier to write the async functionality due to better integration with Python and the THIN approach.

    opened by ESRE-dev 0
  • callproc keyword_parameters needs functools.partial

    callproc keyword_parameters needs functools.partial

    To prevent this error:

    File /path/to/cx_Oracle_async/cursors.py:45, in AsyncCursorWrapper.callproc(self, *args, **kwargs)
         44 async def callproc(self, *args , **kwargs):
    ---> 45     return await self._loop.run_in_executor(self._thread_pool , self._cursor.callproc, *args , **kwargs)
    
    TypeError: BaseEventLoop.run_in_executor() got an unexpected keyword argument 'keyword_parameters'
    
    opened by Jaza 0
  • Missing cursor attribute 'description'

    Missing cursor attribute 'description'

    Other python database packages (including cx_Oracle) include a method to get the column names which involves using cursor.description. It looks like the object 'description' is not available for cx_Oracle_async for the AsyncCursorWrapper. Is there a different method for returning the column names when using this package?

    opened by bbaker1229 1
  • When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs

    When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs

    Describe the bug When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs.

    To Reproduce Run the following with the sixth line of code commented out (as-is) to hang. Run the following with the sixth line of code uncommented to run successfully. (sixth line has the # toggle commenting... comment on it) (ThreadPoolExecutorPlus DEFAULT_MAXIMUM_WORKER_NUM is 32 * num-cores for Linux)

    import cx_Oracle_async
    import asyncio
    
    dbspecs = dict(dsn="WFX742D", user="ars", password="ars")
    
    max_pool_cnxs = 5
    num_acquire_tasks = 32 * os.cpu_count() + max_pool_cnxs
    
    #num_acquire_tasks -= 1 # toggle commenting of this line to hang/run
    queue_len = 0
    
    async def acquire_conn(pool, i):
       global queue_len
       queue_len += 1
       print_pool_info(f't-{i} acquire...', pool)
       conn = await pool.acquire()
       print_pool_info(f't-{i} postacquire', pool)
       queue_len -= 1
       conn._conn.begin()
       cursor = await conn.cursor()
       await cursor.execute('select current_timestamp from dual', [])
       rows = await cursor.fetchall()
       print(f"t-{i} rows: {len(rows)}")
       await conn.commit()
       await pool.release(conn)
    
    async def test_many_threads():
       pool = await cx_Oracle_async.create_pool(**dbspecs, max=max_pool_cnxs)
       print_pool_info('init', pool)
       tasks = [];
       for i in range(0, num_acquire_tasks):
          tasks.append(asyncio.create_task(acquire_conn(pool, i), name=f'task-{i}'))
       await asyncio.wait(tasks)
       print_pool_info('done', pool)
    
    def print_pool_info(action, pool):
       ipool = pool._pool
       print(f'{action} (queued {queue_len}, busy {ipool.busy}/{ipool.max})')
    
    asyncio.run(test_many_threads())
    

    Expected behavior I expected the queries to be queued and run when connections became available.

    Platform:

    • OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
    • Oracle version: 11.2.0.4
    • Python version 3.8.11

    Additional context I worked around this by added a semaphore in my code to limit the number of acquires to the max cnxs. What is the benefit of allowing more threads than cnxs?

    bug 
    opened by ericwaldheim 1
  • Cursor context manager is missing __aexit__

    Cursor context manager is missing __aexit__

    Describe the bug Potential memory leak in cursors context manager.

    To Reproduce Call many times cursor.fetchone().

    Gist with sample code: https://gist.github.com/ilosamart/272a7dadb639f19fd62b7947ae12ab5f It contains three versions of the same method: one async (this lib), one async (this lib) which explicitly closes the cursor, one with pure cx_oracle.

    Expected behavior I expected that the cursor context manager closed the cursor.

    Platform:

    • OS: Ubuntu 20.04
    • Oracle version: 12.2
    • Python version 3.8

    Additional context I used the instructions on https://oracle.github.io/odpi/doc/user_guide/debugging.html to debug memory alloc inside OCI.

    bug 
    opened by ilosamart 1
Releases(0.3.3)
Owner
Data Science, Full Stack Developer.
Dockerized web application on Starlite, SQLAlchemy1.4, PostgreSQL

Production-ready dockerized async REST API on Starlite with SQLAlchemy and PostgreSQL

Artur Shiriev 10 Jan 03, 2023
Online Boutique is a cloud-native microservices demo application

Online Boutique is a cloud-native microservices demo application. Online Boutique consists of a 10-tier microservices application. The application is

Matt Reider 1 Oct 22, 2021
Klein - A micro-framework for developing production-ready web services with Python

Klein, a Web Micro-Framework Klein is a micro-framework for developing production-ready web services with Python. It is 'micro' in that it has an incr

Twisted Matrix Labs 814 Jan 08, 2023
A Python package to easily create APIs in Python.

API_Easy An Python Package for easily create APIs in Python pip install easy-api-builder Requiremnets: = python 3.6 Required modules -- Flask Docume

Envyre-Coding 2 Jan 04, 2022
An alternative serializer implementation for REST framework written in cython built for speed.

drf-turbo An alternative serializer implementation for REST framework written in cython built for speed. Free software: MIT license Documentation: htt

Mng 74 Dec 30, 2022
The little ASGI framework that shines. ?

✨ The little ASGI framework that shines. ✨ Documentation: https://www.starlette.io/ Community: https://discuss.encode.io/c/starlette Starlette Starlet

Encode 7.7k Jan 01, 2023
NO LONGER MAINTAINED - A Flask extension for creating simple ReSTful JSON APIs from SQLAlchemy models.

NO LONGER MAINTAINED This repository is no longer maintained due to lack of time. You might check out the fork https://github.com/mrevutskyi/flask-res

1k Jan 04, 2023
Official mirror of https://gitlab.com/pgjones/quart

Quart Quart is an async Python web microframework. Using Quart you can, render and serve HTML templates, write (RESTful) JSON APIs, serve WebSockets,

Phil Jones 2 Oct 05, 2022
A simple Tornado based framework designed to accelerate web service development

Toto Toto is a small framework intended to accelerate web service development. It is built on top of Tornado and can currently use MySQL, MongoDB, Pos

Jeremy Olmsted-Thompson 61 Apr 06, 2022
The lightning-fast ASGI server. ?

The lightning-fast ASGI server. Documentation: https://www.uvicorn.org Community: https://discuss.encode.io/c/uvicorn Requirements: Python 3.6+ (For P

Encode 6k Jan 03, 2023
Pyrin is an application framework built on top of Flask micro-framework to make life easier for developers who want to develop an enterprise application using Flask

Pyrin A rich, fast, performant and easy to use application framework to build apps using Flask on top of it. Pyrin is an application framework built o

Mohamad Nobakht 10 Jan 25, 2022
Python Wrapper for interacting with the Flutterwave API

Python Flutterwave Description Python Wrapper for interacting with the Flutterwa

William Otieno 32 Dec 14, 2022
PipeLayer is a lightweight Python pipeline framework

PipeLayer is a lightweight Python pipeline framework. Define a series of steps, and chain them together to create modular applications

greaterthan 64 Jul 21, 2022
Phoenix LiveView but for Django

Reactor, a LiveView library for Django Reactor enables you to do something similar to Phoenix framework LiveView using Django Channels. What's in the

Eddy Ernesto del Valle Pino 526 Jan 02, 2023
Serverless Python

Zappa - Serverless Python About Installation and Configuration Running the Initial Setup / Settings Basic Usage Initial Deployments Updates Rollback S

Rich Jones 11.9k Jan 01, 2023
Async Python 3.6+ web server/framework | Build fast. Run fast.

Sanic | Build fast. Run fast. Build Docs Package Support Stats Sanic is a Python 3.6+ web server and web framework that's written to go fast. It allow

Sanic Community Organization 16.7k Dec 28, 2022
JustPy is an object-oriented, component based, high-level Python Web Framework

JustPy Docs and Tutorials Introduction JustPy is an object-oriented, component based, high-level Python Web Framework that requires no front-en

927 Jan 08, 2023
A comprehensive reference for all topics related to building and maintaining microservices

This pandect (πανδέκτης is Ancient Greek for encyclopedia) was created to help you find and understand almost anything related to Microservices that i

Ivan Bilan 64 Dec 09, 2022
The core of a service layer that integrates with the Pyramid Web Framework.

pyramid_services The core of a service layer that integrates with the Pyramid Web Framework. pyramid_services defines a pattern and helper methods for

Michael Merickel 78 Apr 15, 2022
A PC remote controller for YouTube and Twitch

Lazynite Lazynite is a PC remote controller for YouTube and Twitch on Telegram. Features Volume control; Browser fullscreen / video fullscreen; PC shu

Alessio Celentano 46 Nov 12, 2022