A very simple asynchronous wrapper that allows you to get access to the Oracle database in asyncio programs.

Overview

cx_Oracle_async

fury licence pyversions Publish Build Docs

A very simple asynchronous wrapper that allows you to get access to the Oracle database in asyncio programs.

Easy to use , buy may not the best practice for efficiency concern.

Requirements

  • cx_Oracle >= 8.1.0 (Take into consideration that author of cx_Oracle said he's trying to implement asyncio support , APIs maybe change in future version. Switch to 8.1.0 if there's something wrong makes it not gonna work.)
  • ThreadPoolExecutorPlus >= 0.2.0

Install

pip install cx_Oracle_async

Feature

  • Nearly all the same as aiomysql in asynchronous operational approach , with limited cx_Oracle feature support.
  • No automaticly date format transition built-in.
  • AQ feature added , check docs here for further information.
  • You can modify some of the connection properties simply like you're using cx_Oracle.
  • You can do basic insert / select / delete etc.
  • If you're connecting to database which is on a different machine from python process , you need to install oracle client module in order to use this library. Check cx-Oracle's installation guide for further information.

Documentation

https://cx_oracle_async.readthedocs.io

Performance

query type asynchronous multithreading synchronous multithreading synchronous single thread
fast single line query 6259.80 q/s 28906.93 q/s 14805.61 q/s
single line insertion 1341.88 q/s 1898 q/s 1685.17 q/s

/* Test platform: */
AMD Ryzen 3700x
Windows 10 LTSC
Oracle 19c
You can find performance test codes here.

Examples

Before running examples , make sure you've already installed a Oracle Client on your machine.

# basic_usages.py
import asyncio
import cx_Oracle_async

async def main():
    oracle_pool = await cx_Oracle_async.create_pool(
        host='localhost', 
        port='1521',
        user='user', 
        password='password',
        service_name='orcl', 
        min = 2,
        max = 4,
    )

    async with oracle_pool.acquire() as connection:
        async with connection.cursor() as cursor:
            await cursor.execute("SELECT * FROM V$SESSION")
            print(await cursor.fetchall())

    await oracle_pool.close()

if __name__ == '__main__':
    asyncio.run(main())
Comments
  • Default cx_Oracle parameters

    Default cx_Oracle parameters

    Hello,

    1. Sometimes I have got cx_Oracle.DatabaseError: ORA-24422 I want to use FORCE parameter of SessionPool.close(force=False) for to make it true. https://cx-oracle.readthedocs.io/en/latest/api_manual/session_pool.html Could you please add this option.

    2. cx_Oracle.SessionPool defaults is encoding=None, nencoding=None Could you please make it as original values.

    bug 
    opened by sinys15 16
  • Support connection with dsn

    Support connection with dsn

    Could you please add create_pool function with dsn parameter. And it looks better to use cx_Oracle.makedsn to build dsn string instead of your variant "f"{host}:{port}/{db}"". Example from documentation: dsn = cx_Oracle.makedsn("dbhost.example.com", 1521, service_name="orclpdb1") connection = cx_Oracle.connect("hr", userpwd, dsn, encoding="UTF-8")

    opened by sinys15 11
  • Support Rollback and Ping

    Support Rollback and Ping

    Could you please support https://cx-oracle.readthedocs.io/en/latest/api_manual/connection.html Connection.ping() and Connection.rollback() methods. This is must have.

    Connection.module Connection.action Connection.client_identifier Connection.clientinfo Would be very nice too.

    Thank you.

    enhancement 
    opened by sinys15 8
  • Async Close procedure for Oracle pool

    Async Close procedure for Oracle pool

    Seems like the Close method of Oracle pool is missed for async. I'm not shure is it just

        async def close_connection_async(self) -> None:
            if self._connection:
                await self._connection.close()
    

    or should be like

        async def close_connection_async(self) -> None:
            if self._connection:
                self._connection.close()
                await self._connection.wait_closed()
    
    opened by sinys15 4
  • Poetry SolverProblemError

    Poetry SolverProblemError

    When I add cx-Oracle-async by Poetry and call "poetry show" command I got the next error: mydir>poetry show

    SolverProblemError

    Because myproject depends on cx-Oracle-async (^0.1.3) which doesn't match any versions, version solving failed.

    at c:\users\myuser\appdata\local\programs\python\python39-32\lib\site-packages\poetry\puzzle\solver.py:241 in _solve 237│ packages = result.packages 238│ except OverrideNeeded as e: 239│ return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest) 240│ except SolveFailure as e: → 241│ raise SolverProblemError(e) 242│ 243│ results = dict( 244│ depth_first_search( 245│ PackageNode(self._package, packages), aggregate_package_nodes

    opened by sinys15 1
  • AsyncCursorWarper add var

    AsyncCursorWarper add var

    Please add class AsyncCursorWarper: ... async def var(self, args): return await self._loop.run_in_executor(self._thread_pool , self._cursor.var, args)

    And why "Warper", may be "Wrapper"?

    enhancement 
    opened by sinys15 0
  • Consider using the new python-oracledb driver

    Consider using the new python-oracledb driver

    Hi, while I was wrangling with cx_Oracle and my M1, I found out that Oracle released a new way to connect to their database systems: https://oracle.github.io/python-oracledb/

    Maybe that will make it easier to write the async functionality due to better integration with Python and the THIN approach.

    opened by ESRE-dev 0
  • callproc keyword_parameters needs functools.partial

    callproc keyword_parameters needs functools.partial

    To prevent this error:

    File /path/to/cx_Oracle_async/cursors.py:45, in AsyncCursorWrapper.callproc(self, *args, **kwargs)
         44 async def callproc(self, *args , **kwargs):
    ---> 45     return await self._loop.run_in_executor(self._thread_pool , self._cursor.callproc, *args , **kwargs)
    
    TypeError: BaseEventLoop.run_in_executor() got an unexpected keyword argument 'keyword_parameters'
    
    opened by Jaza 0
  • Missing cursor attribute 'description'

    Missing cursor attribute 'description'

    Other python database packages (including cx_Oracle) include a method to get the column names which involves using cursor.description. It looks like the object 'description' is not available for cx_Oracle_async for the AsyncCursorWrapper. Is there a different method for returning the column names when using this package?

    opened by bbaker1229 1
  • When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs

    When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs

    Describe the bug When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs.

    To Reproduce Run the following with the sixth line of code commented out (as-is) to hang. Run the following with the sixth line of code uncommented to run successfully. (sixth line has the # toggle commenting... comment on it) (ThreadPoolExecutorPlus DEFAULT_MAXIMUM_WORKER_NUM is 32 * num-cores for Linux)

    import cx_Oracle_async
    import asyncio
    
    dbspecs = dict(dsn="WFX742D", user="ars", password="ars")
    
    max_pool_cnxs = 5
    num_acquire_tasks = 32 * os.cpu_count() + max_pool_cnxs
    
    #num_acquire_tasks -= 1 # toggle commenting of this line to hang/run
    queue_len = 0
    
    async def acquire_conn(pool, i):
       global queue_len
       queue_len += 1
       print_pool_info(f't-{i} acquire...', pool)
       conn = await pool.acquire()
       print_pool_info(f't-{i} postacquire', pool)
       queue_len -= 1
       conn._conn.begin()
       cursor = await conn.cursor()
       await cursor.execute('select current_timestamp from dual', [])
       rows = await cursor.fetchall()
       print(f"t-{i} rows: {len(rows)}")
       await conn.commit()
       await pool.release(conn)
    
    async def test_many_threads():
       pool = await cx_Oracle_async.create_pool(**dbspecs, max=max_pool_cnxs)
       print_pool_info('init', pool)
       tasks = [];
       for i in range(0, num_acquire_tasks):
          tasks.append(asyncio.create_task(acquire_conn(pool, i), name=f'task-{i}'))
       await asyncio.wait(tasks)
       print_pool_info('done', pool)
    
    def print_pool_info(action, pool):
       ipool = pool._pool
       print(f'{action} (queued {queue_len}, busy {ipool.busy}/{ipool.max})')
    
    asyncio.run(test_many_threads())
    

    Expected behavior I expected the queries to be queued and run when connections became available.

    Platform:

    • OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
    • Oracle version: 11.2.0.4
    • Python version 3.8.11

    Additional context I worked around this by added a semaphore in my code to limit the number of acquires to the max cnxs. What is the benefit of allowing more threads than cnxs?

    bug 
    opened by ericwaldheim 1
  • Cursor context manager is missing __aexit__

    Cursor context manager is missing __aexit__

    Describe the bug Potential memory leak in cursors context manager.

    To Reproduce Call many times cursor.fetchone().

    Gist with sample code: https://gist.github.com/ilosamart/272a7dadb639f19fd62b7947ae12ab5f It contains three versions of the same method: one async (this lib), one async (this lib) which explicitly closes the cursor, one with pure cx_oracle.

    Expected behavior I expected that the cursor context manager closed the cursor.

    Platform:

    • OS: Ubuntu 20.04
    • Oracle version: 12.2
    • Python version 3.8

    Additional context I used the instructions on https://oracle.github.io/odpi/doc/user_guide/debugging.html to debug memory alloc inside OCI.

    bug 
    opened by ilosamart 1
Releases(0.3.3)
Owner
Data Science, Full Stack Developer.
A Python package to easily create APIs in Python.

API_Easy An Python Package for easily create APIs in Python pip install easy-api-builder Requiremnets: = python 3.6 Required modules -- Flask Docume

Envyre-Coding 2 Jan 04, 2022
The source code to the Midnight project

MidnightSniper Started: 24/08/2021 Ended: 24/10/2021 What? This is the source code to a project developed to snipe minecraft names Why release? The ad

Kami 2 Dec 03, 2021
Flask like web framework for AWS Lambda

lambdarest Python routing mini-framework for AWS Lambda with optional JSON-schema validation. ⚠️ A user study is currently happening here, and your op

sloev / Johannes Valbjørn 91 Nov 10, 2022
Flask-Potion is a RESTful API framework for Flask and SQLAlchemy, Peewee or MongoEngine

Flask-Potion Description Flask-Potion is a powerful Flask extension for building RESTful JSON APIs. Potion features include validation, model resource

DTU Biosustain 491 Dec 08, 2022
A shopping list and kitchen inventory management app.

Flask React Project This is the backend for the Flask React project. Getting started Clone this repository (only this branch) git clone https://github

11 Jun 03, 2022
NO LONGER MAINTAINED - A Flask extension for creating simple ReSTful JSON APIs from SQLAlchemy models.

NO LONGER MAINTAINED This repository is no longer maintained due to lack of time. You might check out the fork https://github.com/mrevutskyi/flask-res

1k Jan 04, 2023
The lightning-fast ASGI server. ?

The lightning-fast ASGI server. Documentation: https://www.uvicorn.org Community: https://discuss.encode.io/c/uvicorn Requirements: Python 3.6+ (For P

Encode 6k Jan 03, 2023
An alternative serializer implementation for REST framework written in cython built for speed.

drf-turbo An alternative serializer implementation for REST framework written in cython built for speed. Free software: MIT license Documentation: htt

Mng 74 Dec 30, 2022
O SnakeG é um WSGI feito para suprir necessidadades de perfomance e segurança.

SnakeG O SnakeG é um WSGI feito para suprir necessidadades de perfomance e segurança. Veja o que o SnakeG possui: Multiprocessamento de requisições HT

Jaedson Silva 1 Jul 02, 2022
Restful API framework wrapped around MongoEngine

Flask-MongoRest A Restful API framework wrapped around MongoEngine. Setup from flask import Flask from flask_mongoengine import MongoEngine from flask

Close 525 Jan 01, 2023
CherryPy is a pythonic, object-oriented HTTP framework. https://docs.cherrypy.org/

Welcome to the GitHub repository of CherryPy! CherryPy is a pythonic, object-oriented HTTP framework. It allows building web applications in much the

CherryPy 1.6k Dec 29, 2022
FastAPI framework, high performance, easy to learn, fast to code, ready for production

FastAPI framework, high performance, easy to learn, fast to code, ready for production Documentation: https://fastapi.tiangolo.com Source Code: https:

Sebastián Ramírez 53k Jan 02, 2023
Lemon is an async and lightweight API framework for python

Lemon is an async and lightweight API framework for python . Inspired by Koa and Sanic .

Joway 29 Nov 20, 2022
An effective, simple, and async security library for the Sanic framework.

Sanic Security An effective, simple, and async security library for the Sanic framework. Table of Contents About the Project Getting Started Prerequis

Sunset Dev 72 Nov 30, 2022
A comprehensive reference for all topics related to building and maintaining microservices

This pandect (πανδέκτης is Ancient Greek for encyclopedia) was created to help you find and understand almost anything related to Microservices that i

Ivan Bilan 64 Dec 09, 2022
Pulumi-checkly - Checkly Pulumi Provider With Python

🚨 This project is still in very early stages and is not stable, use at your own

Checkly 16 Dec 15, 2022
You can use the mvc pattern in your flask application using this extension.

You can use the mvc pattern in your flask application using this extension. Installation Run the follow command to install mvc_flask: $ pip install mv

Marcus Pereira 37 Dec 17, 2022
Developer centric, performant and extensible Python ASGI framework

Introduction xpresso is an ASGI web framework built on top of Starlette, Pydantic and di, with heavy inspiration from FastAPI. Some of the standout fe

Adrian Garcia Badaracco 119 Dec 27, 2022
A Flask API REST to access words' definition

A Flask API to access words' definitions

Pablo Emídio S.S 9 Jul 22, 2022
Python AsyncIO data API to manage billions of resources

Introduction Please read the detailed docs This is the working project of the next generation Guillotina server based on asyncio. Dependencies Python

Plone Foundation 183 Nov 15, 2022