A very simple asynchronous wrapper that allows you to get access to the Oracle database in asyncio programs.

Overview

cx_Oracle_async

fury licence pyversions Publish Build Docs

A very simple asynchronous wrapper that allows you to get access to the Oracle database in asyncio programs.

Easy to use , buy may not the best practice for efficiency concern.

Requirements

  • cx_Oracle >= 8.1.0 (Take into consideration that author of cx_Oracle said he's trying to implement asyncio support , APIs maybe change in future version. Switch to 8.1.0 if there's something wrong makes it not gonna work.)
  • ThreadPoolExecutorPlus >= 0.2.0

Install

pip install cx_Oracle_async

Feature

  • Nearly all the same as aiomysql in asynchronous operational approach , with limited cx_Oracle feature support.
  • No automaticly date format transition built-in.
  • AQ feature added , check docs here for further information.
  • You can modify some of the connection properties simply like you're using cx_Oracle.
  • You can do basic insert / select / delete etc.
  • If you're connecting to database which is on a different machine from python process , you need to install oracle client module in order to use this library. Check cx-Oracle's installation guide for further information.

Documentation

https://cx_oracle_async.readthedocs.io

Performance

query type asynchronous multithreading synchronous multithreading synchronous single thread
fast single line query 6259.80 q/s 28906.93 q/s 14805.61 q/s
single line insertion 1341.88 q/s 1898 q/s 1685.17 q/s

/* Test platform: */
AMD Ryzen 3700x
Windows 10 LTSC
Oracle 19c
You can find performance test codes here.

Examples

Before running examples , make sure you've already installed a Oracle Client on your machine.

# basic_usages.py
import asyncio
import cx_Oracle_async

async def main():
    oracle_pool = await cx_Oracle_async.create_pool(
        host='localhost', 
        port='1521',
        user='user', 
        password='password',
        service_name='orcl', 
        min = 2,
        max = 4,
    )

    async with oracle_pool.acquire() as connection:
        async with connection.cursor() as cursor:
            await cursor.execute("SELECT * FROM V$SESSION")
            print(await cursor.fetchall())

    await oracle_pool.close()

if __name__ == '__main__':
    asyncio.run(main())
Comments
  • Default cx_Oracle parameters

    Default cx_Oracle parameters

    Hello,

    1. Sometimes I have got cx_Oracle.DatabaseError: ORA-24422 I want to use FORCE parameter of SessionPool.close(force=False) for to make it true. https://cx-oracle.readthedocs.io/en/latest/api_manual/session_pool.html Could you please add this option.

    2. cx_Oracle.SessionPool defaults is encoding=None, nencoding=None Could you please make it as original values.

    bug 
    opened by sinys15 16
  • Support connection with dsn

    Support connection with dsn

    Could you please add create_pool function with dsn parameter. And it looks better to use cx_Oracle.makedsn to build dsn string instead of your variant "f"{host}:{port}/{db}"". Example from documentation: dsn = cx_Oracle.makedsn("dbhost.example.com", 1521, service_name="orclpdb1") connection = cx_Oracle.connect("hr", userpwd, dsn, encoding="UTF-8")

    opened by sinys15 11
  • Support Rollback and Ping

    Support Rollback and Ping

    Could you please support https://cx-oracle.readthedocs.io/en/latest/api_manual/connection.html Connection.ping() and Connection.rollback() methods. This is must have.

    Connection.module Connection.action Connection.client_identifier Connection.clientinfo Would be very nice too.

    Thank you.

    enhancement 
    opened by sinys15 8
  • Async Close procedure for Oracle pool

    Async Close procedure for Oracle pool

    Seems like the Close method of Oracle pool is missed for async. I'm not shure is it just

        async def close_connection_async(self) -> None:
            if self._connection:
                await self._connection.close()
    

    or should be like

        async def close_connection_async(self) -> None:
            if self._connection:
                self._connection.close()
                await self._connection.wait_closed()
    
    opened by sinys15 4
  • Poetry SolverProblemError

    Poetry SolverProblemError

    When I add cx-Oracle-async by Poetry and call "poetry show" command I got the next error: mydir>poetry show

    SolverProblemError

    Because myproject depends on cx-Oracle-async (^0.1.3) which doesn't match any versions, version solving failed.

    at c:\users\myuser\appdata\local\programs\python\python39-32\lib\site-packages\poetry\puzzle\solver.py:241 in _solve 237│ packages = result.packages 238│ except OverrideNeeded as e: 239│ return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest) 240│ except SolveFailure as e: → 241│ raise SolverProblemError(e) 242│ 243│ results = dict( 244│ depth_first_search( 245│ PackageNode(self._package, packages), aggregate_package_nodes

    opened by sinys15 1
  • AsyncCursorWarper add var

    AsyncCursorWarper add var

    Please add class AsyncCursorWarper: ... async def var(self, args): return await self._loop.run_in_executor(self._thread_pool , self._cursor.var, args)

    And why "Warper", may be "Wrapper"?

    enhancement 
    opened by sinys15 0
  • Consider using the new python-oracledb driver

    Consider using the new python-oracledb driver

    Hi, while I was wrangling with cx_Oracle and my M1, I found out that Oracle released a new way to connect to their database systems: https://oracle.github.io/python-oracledb/

    Maybe that will make it easier to write the async functionality due to better integration with Python and the THIN approach.

    opened by ESRE-dev 0
  • callproc keyword_parameters needs functools.partial

    callproc keyword_parameters needs functools.partial

    To prevent this error:

    File /path/to/cx_Oracle_async/cursors.py:45, in AsyncCursorWrapper.callproc(self, *args, **kwargs)
         44 async def callproc(self, *args , **kwargs):
    ---> 45     return await self._loop.run_in_executor(self._thread_pool , self._cursor.callproc, *args , **kwargs)
    
    TypeError: BaseEventLoop.run_in_executor() got an unexpected keyword argument 'keyword_parameters'
    
    opened by Jaza 0
  • Missing cursor attribute 'description'

    Missing cursor attribute 'description'

    Other python database packages (including cx_Oracle) include a method to get the column names which involves using cursor.description. It looks like the object 'description' is not available for cx_Oracle_async for the AsyncCursorWrapper. Is there a different method for returning the column names when using this package?

    opened by bbaker1229 1
  • When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs

    When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs

    Describe the bug When the number of async tasks awaiting acquire reaches ThreadPoolExecutorPlus max_workers, this program hangs.

    To Reproduce Run the following with the sixth line of code commented out (as-is) to hang. Run the following with the sixth line of code uncommented to run successfully. (sixth line has the # toggle commenting... comment on it) (ThreadPoolExecutorPlus DEFAULT_MAXIMUM_WORKER_NUM is 32 * num-cores for Linux)

    import cx_Oracle_async
    import asyncio
    
    dbspecs = dict(dsn="WFX742D", user="ars", password="ars")
    
    max_pool_cnxs = 5
    num_acquire_tasks = 32 * os.cpu_count() + max_pool_cnxs
    
    #num_acquire_tasks -= 1 # toggle commenting of this line to hang/run
    queue_len = 0
    
    async def acquire_conn(pool, i):
       global queue_len
       queue_len += 1
       print_pool_info(f't-{i} acquire...', pool)
       conn = await pool.acquire()
       print_pool_info(f't-{i} postacquire', pool)
       queue_len -= 1
       conn._conn.begin()
       cursor = await conn.cursor()
       await cursor.execute('select current_timestamp from dual', [])
       rows = await cursor.fetchall()
       print(f"t-{i} rows: {len(rows)}")
       await conn.commit()
       await pool.release(conn)
    
    async def test_many_threads():
       pool = await cx_Oracle_async.create_pool(**dbspecs, max=max_pool_cnxs)
       print_pool_info('init', pool)
       tasks = [];
       for i in range(0, num_acquire_tasks):
          tasks.append(asyncio.create_task(acquire_conn(pool, i), name=f'task-{i}'))
       await asyncio.wait(tasks)
       print_pool_info('done', pool)
    
    def print_pool_info(action, pool):
       ipool = pool._pool
       print(f'{action} (queued {queue_len}, busy {ipool.busy}/{ipool.max})')
    
    asyncio.run(test_many_threads())
    

    Expected behavior I expected the queries to be queued and run when connections became available.

    Platform:

    • OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
    • Oracle version: 11.2.0.4
    • Python version 3.8.11

    Additional context I worked around this by added a semaphore in my code to limit the number of acquires to the max cnxs. What is the benefit of allowing more threads than cnxs?

    bug 
    opened by ericwaldheim 1
  • Cursor context manager is missing __aexit__

    Cursor context manager is missing __aexit__

    Describe the bug Potential memory leak in cursors context manager.

    To Reproduce Call many times cursor.fetchone().

    Gist with sample code: https://gist.github.com/ilosamart/272a7dadb639f19fd62b7947ae12ab5f It contains three versions of the same method: one async (this lib), one async (this lib) which explicitly closes the cursor, one with pure cx_oracle.

    Expected behavior I expected that the cursor context manager closed the cursor.

    Platform:

    • OS: Ubuntu 20.04
    • Oracle version: 12.2
    • Python version 3.8

    Additional context I used the instructions on https://oracle.github.io/odpi/doc/user_guide/debugging.html to debug memory alloc inside OCI.

    bug 
    opened by ilosamart 1
Releases(0.3.3)
Owner
Data Science, Full Stack Developer.
Endpoints is a lightweight REST api framework written in python and used in multiple production systems that handle millions of requests daily.

Endpoints Quickest API builder in the West! Endpoints is a lightweight REST api framework written in python and used in multiple production systems th

Jay Marcyes 30 Mar 05, 2022
A python application to log QSOs directly to QRZ.com from the command line

qrzlogger This script is a QRZ.com command line QSO logger. It does the following: asks the user for a call sign displays available call sign info pul

Michael Clemens 15 Jul 16, 2021
Daniel Vaz Gaspar 4k Jan 08, 2023
The source code to the Midnight project

MidnightSniper Started: 24/08/2021 Ended: 24/10/2021 What? This is the source code to a project developed to snipe minecraft names Why release? The ad

Kami 2 Dec 03, 2021
🔥 Fire up your API with this flamethrower

🔥 Fire up your API. Documentation: https://flama.perdy.io Flama Flama aims to bring a layer on top of Starlette to provide an easy to learn and fast

José Antonio Perdiguero 216 Dec 26, 2022
The comprehensive WSGI web application library.

Werkzeug werkzeug German noun: "tool". Etymology: werk ("work"), zeug ("stuff") Werkzeug is a comprehensive WSGI web application library. It began as

The Pallets Projects 6.2k Jan 01, 2023
Restful API framework wrapped around MongoEngine

Flask-MongoRest A Restful API framework wrapped around MongoEngine. Setup from flask import Flask from flask_mongoengine import MongoEngine from flask

Close 525 Jan 01, 2023
Web3.py plugin for using Flashbots' bundle APIs

This library works by injecting a new module in the Web3.py instance, which allows submitting "bundles" of transactions directly to miners. This is do

Flashbots 293 Dec 31, 2022
A Flask API REST to access words' definition

A Flask API to access words' definitions

Pablo Emídio S.S 9 Jul 22, 2022
The Python micro framework for building web applications.

Flask Flask is a lightweight WSGI web application framework. It is designed to make getting started quick and easy, with the ability to scale up to co

The Pallets Projects 61.5k Jan 06, 2023
A public API written in Python using the Flask web framework to determine the direction of a road sign using AI

python-public-API This repository is a public API for solving the problem of the final of the AIIJC competition. The task is to create an AI for the c

Lev 1 Nov 08, 2021
A simple todo app using flask and sqlachemy

TODO app This is a simple TODO app made using Flask. Packages used: DoodleCSS Special thanks to Chris McCormick (@mccrmx) :) Flask Flask-SQLAlchemy Fl

Lenin 1 Dec 26, 2021
Mini Web Framework on MicroPython (Esp8266)

dupgee Dupgee is a mini web framework developed for micro-python(Tested on esp8266). Installation pip install dupgee Create Project dupgee create newp

ahmet kotan 38 Jul 25, 2022
Official mirror of https://gitlab.com/pgjones/quart

Quart Quart is an async Python web microframework. Using Quart you can, render and serve HTML templates, write (RESTful) JSON APIs, serve WebSockets,

Phil Jones 2 Oct 05, 2022
Library for building WebSocket servers and clients in Python

What is websockets? websockets is a library for building WebSocket servers and clients in Python with a focus on correctness and simplicity. Built on

Aymeric Augustin 4.3k Dec 31, 2022
Sierra is a lightweight Python framework for building and integrating web applications

A lightweight Python framework for building and Integrating Web Applications. Sierra is a Python3 library for building and integrating web applications with HTML and CSS using simple enough syntax. Y

83 Sep 23, 2022
Loan qualifier app - Loan Qualifier Application Built With Python

Loan Qualifier Application This program is designed to automate the discovery pr

Phil Hills 1 Jan 04, 2022
Flask-Potion is a RESTful API framework for Flask and SQLAlchemy, Peewee or MongoEngine

Flask-Potion Description Flask-Potion is a powerful Flask extension for building RESTful JSON APIs. Potion features include validation, model resource

DTU Biosustain 491 Dec 08, 2022
🦍 The Cloud-Native API Gateway

Kong or Kong API Gateway is a cloud-native, platform-agnostic, scalable API Gateway distinguished for its high performance and extensibility via plugi

Kong 33.8k Jan 09, 2023
Async Python 3.6+ web server/framework | Build fast. Run fast.

Sanic | Build fast. Run fast. Build Docs Package Support Stats Sanic is a Python 3.6+ web server and web framework that's written to go fast. It allow

Sanic Community Organization 16.7k Dec 28, 2022