Seamlessly Connecting Notion Database with Python Pandas DataFrame

Overview

notion-df: Seamlessly Connecting Notion Database with Pandas DataFrame

Please Note: This project is currently in pre-alpha stage. The code are not appropriately documented and tested. Please report any issues you find.

Installation

pip install notion-df

Usage

  • Before starting, please follow the instructions to create a new integration and add it to your Notion page or database.

    • We'll refer Internal Integration Token as the api_key below.
  • Pandas-flavored APIs: Just need to add two additional lines of code:

    import notion_df
    notion_df.pandas() #That's it!
    
    import pandas as pd
    df = pd.read_notion(page_url, api_key=api_key)
    df.to_notion(page_url)
  • Download your Notion table as a pandas DataFrame

    import notion_df
    df = notion_df.load(notion_database_url, api_key=api_key)
    # Equivalent to: df = pd.read_notion(notion_database_url, api_key=api_key)
    df.head()
  • Append a local df to a Notion database:

    import notion_df
    notion_df.upload(df, notion_database_url, title="page-title", api_key=api_key)
    # Equivalent to: df.to_notion(notion_database_url, title="page-title", api_key=api_key)
  • Upload a local df to a newly created database in a Notion page:

    import notion_df
    notion_df.upload(df, notion_page_url, title="page-title", api_key=api_key)
    # Equivalent to: df.to_notion(notion_page_url, title="page-title", api_key=api_key)
  • Tired of typing api_key=api_key each time?

    import notion_df
    notion_df.config(api_key=api_key) # Or set an environment variable `NOTION_API_KEY`
    df = notion_df.load(notion_database_url)
    notion_df.upload(df, notion_page_url, title="page-title")
    # Similarly in pandas APIs: df.to_notion(notion_page_url, title="page-title")

TODOs

  • Add tests for
    • load
    • upload
    • values.py
    • configs.py
    • base.py
  • Better class organizations/namings for *Configs and *Values
Comments
  • Upload df with children

    Upload df with children

    This PR allows uploading page content (aka children) alongside the dataframe.

    How to use it?

    import pandas as pd 
    import notion_df
    from notion_df.blocks import *
    
    child1 = ParagraphBlock(paragraph={"rich_text": RichTextObject.encode_string("A")},)
    child2 = ParagraphBlock(paragraph={"rich_text": RichTextObject.encode_string("B")},)
    
    notion_df.upload(
        df = pd.DataFrame([{"Name": "A"}, {"Name": "B"}]), 
        notion_url = "<>", 
        children=[child1, [child2]], #<- this is the update
        client=notion
    )
    

    Note: Given the experimental nature of this function, we only put it in the upload function but not the pd.to_notion API. We will update when it is appropriately tested.

    opened by lolipopshock 0
  • The tables df columns and schema might not be always consistent

    The tables df columns and schema might not be always consistent

    For example, if you have a relation column inside a table, which points to an relation that the Notion integration doesn't have access to, then

    1. the downloaded data will contain that column
    2. the downloaded schema will not contain that column
    opened by lolipopshock 0
  • Get Page ID?

    Get Page ID?

    Is it possible to get the page id of a database item into the data frame? This way I can map relations without setting resolve_relation_values=True and I won't have a problem with relation values that are named the same.

    opened by thielem 0
  • Error when resolve_relation_values=True

    Error when resolve_relation_values=True

    Hey! In contrast to #26 , I can see rollups an relations when resolve_relation_values =False. When I set resolve_relation_values=True, I get the following error:

    Traceback (most recent call last):
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\indexes\base.py", line 3621, in get_loc
        return self._engine.get_loc(casted_key)
      File "pandas\_libs\index.pyx", line 136, in pandas._libs.index.IndexEngine.get_loc
      File "pandas\_libs\index.pyx", line 163, in pandas._libs.index.IndexEngine.get_loc
      File "pandas\_libs\hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
      File "pandas\_libs\hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
    KeyError: 'Beschreibung'
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "a:\xxx\notion_fin_test.py", line 10, in <module>
        df = notion_df.download(page_url, resolve_relation_values=True)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\notion_df\agent.py", line 54, in wrapper
        out = func(client=client, *args, **kwargs)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\notion_df\agent.py", line 216, in download
        relation_df.notion_ids, relation_df[rel_title_col]
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\frame.py", line 3505, in __getitem__
        indexer = self.columns.get_loc(key)
      File "C:\Users\m_thi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pandas\core\indexes\base.py", line 3623, in get_loc
        raise KeyError(key) from err
    KeyError: 'Beschreibung'
    
    opened by thielem 1
  • get user information from database

    get user information from database

    When I try to read a databse from notion, the column that shows the name of the mate assigned to this task is shown as a code but not the name. But the property last edited shows the name. Someone knows what is happening ?

    Thanks

    opened by AlmendrosCarmona 0
  • Uploading pages url issue

    Uploading pages url issue

    I like this package and I am starting to use it to automate some of my note taking. I noticed if I want to add a page into database, only full database url works but not the short version database ID. For downloading, both short ID and full url works. This is a minor issue. I mentioned it just in case you want to fix it.

    Thank you! Eric

    opened by yygou 1
Releases(v0.0.5)
  • v0.0.5(Feb 22, 2022)

    What's Changed

    • Better rich text by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/21

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.4...v0.0.5

    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Feb 12, 2022)

    What's Changed

    • Add rollup by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/15
    • Enable relation resolution by @lolipopshock in https://github.com/lolipopshock/notion-df/pull/16
    • Better string length checking #9

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.3...v0.0.4

    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Feb 1, 2022)

  • v0.0.2(Jan 10, 2022)

    • Better validation for Select Option names #1
    • Allow ignoring errors for uploading #2
    • Handle pagination when downloading #3

    Full Changelog: https://github.com/lolipopshock/notion-df/compare/v0.0.1...v0.0.2

    Source code(tar.gz)
    Source code(zip)
  • v0.0.1(Jan 6, 2022)

    This release implements the basic function of the notion_df library.

    Usage

    • Before starting, please follow the instructions to create a new integration and add it to your Notion page or database.

      • We'll refer Internal Integration Token as the api_key below.
    • Pandas-flavored APIs: Just need to add two additional lines of code:

      import notion_df
      notion_df.pandas() #That's it!
      
      import pandas as pd
      df = pd.read_notion(page_url, api_key=api_key)
      df.to_notion(page_url)
      
    • Download your Notion table as a pandas DataFrame

      import notion_df
      df = notion_df.load(notion_database_url, api_key=api_key)
      # Equivalent to: df = pd.read_notion(notion_database_url, api_key=api_key)
      df.head()
      
    • Append a local df to a Notion database:

      import notion_df
      notion_df.upload(df, notion_database_url, title="page-title", api_key=api_key)
      # Equivalent to: df.to_notion(notion_database_url, title="page-title", api_key=api_key)
      
    • Upload a local df to a newly created database in a Notion page:

      import notion_df
      notion_df.upload(df, notion_page_url, title="page-title", api_key=api_key)
      # Equivalent to: df.to_notion(notion_page_url, title="page-title", api_key=api_key)
      
    • Tired of typing api_key=api_key each time?

      import notion_df
      notion_df.config(api_key=api_key) # Or set an environment variable `NOTION_API_KEY`
      df = notion_df.load(notion_database_url)
      notion_df.upload(df, notion_page_url, title="page-title")
      # Similarly in pandas APIs: df.to_notion(notion_page_url, title="page-title")
      

    Full Changelog: https://github.com/lolipopshock/notion-df/commits/v0.0.1

    Source code(tar.gz)
    Source code(zip)
A Telegram bot to transcribe audio, video and image into text.

Transcriber Bot A Telegram bot to transcribe audio, video and image into text. Deploy to Heroku Local Deploying Install the FFmpeg. Make sure you have

10 Dec 19, 2022
Anchor Protocol Script that can save you from being liquidated!

Why My day job requires a fairly good amount of automation from time to time. Besides, I do like computers to work on what I cannot while I'm sleeping

126 Oct 16, 2022
Discord Token Generator of a project - Some stupids ppl are trying to leak it so i'm leaking faster :)

Original creator: Rolf (dort) HCaptcha Bypasser: h0nde Shark.Solar Discord Token Generator of a project - Some stupids ppl are trying to leak it so i'

Stanley 14 Sep 29, 2021
FAIR Enough Metrics is an API for various FAIR Metrics Tests, written in python

☑️ FAIR Enough metrics for research FAIR Enough Metrics is an API for various FAIR Metrics Tests, written in python, conforming to the specifications

Maastricht University IDS 3 Jul 06, 2022
471 Dec 24, 2022
Discord bot for calculating basic operations and formulas. (Early Development)

MathBot Discord bot for calculating basic operations and formulas. (Early Development) Commits Feel free to contribute to this bot by forking and pull

4 Jul 14, 2022
scrape tiktok/douyin video list from specific user or keyword

get-tiktok-user-video-list scrape tiktok/douyin video list from specific user or keyword 以**https://www.douyin.com/user/MS4wLjABAAAAUpIowEL3ygUAahQB47

wanghaisheng 4 Jul 06, 2022
Python library for Spurwing API to schedule appointments, manage calendars and custom integrations.

Spurwing API Python Library Lightweight Python library for Spurwing's API. Spurwing's API makes it easy to add robust scheduling and booking to your a

Spurwing 1 Jul 14, 2021
Isobot is originally made by notsniped. This is a remix of iso.bot by archisha.

iso6.9-08122021b-1.2beta Isobot is originally made by notsniped#0002. This is a remix of iso.bot by αrchιshα#5518. isobot6.9 is a Discord bot written

Kamilla Youver 3 Jan 11, 2022
It's a discord.py simulator.

DiscordPySimulator It's a discord.py simulator. ⚠️ Things to fix Context As you may know, discord py commands provide the context as the first paramet

Juan Sebastián 11 Oct 24, 2022
An API Client package to access the APIs for NBA.com

nba_api An API Client package to access the APIs for NBA.com Development Version: v1.1.9 nba_api is an API Client for www.nba.com. This package is mea

Swar Patel 1.4k Jan 01, 2023
Powerful spammer bots for telegram made with python and telethon.

Powerful spammer bots for telegram made with python and telethon. We can deploy upto 70 bots at a time.

32 Dec 15, 2022
Thread-safe Python RabbitMQ Client & Management library

AMQPStorm Thread-safe Python RabbitMQ Client & Management library. Introduction AMQPStorm is a library designed to be consistent, stable and thread-sa

Erik Olof Gunnar Andersson 167 Nov 20, 2022
Source code for "Efficient Training of BERT by Progressively Stacking"

Introduction This repository is the code to reproduce the result of Efficient Training of BERT by Progressively Stacking. The code is based on Fairseq

Gong Linyuan 101 Dec 02, 2022
Instant messaging client in tkinter

Concord_client_tk Instant messaging client in tkinter Contributors : Ilade-s [https://github.com/Ilade-s] Doku [https://github.com/D0kuhebi] Descripti

Raphaël Merlet 2 Jun 15, 2022
👾 Telegram Smart Group Assistant 🤖

DarkHelper 🌖 Features ⚡️ Smart anti-apam & anti-NFSW message checker Tag Members , Entertain facility , Welcommer ban , unban , mute , unmute , lock

amirali rajabi 38 Dec 18, 2022
A mass creator for Discord's new channel threads.

discord-thread-flooder A mass creator for Discord's new channel threads. (obv created by https://github.com/imvast) Warning: this may lag ur pc if u h

Vast 6 Nov 04, 2022
💬 Send iMessages using Python through the Shortcuts app.

py-imessage-shortcuts Send iMessages using Python through the Shortcuts app. Requires macOS Monterey (macOS 12) or later. Compatible with Apple Silico

Kevin Schaich 10 Nov 30, 2022
A.I and game for gomoku, working only on windows

Gomoku (A.I of gomoku) The goal of the project is to create an artificial intelligence of gomoku. Goals Beat the opponent. Requirements Python 3.7+ Wo

Luis Rosario 13 Jun 20, 2021
Connects to a local SenseCap M1 Helium Hotspot and pulls API Data.

sensecap_api_checker_HELIUM Connects to a local SenseCap M1 Helium Hotspot and pulls API Data.

Lorentz Factr 1 Nov 03, 2021