A Fork of Gitlab's Permifrost tool for managing Snowflake Permissions

Overview

permifrost-fork

This is a fork of the GitLab permifrost project. As the GitLab team is not currently maintaining the project, we've taken on maintenace of the project to allow for bug fixes and development, while maintaining the spirit of the project that has been loved and used by many.

We welcome contributions, so please feel free to submit PRs or get in touch if you'd like to help in any way.

Installation

Install directly from Github using the following command:

pip install git+https://github.com/hightouchio/permifrost-fork.git

Once we have pypi setup for our first release, those instructions will be added here.

Usage

Use this command to check and manage the permissions of a Snowflake account.

permifrost run <spec_file> [--role] [--dry] [--diff] [--user] [--ignore-memberships]
#>permifrost run --help
Usage: permifrost run [OPTIONS] SPEC

  Grant the permissions provided in the provided specification file for
  specific users and roles

Options:
  --dry        Do not actually run, just check.
  --diff       Show full diff, both new and existing permissions.
  --role TEXT  Run grants for specific roles. Usage: --role testrole --role
               testrole2.

  --user TEXT  Run grants for specific users. Usage: --user testuser --user
               testuser2.

  --ignore-memberships  Do not handle role membership grants/revokes
  --help       Show this message and exit.

Given the parameters to connect to a Snowflake account and a YAML file (a "spec") representing the desired database configuration, this command makes sure that the configuration of that database matches the spec. If there are differences, it will return the sql grant and revoke commands required to make it match the spec. If there are additional permissions set in the database this command will create the necessary revoke commands with the exception of:

  • Object Ownership
  • Warehouse Privileges

Permifrost is heavily inspired by pgbedrock which can be used for managing the permissions in a Postgres database.

spec_file

The YAML specification file is used to define in a declarative way the databases, roles, users and warehouses in a Snowflake account, together with the permissions for databases, schemas and tables for the same account.

All permissions are abbreviated as read or write permissions, with Permifrost generating the proper grants for each type of object. This includes shared databases which have simpler and more limited permissions than non-shared databases.

Tables and views are listed under tables and handled properly behind the scenes.

If * is provided as the parameter for tables the grant statement will use the ALL <object_type>s in SCHEMA syntax. It will also grant to future tables and views. See Snowflake documenation for ON FUTURE

If a schema name includes an asterisk, such as snowplow_*, then all schemas that match this pattern will be included in the grant statement unless it is for ownership, in which case the asterisk is not supported. This can be coupled with the asterisk for table grants to grant permissions on all tables in all schemas that match the given pattern. This is useful for date-partitioned schemas.

All entities must be explicitly referenced. For example, if a permission is granted to a schema or table then the database must be explicitly referenced for permissioning as well. Additionally, role membership must be explicit in the config file. If a role does not have a member_of list, it will have all roles it currently has revoked.

Roles can accept "_" as a role name either alone or nested under the include key. There is optionally an exclude key that can be used if include is used. "_"will grant membership to all roles defined in the spec. Any roles defined inexcludewill be removed from the list defined ininclude.

A specification file has the following structure:

# Databases
databases:
    - db_name:
        shared: boolean
    - db_name:
        shared: boolean
        owner: role_name
    ... ... ...

# Roles
roles:
    - role_name:
        warehouses:
            - warehouse_name
            - warehouse_name
            ...

        member_of:
            - role_name
            - role_name
            ...

            # or

        member_of:
            include:
                - "*"
            exclude:
                - role_name

        privileges:
            databases:
                read:
                    - database_name
                    - database_name
                    ...
                write:
                    - database_name
                    - database_name
                    ...
            schemas:
                read:
                    - database_name.*
                    - database_name.schema_name
                    - database_name.schema_partial_*
                    ...
                write:
                    - database_name.*
                    - database_name.schema_name
                    - database_name.schema_partial_*
                    ...
            tables:
                read:
                    - database_name.*.*
                    - database_name.schema_name.*
                    - database_name.schema_partial_*.*
                    - database_name.schema_name.table_name
                    ...
                write:
                    - database_name.*.*
                    - database_name.schema_name.*
                    - database_name.schema_partial_*.*
                    - database_name.schema_name.table_name
                    ...

        owns:
            databases:
                - database_name
                ...
            schemas:
                - database_name.*
                - database_name.schema_name
                ...
            tables:
                - database_name.*.*
                - database_name.schema_name.*
                - database_name.schema_name.table_name
                ...

    - role_name:
        owner: role_name
    ... ... ...

# Users
users:
    - user_name:
        can_login: boolean
        member_of:
            - role_name
            ...
    - user_name:
        owner: role_name
    ... ... ...

# Warehouses
warehouses:
    - warehouse_name:
        size: x-small
    - warehouse_name:
        size: x-small
        owner: role_name
    ... ... ...

For a working example, you can check the Snowflake specification file that we are using for testing permifrost permissions.

Settings

All settings are declared here with their default values and are described below. These can be added to your spec.yaml file.

require-owner: false

require-owner: Set to true to force having to set the owner property on all objects defined.

--diff

When this flag is set, a full diff with both new and already granted commands is returned. Otherwise, only required commands for matching the definitions on the spec are returned.

--dry

When this flag is set, the permission queries generated are not actually sent to the server and run; They are just returned to the user for examining them and running them manually.

When this flag is not set, the commands will be executed on Snowflake and their status will be returned and shown on the command line.

Connection Parameters

The following environmental variables must be available to connect to Snowflake:

$PERMISSION_BOT_USER
$PERMISSION_BOT_ACCOUNT
$PERMISSION_BOT_WAREHOUSE

Username and Password

To connect using a username and password, also include the following:

$PERMISSION_BOT_PASSWORD
$PERMISSION_BOT_DATABASE
$PERMISSION_BOT_ROLE

Currently, Permifrost assumes you are using the SECURITYADMIN role and will fail validation if you are not.

OAuth

To connect using an OAuth token, also include the following:

$PERMISSION_BOT_OAUTH_TOKEN

Key Pair Authentication

Rather than supplying a password or an oauth token, it's possible to connect via Snowflake's Key Pair authentication by setting the following:

$PERMISSION_BOT_KEY_PATH
$PERMISSION_BOT_KEY_PASSPHRASE

See Snowflake-sqlalchemy for more info.

Contributing

Contributing to Permifrost is easy, and most commands to do so are available within the Makefile.

The easiest way to start developing is to run make permifrost, this will open a shell in a docker container with the local version of Permifrost installed. You can now make changes to the files in your editor and it will be reflected in the commands that you run from the docker shell.

For code checking, you can use make test, make lint,and make typecheck. See the Makefile for more details.

Owner
Hightouch
Hightouch
Bot for tenhou.net riichi mahjong server written in Python

[Tests coverage] Bot was tested with Python 3.7+ and PyPy3, we are not supporting Python 2. What do we have here? Mahjong hands calculation You can fi

190 Jan 06, 2023
Small cloudfoundry client implemented in python

Cloudfoundry python client The cf-python-client repo contains a Python client library for Cloud Foundry. Installing Supported versions warning: Starti

Cloud Foundry Community 50 Sep 08, 2022
A python script that changes our background based on current weather and time of the day.

Desktop background on Windows 10, based on current weather and time A python script that changes our background based on current weather and time of t

Maj Gaberšček 1 Nov 16, 2021
discord vc exploit to lightly lag vcs

discord-vc-reconnector discord vc exploit to lag vcs how to use open the py file, then open devtools on discord, go to network and join a vc, dont sta

Tesco 30 Aug 09, 2022
Python lib to control HottoH based stove devices

Project desciption This library can be used to discuss with HootoH based stove devices Actually tested and validated with a CMG Drum stove. To use thi

3 May 16, 2022
A modular Telegram group management bot running with Python based on Pyrogram.

A modular Telegram group management bot running with Python based on Pyrogram.

Jefanya Efandchris 1 Nov 14, 2022
This Instagram app created as a clone of instagram.Developed during Moringa Core.

Instagram This Instagram app created as a clone of instagram.Developed during Moringa Core. AUTHOR By: Nyagah Isaac Description This web-app allows a

Nyagah Isaac 1 Nov 01, 2021
An API wrapper around Discord API.

NeoCord This project is work in progress not for production use. An asynchronous API wrapper around Discord API written in Python. Features Modern API

Izhar Ahmad 14 Jan 03, 2022
A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources.

A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.

Donne Martin 11.1k Jan 04, 2023
Telegram music & video bot direct play music

Telegram music & video bot direct play music

noinoi-X 1 Dec 28, 2021
Creating a Python API, for the MakeMyTrip Flight Schedules.

MakeMyTripAPI Creating a Python API, for the MakeMyTrip Flight Schedules. Source: MakeMyTrip is an Indian online travel company founded in 2000. Headq

Aman Priyanshu 0 Jan 12, 2022
A Telegram UserBot to Play Radio in Voice Chats. This is also the source code of the userbot which is being used for playing Radio in @AsmSafone Channel.

Telegram Radio Player UserBot A Telegram UserBot to Play Radio in Channel or Group Voice Chats. This is also the source code of the userbot which is b

SAF ONE 44 Nov 12, 2022
AWS SQS event redrive Lambda With Python

AWS SQS event redrive Lambda This repository contains one simple AWS Lambda function in Python to redrive AWS SQS events from source queue to destinat

1 Oct 19, 2021
Discord Unverified Token Gen

Discord-Unverified-Token-Gen This is a token gen that was made in an hour and just generates unverified tokens, most will be locked. Usage: in cmd jus

Aran 2 Oct 23, 2022
A corona statistics and information telegram bot.

A corona statistics and information telegram bot.

Fayas Noushad 15 Oct 21, 2022
Python-based Snapchat score booster using pyautogui module

Snapchat Snapscore Botter Python-based Snapchat score booster using pyautogui module. Click here to report bugs. Usage Download ZIP here and extract t

477 Dec 31, 2022
Turns any script into a telegram bot

pytobot Turns any script into a telegram bot Install pip install --upgrade pytobot Usage Script: while True: message = input() if message == "

Dmitry Kotov 17 Jan 06, 2023
My attempt to reverse the Discord nitro token generation function.

discord-theory-I PART: I My attempt to reverse the Discord nitro token generation function. The Nitro generation tools thing is common in Discord now,

Jakom 29 Aug 14, 2022
A Slash Commands Discord Bot created using Pycord!

Hey, I am Slash Bot. A Bot which works with Slash Commands! Prerequisites Python 3+ Check out. the requirements.txt and install all the pakages. Insta

Saumya Patel 18 Nov 15, 2022
A lightweight, dependency-free Python library (and command-line utility) for downloading YouTube Videos.

24 July 2020 Actively soliciting contributers! Ping @ronncc if you would like to help out! pytube pytube is a very serious, lightweight, dependency-fr

pytube 7.9k Jan 02, 2023