The elegance of Airflow + the power of AWS

Overview

Orkestra

The elegance of Airflow + the power of AWS

Docs Codecov

PyPI PyPI - Downloads PyPI - License PyPI - Python Version GitHub issues Mentioned in Awesome CDK

examples/hello_orkestra.py

list: logger.info(item.dict()) return [item.dict()] * 10 @compose(model=Item, is_map_job=True, **default_args) def double_price(item: Item, context): item.price = item.price * 2 return item.dict() @compose(**default_args) def assert_false(event, context): assert False @compose(**default_args) def do_nothing(event, context): logger.info({"doing": "nothing"}) @compose(**default_args) def say_hello(event, context): return "hello, world" @compose(**default_args) def say_goodbye(event, context): return "goodbye" @compose(**default_args) def random_int(event, context): return random.randrange(100) @compose(**default_args) def random_float(event, context): return float(random_int(event, context)) dag() ">
import random
from typing import *
from uuid import uuid4

from aws_lambda_powertools import Logger, Tracer
from pydantic import BaseModel

from orkestra import compose
from orkestra.interfaces import Duration


def dag():
    (
        generate_item
        >> add_price
        >> copy_item
        >> double_price
        >> (do_nothing, assert_false)
        >> say_hello
        >> [random_int, random_float]
        >> say_goodbye
    )


class Item(BaseModel):
    id: str
    name: str
    price: Optional[float] = None

    @classmethod
    def random(cls):
        return cls(
            id=str(uuid4()),
            name=random.choice(
                [
                    "potato",
                    "moon rock",
                    "hat",
                ]
            ),
        )


logger = Logger()

tracer = Tracer()


default_args = dict(
    enable_powertools=True,
    timeout=Duration.seconds(6),
)


@compose(**default_args)
def generate_item(event, context):
    logger.info("generating random item")
    item = Item.random().dict()
    logger.info(item)
    tracer.put_metadata("GenerateItem", "SUCCESS")
    return item


@compose(model=Item, **default_args)
def add_price(item: Item, context):
    price = 3.14
    logger.info(
        "adding price to item",
        extra={
            "item": item.dict(),
            "price": price,
        },
    )
    item.price = price
    return item.dict()


@compose(model=Item, **default_args)
def copy_item(item: Item, context) -> list:
    logger.info(item.dict())
    return [item.dict()] * 10


@compose(model=Item, is_map_job=True, **default_args)
def double_price(item: Item, context):
    item.price = item.price * 2
    return item.dict()


@compose(**default_args)
def assert_false(event, context):
    assert False


@compose(**default_args)
def do_nothing(event, context):
    logger.info({"doing": "nothing"})


@compose(**default_args)
def say_hello(event, context):
    return "hello, world"


@compose(**default_args)
def say_goodbye(event, context):
    return "goodbye"


@compose(**default_args)
def random_int(event, context):
    return random.randrange(100)


@compose(**default_args)
def random_float(event, context):
    return float(random_int(event, context))


dag()

app.py

#!/usr/bin/env python3
from aws_cdk import core as cdk

from examples.hello_orkestra import generate_item


class HelloOrkestra(cdk.Stack):
    def __init__(self, scope, id, **kwargs):

        super().__init__(scope, id, **kwargs)

        generate_item.schedule(
            self,
            expression="rate(5 minutes)",
            state_machine_name="hello_orkestra",
        )


app = cdk.App()


app.synth()

state machine

Owner
Stephan Fitzpatrick
“Talk is cheap. Show me the code.”
Stephan Fitzpatrick
Python function to construct an ODS spreadsheet on the fly - without having to store the entire file in memory or disk

stream-write-ods Python function to construct an ODS (OpenDocument Spreadsheet) on the fly - without having to store the entire file in memory or disk

Department for International Trade 1 Oct 09, 2022
Python wrapper for Revolt API

defectio is a direct implementation of the entire Revolt API and provides a way to authenticate and start communicating with Revolt servers. Similar interface to discord.py

Leon Bowie 26 Sep 18, 2022
A Telegram bot that can stream Telegram files to users over HTTP.

T.ME_FILE_TO_LINK Hi iam a file to link bot....best Bot telegram Telegram File To Link Generation Bot A Telegram bot that can stream Telegram files to

1 Oct 24, 2021
Weee - Advanced project's versions bumper

Weee - Advanced project's versions bumper

Yan Kurbatov 2 Jun 06, 2022
DIAL(Did I Alert Lambda?) is a centralised security misconfiguration detection framework which completely runs on AWS Managed services like AWS API Gateway, AWS Event Bridge & AWS Lambda

DIAL(Did I Alert Lambda?) is a centralised security misconfiguration detection framework which completely runs on AWS Managed services like AWS API Gateway, AWS Event Bridge & AWS Lambda

CRED 71 Dec 29, 2022
Advance Anonymous Sender bot with Caption Editor

AnonyMous Sender 👨‍💻 Advanced Anonymous Sender with Caption Editor Join @DaisySupport_Official 🎵 for help Features Get forwarded messages without f

Inuka Asith 13 Oct 09, 2022
A Python Library to interface with Tumblr v2 REST API & OAuth

Tumblpy Tumblpy is a Python library to help interface with Tumblr v2 REST API & OAuth Features Retrieve user information and blog information Common T

Mike Helmick 125 Jun 20, 2022
A high level library for building Discord bots.

Qord A high level library for building Discord bots. 🚧 This library is currently in development. Questions that you are having What is this? This is

Izhar Ahmad 16 May 14, 2022
A simple, fast, and awesome discord nuke bot! The only thing you need to add is your bot token.

SimpleNukeBot A simple, fast, and awesome discord nuke bot! The only thing you need to add is your bot token. Instructions: All you need to do is crea

Bisc 1 Apr 18, 2022
Discord bot script for sending multiple media files to a discord channel according to discord limitations.

Discord Bulk Image Sending Bot Send bulk images to Discord channel. This is a bot script that will allow you to send multiple images to Discord channe

Nikola Arbov 1 Jan 13, 2022
NiceHash Python Library and Command Line Rest API

NiceHash Python Library and Command Line Rest API Requirements / Modules pip install requests Required data and where to get it Following data is nee

Ashlin Darius Govindasamy 2 Jan 02, 2022
Mass Instagram Checker

Mass Instagram Checker

X - MrG3P5 5 Nov 09, 2022
Flask-SQLAlchemy API for daisuki-web

💟 Anime Daisuki! API API de animes com cadastro de usuários. O usuário autenticado pode avaliar e favoritar animes, comentar episódios e verificar o

Paulo Thor 1 Nov 04, 2021
streamlit translator is used to detect and translate between languages created using gTTS, googletrans, pillow and streamlit python packages

Streamlit Translator Streamlit Translator is a simple translator app to detect and translate between languages. Streamlit Translator gets text and lan

Siva Prakash 5 Apr 05, 2022
Most Powerful Chatbot On Telegram Bot

About Hello, I am Lycia [リュキア], An Intelligent ChatBot. If You Are Feeling Lonely, You can Always Come to me and Chat With Me! How To Host The easiest

RedAura 8 May 26, 2021
A discord bot to check if messages have the correct code formatting.

discord-code-formatter A discord bot to check if messages have the correct code formatting. This was a basic project to help me learn Python and learn

Nash Boisvert 1 Nov 23, 2021
Async wrapper over hentaichan.live

hentai-chan-api-async is a small asynchronous parser library that will allow you to easily use manga from https://hentaichan.live Recommended to use python3.7+

7 Dec 15, 2022
Telegram client written in GTK & Python

Meowgram GTK Telegram Client 🐱 Why Meogram? Meowgram = Meow + Gram :D Meow - Talking cats sound. It's a symbol of unique and user friendly UI of clie

Artem Prokop 71 May 04, 2022
Osmopy - osmo python client library

osmopy Version 0.0.2 Tools for Osmosis wallet management and offline transaction

5 May 22, 2022