Simple example of FastAPI + Celery + Triton for benchmarking

Overview

You can see the previous work from:

Benchmark FastAPI + Celery with / without Triton

with Triton Server

without Triton Server

Benchmark Results

See Benchmark Results

Preparation

1. Setup packages

Install Anaconda and execute the following commands:

$ make env        # create a conda environment (need only once)
$ source init.sh  # activate the env
$ make setup      # setup packages (need only once)

2. Train a CNN model (Recommended on GPU)

$ source create_model.sh

3. Check the model repository created

$ tree model_repository

model_repository
└── mnist_cnn
    ├── 1
    │   └── model.pt
    └── config.pbtxt

2 directories, 2 files

How to play

Server (Option 1 - On your Local)

Install Redis & Docker, and run the following commands:

$ make triton     # run triton server
$ make broker     # run redis broker
$ make worker     # run celery worker
$ make api        # run fastapi server
$ make dashboard  # run dashboard that monitors celery

Server (Option 2 - Docker Compose available on GPU devices)

Install Docker & Docker Compose, and run the following command:

$ docker-compose up

[Optional] Additional Triton Servers

You can start up additional Triton servers on other devices.

$ make triton

[Optional] Additional Workers

You can start up additional workers on other devices.

$ export BROKER_URL=redis://redis-broker-ip:6379    # default is localhost
$ export BACKEND_URL=redis://redis-backend-ip:6379  # default is localhost
$ export TRITON_SERVER_URL=http://triton-server-ip:9000   # default is localhost
$ make worker
  • NOTE: Worker needs to run on the machine which Triton runs on due to shared memory settings.

Dashboard for Celery (Flower)

http://0.0.0.0:5555/ image

Load Test (w/ Locust)

Execute Locust

$ make load  # for load test without Triton
or
$ make load-triton  # for  load test with Triton

Open http://0.0.0.0:8089

Type url for the API server.

Issue Handling

Redis Error 8 connecting localhost:6379. nodename nor servname provided, or not known.

$ ulimit -n 1024

Docker's network_mode=bridgedegrades the network performance.

We recommend to use Linux server if you would like to run docker-compose up.

For Developers

$ make setup-dev      # setup for developers
$ make format         # format scripts
$ make lint           # lints scripts
$ make utest          # runs unit tests
$ make cov            # opens unit test coverage information
Owner
Jinwoo Park (Curt)
A domain-independent problem-solver
Jinwoo Park (Curt)
🐍 Simple FastAPI template with factory pattern architecture

Description This is a minimalistic and extensible FastAPI template that incorporates factory pattern architecture with divisional folder structure. It

Redowan Delowar 551 Dec 24, 2022
FastAPI client generator

FastAPI-based API Client Generator Generate a mypy- and IDE-friendly API client from an OpenAPI spec. Sync and async interfaces are both available Com

David Montague 283 Jan 04, 2023
Mixer -- Is a fixtures replacement. Supported Django, Flask, SqlAlchemy and custom python objects.

The Mixer is a helper to generate instances of Django or SQLAlchemy models. It's useful for testing and fixture replacement. Fast and convenient test-

Kirill Klenov 871 Dec 25, 2022
Full stack, modern web application generator. Using FastAPI, PostgreSQL as database, Docker, automatic HTTPS and more.

Full Stack FastAPI and PostgreSQL - Base Project Generator Generate a backend and frontend stack using Python, including interactive API documentation

Sebastián Ramírez 10.8k Jan 08, 2023
Cube-CRUD is a simple example of a REST API CRUD in a context of rubik's cube review service.

Cube-CRUD is a simple example of a REST API CRUD in a context of rubik's cube review service. It uses Sqlalchemy ORM to manage the connection and database operations.

Sebastian Andrade 1 Dec 11, 2021
FastAPI native extension, easy and simple JWT auth

fastapi-jwt FastAPI native extension, easy and simple JWT auth

Konstantin Chernyshev 19 Dec 12, 2022
User authentication fastapi with python

user-authentication-fastapi Authentication API Development Setup environment You should create a virtual environment and activate it: virtualenv venv

Sabir Hussain 3 Mar 03, 2022
Basic fastapi blockchain - An api based blockchain with full functionality

Basic fastapi blockchain - An api based blockchain with full functionality

1 Nov 27, 2021
FastAPI IPyKernel Sandbox

FastAPI IPyKernel Sandbox This repository is a light-weight FastAPI project that is meant to provide a wrapper around IPyKernel interactions. It is in

Nick Wold 2 Oct 25, 2021
FastAPI + PeeWee = <3

FastAPIwee FastAPI + PeeWee = 3 Using Python = 3.6 🐍 Installation pip install FastAPIwee 🎉 Documentation Documentation can be found here: https://

16 Aug 30, 2022
Adds GraphQL support to your Flask application.

Flask-GraphQL Adds GraphQL support to your Flask application. Usage Just use the GraphQLView view from flask_graphql from flask import Flask from flas

GraphQL Python 1.3k Dec 31, 2022
A set of demo of deploying a Machine Learning Model in production using various methods

Machine Learning Model in Production This git is for those who have concern about serving your machine learning model to production. Overview The tuto

Vo Van Tu 53 Sep 14, 2022
Hook Slinger acts as a simple service that lets you send, retry, and manage event-triggered POST requests, aka webhooks

Hook Slinger acts as a simple service that lets you send, retry, and manage event-triggered POST requests, aka webhooks. It provides a fully self-contained docker image that is easy to orchestrate, m

Redowan Delowar 96 Jan 02, 2023
Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

Tomasz Wójcik 300 Dec 26, 2022
MS Graph API authentication example with Fast API

MS Graph API authentication example with Fast API What it is & does This is a simple python service/webapp, using FastAPI with server side rendering,

Andrew Hart 4 Aug 11, 2022
🚀 Cookiecutter Template for FastAPI + React Projects. Using PostgreSQL, SQLAlchemy, and Docker

FastAPI + React · A cookiecutter template for bootstrapping a FastAPI and React project using a modern stack. Features FastAPI (Python 3.8) JWT authen

Gabriel Abud 1.4k Jan 02, 2023
Hyperlinks for pydantic models

Hyperlinks for pydantic models In a typical web application relationships between resources are modeled by primary and foreign keys in a database (int

Jaakko Moisio 10 Apr 18, 2022
A dynamic FastAPI router that automatically creates CRUD routes for your models

⚡ Create CRUD routes with lighting speed ⚡ A dynamic FastAPI router that automatically creates CRUD routes for your models Documentation: https://fast

Adam Watkins 943 Jan 01, 2023
FastAPI interesting concepts.

fastapi_related_stuffs FastAPI interesting concepts. FastAPI version :- 0.70 Python3 version :- 3.9.x Steps Test Django Like settings export FASTAPI_S

Mohd Mujtaba 3 Feb 06, 2022
Cookiecutter API for creating Custom Skills for Azure Search using Python and Docker

cookiecutter-spacy-fastapi Python cookiecutter API for quick deployments of spaCy models with FastAPI Azure Search The API interface is compatible wit

Microsoft 379 Jan 03, 2023