Simple example of FastAPI + Celery + Triton for benchmarking

Overview

You can see the previous work from:

Benchmark FastAPI + Celery with / without Triton

with Triton Server

without Triton Server

Benchmark Results

See Benchmark Results

Preparation

1. Setup packages

Install Anaconda and execute the following commands:

$ make env        # create a conda environment (need only once)
$ source init.sh  # activate the env
$ make setup      # setup packages (need only once)

2. Train a CNN model (Recommended on GPU)

$ source create_model.sh

3. Check the model repository created

$ tree model_repository

model_repository
└── mnist_cnn
    ├── 1
    │   └── model.pt
    └── config.pbtxt

2 directories, 2 files

How to play

Server (Option 1 - On your Local)

Install Redis & Docker, and run the following commands:

$ make triton     # run triton server
$ make broker     # run redis broker
$ make worker     # run celery worker
$ make api        # run fastapi server
$ make dashboard  # run dashboard that monitors celery

Server (Option 2 - Docker Compose available on GPU devices)

Install Docker & Docker Compose, and run the following command:

$ docker-compose up

[Optional] Additional Triton Servers

You can start up additional Triton servers on other devices.

$ make triton

[Optional] Additional Workers

You can start up additional workers on other devices.

$ export BROKER_URL=redis://redis-broker-ip:6379    # default is localhost
$ export BACKEND_URL=redis://redis-backend-ip:6379  # default is localhost
$ export TRITON_SERVER_URL=http://triton-server-ip:9000   # default is localhost
$ make worker
  • NOTE: Worker needs to run on the machine which Triton runs on due to shared memory settings.

Dashboard for Celery (Flower)

http://0.0.0.0:5555/ image

Load Test (w/ Locust)

Execute Locust

$ make load  # for load test without Triton
or
$ make load-triton  # for  load test with Triton

Open http://0.0.0.0:8089

Type url for the API server.

Issue Handling

Redis Error 8 connecting localhost:6379. nodename nor servname provided, or not known.

$ ulimit -n 1024

Docker's network_mode=bridgedegrades the network performance.

We recommend to use Linux server if you would like to run docker-compose up.

For Developers

$ make setup-dev      # setup for developers
$ make format         # format scripts
$ make lint           # lints scripts
$ make utest          # runs unit tests
$ make cov            # opens unit test coverage information
Owner
Jinwoo Park (Curt)
A domain-independent problem-solver
Jinwoo Park (Curt)
FastAPI with async for generating QR codes and bolt11 for Lightning Addresses

sendsats An API for getting QR codes and Bolt11 Invoices from Lightning Addresses. Share anywhere; as a link for tips on a twitter profile, or via mes

Bitkarrot 12 Jan 07, 2023
Fastapi performans monitoring

Fastapi-performans-monitoring This project is a simple performance monitoring for FastAPI. License This project is licensed under the terms of the MIT

bilal alpaslan 11 Dec 31, 2022
A rate limiter for Starlette and FastAPI

SlowApi A rate limiting library for Starlette and FastAPI adapted from flask-limiter. Note: this is alpha quality code still, the API may change, and

Laurent Savaete 562 Jan 01, 2023
A comprehensive CRUD API generator for SQLALchemy.

FastAPI Quick CRUD Introduction Advantage Constraint Getting started Installation Usage Design Path Parameter Query Parameter Request Body Upsert Intr

192 Jan 06, 2023
fastapi-admin2 is an upgraded fastapi-admin, that supports ORM dialects, true Dependency Injection and extendability

FastAPI2 Admin Introduction fastapi-admin2 is an upgraded fastapi-admin, that supports ORM dialects, true Dependency Injection and extendability. Now

Glib 14 Dec 05, 2022
A server hosts a FastAPI application and multiple clients can be connected to it via SocketIO.

FastAPI_and_SocketIO A server hosts a FastAPI application and multiple clients can be connected to it via SocketIO. Executing server.py sets up the se

Ankit Rana 2 Mar 04, 2022
Toolkit for developing and maintaining ML models

modelkit Python framework for production ML systems. modelkit is a minimalist yet powerful MLOps library for Python, built for people who want to depl

140 Dec 27, 2022
Full stack, modern web application generator. Using FastAPI, PostgreSQL as database, Docker, automatic HTTPS and more.

Full Stack FastAPI and PostgreSQL - Base Project Generator Generate a backend and frontend stack using Python, including interactive API documentation

Sebastián Ramírez 10.8k Jan 08, 2023
CLI and Streamlit applications to create APIs from Excel data files within seconds, using FastAPI

FastAPI-Wrapper CLI & APIness Streamlit App Arvindra Sehmi, Oxford Economics Ltd. | Website | LinkedIn (Updated: 21 April, 2021) fastapi-wrapper is mo

Arvindra 49 Dec 03, 2022
Local Telegram Bot With FastAPI & Ngrok

An easy local telegram bot server with python, fastapi and ngrok.

Ömer Faruk Özdemir 7 Dec 25, 2022
Flood Detection with Google Earth Engine

ee-fastapi: Flood Detection System A ee-fastapi is a simple FastAPI web application for performing flood detection using Google Earth Engine in the ba

Cesar Aybar 69 Jan 06, 2023
Mixer -- Is a fixtures replacement. Supported Django, Flask, SqlAlchemy and custom python objects.

The Mixer is a helper to generate instances of Django or SQLAlchemy models. It's useful for testing and fixture replacement. Fast and convenient test-

Kirill Klenov 871 Dec 25, 2022
An alternative implement of Imjad API | Imjad API 的开源替代

HibiAPI An alternative implement of Imjad API. Imjad API 的开源替代. 前言 由于Imjad API这是什么?使用人数过多, 致使调用超出限制, 所以本人希望提供一个开源替代来供社区进行自由的部署和使用, 从而减轻一部分该API的使用压力 优势

Mix Technology 450 Dec 29, 2022
Opentracing support for Starlette and FastApi

Starlette-OpenTracing OpenTracing support for Starlette and FastApi. Inspired by: Flask-OpenTracing OpenTracing implementations exist for major distri

Rene Dohmen 63 Dec 30, 2022
Learn to deploy a FastAPI application into production DigitalOcean App Platform

Learn to deploy a FastAPI application into production DigitalOcean App Platform. This is a microservice for our Try Django 3.2 project. The goal is to extract any and all text from images using a tec

Coding For Entrepreneurs 59 Nov 29, 2022
Cookiecutter API for creating Custom Skills for Azure Search using Python and Docker

cookiecutter-spacy-fastapi Python cookiecutter API for quick deployments of spaCy models with FastAPI Azure Search The API interface is compatible wit

Microsoft 379 Jan 03, 2023
The template for building scalable web APIs based on FastAPI, Tortoise ORM and other.

FastAPI and Tortoise ORM. Powerful but simple template for web APIs w/ FastAPI (as web framework) and Tortoise-ORM (for working via database without h

prostomarkeloff 95 Jan 08, 2023
fastapi-mqtt is extension for MQTT protocol

fastapi-mqtt MQTT is a lightweight publish/subscribe messaging protocol designed for M2M (machine to machine) telemetry in low bandwidth environments.

Sabuhi 144 Dec 28, 2022
Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

Tomasz Wójcik 300 Dec 26, 2022
京东图片点击验证码识别

京东图片验证码识别 本项目是@yqchilde 大佬的 JDMemberCloseAccount 识别图形验证码(#45)思路验证,若你也有思路可以提交Issue和PR也可以在 @yqchilde 的 TG群 找到我 声明 本脚本只是为了学习研究使用 本脚本除了采集处理验证码图片没有其他任何功能,也

AntonVanke 37 Dec 22, 2022