Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Last update: Jul 27, 2022

Overview

OpenVINO Model Server wrapper API for Python

Description

This project provides a Python wrapper class for OpenVINO Model Server.
User can submit DL inference request to OVMS with just a few lines of code.

Sample code

from ovms_wrapper.ovms_wrapper import OpenVINO_Model_Server

import cv2
import numpy as np

ovms = OpenVINO_Model_Server()
ovms.connect('127.0.0.1', 9000)
#print(ovms.get_model_status('resnet_50'))
model = ovms.open_model('resnet_50')
print(model.inputs, model.outputs)

image_file  = 'ovms/lib/python3.8/site-packages/skimage/data/rocket.jpg'
img = cv2.imread(image_file)                # Read an image
res = model.single_image_infer(img)         # Infer
result = res[model.outputs[0]['name']]

# display result
nu = np.array(result)
ma = np.argmax(nu)
print("Result:", ma)

How to setup OpenVINO Model Server (Ubuntu)

Note: OVMS can run on Windows too. Please refer to the official OVMS document for details.

sudo apt update && sudo apt install -y python3-venv
python -m pip install tensorflow tensorflow-serving-api

Create Python virtual env, install OpenVINO, and prepare an IR model
Installing OpenVINO just for downloading a DL model and converting it into OpenVINO IR model. This is not required if you already have the IR models.

python3 -m venv ovms
. ovms/bin/activate
python -m pip install openvino-dev
omz_downloader --name resnet-50-tf
omz_converter --name resnet-50-tf --precisions FP16
deactivate

Start OpenVINO Model Server as Docker container

docker run -d --rm \
  -v $PWD/public/resnet-50-tf/FP16:/models/resnet50/1 \
  -p 9000:9000 \
  openvino/model_server:latest \
  --model_path /models/resnet50 \
  --model_name resnet_50 \
  --port 9000

OVMS will start serving the Resnet-50 model as model-name='resnet_50', model-version=1, and gRPC-port=9000.

Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Related tags

Overview

OpenVINO Model Server wrapper API for Python

Description

Sample code

How to setup OpenVINO Model Server (Ubuntu)

Owner

Yasunori Shimura

Additional code for Stable-baselines3 to load and upload models from the Hub.

KSAI Lite is a deep learning inference framework of kingsoft, based on tensorflow lite

Character-Input - Create a program that asks the user to enter their name and their age

Real-Time Multi-Contact Model Predictive Control via ADMM

Code for KDD'20 "Generative Pre-Training of Graph Neural Networks"

High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

An open source Jetson Nano baseboard and tools to design your own.

DeepFashion2 is a comprehensive fashion dataset.

Deep Learning for Human Part Discovery in Images - Chainer implementation

MTCNN face detection implementation for TensorFlow, as a PIP package.

Is RobustBench/AutoAttack a suitable Benchmark for Adversarial Robustness?

FCOS: Fully Convolutional One-Stage Object Detection (ICCV'19)

Multi-Target Adversarial Frameworks for Domain Adaptation in Semantic Segmentation

MSG-Transformer: Exchanging Local Spatial Information by Manipulating Messenger Tokens

Feedback is important: response-aware feedback mechanism for background based conversation

Code for "Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification", ECCV 2020 Spotlight

This repository is dedicated to developing and maintaining code for experiments with wide neural networks.

GenGNN: A Generic FPGA Framework for Graph Neural Network Acceleration

Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

Source code for EquiDock: Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking (ICLR 2022)