Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Last update: Jul 27, 2022

Overview

OpenVINO Model Server wrapper API for Python

Description

This project provides a Python wrapper class for OpenVINO Model Server.
User can submit DL inference request to OVMS with just a few lines of code.

Sample code

from ovms_wrapper.ovms_wrapper import OpenVINO_Model_Server

import cv2
import numpy as np

ovms = OpenVINO_Model_Server()
ovms.connect('127.0.0.1', 9000)
#print(ovms.get_model_status('resnet_50'))
model = ovms.open_model('resnet_50')
print(model.inputs, model.outputs)

image_file  = 'ovms/lib/python3.8/site-packages/skimage/data/rocket.jpg'
img = cv2.imread(image_file)                # Read an image
res = model.single_image_infer(img)         # Infer
result = res[model.outputs[0]['name']]

# display result
nu = np.array(result)
ma = np.argmax(nu)
print("Result:", ma)

How to setup OpenVINO Model Server (Ubuntu)

Note: OVMS can run on Windows too. Please refer to the official OVMS document for details.

sudo apt update && sudo apt install -y python3-venv
python -m pip install tensorflow tensorflow-serving-api

Create Python virtual env, install OpenVINO, and prepare an IR model
Installing OpenVINO just for downloading a DL model and converting it into OpenVINO IR model. This is not required if you already have the IR models.

python3 -m venv ovms
. ovms/bin/activate
python -m pip install openvino-dev
omz_downloader --name resnet-50-tf
omz_converter --name resnet-50-tf --precisions FP16
deactivate

Start OpenVINO Model Server as Docker container

docker run -d --rm \
  -v $PWD/public/resnet-50-tf/FP16:/models/resnet50/1 \
  -p 9000:9000 \
  openvino/model_server:latest \
  --model_path /models/resnet50 \
  --model_name resnet_50 \
  --port 9000

OVMS will start serving the Resnet-50 model as model-name='resnet_50', model-version=1, and gRPC-port=9000.

Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Related tags

Overview

OpenVINO Model Server wrapper API for Python

Description

Sample code

How to setup OpenVINO Model Server (Ubuntu)

Owner

Yasunori Shimura

Official codebase for Pretrained Transformers as Universal Computation Engines.

A Differentiable Recipe for Learning Visual Non-Prehensile Planar Manipulation

You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks.

Official code repository for the publication "Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons"

StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

Cleaned up code for DSTC 10: SIMMC 2.0 track: subtask 2: multimodal coreference resolution

A Research-oriented Federated Learning Library and Benchmark Platform for Graph Neural Networks. Accepted to ICLR'2021 - DPML and MLSys'21 - GNNSys workshops.

Computations and statistics on manifolds with geometric structures.

The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training

Domain Adaptation with Invariant RepresentationLearning: What Transformations to Learn?

Cross-modal Deep Face Normals with Deactivable Skip Connections

Drone detection using YOLOv5

git《FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding》(CVPR 2021) GitHub: [fig8]

Light-Head R-CNN

Paaster is a secure by default end-to-end encrypted pastebin built with the objective of simplicity.

Spatial Intention Maps for Multi-Agent Mobile Manipulation (ICRA 2021)

Official implementation of the article "Unsupervised JPEG Domain Adaptation For Practical Digital Forensics"

Repository for paper "Non-intrusive speech intelligibility prediction from discrete latent representations"

Vision-and-Language Navigation in Continuous Environments using Habitat