1. OS Ubuntu 18.04

2. OS Architecture x86_64

3. Version of OpenVINO 2021.2.185

4. Version of tflite2tensorflow 1.8.0

11. URL or source code for simple inference testing code: https://github.com/geaxgx/openvino_blazepose

12. Issue Details

Hi Katsuya, I would like to have your opinion on the following problem: I have used tflite2tensorflow 1.8.0 to convert the new version of mediapipe Blazepose models (mediapipe version 0.8.4). In this new version, now we have 3 pose landmark models (full, lite and heavy) and one common pose detection model. Then I try to test my code BlazeposeOpenvino.py with each model first on CPU then on MYRIAD (OAK-D). Everything seems to work great except when I run the heavy model on the MYRIAD. For instance, below is the outuput of the heavy model running on the CPU (which looks accurate): And below is the output of the same heavy model running on the MYRIADX : We can see the skeleton is kind of distorded. Actually, in many cases, with other images, we don't even have a skeleton drawn because the score given by the model is too low.

Do you have an idea of what the problem is ? The heavy model is much bigger (27Mo for the bin fle) than the other models. It takes about 30s to load on the MyriadX. But I guess the size is still acceptable otherwise I would get some error messages.

In case, you want to reproduce the problem:

# Clone my repo
git clone https://github.com/geaxgx/openvino_blazepose.git
cd openvino_blazepose
# Start your tflite2tensorflow docker
./docker_tflite2tensorflow.sh
cd workdir/models
# In my repo, there are only the FP32 IR version of the models, so we need to download the original tflite file 
# and then convert it using tflite2tensorflow. All is done with the following command:
./get_and_convert_heavy.sh
# Exit docker container
exit

# To test with the heavy model running on the CPU :
python BlazeposeOpenvino.py --lm_xml models/pose_landmark_heavy_FP16.xml -i img/yoga.jpg
# To test on the MYRIAD
python BlazeposeOpenvino.py --lm_xml models/pose_landmark_heavy_FP16.xml -i img/yoga.jpg --lm_device MYRIAD 

You can test also with the image img/yoga2.jpg (no skeleton detected).

Thank you.

1. OS Ubuntu 18.04

2. OS Architecture x86_64

3. Version of OpenVINO 2021.2.185 (the one from your dockerfile)

4. Version of TensorFlow e.g. v2.4.1 (the one from your dockerfile)

9. Download URL for .tflite : https://github.com/google/mediapipe/blob/master/mediapipe/modules/palm_detection/palm_detection.tflite

Hi Pinto ! First of all, I want to thank you for your last version of tflite2tensorflow. The dockerfile will surely make users life much easier !

I have installed and run the docker image of tflite2tensorflow to convert the Mediapipe palm detection model (see link above) into Openvino IR format. This model takes 128x128 images as input, whereas the previous model took 256x256 images. When running the FP32 model on my cpu, I noticed that sometimes the palm bounding box seemed a bit off. When comparing with the output from the original tflite model, we can see the bounding boxes are not the same: Below is the output from the FP32 openvino model:

Below is the output from the tflite model:

Note that if I compare the outputs of the older 256x256 models, there are no differences between tflite and Openvino versions.

Do you have an idea of what could explain the different ouputs ? Using Netron, I can see that the new tflite model now uses Prelu and ResizeBilinear that were not used in the older model. I don't see how Prelu could cause differences in the conversion process, but ResizeBilinear may be trickier (converted into Interpolate). Do you have any thoughts about that ?

Thanks for your help! I would like to use the new model, which is much faster than the previous version.

I can send you the code to reproduce the problem if you want.

I am trying to convert these tflite models: https://github.com/breizhn/DTLN-aec/tree/main/pretrained_models The command I ran was: tflite2tensorflow --model_path dtln_aec_128_2.tflite --flatc_path ../flatc --schema_path ../schema.fbs --output_pb But ran into this error:

ValueError: Dimension size, given by scalar input 1 must be in range [-1, 1) for '{{node split_functional_5/lstm_6/lstm_cell_6/StatefulPartitionedCall/split}} = Split[T=DT_FLOAT, num_split=4](split_functional_5/lstm_6/lstm_cell_6/StatefulPartitionedCall/split/split_dim, BiasAdd_functional_5/lstm_6/lstm_cell_6/StatefulPartitionedCall/BiasAdd)' with input shapes: [], [512] and with computed input tensors: input[0] = <1>.

Tried with both pip installation and docker version. Is it a bug or something that not supported?

Issue Type

Bug

OS

Ubuntu

OS architecture

x86_64

Programming Language

Python

Framework

ONNX

Download URL for tflite file

VertexAI Auto ML classification TFlite Format model

Convert Script

●tflite2tensorflow

docker run -it --rm --gpus all -v pwd:/home/user/workdir ghcr.io/pinto0309/tflite2tensorflow:latest

tflite2tensorflow
--model_path model.tflite
--flatc_path ../flatc
--schema_path ../schema.fbs
--output_pb
--optimizing_for_openvino_and_myriad
--rigorous_optimization_for_myriad

tflite2tensorflow
--model_path model.tflite
--flatc_path ../flatc
--schema_path ../schema.fbs
--output_onnx
--onnx_opset 13
--output_openvino_and_myriad

●tensorflow2onnx pip install git+https://github.com/onnx/tensorflow-onnx

python -m tf2onnx.convert --opset 13 --tflite model.tflite --output model.onnx --dequantize

Description

I'm trying to convert the attached file from tflite to onnx. (Eventually I want to run it in myriad). In the conversion source tflite, the input to sortmax is [1×8], but in tflite2tensorflow, it is [7×7×8]. If you try tensorflow2tflite, the input to sortmax becomes [1×8] as in tflite. I think [1×8] is correct because it is an 8-class classification model.

[tflite]

[tflite2tensorflow(savedmodel and onnx)]

[tensorflow2onnx]

Relevant Log Output

None.

Source code for simple inference testing code

No response

Hi , thanks for your great work.

recently, I am working on pose estimation with mediapipe. I convert the pose_detection.tflite model to onnx with your tflite2tensorflow, the coversion processing is ok, the log info shows that the converison is success. But when I use the converted .onnx model the outpt value, seems not correct, and is different from what the orginal .tflite model dose.

In the original tflite model, the max confidence value of bounxing box is 0.9, but using the converted model , the max value of bounxing box is only 0.078, which is not correct. And I also tried the model you've already converted, the result is also not right. Is there something wrong in my step or code?

1. WIndows10

2. x86_64

3. Version of OpenVINO : none

4. Version of TensorFlow e.g. v2.6.0

5. Version of TensorRT : none

6. Version of TFJS : none

7. Version of coremltools : none

8. Version of ONNX : 1.10.1

9. Download URL for .tflite IR model

10. URL of the repository from which the transformed model was taken : https://github.com/google/mediapipe/tree/master/mediapipe/modules/pose_detection

11. URL or source code for simple inference testing code

def image_preprocess(img):
    img = cv2.resize(img, dsize=(224, 224), interpolation=cv2.INTER_AREA)
    img = img.astype("float32")
    img /= 255.0
    print(img.shape)
    img.resize((1, 224, 224, 3))
    return img

def inference(img):
    # Test the model on random input data.
    input_shape = input_details[0]['shape']
    input_data = np.array(img, dtype=np.float32)
    interpreter.set_tensor(input_details[0]['index'], input_data)

    interpreter.invoke()

    # The function `get_tensor()` returns a copy of the tensor data.
    # Use `tensor()` in order to get a pointer to the tensor.
    output_data = interpreter.get_tensor(output_details[0]['index'])
    output_data1 = interpreter.get_tensor(output_details[1]['index'])

    result = [output_data, output_data1]

    print("result[0] shape:", result[0].shape)  # (1, 2254, 12)
    print("result[1] shape:", result[1].shape)  # (1, 2254, 1)
    return result

model = "model_float32.onnx"

# load model
session = onnxruntime.InferenceSession(model, None)
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].name
print(input_name)
print(output_name)

12. Issue Details

1. Ubuntu 18.04

2. OS Architecture x86_64

3. OpenVINO e.g. 2021.4.582

9. Download URL for .tflite IR model https://github.com/google/mediapipe/blob/master/mediapipe/modules/pose_detection/pose_detection.tflite

Hi @PINTO0309 ! New mediapipe version 0.8.6 comes with new models for Blazepose (that's a never ending story :-) The size of the pose detection model (link above) has been significantly reduced (from ~7.5MB to ~3MB) but unfortunately the model is using a layer named Densify that is not implemented in tflite2tensorflow. I guess it is a relatively new layer. When trying to visualize its data in Netron, I get an "Invalid tensor data size" message.

Do you think Densify can be easily implemented in your tools ? Note that it is not something I am eagerly waiting for since I can do without it by using the previous version of the pose detection model.

output json command = /content/flatbuffers/build/flatc -t --strict-json --defaults-json -o . /content/flatbuffers/build/schema.fbs -- /content/hair_segmentation.tflite Traceback (most recent call last): File "/usr/local/bin/tflite2tensorflow", line 2882, in main() File "/usr/local/bin/tflite2tensorflow", line 2556, in main ops, op_types = parse_json(jsonfile_path) File "/usr/local/bin/tflite2tensorflow", line 247, in parse_json j = json.load(open(jsonfile_path)) FileNotFoundError: [Errno 2] No such file or directory: './/content/hair_segmentation.json'

Issue Type

Feature Request

OS

Other

OS architecture

Other

Programming Language

Other

Framework

OpenVINO, Myriad Inference Engine

Download URL for tflite file

None

Convert Script

None

Description

Replace tf.math.reduce_prod with tf.math.multiply https://www.tensorflow.org/api_docs/python/tf/math/reduce_prod

https://github.com/PINTO0309/PINTO_model_zoo/tree/main/282_face_landmark_with_attention

https://github.com/iwatake2222/play_with_tflite/tree/master/pj_tflite_face_landmark_with_attention

Relevant Log Output

None

Source code for simple inference testing code

No response

Issue Type

Bug

OS

Mac OS

OS architecture

x86_64

Programming Language

Python

Framework

CoreML

Download URL for tflite file

https://github.com/google/mediapipe/blob/master/mediapipe/modules/palm_detection/palm_detection_full.tflite

Convert Script

docker run -it --rm \
  -v `pwd`:/home/user/workdir \
  ghcr.io/pinto0309/tflite2tensorflow:latest

tflite2tensorflow \
  --model_path palm_detection_full.tflite \
  --flatc_path ../flatc \
  --schema_path ../schema.fbs \
  --output_pb

tflite2tensorflow \
  --model_path palm_detection_full.tflite \
  --flatc_path ../flatc \
  --schema_path ../schema.fbs \
  --output_no_quant_float32_tflite \
  --output_dynamic_range_quant_tflite \
  --output_weight_quant_tflite \
  --output_float16_quant_tflite \
  --output_integer_quant_tflite \
  --string_formulas_for_normalization 'data / 255.0' \
  --output_tfjs \
  --output_coreml \
  --output_tftrt_float32 \
  --output_tftrt_float16 \
  --output_onnx \
  --onnx_opset 13 \
  --output_openvino_and_myriad

Description

when I deployed the converted mp palm detection coreml model on IOS app, I got error like:

2022-02-15 16:31:50.577312+0000 MLModelCamera[12602:4637828] [espresso] [Espresso::handle_ex_plan] exception=Espresso exception: "Generic error": [Exception from layer 240: model_1/model/add_24/add]Espresso exception: "Invalid blob shape": elementwise_kernel_cpu: Cannot broadcast [12, 2, 1, 256, 1] and [12, 1, 1, 256, 1] status=-1

seems like the resizeBilinear op causes this problem. Is there any way to fix this issue? Thanks in advance.

The related code in tflite2tensorlfow.py is:

        elif op_type == 'RESIZE_BILINEAR':
            input_tensor = tensors[op['inputs'][0]]
            size_detail = interpreter._get_tensor_details(op['inputs'][1])
            size = interpreter.get_tensor(size_detail['index'])
            size_height = size[0]
            size_width  = size[1]

            options = op['builtin_options']
            align_corners = options['align_corners']
            half_pixel_centers = options['half_pixel_centers']

            def upsampling2d_bilinear(x, size_height, size_width, align_corners, half_pixel_centers):
                if optimizing_for_edgetpu_flg:
                    return tf.image.resize_bilinear(x, (size_height, size_width))
                else:
                    if optimizing_for_openvino_and_myriad:
                        if half_pixel_centers:
                            return tf.image.resize_bilinear(
                                x,
                                (size_height, size_width),
                                align_corners=False,
                                half_pixel_centers=half_pixel_centers
                            )
                        else:
                            return tf.image.resize_bilinear(
                                x,
                                (size_height, size_width),
                                align_corners=True,
                                half_pixel_centers=half_pixel_centers
                            )
                    else:
                        return tfv2.image.resize(
                            x,
                            [size_height, size_width],
                            method='bilinear'
                        )

            output_detail = interpreter._get_tensor_details(op['outputs'][0])
            lambda_name = get_op_name(output_detail['name']) + '_lambda'
            output_tensor_lambda = tf.keras.layers.Lambda(
                upsampling2d_bilinear,
                arguments={'size_height': size_height,
                            'size_width': size_width,
                            'align_corners': align_corners,
                            'half_pixel_centers': half_pixel_centers},
                name=lambda_name
            )(input_tensor)
            output_tensor = tf.identity(
                output_tensor_lambda,
                name=get_op_name(output_detail['name'])
            )
            tensors[output_detail['index']] = output_tensor

Relevant Log Output

2022-02-15 16:31:50.577312+0000 MLModelCamera[12602:4637828] [espresso] [Espresso::handle_ex_plan] exception=Espresso exception: "Generic error": [Exception from layer 240: model_1/model/add_24/add]Espresso exception: "Invalid blob shape": elementwise_kernel_cpu: Cannot broadcast [12, 2, 1, 256, 1] and [12, 1, 1, 256, 1] status=-1

Source code for simple inference testing code

No response

Hello Pinto, I am following the tutorial form here: https://github.com/geaxgx/openvino_hand_tracker Can it be that in the latest docker version the API has changed?

This is what I do: [email protected]:/tmp/depthai_hand_tracker$ docker run --gpus all -it --rm
-v pwd:/workspace/resources
-e LOCAL_UID=$(id -u $USER)
-e LOCAL_GID=$(id -g $USER)
pinto0309/tflite2tensorflow:latest bash

NVIDIA Release 20.09 (build 15985252)

NVIDIA TensorRT 7.1.3 (c) 2016-2020, NVIDIA CORPORATION. All rights reserved. Container image (c) 2020, NVIDIA CORPORATION. All rights reserved.

https://developer.nvidia.com/tensorrt

To install Python sample dependencies, run /opt/tensorrt/python/python_setup.sh

To install open source parsers, plugins, and samples, run /opt/tensorrt/install_opensource.sh. See https://github.com/NVIDIA/TensorRT for more information.

error: XDG_RUNTIME_DIR not set in the environment. [setupvars.sh] OpenVINO environment initialized

wget https://github.com/google/mediapipe/blob/master/mediapipe/modules/hand_landmark/hand_landmark.tflite wget https://github.com/google/mediapipe/blob/master/mediapipe/modules/palm_detection/palm_detection.tflite

tflite2tensorflow --model_path ./hand_landmark.tflite --model_output_path palm_detection --flatc_path /home/user/flatc --schema_path /home/user/schema.fbs --output_pb True tflite2tensorflow: error: unrecognized arguments: True

tflite2tensorflow --model_path ./hand_landmark.tflite --model_output_path palm_detection --flatc_path /home/user/flatc --schema_path /home/user/schema.fbs --output_pb
output json command = /home/user/flatc -t --strict-json --defaults-json -o . /home/user/schema.fbs -- ./hand_landmark.tflite /home/user/flatc: error: binary "./hand_landmark.tflite" does not have expected file_identifier "TFL3", use --raw-binary to read this file anyway.

Thanks Marco

Running tflite2tensorflow in docker container, against tflite model generated in the AutoML Vision service in google cloud, produces:

sh-5.0$ tflite2tensorflow \

--model_path model.tflite
--flatc_path ../flatc
--schema_path ../schema.fbs
--output_pb
--optimizing_for_openvino_and_myriad Traceback (most recent call last): File "/usr/local/bin/tflite2tensorflow", line 6201, in main() File "/usr/local/bin/tflite2tensorflow", line 5592, in main ops, json_tensor_details, op_types, full_json = parse_json(jsonfile_path) File "/usr/local/bin/tflite2tensorflow", line 265, in parse_json op_types = [v['builtin_code'] for v in j['operator_codes']] KeyError: 'operator_codes'

The tflite output from AutoML Vision was three files: model.tflite, tflite_metadata.json, and dict.txt (which contains the list of labels).

I first received the error that it could not find model.json, so I renamed tflite_metadata.json to model.json and thus the error above.

Here is contents of model.json (tflite_metadata.json) - obviously it does not include operator codes.

{ "inferenceType": "QUANTIZED_UINT8", "inputShape": [ 1, 320, 320, 3 ], "inputTensor": "normalized_input_image_tensor", "maxDetections": 40, "outputTensorRepresentation": [ "bounding_boxes", "class_labels", "class_confidences", "num_of_boxes" ], "outputTensors": [ "TFLite_Detection_PostProcess", "TFLite_Detection_PostProcess:1", "TFLite_Detection_PostProcess:2", "TFLite_Detection_PostProcess:3" ] }

Is tflite2tensorflow expecting a different set of files?

OS you are using: MacOS 11.4

Version of TensorFlow: v2.5.0

Environment: Docker

Under tf 2.5.0, I converted my pre-trained model from saved_model to tflite.

Afterwards, in Docker container, when I was converting this tflite model to pb format using tflite2tensorflow, the following error occured:

ERROR: The UNIDIRECTIONAL_SEQUENCE_LSTM layer is not yet implemented.

(In this experiment, I did not perform quantization/optimization, but later on I do plan to use tflite to quantize my model that is to be saved as .tflite, which is why I did not directly convert saved_model to pb)