An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Related tags

Image Processingexecutor-3d-encoder
Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

  • PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
  • PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

  • PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
  • PointConv: Deep Convolutional Networks on 3D Point Clouds
Comments
  • which version should the `finetuner` module be?

    which version should the `finetuner` module be?

    I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

    from finetuner.tuner.callback import BestModelCheckpoint
from finetuner.tuner.pytorch.losses import TripletLoss
from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

    It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

    Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.

    opened by elenacliu 4
  • fix: fix previous structure

    fix: fix previous structure

    I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

    opened by elenacliu 2
  • Add dataset, preprocess and pointnet2

    Add dataset, preprocess and pointnet2

    Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

    @numb3r3 @jemmyshin

    opened by elenacliu 1
  • In `farthest_point_sample` function, the first centroid sampled is not random.

    In `farthest_point_sample` function, the first centroid sampled is not random.

    I'm reading your codes.

    https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

    The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

    opened by elenacliu 1
  • Provide diffusion encoder

    Provide diffusion encoder

    https://github.com/nmwsharp/diffusion-net

    I'm unfamiliar with the jina ecosystem. So this is my first issue.

    Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

    opened by fire 2
Releases(v0.3.0)
Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
GitHub Repository
Napari plugin for iteratively improving 3D instance segmentation of cells (u-net x watershed)

iterseg napari plugin for iteratively improving unet-watershed segmentation This napari plugin was generated with Cookiecutter using @napari's cookiec

Abigail McGovern 3 May 16, 2022
This is a python project which detects color of an image when you double click on it.

This is a python project which detects color of an image when you double click on it. You have to press ESC button to close the pop-up Image window. There are mainly two library CV2 and Pandas that a

Yashwant Kumar Singh 0 Aug 16, 2022
Open source software for image correlation, distance and analysis

Douglas-Quaid Project Open source software for image correlation, distance and analysis. Strongly related to : Carl-Hauser Problem statement (@CIRCL)

Dominik Dancs 2 May 01, 2022
Digital image process Basic algorithm

These are some basic algorithms that I have implemented by my hands in the process of learning digital image processing, such as mean and median filtering, sharpening algorithms, interpolation scalin

JingYu 2 Nov 03, 2022
🖼️ Draw Images or GIFs in your terminal

Drawitor Draw Images/GIFs in your terminal. Install pip install drawitor CLI Tool drawitor cat_dancing.gif Library The library is written in a simple

Eliaz Bobadilla 7 Dec 15, 2022
Automatic picture transmission(APT) protocol decoder for NOAA weather satellites

Automatic picture transmission(APT) decoder Automatic picture transmission protocol is used by NOAA satellites. They constantly send a frequency modul

Jayachandra Kasarla 25 Aug 05, 2022
Visage Differentiation is a GUI application for outlining and labeling the visages in an image.

Visage Differentiation Visage Differentiation is a GUI application for outlining and labeling the visages in an image. The main functionality is provi

Grant Randa 0 Jan 13, 2022
An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

3D Mesh Encoder An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings o

Jina AI 11 Dec 14, 2022
Dynamic image server for web and print

Quru Image Server - dynamic imaging for web and print QIS is a high performance web server for creating and delivering dynamic images. It is ideal for

Quru 84 Jan 03, 2023
PyLibTiff - a wrapper to the libtiff library to Python using ctypes

PyLibTiff is a package that provides: a wrapper to the libtiff library to Python using ctypes. a pure Python module for reading and writing TIFF and L

Pearu Peterson 105 Dec 21, 2022
Tool that takes your photo and generates a pixelated color by number photo.

Color by number Tool that takes your photo and generates a pixelated color by number photo. Requirements You need to have python installed on your com

1 Dec 18, 2021
Vignette is a Python library to create and manage thumbnails following the FreeDesktop standard.

Vignette Vignette is a Python library to create and manage thumbnails following the FreeDesktop standard. Thumbnails are stored in a shared directory

3 Feb 06, 2022
Python Script to generate posters out of the images in directory.

Poster-Maker Python Script to generate posters out of the images in directory. This version is very basic ligthweight code to combine organize images

1 Feb 02, 2022
python binding for libvips using cffi

README PyPI package: https://pypi.python.org/pypi/pyvips conda package: https://anaconda.org/conda-forge/pyvips We have formatted docs online here: ht

libvips 467 Dec 30, 2022
Sombra is simple Raytracer written in pure Python.

Sombra Sombra is simple Raytracer written in pure Python. It's main purpose is to help understand how raytracing works with a clean code. If you are l

Hernaldo Jesus Henriquez Nuñez 10 Jul 16, 2022
Png-to-stl - Converts PNG and text to SVG, and then extrudes that based on parameters

have ansible installed locally run ansible-playbook setup_application.yml this sets up directories, installs system packages, and sets up python envir

1 Jan 03, 2022
This will help to read QR codes using Raspberry Pi and Pi Camera

Raspberry-Pi-Generate-and-Read-QR-code This will help to read QR codes using Raspberry Pi and Pi Camera Install the required libraries first in your T

Raspberry_Pi Pakistan 2 Nov 06, 2021
Fixed Version Of Blender Low Poly Rock Generator For Blender 3.0.0

Blender (3.0.0) - Low Poly Rock Generator This is an addon for Blender 3.0.0 to generate low poly rocks. It was based on an addon that unfortunately h

3 Mar 24, 2022
HTML2Image is a lightweight Python package that acts as a wrapper around the headless mode of existing web browsers to generate images from URLs and from HTML+CSS strings or files.

A package acting as a wrapper around the headless mode of existing web browsers to generate images from URLs and from HTML+CSS strings or files.

176 Jan 01, 2023
Python Image Morpher (PIM) is a program that can take two images and blend them to whatever extent or precision that you like

Python Image Morpher (PIM) is a program that can take two images and blend them to whatever extent or precision that you like! It is designed to emulate some of Python's OpenCV image processing from

David Dowd 108 Dec 19, 2022