An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Last update: Dec 14, 2022

Related tags

Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
PointConv: Deep Convolutional Networks on 3D Point Clouds

Comments

which version should the `finetuner` module be?
I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

from finetuner.tuner.callback import BestModelCheckpoint from finetuner.tuner.pytorch.losses import TripletLoss from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.
opened by elenacliu 4
fix: fix previous structure

I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

opened by elenacliu 2
Add dataset, preprocess and pointnet2

Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

@numb3r3 @jemmyshin

opened by elenacliu 1
In `farthest_point_sample` function, the first centroid sampled is not random.

I'm reading your codes.

https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

opened by elenacliu 1
Provide diffusion encoder

https://github.com/nmwsharp/diffusion-net

I'm unfamiliar with the jina ecosystem. So this is my first issue.

Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

opened by fire 2

Releases(v0.3.0)

v0.3.0(May 6, 2022)

Source code(tar.gz)
Source code(zip)
v0.2.0(Feb 21, 2022)

Source code(tar.gz)
Source code(zip)

Owner

Jina AI

A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.

GitHub Repository

Depix is a tool for recovering passwords from pixelized screenshots.

This implementation works on pixelized images that were created with a linear box filter. In this article I cover background information on pixelization and similar research.

23.1k Jan 04, 2023

Python implementation of image filters (such as brightness, contrast, saturation, etc.)

PyPhotoshop Python implementation of image filters Use Python to adjust brightness and contrast, add blur, and detect edges! Follow along tutorial: ht

87 Dec 15, 2022

Open source software for image correlation, distance and analysis

Douglas-Quaid Project Open source software for image correlation, distance and analysis. Strongly related to : Carl-Hauser Problem statement (@CIRCL)

2 May 01, 2022

Fast Image Retrieval (FIRe) is an open source image retrieval project

Fast Image Retrieval (FIRe) is an open source image retrieval project release by Center of Image and Signal Processing Lab (CISiP Lab), Universiti Malaya. This project implements most of the major bi

39 Nov 25, 2022

GIMP script to export bitmap as GRAPHICS 4 file (aka SCREEN 5)

gimpfu-msx-gr4.py GIMP script to export bitmap as GRAPHICS 4 file (aka SCREEN 5). GRAPHICS 4 specs are: 256x212 (or 256x192); 16 color palette (from 5

4 Oct 17, 2022

LSB Image Steganography Using Python

Steganography is the science that involves communicating secret data in an appropriate multimedia carrier, e.g., image, audio, and video files

2 Nov 04, 2021

A simple image-level annotation tool supporting multi-channel images for napari.

napari-labelimg4classification A simple image-level annotation tool supporting multi-channel images for napari. This napari plugin was generated with

4 May 16, 2022

Image Processing - Make noise images clean

影像處理-影像降躁化(去躁化) (Image Processing - Make Noise Images Clean) 得力於電腦效能的大幅提升以及GPU的平行運算架構，讓我們能夠更快速且有效地訓練AI，並將AI技術應用於不同領域。本篇將帶給大家的是「將深度學習應用於影像處理中的影像降躁化」，

2 Aug 04, 2022

A script to generate a profile picture and a banner that show the same image on Discord.

Discord profile picture & banner generator A script to generate a profile picture and a banner that show the same image on Discord. Installation / Upd

9 Nov 27, 2022

Python Interface of P3D

pyp3d 介绍: pyp3d是一个可在python上使用的工具包，它提供了一种可使用python脚本驱动创建模型的方法，为三维建模提供了全新的思路。 pyp3d中定义了一系列建模相关的必要的数据类型，例如球体、圆锥台、四棱锥台、拉伸体、圆角管等几何体，pyp3d还提供了许多函数来实现放置集合体、

20 Sep 07, 2022

cmdpxl: a totally practical command-line image editor

cmdpxl: a totally practical command-line image editor Features cmdpxl has many exciting functionalities, including Editing pixels one at a time! Savin

475 Dec 23, 2022

Py3D - A 3d rendering engine written entirely in python

Py3D is a 3d rendering engine written entirely in python. It is a simple and eas

2 Nov 14, 2022

This repository will help you get label for images in Stanford Cars Dataset.

STANFORD CARS DATASET stanford-cars "The Cars dataset contains 16,185 images of 196 classes of cars. The data is split into 8,144 training images and

3 Sep 20, 2022

QR-Generator - An awesome QR Generator to create or customize your QR's

QR Generator An awesome QR Generator to create or customize your QR's! Table of

1 Jan 28, 2022

SimpleITK is an image analysis toolkit with a large number of components supporting general filtering operations, image segmentation and registration

672 Jan 05, 2023