An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Last update: Dec 14, 2022

Related tags

Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
PointConv: Deep Convolutional Networks on 3D Point Clouds

Comments

which version should the `finetuner` module be?
I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

from finetuner.tuner.callback import BestModelCheckpoint from finetuner.tuner.pytorch.losses import TripletLoss from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.
opened by elenacliu 4
fix: fix previous structure

I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

opened by elenacliu 2
Add dataset, preprocess and pointnet2

Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

@numb3r3 @jemmyshin

opened by elenacliu 1
In `farthest_point_sample` function, the first centroid sampled is not random.

I'm reading your codes.

https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

opened by elenacliu 1
Provide diffusion encoder

https://github.com/nmwsharp/diffusion-net

I'm unfamiliar with the jina ecosystem. So this is my first issue.

Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

opened by fire 2

Releases(v0.3.0)

v0.3.0(May 6, 2022)

Source code(tar.gz)
Source code(zip)
v0.2.0(Feb 21, 2022)

Source code(tar.gz)
Source code(zip)

Owner

Jina AI

A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.

GitHub Repository

Easy to use Python module to extract Exif metadata from digital image files.

719 Jan 05, 2023

Python Program that lets you write in your handwriting!

Handwriting with Python Python Program that lets you write in your handwriting! Inspired by: thaisribeiro.in How to run? Install Unidecode and Pillow

2 Oct 25, 2021

Py3D - A 3d rendering engine written entirely in python

Py3D is a 3d rendering engine written entirely in python. It is a simple and eas

2 Nov 14, 2022

NFT collection generator. Generates layered images

NFT collection generator Generates layered images, whole collections. Provides additional functionality. Repository includes three scripts generate.py

10 Nov 15, 2022

A linear stairs generation add-on for Blender

Linear Stairs Generator Table of Contents Installation Usage Screenshots Important Notes Requirements Blender 3.0 or newer. Installation: Download a z

4 May 17, 2022

A minimal, standalone viewer for 3D animations stored as stop-motion sequences of individual .obj mesh files.

ObjSequenceViewer V0.5 A minimal, standalone viewer for 3D animations stored as stop-motion sequences of individual .obj mesh files. Installation: pip

2 Aug 04, 2022

A Icon Maker GUI Made - Convert your image into icon ( .ico format ).

Icon-Maker-GUI A Icon Maker GUI Made Using Python 3.9.0 . It will take any image and convert it to ICO file, for web site favicon or Windows applicati

12 Dec 15, 2021

With this simple py script you will be able to get all the .png from a folder and generate a yml for Oraxen

Oraxen-item-to-yml With this simple py script you will be able to get all the .png from a folder and generate a yml for Oraxen How to use Install the

1 Dec 29, 2021

DP2 graph edit codes.

必要なソフト・パッケージ Python3 Numpy JSON Matplotlib 動作確認環境 MacBook Air M1 Python 3.8.2 (arm64) Numpy 1.22.0 Matplotlib 3.5.1 JSON 2.0.9 使い方 draw_time_histgram(

1 Feb 19, 2022

Python avatar generator for absolute nerds

pagan Welcome to the Python Avatar Generator for Absolute Nerds. Current version: 0.4.3 View the change history here. Remember those good old days whe

280 Dec 16, 2022

QR Generator using GUI with Tinker

BinCat Token System Very simple python script with GUI that generates QR codes. It don't include a QR "decription" tool. It only generate-it and thats

1 Nov 06, 2021

Glyph-graph - A simple, yet versatile, package for graphing equations on a 2-dimensional text canvas

Glyth Graph Revision for 0.01 A simple, yet versatile, package for graphing equations on a 2-dimensional text canvas List of contents: Brief Introduct

2 Oct 21, 2022

A simple programme for converting url into a qr code (.png file)

QrTk A simple lightweight programme for converting url into a qr code (.png file) Pre-Requisites Before installing the programme , you need to run the

4 Nov 08, 2021

Python Script to generate posters out of the images in directory.

Poster-Maker Python Script to generate posters out of the images in directory. This version is very basic ligthweight code to combine organize images

1 Feb 02, 2022

A Python package implementing various CFA (Colour Filter Array) demosaicing algorithms and related utilities.

Colour - Demosaicing A Python package implementing various CFA (Colour Filter Array) demosaicing algorithms and related utilities. It is open source a

218 Dec 04, 2022

A tool for making simple-style text posters or wallpapers with high resolution.

PurePoster PurePoster is a fancy tool for making arbitrary-resolution, simple-style posters or wallpapers with text in center. Functionality PurePoste

4 Jul 09, 2022

Python Interface of P3D

pyp3d 介绍: pyp3d是一个可在python上使用的工具包，它提供了一种可使用python脚本驱动创建模型的方法，为三维建模提供了全新的思路。 pyp3d中定义了一系列建模相关的必要的数据类型，例如球体、圆锥台、四棱锥台、拉伸体、圆角管等几何体，pyp3d还提供了许多函数来实现放置集合体、

20 Sep 07, 2022

Open source software for image correlation, distance and analysis

Douglas-Quaid Project Open source software for image correlation, distance and analysis. Strongly related to : Carl-Hauser Problem statement (@CIRCL)

2 May 01, 2022

Small wrapper around 3dmol.js and html2canvas for creating self-contained HTML files that display a 3D molecular representation.

Description Small wrapper around 3dmol.js and html2canvas for creating self-contained HTML files that display a 3D molecular representation. Double cl

1 Dec 02, 2021

Maze generator with most popular shapes - hexagon, triangle, square

Maze-Generator Maze generator with most popular shapes - hexagon, triangle, square (sqaure not implemented yet): Theory: Planar Graph https://en.wikip

2 Dec 28, 2021