An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Last update: Dec 14, 2022

Related tags

Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
PointConv: Deep Convolutional Networks on 3D Point Clouds

Comments

which version should the `finetuner` module be?
I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

from finetuner.tuner.callback import BestModelCheckpoint from finetuner.tuner.pytorch.losses import TripletLoss from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.
opened by elenacliu 4
fix: fix previous structure

I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

opened by elenacliu 2
Add dataset, preprocess and pointnet2

Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

@numb3r3 @jemmyshin

opened by elenacliu 1
In `farthest_point_sample` function, the first centroid sampled is not random.

I'm reading your codes.

https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

opened by elenacliu 1
Provide diffusion encoder

https://github.com/nmwsharp/diffusion-net

I'm unfamiliar with the jina ecosystem. So this is my first issue.

Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

opened by fire 2

Releases(v0.3.0)

v0.3.0(May 6, 2022)

Source code(tar.gz)
Source code(zip)
v0.2.0(Feb 21, 2022)

Source code(tar.gz)
Source code(zip)

Owner

Jina AI

A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.

GitHub Repository

NFT collection generator. Generates layered images

NFT collection generator Generates layered images, whole collections. Provides additional functionality. Repository includes three scripts generate.py

10 Nov 15, 2022

Simple utility to tinker with OPlus images

OPlus image utilities Prerequisites Linux running kernel 5.4 or up (check with uname -r) Image rebuilding Used to rebuild read-only erofs images into

15 Dec 28, 2022

Archive of the image generator stuff from my API

alex_api_archive Archive of the image generator stuff from my API FAQ Q: Why? A: Because I am removing these components from the API Q: How do I run i

26 Nov 17, 2022

Transfers a image file(.png) to an Excel file(.xlsx)

7 Feb 11, 2022

🎨 Generate and change color-schemes on the fly.

Generate and change color-schemes on the fly. Pywal is a tool that generates a color palette from the dominant colors in an image. It then applies the

6.9k Jan 03, 2023

Computer art based on joining transparent images

Computer Art There is no must in art because art is free. Introduction The following tutorial exaplains how to generate computer art based on a series

12 Jul 30, 2022

This is a tool to automate the icons generation from sets of svg files into fonts and atlases.

SVG Icon processing tool for C++

66 Dec 14, 2022

Pixel Brush Processing Unit

Pixel Brush Processing Unit The Pixel Brush Processing Unit (PBPU for short) is a simple 4-Bit CPU I designed in Logisim while I was still in school a

2 Nov 03, 2022

3D Model files and source code for rotating turntable. Raspberry Pi, DC servo and PWM modulator required.

3DSimpleTurntable 3D Model files and source code for rotating turntable. Raspberry Pi, DC servo and PWM modulator required. Preview Construction Print

1 Feb 13, 2022

MaryJane is a simple MJPEG server written in Python.

152 Dec 13, 2022

Bot by image recognition simulating (random) human clicks

bbbot22 bot por reconhecimento de imagem simulando cliques humanos (aleatórios) inb4: sim, esse é basicamente o mesmo bot de 2021 porque a Globo não t

2 Apr 05, 2022

Hacking github graph with a easy python script

Hacking-Github-Graph Hacking github graph with a easy python script Requirements git latest version installed. A text editor (eg: vs code, sublime tex

1 Nov 01, 2021

Depix is a tool for recovering passwords from pixelized screenshots.

This implementation works on pixelized images that were created with a linear box filter. In this article I cover background information on pixelization and similar research.

23.1k Jan 04, 2023

API to help generating QR-code for ZATCA's e-invoice known as Fatoora with any programming language

You can try it @ api-fatoora api-fatoora API to help generating QR-code for ZATCA's e-invoice known as Fatoora with any programming language Disclaime

12 Oct 05, 2022

An API which would colorize a black and white image

Image Colorization API Machine Learning Model used- https://github.com/richzhang/colorization/tree/caffe Paper - https://arxiv.org/abs/1603.08511 Step

4 Nov 23, 2021

Convert photos to paintings with python

Convert-photos-to-paintings Before the changes After the changes Before the changes After the changes This code is written in the Python programming l

3 May 31, 2022

An agnostic Canvas API for the browser-less and insane

Apollo An agnostic Canvas API for the browser-less and mildly insane. Project Apollo is a Pythonic re-imagining of HTML Canvas element implementati

1 Jan 13, 2022

Django helper application to easily and non-destructively crop arbitrarily large images in admin and frontend.

django-image-cropping django-image-cropping is an app for cropping uploaded images via Django's admin backend using Jcrop. Screenshot: django-image-cr

546 Jan 03, 2023

HtmlWebShot - A python3 package which Can Create Images From url, Html-CSS, Svg and from any readable file and texts with many setup features.

A python3 package which Can Create Images From url, Html-CSS, Svg and from any readable file and texts with many setup features

24 Dec 14, 2022

An executor that performs standard pre-processing and normalization on images.

6 Jun 30, 2022