Omnivore, a non picky AI model, focuses on images, videos and 3D data!

publisher ：Towhee Technical team

Tired of using different models for different data ？ Have you ever thought that one model can handle data of different modes ？ finally , stay 2022 Beginning of the year Meta AI Launched “ Omnivores ” Omnivore, One model handles different visual modal data , The image can be 、 video 、3D Data are classified .Omnivore Not only compatible with many types of data , It is also among the best in the data sets of different tasks .Omnivore In the image classification dataset ImageNet Up to 86.0% precision ; In... For action recognition Kinetics Data sets can achieve 84.1% precision ; For single view 3D Scene classification SUN RGB-D On dataset , The accuracy is also as high as 67.1% .

Omnivore: Multiple visual modalities

Omnivore Convert the data of different visual modes into general vector format , And then use it Transformer Unique flexibility , Carry out joint training for classification tasks of different modes . Whether it's training from scratch , Or fine tune the pre training model , Just use Omnivore And ready-made standard data sets , It can make its performance reach or even exceed the corresponding single model .

Reference material ：

Model use case ：action-classification/omnivore

Address of thesis ：OMNIVORE: A Single Model for Many Visual Modalities

More information ： Facebook AI Introduction “ Supermodel ”： Get the image 、 Video and 3D Three classification tasks of data , Performance is not inferior to independent models

For more project updates and details, please pay attention to our project ( https://github.com/towhee-io/towhee/blob/main/towhee/models/README_CN.md) , Your attention is a powerful driving force for us to generate electricity with love , welcome star, fork, slack Three even :)