CLIP: Connecting Text and Image (Learning Transferable Visual Models From Natural Language Supervision)

Last update: Jan 07, 2023

Overview

CLIP (Contrastive Language–Image Pre-training)

Experiments (Evaluation)

Model	Dataset	Acc (%)
ViT-B/32 (Paper)	CIFAR100	65.1
ViT-B/32 (Our)	CIFAR100	61.71
ViT-B/32 (Paper	CIFAR10	91.3
ViT-B/32 (Our)	CIFAR10	88.8

Overview

Training

Work In Process

Usage

Evaluation

python evaluation.py --dataset CIFAR100 --cuda True

args
- dataset (str): CIFAR10, CIFAR100 (default: CIFAR100)
- num_workers (int): default: 0
- batch_size (int): default: 128
- cuda (bool): False
Training
- Prepare Data
  - Visual Genome Dataset link
  - Download (images, region descriptions)
- training
```
python main.py --base_dir ./ --cuda True
```

Reference

paper link
Author: Alec Radford, Jong Wook Kim, Chris Hallacy, Girish Sastry, Amanda Askell, Pamela Mishkin, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Jack Clark, Gretchen Krueger, Ilya Sutskever
OpenAI

Owner

Myeongjun Kim

Computer Vision Research using Deep Learning

GitHub Repository

2021-MICCAI-Progressively Normalized Self-Attention Network for Video Polyp Segmentation

2021-MICCAI-Progressively Normalized Self-Attention Network for Video Polyp Segmentation Authors: Ge-Peng Ji*, Yu-Cheng Chou*, Deng-Ping Fan, Geng Che

85 Dec 30, 2022

TensorFlow implementation of ENet, trained on the Cityscapes dataset.

segmentation TensorFlow implementation of ENet (https://arxiv.org/pdf/1606.02147.pdf) based on the official Torch implementation (https://github.com/e

248 Dec 16, 2022

SMIS - Semantically Multi-modal Image Synthesis(CVPR 2020)

Semantically Multi-modal Image Synthesis Project page / Paper / Demo Semantically Multi-modal Image Synthesis(CVPR2020). Zhen Zhu, Zhiliang Xu, Anshen

316 Dec 01, 2022

Python implementation of NARS (Non-Axiomatic-Reasoning-System)

11 Dec 20, 2022

CVPR 2021 Official Pytorch Code for UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training

UC2 UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu,

28 Dec 30, 2022

Code repo for "Towards Interpretable Deep Networks for Monocular Depth Estimation" paper.

InterpretableMDE A PyTorch implementation for "Towards Interpretable Deep Networks for Monocular Depth Estimation" paper. arXiv link: https://arxiv.or

16 Aug 12, 2022

Single cell current best practices tutorial case study for the paper:Luecken and Theis, "Current best practices in single-cell RNA-seq analysis: a tutorial"

Scripts for "Current best-practices in single-cell RNA-seq: a tutorial" This repository is complementary to the publication: M.D. Luecken, F.J. Theis,

968 Dec 28, 2022

FedGS: A Federated Group Synchronization Framework Implemented by LEAF-MX.

FedGS: Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT Preparation For instructions on generating data, plea

9 Dec 22, 2022

Blind Image Super-resolution with Elaborate Degradation Modeling on Noise and Kernel

Blind Image Super-resolution with Elaborate Degradation Modeling on Noise and Kernel This repository is the official PyTorch implementation of BSRDM w

69 Jan 05, 2023

Pytoydl: A toy deep learning framework built upon numpy.

Documents: https://pytoydl.readthedocs.io/zh/latest/ Pytoydl A toy deep learning framework built upon numpy. You can star this repository to keep trac

28 Dec 10, 2022

An experiment to bait a generalized frontrunning MEV bot

Honeypot 🍯 A simple experiment that: Creates a honeypot contract Baits a generalized fronturnning bot with a unique transaction Analyze bot behaviour

14 Nov 24, 2022

Optimizing Value-at-Risk and Conditional Value-at-Risk of Black Box Functions with Lacing Values (LV)

BayesOpt-LV Optimizing Value-at-Risk and Conditional Value-at-Risk of Black Box Functions with Lacing Values (LV) About This repository contains the s

1 Nov 11, 2021

This is the code for the paper "Motion-Focused Contrastive Learning of Video Representations" (ICCV'21).

Motion-Focused Contrastive Learning of Video Representations Introduction This is the code for the paper "Motion-Focused Contrastive Learning of Video

11 Sep 23, 2022

[CVPR 2022 Oral] Versatile Multi-Modal Pre-Training for Human-Centric Perception

Versatile Multi-Modal Pre-Training for Human-Centric Perception Fangzhou Hong1 Liang Pan1 Zhongang Cai1,2,3 Ziwei Liu1* 1S-Lab, Nanyang Technologic

96 Jan 03, 2023

HuSpaCy: industrial-strength Hungarian natural language processing

HuSpaCy: Industrial-strength Hungarian NLP HuSpaCy is a spaCy model and a library providing industrial-strength Hungarian language processing faciliti

120 Dec 14, 2022

Which Style Makes Me Attractive? Interpretable Control Discovery and Counterfactual Explanation on StyleGAN

Interpretable Control Exploration and Counterfactual Explanation (ICE) on StyleGAN Which Style Makes Me Attractive? Interpretable Control Discovery an

11 Dec 01, 2022

The official repository for our paper "The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers". We significantly improve the systematic generalization of transformer models on a variety of datasets using simple tricks and careful considerations.

Codebase for training transformers on systematic generalization datasets. The official repository for our EMNLP 2021 paper The Devil is in the Detail:

57 Nov 21, 2022

Remote sensing change detection tool based on PaddlePaddle

PdRSCD PdRSCD（PaddlePaddle Remote Sensing Change Detection）是一个基于飞桨PaddlePaddle的遥感变化检测的项目，pypi包名为ppcd。目前0.2版本，最新支持图像列表输入的训练和预测，如多期影像、多源影像甚至多期多源影像。可以快速完

38 Aug 31, 2022

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

The Official PyTorch Implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

3 Oct 15, 2021

Softlearning is a reinforcement learning framework for training maximum entropy policies in continuous domains. Includes the official implementation of the Soft Actor-Critic algorithm.

Softlearning Softlearning is a deep reinforcement learning toolbox for training maximum entropy policies in continuous domains. The implementation is

997 Dec 30, 2022

CLIP: Connecting Text and Image (Learning Transferable Visual Models From Natural Language Supervision)

Related tags

Overview

CLIP (Contrastive Language–Image Pre-training)

Experiments (Evaluation)

Overview

Training

Usage

Reference

Owner

Myeongjun Kim

2021-MICCAI-Progressively Normalized Self-Attention Network for Video Polyp Segmentation

TensorFlow implementation of ENet, trained on the Cityscapes dataset.

SMIS - Semantically Multi-modal Image Synthesis(CVPR 2020)

Python implementation of NARS (Non-Axiomatic-Reasoning-System)

CVPR 2021 Official Pytorch Code for UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training

Code repo for "Towards Interpretable Deep Networks for Monocular Depth Estimation" paper.

Single cell current best practices tutorial case study for the paper:Luecken and Theis, "Current best practices in single-cell RNA-seq analysis: a tutorial"

FedGS: A Federated Group Synchronization Framework Implemented by LEAF-MX.

Blind Image Super-resolution with Elaborate Degradation Modeling on Noise and Kernel

Pytoydl: A toy deep learning framework built upon numpy.

An experiment to bait a generalized frontrunning MEV bot

Optimizing Value-at-Risk and Conditional Value-at-Risk of Black Box Functions with Lacing Values (LV)

This is the code for the paper "Motion-Focused Contrastive Learning of Video Representations" (ICCV'21).

[CVPR 2022 Oral] Versatile Multi-Modal Pre-Training for Human-Centric Perception

HuSpaCy: industrial-strength Hungarian natural language processing

Which Style Makes Me Attractive? Interpretable Control Discovery and Counterfactual Explanation on StyleGAN

The official repository for our paper "The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers". We significantly improve the systematic generalization of transformer models on a variety of datasets using simple tricks and careful considerations.

Remote sensing change detection tool based on PaddlePaddle

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

Softlearning is a reinforcement learning framework for training maximum entropy policies in continuous domains. Includes the official implementation of the Soft Actor-Critic algorithm.