[CVPR 2022] Structured Sparse R-CNN for Direct Scene Graph Generation

Related tags

Deep LearningStructured-Sparse-RCNN
Overview

Structured Sparse R-CNN for Direct Scene Graph Generation

Our paper Structured Sparse R-CNN for Direct Scene Graph Generation has been accepted by CVPR 2022.

Requirements

Environments: python 3.8, cuda 10.1, pytorch 1.7.1

To install requirements:

conda create --name scene_graph_benchmark
conda activate scene_graph_benchmark

pip install --user ipython
pip install --user scipy
pip install --user h5py
pip install --user pyyaml
pip install --user yacs
pip install --user scipy
pip install --user h5py
pip install --user tqdm
pip install --user opencv-python

pip install --user ninja yacs cython matplotlib tqdm opencv-python overrides


conda install pytorch==1.7.1 torchvision==0.8.2 cudatoolkit=10.1 -c pytorch

git clone https://github.com/cocodataset/cocoapi.git
cd cocoapi/PythonAPI
python setup.py build_ext install

git clone https://github.com/NVIDIA/apex.git
cd apex
python setup.py install --cuda_ext --cpp_ext

# put our code SGGbench into your directory, such as /home/username
cd /home/username/SGGbench 
python setup.py build develop

Datasets

VG

For the dataset and pretrained backbone weights preparation, please follow: Scene-Graph-Benchmark.pytorch

OI v4, v6

For the datasets and pretrained backbone weights preparation, please follow: BGNN-SGG. Actually, BGNN-SGG is also compatible with Scene-Graph-Benchmark.pytorch.

Training

VG

On VG, we notice ~0.2 [email protected] noise for our model with 300 queries.

To train the model with 300 triplet queries in the paper, run this command. The results are [email protected]: 36.9; [email protected]: 3.7; [email protected]: 10.0: (8 RTX 2080ti 11G)

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --master_port 10020 --nproc_per_node=8 tools/relation_train_net.py --config-file "configs/simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 8 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/pretrained_faster_rcnn/model_final.pth OUTPUT_DIR Outputs/real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 256 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

To train the model with 800 triplet queries in the paper, run this command. The results are [email protected]: 38.4; [email protected]: 4.0; [email protected]: 10.3: (8 V100 32G)

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --master_port 10020 --nproc_per_node=8 tools/relation_train_net.py --config-file "configs/simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 8 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/pretrained_faster_rcnn/model_final.pth OUTPUT_DIR Outputs/real025kl_newe2rposition_fullobjbranch_800q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 800 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 256 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

To train the model with 300 triplet queries and backbone, run this command. The results are [email protected]: 36.7; [email protected]: 3.8; [email protected]: 10.1: (8 V100 32G)

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --master_port 10020 --nproc_per_node=8 tools/relation_train_net.py --config-file "configs/simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 8 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" SOLVER.BACKBONE_MULTIPLIER 0.1 MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/pretrained_faster_rcnn/model_final.pth OUTPUT_DIR Outputs/real025kl_newe2rposition_fullobjbranch_800q_smalldyconv_prequeryaug9_norel_hardlabel_bkb MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE False MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 256 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

OI V4

To train the model with 300 triplet queries in the paper, run this command: (8 RTX 2080ti 11G)

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --master_port 10020 --nproc_per_node=8 tools/relation_train_net.py --config-file "configs/oiv4_simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 8 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/iov4_pretrain_faster_rcnn/model_final.pth OUTPUT_DIR Outputs/oiv4_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 1024 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

OI V6

To train the model with 300 triplet queries in the paper, run this command: (8 RTX 2080ti 11G)

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --master_port 10020 --nproc_per_node=8 tools/relation_train_net.py --config-file "configs/oiv6_simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 8 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/iov6_pretrain_faster_rcnn/model_final.pth OUTPUT_DIR Outputs/oiv6_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 1024 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

Evaluation

VG

300 queries

To evaluate my model, run:

CUDA_VISIBLE_DEVICES=4,5,6,7 python -m torch.distributed.launch --master_port 10029 --nproc_per_node=4 tools/relation_test_net.py --config-file "configs/simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 4 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.WEIGHT Outputs/real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel/model_final.pth OUTPUT_DIR Outputs/eva_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 256 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

📋 Before testing, create a new directory, named 'eva_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel', in ./SGGbench/Outputs/ .

To evaluate my model with TDE, run:

CUDA_VISIBLE_DEVICES=2,3,4,5 python -m torch.distributed.launch --master_port 10029 --nproc_per_node=4 tools/relation_test_net.py --config-file "configs/simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 4 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.WEIGHT Outputs/real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel/model_final.pth OUTPUT_DIR Outputs/tde_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS True MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 256 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

📋 Before testing, create a new directory, named 'tde_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel', in ./SGGbench/Outputs/ .

To evaluate my model with our LA, run:

CUDA_VISIBLE_DEVICES=4,5,6,7 python -m torch.distributed.launch --master_port 10029 --nproc_per_node=4 tools/relation_test_net.py --config-file "configs/simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 4 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.WEIGHT Outputs/real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel/model_final.pth OUTPUT_DIR Outputs/la03_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 256 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True MODEL.SimrelRCNN.REL_LOGITS_ADJUSTMENT True MODEL.SimrelRCNN.LOGIT_ADJ_TAU 0.3

📋 Before testing, create a new directory, named 'la03_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel', in ./SGGbench/Outputs/ .

800 queries

To evaluate my model, run:

CUDA_VISIBLE_DEVICES=3,5,6,7 python -m torch.distributed.launch --master_port 10029 --nproc_per_node=4 tools/relation_test_net.py --config-file "configs/simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 4 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.WEIGHT Outputs/real025kl_newe2rposition_fullobjbranch_800q_smalldyconv_prequeryaug9_norel_hardlabel/model_final.pth OUTPUT_DIR Outputs/eva_real025kl_newe2rposition_fullobjbranch_800q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 800 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 256 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

📋 Before testing, create a new directory, named 'eva_real025kl_newe2rposition_fullobjbranch_800q_smalldyconv_prequeryaug9_norel_hardlabel', in ./SGGbench/Outputs/ .

To evaluate my model with TDE, run:

CUDA_VISIBLE_DEVICES=2,3,4,5 python -m torch.distributed.launch --master_port 10029 --nproc_per_node=4 tools/relation_test_net.py --config-file "configs/simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 4 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.WEIGHT Outputs/real025kl_newe2rposition_fullobjbranch_800q_smalldyconv_prequeryaug9_norel_hardlabel/model_final.pth OUTPUT_DIR Outputs/tde_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 800 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS True MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 256 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

📋 Before testing, create a new directory, named 'tde_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel', in ./SGGbench/Outputs/ .

To evaluate my model with our LA, run:

CUDA_VISIBLE_DEVICES=3,5,6,7 python -m torch.distributed.launch --master_port 10029 --nproc_per_node=4 tools/relation_test_net.py --config-file "configs/simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 4 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.WEIGHT Outputs/real025kl_newe2rposition_fullobjbranch_800q_smalldyconv_prequeryaug9_norel_hardlabel/model_final.pth OUTPUT_DIR Outputs/LA03_real025kl_newe2rposition_fullobjbranch_800q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 800 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 256 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True MODEL.SimrelRCNN.REL_LOGITS_ADJUSTMENT True MODEL.SimrelRCNN.LOGIT_ADJ_TAU 0.3

📋 Before testing, create a new directory, named 'LA03_real025kl_newe2rposition_fullobjbranch_800q_smalldyconv_prequeryaug9_norel_hardlabel', in ./SGGbench/Outputs/ .

OI V4

300 queries

To evaluate my model, run:

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --master_port 10020 --nproc_per_node=8 tools/relation_test_net.py --config-file "configs/oiv4_simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 8 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.WEIGHT Outputs/oiv4_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel/model_final.pth OUTPUT_DIR Outputs/eva_oiv4_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 1024 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

📋 Before testing, create a new directory, named 'eva_oiv4_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel', in ./SGGbench/Outputs/ .

To evaluate my model with our LA, run:

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --master_port 10020 --nproc_per_node=8 tools/relation_test_net.py --config-file "configs/oiv4_simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 8 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.WEIGHT Outputs/oiv4_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel/model_final.pth OUTPUT_DIR Outputs/la_oiv4_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 1024 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True MODEL.SimrelRCNN.REL_LOGITS_ADJUSTMENT True MODEL.SimrelRCNN.LOGIT_ADJ_TAU 0.3

📋 Before testing, create a new directory, named 'la_oiv4_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel', in ./SGGbench/Outputs/ .

OI V6

300 queries

To evaluate my model, run:

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --master_port 10020 --nproc_per_node=8 tools/relation_test_net.py --config-file "configs/oiv6_simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 8 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.WEIGHT Outputs/oiv6_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel/model_final.pth OUTPUT_DIR Outputs/eva_oiv6_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 1024 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True

📋 Before testing, create a new directory, named 'eva_oiv6_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel', in ./SGGbench/Outputs/ .

To evaluate my model with our LA, run:

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --master_port 10020 --nproc_per_node=8 tools/relation_test_net.py --config-file "configs/oiv6_simrel_e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE none MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs SOLVER.IMS_PER_BATCH 8 TEST.IMS_PER_BATCH 8 SOLVER.VAL_PERIOD 12000 SOLVER.MAX_ITER 80000 SOLVER.STEPS '(47000, 64000)' SOLVER.BASE_LR 0.000008 SOLVER.OPTIMIZER "ADAMW" MODEL.WEIGHT Outputs/oiv6_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel/model_final.pth OUTPUT_DIR Outputs/la_oiv6_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel MODEL.ROI_BOX_HEAD.MLP_HEAD_DIM 1024 MODEL.SimrelRCNN.ENABLE_BG_OBJ False MODEL.SimrelRCNN.ENABLE_REL_X2Y True MODEL.SimrelRCNN.REL_DIM 256 MODEL.SimrelRCNN.NUM_PROPOSALS 300 MODEL.SimrelRCNN.NUM_HEADS 6 MODEL.SimrelRCNN.REL_STACK_NUM 6 MODEL.SimrelRCNN.TRIPLET_MASK_WEIGHT 1.0 MODEL.SimrelRCNN.FREEZE_BACKBONE True MODEL.SimrelRCNN.CROSS_OBJ_FEAT_FUSION False MODEL.SimrelRCNN.CLASS_WEIGHT 1.333 MODEL.SimrelRCNN.L1_WEIGHT 5.0 MODEL.SimrelRCNN.GIOU_WEIGHT 2.0 GLOVE_DIR '' MODEL.SimrelRCNN.ENABLE_FREQ False MODEL.SimrelRCNN.ENABLE_QUERY_REVERSE False MODEL.SimrelRCNN.USE_REFINE_OBJ_FEATURE True MODEL.SimrelRCNN.FREEZE_PUREE_OBJDET False MODEL.SimrelRCNN.PURE_ENT_NUM_PROPOSALS 100 MODEL.SimrelRCNN.ENABLE_MASK_BRANCH False MODEL.SimrelRCNN.KL_BRANCH_WEIGHT 0.25 MODEL.SimrelRCNN.ENABLE_KL_BRANCH True MODEL.SimrelRCNN.POSI_ENCODE_DIM 64 MODEL.SimrelRCNN.REL_CLASS_WEIGHT 1.334 MODEL.SimrelRCNN.PURE_ENT_CLASS_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_GIOU_WEIGHT 2.0 MODEL.SimrelRCNN.PURE_ENT_L1_WEIGHT 5.0 MODEL.SimrelRCNN.AUXILIARY_BRANCH True MODEL.SimrelRCNN.AUXILIARY_BRANCH_SELECT_ENT_MAX_NUM 25 MODEL.SimrelRCNN.AUXILIARY_BRANCH_START 11 MODEL.SimrelRCNN.ENABLE_ENT_PROP True MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_ANALYSIS False MODEL.SimrelRCNN.USE_CROSS_RANK False MODEL.SimrelRCNN.DISABLE_KQ_FUSION_SELFATTEN False MODEL.SimrelRCNN.DIM_ENT_PRE_CLS 1024 MODEL.SimrelRCNN.DIM_ENT_PRE_REG 256 MODEL.SimrelRCNN.ONE_REL_CONV True MODEL.SimrelRCNN.ENABLE_BATCH_REDUCTION True MODEL.SimrelRCNN.USE_HARD_LABEL_KLMATCH True MODEL.SimrelRCNN.DISABLE_OBJ2REL_LOSS True MODEL.SimrelRCNN.REL_LOGITS_ADJUSTMENT True MODEL.SimrelRCNN.LOGIT_ADJ_TAU 0.3

📋 Before testing, create a new directory, named 'la_oiv6_real025kl_newe2rposition_fullobjbranch_300q_smalldyconv_prequeryaug9_norel_hardlabel', in ./SGGbench/Outputs/ .

Pre-trained Models

You can download pretrained models here:

VG

300 queries

800 queries

OI V4

300 queries

OI V6

300 queries

Results

Our model achieves the following performance on :

Visual Genome

* means 800 queries.

Models SGGen [email protected] SGGen [email protected] SGGen [email protected] SGGen [email protected] SGGen [email protected] SGGen [email protected] SGGen [email protected] SGGen [email protected] SGGen [email protected]
Our model 25.8 32.7 36.9 1.5 2.7 3.7 6.1 8.4 10.0
Our model* 26.1 33.5 38.4 1.5 2.7 4.0 6.2 8.6 10.3
Our model+TDE 14.5 18.3 21.0 1.8 2.7 3.6 10.8 15.0 18.5
Our model*+TDE 15.0 19.7 22.9 1.6 2.7 3.8 9.8 14.6 18.0
Our model+LA 18.4 23.3 26.5 1.9 2.9 4.0 13.5 17.9 21.4
Our model*+LA 18.2 23.7 27.3 2.0 3.1 4.5 13.7 18.6 22.5

Acknowledgement

Our code is mainly based on: Scene-Graph-Benchmark.pytorch, SparseR-CNN and BGNN-SGG.

For this paper, I'm extremely grateful to my advisor Prof. Limin Wang. I should also appreciate my group members for discussing with me: Jing Tan, Ziteng Gao and Jiaqi Tang.

Citations

@inproceedings{ssrcnnsgg22cvpr,
  author    = {Yao Teng and
               Limin Wang},
  title     = {Structured Sparse {R-CNN} for Direct Scene Graph Generation},
  booktitle = {{CVPR}},
  year      = {2022}
}
Owner
Multimedia Computing Group, Nanjing University
Multimedia Computing Group, Nanjing University
GitHub Repository
Python scripts for performing stereo depth estimation using the MobileStereoNet model in ONNX

ONNX-MobileStereoNet Python scripts for performing stereo depth estimation using the MobileStereoNet model in ONNX Stereo depth estimation on the cone

Ibai Gorordo 23 Nov 29, 2022
Code repo for "Transformer on a Diet" paper

Transformer on a Diet Reference: C Wang, Z Ye, A Zhang, Z Zhang, A Smola. "Transformer on a Diet". arXiv preprint arXiv (2020). Installation pip insta

cgraywang 31 Sep 26, 2021
FaceQgen: Semi-Supervised Deep Learning for Face Image Quality Assessment

FaceQgen FaceQgen: Semi-Supervised Deep Learning for Face Image Quality Assessment This repository is based on the paper: "FaceQgen: Semi-Supervised D

Javier Hernandez-Ortega 3 Aug 04, 2022
Pytorch port of Google Research's LEAF Audio paper

leaf-audio-pytorch Pytorch port of Google Research's LEAF Audio paper published at ICLR 2021. This port is not completely finished, but the Leaf() fro

Dennis Fedorishin 80 Oct 31, 2022
Notebooks, slides and dataset of the CorrelAid Machine Learning Winter School

CorrelAid Machine Learning Winter School Welcome to the CorrelAid ML Winter School! Task The problem we want to solve is to classify trees in Roosevel

CorrelAid 12 Nov 23, 2022
基于深度强化学习的原神自动钓鱼AI

原神自动钓鱼AI由YOLOX, DQN两部分模型组成。使用迁移学习，半监督学习进行训练。 模型也包含一些使用opencv等传统数字图像处理方法实现的不可学习部分。

4.2k Jan 01, 2023
Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

LiDAR fog simulation Created by Martin Hahner at the Computer Vision Lab of ETH Zurich. This is the official code release of the paper Fog Simulation

Martin Hahner 110 Dec 30, 2022
TraND: Transferable Neighborhood Discovery for Unsupervised Cross-domain Gait Recognition.

TraND This is the code for the paper "Jinkai Zheng, Xinchen Liu, Chenggang Yan, Jiyong Zhang, Wu Liu, Xiaoping Zhang and Tao Mei: TraND: Transferable

Jinkai Zheng 32 Apr 04, 2022
Scalable Multi-Agent Reinforcement Learning

Scalable Multi-Agent Reinforcement Learning 1. Featured algorithms: Value Function Factorization with Variable Agent Sub-Teams (VAST) [1] 2. Implement

3 Aug 02, 2022
The Python ensemble sampling toolkit for affine-invariant MCMC

emcee The Python ensemble sampling toolkit for affine-invariant MCMC emcee is a stable, well tested Python implementation of the affine-invariant ense

Dan Foreman-Mackey 1.3k Dec 31, 2022
KITTI-360 Annotation Tool is a framework that developed based on python(cherrypy + jinja2 + sqlite3) as the server end and javascript + WebGL as the front end.

KITTI-360 Annotation Tool is a framework that developed based on python(cherrypy + jinja2 + sqlite3) as the server end and javascript + WebGL as the front end.

86 Dec 12, 2022
PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence) and pre-trained model on ImageNet dataset

Reference-Based-Sketch-Image-Colorization-ImageNet This is a PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization usin

Yuzhi ZHAO 11 Jul 28, 2022
Generate images from texts. In Russian. In PaddlePaddle

ruDALL-E PaddlePaddle ruDALL-E in PaddlePaddle. Install: pip install rudalle_paddle==0.0.1rc1 Run with free v100 on AI Studio. Original Pytorch versi

AgentMaker 20 Oct 18, 2022
SGPT: Multi-billion parameter models for semantic search

SGPT: Multi-billion parameter models for semantic search This repository contains code, results and pre-trained models for the paper SGPT: Multi-billi

Niklas Muennighoff 182 Dec 29, 2022
An Efficient Training Approach for Very Large Scale Face Recognition or F²C for simplicity.

Fast Face Classification (F²C) This is the code of our paper An Efficient Training Approach for Very Large Scale Face Recognition or F²C for simplicit

33 Jun 27, 2021
Some code of the implements of Geological Modeling Using 3D Pixel-Adaptive and Deformable Convolutional Neural Network

3D-GMPDCNN Geological Modeling Using 3D Pixel-Adaptive and Deformable Convolutional Neural Network PyTorch implementation of "Geological Modeling Usin

5 Nov 21, 2022
PaddleBoBo是基于PaddlePaddle和PaddleSpeech、PaddleGAN等开发套件的虚拟主播快速生成项目

PaddleBoBo - 元宇宙时代，你也可以动手做一个虚拟主播。 PaddleBoBo是基于飞桨PaddlePaddle深度学习框架和PaddleSpeech、PaddleGAN等开发套件的虚拟主播快速生成项目。PaddleBoBo致力于简单高效、可复用性强，只需要一张带人像的图片和一段文字，就能

502 Jan 08, 2023
Technical Analysis Indicators - Pandas TA is an easy to use Python 3 Pandas Extension with 130+ Indicators

Pandas TA - A Technical Analysis Library in Python 3 Pandas Technical Analysis (Pandas TA) is an easy to use library that leverages the Pandas package

Kevin Johnson 3.2k Jan 09, 2023
Numbering permanent and deciduous teeth via deep instance segmentation in panoramic X-rays

Numbering permanent and deciduous teeth via deep instance segmentation in panoramic X-rays In this repo, you will find the instructions on how to requ

Intelligent Vision Research Lab 4 Jul 21, 2022
Implement face detection, and age and gender classification, and emotion classification.

YOLO Keras Face Detection Implement Face detection, and Age and Gender Classification, and Emotion Classification. (image from wider face dataset) Ove

Chloe 10 Nov 14, 2022