Med-VQA
In this repository we have tested 3 VQA models on the ImageCLEF-2019 dataset. Two of these are made on top of Facebook AI Reasearch's Multi-Modal Framework (MMF).
| Model Name | Accuracy | Number of Epochs |
|---|---|---|
| Hierarchical Question-Image Co-attention | 48.32% | 42 |
| MMF Transformer | 51.76% | 30 |
| MMBT | 86.78% | 30 |
Test them for yourself !
Download the dataset from here and place it in a directory named /dataset/med-vqa-data/ in the directory where this repository is cloned.
MMF Transformer:
mmf_run config=projects/hateful_memes/configs/mmf_transformer/defaults.yaml model=mmf_transformer dataset=hateful_memes training.checkpoint_interval=100 training.max_updates=3000
MMBT :
mmf_run config=projects/hateful_memes/configs/mmbt/defaults.yaml model=mmbt dataset=hateful_memes training.checkpoint_interval=100 training.max_updates=3000
Heirarchical Question-Image Co-attention:
cd hierarchical \
python main.py
4 Feb 18, 2022
1 Nov 26, 2022
13 Sep 08, 2022
3 May 12, 2022
289 Jan 06, 2023
20 Dec 29, 2022
21 Dec 17, 2022
49 Aug 24, 2022
27 Jan 05, 2023
116 Dec 27, 2022
41 Dec 30, 2022
3k Jan 06, 2023
1 Nov 02, 2021
3 Jan 12, 2022
29 Oct 16, 2022
15 Oct 24, 2022
20 Dec 13, 2022
255 Dec 27, 2022
193 Jan 04, 2023
1 Dec 23, 2022