Skip to content

RAdiological Text Captioning for Human Examined Thoraxes

License

Notifications You must be signed in to change notification settings

farrell236/RATCHET

Repository files navigation

RATCHET: RAdiological Text Captioning for Human Examined Thoraxes

RATCHET is a Medical Transformer for Chest X-ray Diagnosis and Reporting. Based on the architecture featured in Attention Is All You Need. This network is trained and validated on the MIMIC-CXR v2.0.0 dataset.

Architecture

RATCHET Architecture

Run the code

Download pretrained weights and put in ./checkpoints folder.

Start streamlit to run the webapp:

streamlit run web_demo.py
Environment:
Python 3.9.10
Packages:
imageio                  2.26.0
matplotlib               3.7.1
numpy                    1.23.5
pandas                   1.5.3
scikit-image             0.20.0
streamlit                1.20.0
tensorflow               2.11.0
tokenizers               0.13.2
tqdm                     4.64.1

Docker Container

Build the docker container:

docker build -t ratchet ./Dockerfile

Run the docker image on CXR images:

docker run --user $(id -u):$(id -g) \
-v /path/to/image_input_folder:/code/RATCHET/inp_folder \
-v /path/to/report_output_folder:/code/RATCHET/out_folder:rw \
-i -t ratchet python run_model.py

Each image in inp_folder would have a corresponding .txt report saved in out_folder.

Results

     Cardiomegaly           Cardiomegaly Attention Plot     

Generated Text:

In comparison with the study of ___, there is little overall change. Again there is substantial enlargement of the cardiac silhouette with a dual-channel pacer device in place. No evidence of vascular congestion or acute focal pneumonia. Blunting of the costophrenic angles is again seen.

More Examples

More Captioning Examples