Hi ~ @Green-Wood.

I feel grateful mind for your help. I wanna get predict.py code that prints latex from an input image. If this code is provided, it will be very useful to others as well.

Best regards.

hello!thanks for your time! When I transfer some code in decoder or use it directly,the val_exprate are always be 0.000,I don't know why. Another problem is,I noticed that this code don't have the function to save checkpoint or something.Can you give me some help?Thanks again!

When I retrained the model according to the instruction, the val_exprate was always 0.00, did anyone encounter this problem, thank you! (I has not modified any codes) @Green-Wood

When I run test.py code, the following error occurs. Can i get some helps?

in test.py code test_year = "2016" ckp_path = "pretrained model"

GPU available: True, used: True
TPU available: False, using: 0 TPU cores
Load data from: /home/motive/PycharmProjects/BTTR/bttr/datamodule/../../data.zip
Extract data from: 2016, with data size: 1147
total  1147 batch data loaded
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
Testing: 100%|██████████| 1147/1147 [07:34<00:00,  2.01s/it]ExpRate: 0.32258063554763794
length of total file: 1147
Testing: 100%|██████████| 1147/1147 [07:34<00:00,  2.52it/s]
--------------------------------------------------------------------------------
DATALOADER:0 TEST RESULTS
{}
--------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/motive/PycharmProjects/BTTR/test.py", line 17, in <module>
    trainer.test(model, datamodule=dm)
  File "/home/motive/anaconda3/envs/bttr/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 579, in test
    results = self._run(model)
  File "/home/motive/anaconda3/envs/bttr/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 759, in _run
    self.post_dispatch()
  File "/home/motive/anaconda3/envs/bttr/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 789, in post_dispatch
    self.accelerator.teardown()
  File "/home/motive/anaconda3/envs/bttr/lib/python3.7/site-packages/pytorch_lightning/accelerators/gpu.py", line 51, in teardown
    self.lightning_module.cpu()
  File "/home/motive/anaconda3/envs/bttr/lib/python3.7/site-packages/pytorch_lightning/utilities/device_dtype_mixin.py", line 141, in cpu
    return super().cpu()
  File "/home/motive/anaconda3/envs/bttr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 471, in cpu
    return self._apply(lambda t: t.cpu())
  File "/home/motive/anaconda3/envs/bttr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 359, in _apply
    module._apply(fn)
  File "/home/motive/anaconda3/envs/bttr/lib/python3.7/site-packages/torchmetrics/metric.py", line 317, in _apply
    setattr(this, key, [fn(cur_v) for cur_v in current_val])
  File "/home/motive/anaconda3/envs/bttr/lib/python3.7/site-packages/torchmetrics/metric.py", line 317, in <listcomp>
    setattr(this, key, [fn(cur_v) for cur_v in current_val])
  File "/home/motive/anaconda3/envs/bttr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 471, in <lambda>
    return self._apply(lambda t: t.cpu())
AttributeError: 'tuple' object has no attribute 'cpu'

Hi, thank you for great repository!

How long does it take to train for your experiment in the paper? I mean training on CROHME 2014/2016/2019 on four NVIDIA 1080Ti GPUs.

Thanks,

Hi~ @Green-Wood

I wanna apply trasnfer learning using pretrained model.

but, LightningCLI() is wrapped and difficult to customize.

Thanks & best regards.

Hi, I wanna test your BTTR model but, it need to training process which will take a lot of time. So, can you give me a pretrained model link?

Best regards.

Hi , getting error after adding new token in dictionary.txt

Error(s) in loading state_dict for LitBTTR: size mismatch for bttr.decoder.word_embed.0.weight: copying a param with shape torch.Size([113, 256]) from checkpoint, the shape in current model is torch.Size([115, 256]). size mismatch for bttr.decoder.proj.weight: copying a param with shape torch.Size([113, 256]) from checkpoint, the shape in current model is torch.Size([115, 256]). size mismatch for bttr.decoder.proj.bias: copying a param with shape torch.Size([113]) from checkpoint, the shape in current model is torch.Size([115]).

Kindly help me out how can i fix this error.

Could you tell me how to generate the offline math expression image from inkml file? My experiment show that a large scale image could improve the result obviously，so I'd like to know if there is unified offline data for academic research.

Hi ,

As this model is a bit slower compared to the existing state-of-the-art model on CPU. So I tried to make predictions on GPU and surprisingly it slower on Gpu compare to CPU as well.

I am attaching a code snapshot here

device = torch.device('cuda')if torch.cuda.is_available() else torch.device('cpu')

model = LitBTTR.load_from_checkpoint('pretrained-2014.ckpt',map_location=device)

img = Image.open(img_path) img = ToTensor()(img) img.to(device)

t1 = time.time() hyp = model.beam_search(img) t2 = time.time()

Kindly help me out here how i can reduce prediction time

FYI - using GPU on aws g4dn.xlarge configuration machine

hello i don't know how to add scalar to TensorBoard? I want to do this kind of topic, hoping to improve some ExpRate, but I don’t know much about lightning TensorBoard.