Applying PVT detection framework, I tried a CycleMLP-B1 based detector with RetinaNet 1x. I got AP=27.1, fairly inferior to the reported 38.6. Could you give some advices to reproduce the reported result?

The specific configure is as follows

base = [ 'base/models/retinanet_r50_fpn.py', 'base/datasets/coco_detection.py', 'base/schedules/schedule_1x.py', 'base/default_runtime.py' ] #optimizer model = dict( pretrained='./pretrained/CycleMLP_B1.pth', backbone=dict( type='CycleMLP_B1_feat', style='pytorch'), neck=dict( type='FPN', in_channels=[64, 128, 320, 512], out_channels=256, start_level=1, add_extra_convs='on_input', num_outs=5)) #optimizer optimizer = dict(delete=True, type='AdamW', lr=0.0001, weight_decay=0.0001) optimizer_config = dict(grad_clip=None)

find_unused_parameters = True

Thank you for this great repo and interesting paper.

I tried compiling CycleMLP to onnx and not surpassingly the process failed since CycleMLP include dynamic offset creation in https://github.com/ShoufaChen/CycleMLP/blob/main/cycle_mlp.py#L132 and as such cannot be converted to a frozen graph. Were you able to convert CycleMLP to onnx or any other frozen graph framework?

Thanks in advance.

Hi, thanks for your wonderful work.

I'm currently studying your work, and come up with some question about the offset calculations.

I understood the offset calculation mentioned on the paper, but can't understand about how generated offset is being used in the code.

For ex) if $S_H \times S_W : 3 \times 1$; I understood how the offset is applied in this figure

by calculate like this:

However, when I run the offset generating code, I can't figure out how this offset is being used in deform_conv2d

Can you provide more detailed information about this??

And also, the paper contains how $S_H \times S_W: 3 \times 3$ works, but in the code, it seems like either one ofkernel_size[0] or kernel_size[1] has to be 1. So, if I want to use $S_H \times S_W : 3 \times 3$, do I have to make $3 \times 1$ and $1 \times 3$ offsets and add those together?

Thank you again for your work. I really learned a lot.

Hello.

First of all, thank you for curating this interesting work. I was wondering, are there any working examples of how I can use CycleMLP for dense prediction while maintaining the original input size (e.g., predict a 0 or 1 value for each pixel in an input image)? In addition, I am interested in only a single ("annotated") output image, although I noticed the model definitions given in this repository output multiple downsampled versions of the original input image. Any thoughts on this?

Thank you in advance for your time.

For classificaion on ImageNet-1k, the acuracy of Swin-B is 83.5, which is 0.1 higher than the proposed CycleMLP-B. But, in this paper, the authors reprot that the accuracy of Swin-B is 83.3, which is 0.1 lower than the proposed CycleMLP-B. Why are these accuracies different?

Thanks for your work!

The implementation of this code inspired me. But the calculation of offset here is confusing. Although this issue (https://github.com/ShoufaChen/CycleMLP/issues/10) has asked similar questions, I haven't found a reasonable explanation.

https://github.com/ShoufaChen/CycleMLP/blob/2f76a1f6e3cc6672143fdac46e3db5f9a7341253/cycle_mlp.py#L127-L136

kernel_size = (1, 3)
start_idx = (kernel_size[0] * kernel_size[1]) // 2
for i in range(num_channels):
    offset[0, 2 * i + 0, 0, 0] = 0
    # relative offset
    offset[0, 2 * i + 1, 0, 0] = (i + start_idx) % kernel_size[1] - (kernel_size[1] // 2)
offset.reshape(num_channels, 2)

tensor([[ 0.,  0.],
        [ 0.,  1.],
        [ 0., -1.],
        [ 0.,  0.],
        [ 0.,  1.],
        [ 0., -1.]])

the results are different with the figure in paper:

Some codes for verification:

import torch
from torchvision.ops import deform_conv2d

num_channels = 6

data = torch.arange(1, 6).reshape(1, 1, 1, 5).expand(-1, num_channels, -1, -1)
data
"""
tensor([[[[1, 2, 3, 4, 5]],
         [[1, 2, 3, 4, 5]],
         [[1, 2, 3, 4, 5]],
         [[1, 2, 3, 4, 5]],
         [[1, 2, 3, 4, 5]],
         [[1, 2, 3, 4, 5]]]])
"""

weight = torch.eye(num_channels).reshape(num_channels, num_channels, 1, 1)
weight.reshape(num_channels, num_channels)
"""
tensor([[1., 0., 0., 0., 0., 0.],
        [0., 1., 0., 0., 0., 0.],
        [0., 0., 1., 0., 0., 0.],
        [0., 0., 0., 1., 0., 0.],
        [0., 0., 0., 0., 1., 0.],
        [0., 0., 0., 0., 0., 1.]])
"""

offset = torch.empty(1, 2 * num_channels * 1 * 1, 1, 1)
kernel_size = (1, 3)
start_idx = (kernel_size[0] * kernel_size[1]) // 2
for i in range(num_channels):
    offset[0, 2 * i + 0, 0, 0] = 0
    # relative offset
    offset[0, 2 * i + 1, 0, 0] = (
        (i + start_idx) % kernel_size[1] - (kernel_size[1] // 2)
    )
offset.reshape(num_channels, 2)
"""
tensor([[ 0.,  0.],
        [ 0.,  1.],
        [ 0., -1.],
        [ 0.,  0.],
        [ 0.,  1.],
        [ 0., -1.]])
"""

deform_conv2d(
    data.float(), 
    offset=offset.expand(-1, -1, -1, 5).float(), 
    weight=weight.float(), 
    bias=None,
)
"""
tensor([[[[1., 2., 3., 4., 5.]],
         [[2., 3., 4., 5., 0.]],
         [[0., 1., 2., 3., 4.]],
         [[1., 2., 3., 4., 5.]],
         [[2., 3., 4., 5., 0.]],
         [[0., 1., 2., 3., 4.]]]])
"""

Hi, thank you very much for your excellent work. In Fig.4 of your paper, you show the pseudo-kernel when kernel size is 1x3. But I when I find that function "gen_offset" does not generate the same offset as Fig.4. The offset it generates is "0,1,0,-1,0,0,0,1..." instead of "0,1,0,-1,0,1,0,-1', which is shown in Fig.4. So could you please tell me the reason?

Hi, thanks for your great work and code. I wonder the parameter crop_pct actually works in which part of code. When I go throught the timm, I can't find out how this crop_pct is loaded.

Thank you very much for such a wonderful work!

After learning the cycle_mlp source code in the repository, I am very confused to deploy CycleMLP Block based on Swin Transformer. Is it convenient for you to release swin-based CycleMLP? Looking forward to your reply, Thanks!