Hello：
There is a obvious difference between the actual evaluation results of the algorithm and those described in the paper, the cause of the problem has not been found yet, please give me some suggestions.
1.Evaluation results for the model provided in github of SamsungLabs/fcaf3d：(12 epoches)
1.1 sunrgbd
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/5050, elapsed: [>>>>>>>>>>>>>>>>] 5050/5050, 15.9 task/s, elapsed: 318s, ETA: 0s
+-------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed | 0.0496 | 0.7612 | 0.0094 | 0.5883 |
| table | 0.1570 | 0.8514 | 0.1222 | 0.7909 |
| sofa | 0.2261 | 0.8469 | 0.1440 | 0.7735 |
| chair | 0.1985 | 0.7662 | 0.1671 | 0.7272 |
| toilet | 0.3586 | 0.9379 | 0.2778 | 0.8552 |
| desk | 0.0970 | 0.7641 | 0.0665 | 0.6982 |
| dresser | 0.2063 | 0.9128 | 0.1956 | 0.8670 |
| night_stand | 0.4174 | 0.9608 | 0.4127 | 0.9451 |
| bookshelf | 0.1585 | 0.8865 | 0.1489 | 0.8511 |
| bathtub | 0.3658 | 0.9592 | 0.3561 | 0.8571 |
+-------------+---------+---------+---------+---------+
| Overall | 0.2235 | 0.8647 | 0.1900 | 0.7954 |
+-------------+---------+---------+---------+---------+
{'bed_AP_0.25': 0.049614179879426956, 'table_AP_0.25': 0.15698249638080597, 'sofa_AP_0.25': 0.2261233627796173, 'chair_AP_0.25': 0.19846875965595245, 'toilet_AP_0.25': 0.35864582657814026, 'desk_AP_0.25': 0.09701932221651077, 'dresser_AP_0.25': 0.20626720786094666, 'night_stand_AP_0.25': 0.41742968559265137, 'bookshelf_AP_0.25': 0.15852327644824982, 'bathtub_AP_0.25': 0.3657997250556946, 'mAP_0.25': 0.22348737716674805, 'bed_rec_0.25': 0.7611650485436893, 'table_rec_0.25': 0.8513628620102215, 'sofa_rec_0.25': 0.84688995215311, 'chair_rec_0.25': 0.7661741214057508, 'toilet_rec_0.25': 0.9379310344827586, 'desk_rec_0.25': 0.7640807651434643, 'dresser_rec_0.25': 0.9128440366972477, 'night_stand_rec_0.25': 0.9607843137254902, 'bookshelf_rec_0.25': 0.8865248226950354, 'bathtub_rec_0.25': 0.9591836734693877, 'mAR_0.25': 0.8646940630326154, 'bed_AP_0.50': 0.009411284700036049, 'table_AP_0.50': 0.12223674356937408, 'sofa_AP_0.50': 0.14403118193149567, 'chair_AP_0.50': 0.1670922487974167, 'toilet_AP_0.50': 0.27775609493255615, 'desk_AP_0.50': 0.06645272672176361, 'dresser_AP_0.50': 0.19564399123191833, 'night_stand_AP_0.50': 0.41268840432167053, 'bookshelf_AP_0.50': 0.14886672794818878, 'bathtub_AP_0.50': 0.35609957575798035, 'mAP_0.50': 0.19002790749073029, 'bed_rec_0.50': 0.5883495145631068, 'table_rec_0.50': 0.790885860306644, 'sofa_rec_0.50': 0.773524720893142, 'chair_rec_0.50': 0.7272364217252396, 'toilet_rec_0.50': 0.8551724137931035, 'desk_rec_0.50': 0.6981934112646121, 'dresser_rec_0.50': 0.8669724770642202, 'night_stand_rec_0.50': 0.9450980392156862, 'bookshelf_rec_0.50': 0.851063829787234, 'bathtub_rec_0.50': 0.8571428571428571, 'mAR_0.50': 0.7953639545755846}
1.2 scannet
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune
the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/312, elapsed: 0[>>>>>>>>>>>>>>>>>>>] 312/312, 10.4 task/s, elapsed: 30s, ETA: 0s
+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.0077 | 0.2742 | 0.0052 | 0.2419 |
| bed | 0.0003 | 0.0370 | 0.0000 | 0.0247 |
| chair | 0.0193 | 0.2661 | 0.0138 | 0.2354 |
| sofa | 0.0015 | 0.0825 | 0.0005 | 0.0825 |
| table | 0.0016 | 0.1429 | 0.0011 | 0.1229 |
| door | 0.0104 | 0.2441 | 0.0079 | 0.2313 |
| window | 0.0028 | 0.1773 | 0.0021 | 0.1631 |
| bookshelf | 0.0081 | 0.1558 | 0.0042 | 0.1429 |
| picture | 0.0028 | 0.2658 | 0.0025 | 0.2568 |
| counter | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| desk | 0.0012 | 0.0866 | 0.0010 | 0.0709 |
| curtain | 0.0000 | 0.0448 | 0.0000 | 0.0448 |
| refrigerator | 0.0000 | 0.0351 | 0.0000 | 0.0000 |
| showercurtrain | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| toilet | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| sink | 0.0000 | 0.0102 | 0.0000 | 0.0000 |
| bathtub | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| garbagebin | 0.0048 | 0.2113 | 0.0039 | 0.1811 |
+----------------+---------+---------+---------+---------+
| Overall | 0.0034 | 0.1130 | 0.0024 | 0.0999 |
+----------------+---------+---------+---------+---------+
{'cabinet_AP_0.25': 0.007672073319554329, 'bed_AP_0.25': 0.0003270927118137479, 'chair_AP_0.25': 0.019300922751426697, 'sofa_AP_0.25': 0.0015425414312630892, 'table_AP_0.25': 0.0016294954111799598, 'door_AP_0.25': 0.010358083993196487, 'window_AP_0.25': 0.0027569225057959557, 'bookshelf_AP_0.25': 0.008131678216159344, 'picture_AP_0.25': 0.002778079127892852, 'counter_AP_0.25': 0.0, 'desk_AP_0.25': 0.0011750475969165564, 'curtain_AP_0.25': 2.2466720110969618e-05, 'refrigerator_AP_0.25': 1.3386352293309756e-05, 'showercurtrain_AP_0.25': 0.0, 'toilet_AP_0.25': 0.0, 'sink_AP_0.25': 1.440034111510613e-06, 'bathtub_AP_0.25': 0.0, 'garbagebin_AP_0.25': 0.00475729675963521, 'mAP_0.25': 0.003359251655638218, 'cabinet_rec_0.25': 0.27419354838709675, 'bed_rec_0.25': 0.037037037037037035, 'chair_rec_0.25': 0.26608187134502925, 'sofa_rec_0.25': 0.08247422680412371, 'table_rec_0.25': 0.14285714285714285, 'door_rec_0.25': 0.24411134903640258, 'window_rec_0.25': 0.1773049645390071, 'bookshelf_rec_0.25': 0.15584415584415584, 'picture_rec_0.25': 0.26576576576576577, 'counter_rec_0.25': 0.0, 'desk_rec_0.25': 0.08661417322834646, 'curtain_rec_0.25': 0.04477611940298507, 'refrigerator_rec_0.25': 0.03508771929824561, 'showercurtrain_rec_0.25': 0.0, 'toilet_rec_0.25': 0.0, 'sink_rec_0.25': 0.01020408163265306, 'bathtub_rec_0.25': 0.0, 'garbagebin_rec_0.25': 0.21132075471698114, 'mAR_0.25': 0.11298182832749847, 'cabinet_AP_0.50': 0.005249418318271637, 'bed_AP_0.50': 7.024568276392529e-06, 'chair_AP_0.50': 0.013795703649520874, 'sofa_AP_0.50': 0.0005232820985838771, 'table_AP_0.50': 0.001064108102582395, 'door_AP_0.50': 0.007914243265986443, 'window_AP_0.50': 0.0021169877145439386, 'bookshelf_AP_0.50': 0.0042017120867967606, 'picture_AP_0.50': 0.0025250331964343786, 'counter_AP_0.50': 0.0, 'desk_AP_0.50': 0.0010463166981935501, 'curtain_AP_0.50': 2.2466720110969618e-05, 'refrigerator_AP_0.50': 0.0, 'showercurtrain_AP_0.50': 0.0, 'toilet_AP_0.50': 0.0, 'sink_AP_0.50': 0.0, 'bathtub_AP_0.50': 0.0, 'garbagebin_AP_0.50': 0.003941362723708153, 'mAP_0.50': 0.0023559811525046825, 'cabinet_rec_0.
50': 0.24193548387096775, 'bed_rec_0.50': 0.024691358024691357, 'chair_rec_0.50': 0.23538011695906433, 'sofa_rec_0.50': 0.08247422680412371,
'table_rec_0.50': 0.12285714285714286, 'door_rec_0.50': 0.23126338329764454, 'window_rec_0.50': 0.16312056737588654, 'bookshelf_rec_0.50':
0.14285714285714285, 'picture_rec_0.50': 0.25675675675675674, 'counter_rec_0.50': 0.0, 'desk_rec_0.50': 0.07086614173228346, 'curtain_rec_0.
50': 0.04477611940298507, 'refrigerator_rec_0.50': 0.0, 'showercurtrain_rec_0.50': 0.0, 'toilet_rec_0.50': 0.0, 'sink_rec_0.50': 0.0, 'batht
ub_rec_0.50': 0.0, 'garbagebin_rec_0.50': 0.1811320754716981, 'mAR_0.50': 0.0998950286339104}
2.Evaluation results for the model trained by myself：(12 epoches)
2.1 sunrgbd
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/5050, elapsed: [>>>>>>>>>>>>>>>] 5050/5050, 0.3 task/s, elapsed: 16300s, ETA: 0s
+-------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed | 0.5560 | 0.9903 | 0.4873 | 0.8738 |
| table | 0.3110 | 0.9817 | 0.2184 | 0.7785 |
| sofa | 0.4583 | 0.9841 | 0.4016 | 0.8740 |
| chair | 0.5615 | 0.9633 | 0.4967 | 0.8643 |
| toilet | 0.6838 | 0.9931 | 0.6078 | 0.8966 |
| desk | 0.1938 | 0.9644 | 0.0907 | 0.6791 |
| dresser | 0.2366 | 0.9541 | 0.1876 | 0.7752 |
| night_stand | 0.4379 | 0.9765 | 0.3945 | 0.8824 |
| bookshelf | 0.2321 | 0.9184 | 0.0987 | 0.5426 |
| bathtub | 0.3912 | 0.9796 | 0.3296 | 0.8367 |
+-------------+---------+---------+---------+---------+
| Overall | 0.4062 | 0.9705 | 0.3313 | 0.8003 |
+-------------+---------+---------+---------+---------+
{'bed_AP_0.25': 0.5559849143028259, 'table_AP_0.25': 0.3109968602657318, 'sofa_AP_0.25': 0.4582793712615967, 'chair_AP_0.25': 0.5614634156227112, 'toilet_AP_0.25': 0.6837515830993652, 'desk_AP_0.25': 0.1938285529613495, 'dresser_AP_0.25': 0.23655451834201813, 'night_stand_AP_0.25': 0.43787485361099243, 'bookshelf_AP_0.25': 0.2321251481771469, 'bathtub_AP_0.25': 0.3912363648414612, 'mAP_0.25': 0.4062095582485199, 'bed_rec_0.25': 0.9902912621359223, 'table_rec_0.25': 0.981686541737649, 'sofa_rec_0.25': 0.9840510366826156, 'chair_rec_0.25': 0.963258785942492, 'toilet_rec_0.25': 0.993103448275862, 'desk_rec_0.25': 0.9643995749202976, 'dresser_rec_0.25': 0.9541284403669725, 'night_stand_rec_0.25': 0.9764705882352941, 'bookshelf_rec_0.25': 0.9184397163120568, 'bathtub_rec_0.25': 0.9795918367346939, 'mAR_0.25': 0.9705421231343856, 'bed_AP_0.50': 0.4872896671295166, 'table_AP_0.50': 0.21839171648025513, 'sofa_AP_0.50': 0.40156134963035583, 'chair_AP_0.50': 0.4966593384742737, 'toilet_AP_0.50': 0.607752799987793, 'desk_AP_0.50': 0.09067006409168243, 'dresser_AP_0.50': 0.18762624263763428, 'night_stand_AP_0.50': 0.39452967047691345, 'bookshelf_AP_0.50': 0.09874459356069565, 'bathtub_AP_0.50': 0.32958272099494934, 'mAP_0.50': 0.33128082752227783, 'bed_rec_0.50': 0.8737864077669902, 'table_rec_0.50': 0.778534923339012, 'sofa_rec_0.50': 0.8740031897926634, 'chair_rec_0.50': 0.8643170926517572, 'toilet_rec_0.50': 0.896551724137931, 'desk_rec_0.50': 0.6790648246546227, 'dresser_rec_0.50': 0.7752293577981652, 'night_stand_rec_0.50': 0.8823529411764706, 'bookshelf_rec_0.50': 0.5425531914893617, 'bathtub_rec_0.50': 0.8367346938775511, 'mAR_0.50': 0.8003128346684527}
2.2 scannet
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tun$
the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/312, elapsed: 0[>>>>>>> ] 116/312, 0.2 task/s, elapsed[ ] 117/312, 0.2 t$
sk/s, elapsed: 547s,[ ] 118/312, 0.2 task/s, elapsed: 547s, ETA: 89[>>>>>>>>>>>>> ] 240/312, [> ] 241/312, 0.2 task/s, elapsed: 1262$
[> ] 242/312, 0.2 task/s, elapsed: 1262s[> ] 243/312, 0.2 task/s, elap[>>>>>>>>>>>>>>>>>>] 312/312, 0.2 task/s, elapsed: 1531s, ETA: 0s
+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.0089 | 0.2500 | 0.0071 | 0.2419 |
| bed | 0.0022 | 0.0494 | 0.0005 | 0.0123 |
| chair | 0.0195 | 0.2661 | 0.0131 | 0.2383 |
| sofa | 0.0009 | 0.0825 | 0.0000 | 0.0309 |
| table | 0.0030 | 0.1400 | 0.0021 | 0.1257 |
| door | 0.0072 | 0.2698 | 0.0066 | 0.2548 |
| window | 0.0030 | 0.1844 | 0.0020 | 0.1525 |
| bookshelf | 0.0041 | 0.1429 | 0.0031 | 0.1429 |
| picture | 0.0049 | 0.2523 | 0.0045 | 0.2342 |
| counter | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| desk | 0.0041 | 0.0709 | 0.0041 | 0.0551 |
| curtain | 0.0001 | 0.0597 | 0.0001 | 0.0597 |
| refrigerator | 0.0001 | 0.0877 | 0.0001 | 0.0877 |
| showercurtrain | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| toilet | 0.0005 | 0.0345 | 0.0001 | 0.0345 |
| sink | 0.0000 | 0.0408 | 0.0000 | 0.0408 |
| bathtub | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| garbagebin | 0.0076 | 0.2038 | 0.0063 | 0.1887 |
+----------------+---------+---------+---------+---------+
| Overall | 0.0037 | 0.1186 | 0.0028 | 0.1056 |
+----------------+---------+---------+---------+---------+
{'cabinet_AP_0.25': 0.0088879419490695, 'bed_AP_0.25': 0.002196365036070347, 'chair_AP_0.25': 0.019513951614499092, 'sofa_AP_0.25': 0.00092$
0919025614858, 'table_AP_0.25': 0.0029890856239944696, 'door_AP_0.25': 0.007219596300274134, 'window_AP_0.25': 0.002968191634863615, 'books$
elf_AP_0.25': 0.0041231452487409115, 'picture_AP_0.25': 0.004887890536338091, 'counter_AP_0.25': 0.0, 'desk_AP_0.25': 0.004088526125997305,
'curtain_AP_0.25': 0.0001449216069886461, 'refrigerator_AP_0.25': 0.00010905037925112993, 'showercurtrain_AP_0.25': 0.0, 'toilet_AP_0.25': $
.0005089639453217387, 'sink_AP_0.25': 3.358132016728632e-05, 'bathtub_AP_0.25': 0.0, 'garbagebin_AP_0.25': 0.007584888022392988, 'mAP_0.25'$
0.003676677355542779, 'cabinet_rec_0.25': 0.25, 'bed_rec_0.25': 0.04938271604938271, 'chair_rec_0.25': 0.26608187134502925, 'sofa_rec_0.25$
: 0.08247422680412371, 'table_rec_0.25': 0.14, 'door_rec_0.25': 0.2698072805139186, 'window_rec_0.25': 0.18439716312056736, 'bookshelf_rec_0
.25': 0.14285714285714285, 'picture_rec_0.25': 0.25225225225225223, 'counter_rec_0.25': 0.0, 'desk_rec_0.25': 0.07086614173228346, 'curtain_
rec_0.25': 0.05970149253731343, 'refrigerator_rec_0.25': 0.08771929824561403, 'showercurtrain_rec_0.25': 0.0, 'toilet_rec_0.25': 0.034482758
620689655, 'sink_rec_0.25': 0.04081632653061224, 'bathtub_rec_0.25': 0.0, 'garbagebin_rec_0.25': 0.2037735849056604, 'mAR_0.25': 0.118589569
75081056, 'cabinet_AP_0.50': 0.007079910486936569, 'bed_AP_0.50': 0.0005367686389945447, 'chair_AP_0.50': 0.013134335167706013, 'sofa_AP_0.5
0': 4.525698022916913e-05, 'table_AP_0.50': 0.0021134275011718273, 'door_AP_0.50': 0.00658177025616169, 'window_AP_0.50': 0.0019997931085526
943, 'bookshelf_AP_0.50': 0.0031221157405525446, 'picture_AP_0.50': 0.004526467993855476, 'counter_AP_0.50': 0.0, 'desk_AP_0.50': 0.00407800
916582346, 'curtain_AP_0.50': 0.0001449216069886461, 'refrigerator_AP_0.50': 0.0001018800976453349, 'showercurtrain_AP_0.50': 0.0, 'toilet_A
P_0.50': 5.409148434409872e-05, 'sink_AP_0.50': 3.358132016728632e-05, 'bathtub_AP_0.50': 0.0, 'garbagebin_AP_0.50': 0.006303954869508743, '
mAP_0.50': 0.0027697933837771416, 'cabinet_rec_0.50': 0.24193548387096775, 'bed_rec_0.50': 0.012345679012345678, 'chair_rec_0.50': 0.2383040
9356725146, 'sofa_rec_0.50': 0.030927835051546393, 'table_rec_0.50': 0.12571428571428572, 'door_rec_0.50': 0.25481798715203424, 'window_rec_
0.50': 0.1524822695035461, 'bookshelf_rec_0.50': 0.14285714285714285, 'picture_rec_0.50': 0.23423423423423423, 'counter_rec_0.50': 0.0, 'des
k_rec_0.50': 0.05511811023622047, 'curtain_rec_0.50': 0.05970149253731343, 'refrigerator_rec_0.50': 0.08771929824561403, 'showercurtrain_rec
144 Jan 04, 2023
20 Aug 31, 2022
28 Nov 18, 2022
25 Dec 08, 2022
34 Dec 29, 2022
1 Jan 03, 2022
2 Jun 18, 2022
79 Dec 16, 2022
17 May 20, 2022
151 Dec 28, 2022
336 Dec 30, 2022
4 Aug 10, 2022
109 Dec 20, 2022
24 Jul 07, 2022
1 Feb 13, 2022
21 Dec 27, 2022
5 Dec 21, 2022
2.8k Dec 31, 2022
59 Oct 31, 2022
237 Jan 05, 2023