Abstract

Self supervised pre training paradigm for scalable dynamic routing SDRNet, In classification and detection tasks SOTA！

（state-of-the-art Best at present / State-of-the-art ）

A common practice of self supervised pre training is to use as much data as possible .

For specific downstream tasks , stay Preliminary training It's about Irrelevant data May be Reduce downstream performance .

For the existing SSL Method , Used in pre training for different tasks Customized data sets for different downstream tasks More complicated .

Put forward a kind called Scalable dynamic routing （Scalable Dynamic Routing,SDR） The new SSL normal form ,

This paradigm can be trained once , And use the pre training model of task customization to effectively deploy to different downstream tasks .

Construct with different subnets SDRnet, And gradually train through data perception , Just use A subset of data trains each subnet .

When the downstream task arrives , Route between all pre trained subnets , To obtain the best subnet and its corresponding weight .

SDR First, the training data are clustered , Then through the progressive training framework of data perception

Use different clusters to train each subnet .

Last , Deploy customized subnets to different downstream tasks effectively

Reference link ：

Pre training data is not the more the better ？ HKUST & Huawei Noah proposes a self supervised pre training paradigm with scalable dynamic routing SDRNet, In classification and detection tasks SOTA！