About NLP self supervised learning, I was asked to collapse during the interview!

Don't miss the benefits at the end of the article

Literacy is self supervised and unsupervised

（ Many students can't answer the real difference between supervised and unsupervised , I'm not targeting anyone .）

Unsupervised learning ： Unsupervised learning does not rely on any tag values , By mining the inherent characteristics of data , Find the relationship between the samples , Such as clustering related tasks .

Self supervised learning ： Unlike unsupervised learning , Self supervised learning mainly uses auxiliary tasks (pretext） From large-scale unsupervised data mining their own supervision information , The network is trained by the supervision information of this structure , Thus we can learn valuable representations of downstream tasks .

summary ： The supervision information of self supervised learning is not manually marked , But through auxiliary tasks (pretext） Automatically construct supervision information in large-scale unsupervised data , Labels obtained by , You can train like supervised learning .

Self supervised and unsupervised development

Self supervised learning cv Direction video , Medical images , The development trend of autonomous driving and other fields is very good , For example, self supervised learning method based on comparative learning 《moco》 Yes, it is cv Relevant research in the field has had a far-reaching impact .

Empathy , Self supervised learning is promoting natural language processing (NLP) The field has been a great success , Include Collobert-Weston 2008 model, Word2Vec, GloVE, fastText etc. . The performance of pre training in this way is much higher than that of training only in the supervised way .

nlp Where to start self supervised learning

Master self supervised unsupervised learning in nlp Technological evolution in the field , Is laying the foundation ！16 The eye of depth invited to 5 Years of deep learning and machine learning experience , stay ICDM Mr. Zhao, who published many academic papers at the meeting Let's explain to you that self-monitoring and unsupervised are nlp Key points of Technological Development in the field , And the focus of comparative learning . From classic papers cpc（《Representation Learning with Contrastive Predictive Coding》） Starting with , Take you through nlp Self supervised learning . Finally, nlp Self supervised learning Learning path recommend ！ rich content , Super value dry goods , Not to miss .

Below scan QR code

0.1 Yuangou course

Add a small enjoyment after class

So as not to miss it Live notification

And receive benefits （ At the end of the article ）

Live spoilers

1. The development and direction of self-monitoring in natural language processing

2. Technological evolution path

3.NLP Pre training technology talk

4.NLP Comparative learning in English

5. Classic papers CPC Explain

6. Learning path recommendation

Below scan QR code

0.1 Yuangou course

Add a small enjoyment after class

So as not to miss it Live notification

And receive benefits （ At the end of the article ）

Welfare at the end of article

value 198 yuan AI Environment configuration tutorial ,

After purchasing the supervised course add Enjoy free of charge receive ！

Below scan QR code

0.1 Yuan purchase Self supervision course

Add a small enjoyment after class

receive AI Environment construction learning video

Part of the content of this article comes from network intrusion and deletion