Lychee sound quality high fidelity AI noise reduction technology sharing

“ The goal of litchi audio processing is two words ： quiet —— Clear 、 quiet . Let users hear more clearly 、 More real 、 Better .” Liuxiaoyu, vice president of technology of litchi group, mentioned pointedly when talking about several difficulties to be overcome in audio technology .

With the live broadcast of the epidemic 、 Online social networking 、 Online classroom 、 The rapid development of online conferences and the continuous growth of the meta universe industry , Audio technology plays an increasingly important role . But the current popular live video 、 Audio and video group chat 、1 Yes 1 Voice matching chat and other social scenes , But it is often accompanied by noisy environmental noise , Such as keyboard tapping 、 Pets at home are noisy 、 Children crying, etc , These will be transmitted to the receiver through the interactive scene , Voice social process is full of interference .

In recent days, , Located in Dawan District “ The first Chinese audio share ” Lichi group learned , The company's big bay audio technology team uses hardware or software to reduce noise 、 Different software noise reduction algorithms 、 Combination of noise reduction and scene , introduce AI Noise reduction , It can effectively suppress the background noise in the process of audio and video calls in interactive entertainment scenes , And ensure that the voice is not damaged , Finally, it can effectively improve the real-time interactive experience in a variety of complex scenarios . at present , Litchi's high fidelity noise reduction technology leads the world .

Liu Xiaoyu, vice president of technology of Lichi group, attended the Huawei Developer Conference

1. Dawan district team AI Noise reduction to achieve strong noise reduction 、 high fidelity , Leading the world

With the epidemic, online interactive entertainment is popular , The importance of live interactive entertainment scenes is highlighted . Weidunxiao, head of audio technology of litchi group, introduced , Different online scenes have different needs for audio high-quality experience . For example, in educational scenes , It focuses on knowledge acquisition and sound clarity , Interact in time ; Conference scenes value the fluency and clarity of speech ; And in the entertainment scene , In addition to interesting content to attract outdoor , High quality experience and interactive function of audio , It is one of the most important factors for users to be willing to participate continuously .

As computing power continues to grow , Based on big data training AI Speech noise reduction algorithm has strong ability , Make real time AI Speech noise reduction algorithm becomes possible in interactive entertainment scenes . Compared with the traditional noise reduction algorithm , Developed by litchi technology team AI The effect of noise reduction has been greatly improved , For live broadcast scenes, you may often encounter typing 、 refresh 、 Background discussion and other noises can be effectively suppressed and even reduced to the lowest impact .

“ In the interactive entertainment business scenario, noise reduction is required for the full band , in consideration of CPU Performance and noise reduction processing time , A hybrid architecture is used to reduce noise in the full band , Low frequency uses AI Model processing , High frequency adopts traditional noise reduction .” Wei dunxiao said .

In the use and feedback of a large number of users , Litchi audio technology team found , In the use scenario of interactive entertainment social products , Transient noise accounts for more , Especially the touch sound 、 Such sounds as eating potato chips and other home scenes account for a large proportion .

Litchi technology team uses massive voice samples in the station , Training this AI Noise reduction model , It can filter out unwanted sounds , Therefore, everyone's audio can be transmitted more clearly to the receiver's ears , Even if everyone speaks at the same time , Especially litchi App In the scene of multi person voice connected with wheat .“AI Compared with traditional noise reduction , It has stronger noise reduction ability , But the possibility of speech damage is greater , But litchi AI Noise reduction has little damage to speech , Make everyone's voice transmitted with high fidelity .”

Besides , Litchi audio R & D personnel choose the top business 10 A lot of experiments and feedback have been carried out on mobile phone models , Ensure that the mainstream platform is damaged by bass 、 High performance 、 Low power operation , Make the user's device not stuck 、 Not hot .

According to introducing , The sound quality of Litchi in the audio interactive entertainment scene is high fidelity AI Noise reduction technology has led the world , It has laid a good foundation for the next step of audio entertainment immersive experience development under interactive entertainment scenes in Dawan district and even in China .

2． New breakthroughs in understanding interactive entertainment scenes

Audio industry AI Technology has developed to the present , Algorithm 、 Out of data scenarios and industry knowledge have become a key . Development is to let the voice do “ Near the border ”. Eliminate all factors that will affect the sense of scene, such as noise 、 Echoes 、 Noise, etc , Then according to the real or virtual environment , Reshape the sound source and spatial perception .

litchi APP The common scene is live broadcast + The scene of Lian mai , That is, most of the time, the anchor is a single person live broadcast , Users usually listen as listeners , But sometimes you can also click the button representing Lianmai to go online , After the anchor receives the request for connection , If through , Then this user can work with the anchor on RTC Real time interaction in the system .

The anchor can rely on a powerful anchor engine to add music or sound effects to the live broadcast 、 You can also call the mixer to beautify the sound or enhance the entertainment of interaction by changing the sound . In this scenario , Multiple anchors perform interactive or entertaining performances in the room , And users can listen 、 You can also interact and socialize with anchors on the Internet . The anchor or user is in a RTC In the system , And the audience can join RTC System , It can also be done through CDN Pull the flow .

To reduce noise, the first thing is to understand sound , Analyze all kinds of audio in the scene through sound understanding . When users play lychee social products , I like eating potato chips 、 Typing on the keyboard 、 Drinking iced soda , Then all kinds of touch sounds . There are many types of noise in life , Even the sound of cooking at home 、 Household appliances sweep the floor 、 Typhoon weather wind noise . If these sounds need to be handled well , Technology is recognized as the most difficult in the industry .

“ To deeply understand noise reduction, we need to first understand what noise our products need to solve , Then reduce noise and suppress these noises , This is a creation that fits the business scenario very well .”

Wei dunxiao introduced , Compared with other scenes, interactive entertainment scenes , The technical difference is mainly in the access of different peripherals 、 Multi channel support 、AI Sound change demand 、 Sound understanding and link sound quality improvement . It is different from the main source acquisition input channel of the conference scene sound source , Entertainment scenes are for entertainment , Support music playing channels at the anchor 、 Sound playback channel 、 Screen sharing channel, etc . When the anchor performs a talent show or plays music , The whole interactive entertainment scene will have higher requirements for sound quality . In terms of audio experience , Let users immerse themselves in the interactive scene as if they were local , Free from all kinds of noise input around , This has also become a major technical difficulty in the audio industry .

“ Litchi audio AI Noise reduction is to find out the characteristics of those noises for targeted reduction .AI Just feed it something , It can do anything . Let's knock 、 Collision sound 、 Noise pours into the learning system ,AI Know this thing , It can be disposed of later .” Litchi technicians will record some sound training algorithms .

however , Liu Xiaoyu also added , On the main voice scene , The difference brought by the algorithm is not big （ Hardware will cover up the gap ）, In some scenarios that are not covered by hardware , Such as music scenes , Video and sound scenes in screen sharing , Have high requirements for sound quality , This requires a breakthrough in the core algorithm ,“ Now look at , This is a big challenge for the whole industry , The team is doing relevant technical research to deal with future scenarios .”

A senior person in the industry from a large factory commented on this technology and said , Litchi AI Noise reduction has achieved “ Unexpectedly high level ”.

Liuxiaoyu, vice president of litchi technology, summarized , With the advent of the meta universe , Users' perception of sound quality 、 Immersive experience requires more and more , The effect of the access device 、 Low delay 、 Spatial audio technology 、 Environmental acoustic simulation, etc , These are the difficulties that need to be overcome in the current audio interactive entertainment . Litchi technology team is constantly striving forward , Continue to promote China's Internet audio social technology to be a world leader .