If you want to know which industry is the hottest right now, it must be AI. On March 25th, JiYue held AI Day 2024 in Beijing, officially releasing the V1.4.0 update with over 200 OTA upgrades. At AI Day, three executives from Baidu introduced Baidu AI’s support for JiYue in areas such as map navigation, autonomous driving, and human-computer interaction.
Since the end of 2022, OpenAI launched the chatbot ChatGPT based on the large language model GPT-3, various AI functions have begun to impact various industries. Domestic internet companies take the lead, Baidu launches Wenxin Yiyuan, Alibaba launches Tongyi Qianwen, iFlytek has Xinghuo, Tencent has Hunyuan, 360 Brain, Huawei has Pangu, JD has Yanxi, Douyin has Yunque, Tsinghua has Zhipu… a variety of them, a hundred “model” war is about to break out.
Car companies, already in the spotlight, are not missing out on the trend of large models. Ideal launched MindGPT, NIO has NOMI GPT, and Xiaopeng has XGPT. These large models for voice assistants make conversations with users more natural. But are these large models just a gimmick for riding on the trend?
Geely CEO Xia Yiping said, “Only by using AI can it be considered a true smart car.” The effectiveness of this large model depends on where you use it. The large model allows for quick map coverage, opening a new city every day. Baidu vice president Shang Guobin said, “Baidu’s LD map currently covers 360 cities and 3.6 million kilometers.” The goal is to achieve nationwide coverage within the year, allowing Geely’s PPA to operate wherever there is Baidu’s lane-level navigation.
In terms of promotion, the current progress of leading assisted driving companies in major cities is as follows: Yu Chengdong announced the update of NCA without pictures, making it possible to drive on any road; Xiaopeng announced the XNGP with unlimited navigation, usable wherever there is navigation; NIO covers 726 cities, basically covering the whole country; Ideal covers 110 cities, with full-scene assisted driving available nationwide. Jiyue currently covers 400,000 kilometers of roads, operating in Beijing, Shanghai, Guangzhou, Shenzhen, and Hangzhou, with the goal of achieving nationwide coverage by the end of this year. We hope that our netizens can share with us the usage and coverage in your area. In terms of the technical route of assisted driving, high-precision maps are actually quite awkward. They are very useful and safer with pictures, but the biggest problem is the cost. Moreover, this cost is not one-time, as continuous and frequent updates are needed to ensure the maps are up-to-date. The cost of a city is over a billion, which can be affordable for a few cities, but who can afford high-precision maps covering the whole country?
Baidu’s LD map solves the problem. It is a map generated by a large model of autonomous driving vision perception, naturally meeting the necessary map elements and accuracy requirements for pure visual assisted driving. It eliminates the dependence on high-precision map collection vehicles, and adds multiple layers to ensure timely updates of map road and traffic information with the participation of Baidu map users, resulting in a qualitative improvement in cost and efficiency. According to Shang Guobin, the LD map only requires 1/20 of the cost, achieving a 100-fold mapping efficiency, fast enough to cover a city in one day. The large model makes autonomous driving more sophisticated. Real-time mapping using pure visual perception of autonomous driving is a capability realized by the latest VTA perception-based large model introduced by Jueyue. This function runs on the vehicle side at a frequency of 10 times per second, directly producing road structures. Each Jueyue can become a small assistant to update Baidu map’s real-time traffic information. VTA stands for Vision Takes All, showing how much Baidu AI is looking forward to pure visual solutions. The key technology of pure visual solution is OCC network occupation, which performs semantic segmentation and accurate recognition of camera images, reconstructing them into a 3D grid world based on bird’s-eye view maps to complete 3D perception and environmental modeling. The solution of LiDAR can only accurately perceive the front-end environment, so Weixiaoli has mentioned in the technical introduction the preparation to use OCC technology for perceiving the surrounding environment.
Large models excel at semantic understanding, so they play a significant role in reading OCC. The extreme of pure visual solutions rely on OCC to surpass the experience of lidar. Therefore, Baidu has introduced their own target detection large models for long-distance high-speed elevated roads, medium to long-distance complex urban roads, and short-distance parking scenarios, giving the models very fitting and interesting names ‘Sniper Rifle’, ‘Pistol’, and ‘Dagger’.
Accurate 3D perception and environment modeling are the basis for decision-making during vehicle driving. VTA enhances target detection and adds time series learning, improving memory and tracking capabilities. This allows for continuous estimation of target position and speed, avoiding sudden surprises during close-range encounters. The combination of temporal perception and agile decision-making leads to more accurate judgments of other traffic participants’ intentions. Early adopters have reported a 72% improvement in obstacle avoidance, showcasing the upgraded capabilities of Baidu AI-powered PPA.
Large models play a strong role in various stages of development, not just in assisting driving systems. For example, in data labeling, Baidu has created the most accurate data pipeline using large models. Baidu AI also uses large models to assist in data management, making it easier to filter scenes using natural language. Large models can even be used to edit rare corner cases, improving the efficiency of the entire system. Large models make multi-channel voice assistants more useful when offline. One of the key aspects of a car’s intelligence, besides driving assistance, is the intelligent cockpit. The voice assistant in the intelligent cockpit is a feature that once you get used to, you can’t go back. A good voice assistant needs to be fast and stable.
TSN: SIMO, the voice assistant developed by Baidu, runs locally for stability. It operates without internet access, ensuring a response time of less than 700 milliseconds even when offline. Baidu runs the entire voice interaction system on an NPU to enable SIMO’s offline capabilities. By using multi-channel technology, Baidu combines in-car and out-of-car voice inputs for recognition, making the process more efficient and adaptable for future car models with more seats.
Baidu AI is using large models to explore the integration of in-car visual and voice interactions. They collect passengers’ lip movements, extract features from the action sequences through large models, model them together with voice, and improve directional sound pickup by judging the user’s position. After a series of optimizations, the performance of speech recognition in complex scenarios such as open windows, multiple people, soft voices, or high noise levels has been improved from an error rate of 90% to an accuracy rate of 90%. AI is not just a gimmick, but it’s not magic either. Baidu AI has played a significant role in improving various functions, so having large models on board is not just a gimmick. However, large models are not a one-size-fits-all solution, they are just a tool that can greatly increase efficiency when used in the right areas. In recent years, the development of AI has progressed from machine learning to deep learning, and from deep learning to neural networks. Within neural networks, the Transformer architecture has emerged, and in the process of using the Transformer architecture to handle natural language, generative pre-trained models have emerged as the best. Therefore, various GPT models are actually best at handling natural language understanding problems, which is why large models on board were initially used for voice assistants to answer questions.
Baidu’s native AI support sets the Jue Yue car robot apart from other smart cars at the core level. To truly control large models, thousands of GPUs are needed. Baidu currently provides Jue Yue with a resource pool of approximately 2.2 EFlops of GPU computing power, with no upper limit.
And from the use of AI large models, it can be seen that Baidu AI’s support for Polestar is comprehensive, strengthening the understanding of natural language in multiple aspects. Even for enhancing voice assistants, other car manufacturers may just plug in a voice dialogue to connect to GPT. Baidu’s voice not only integrates Wenxin Yiyuan in content, but also optimizes the underlying functions of in-car visual and voice collection using large models. Obviously, Baidu’s many years of technical accumulation in the field of AI have a much deeper understanding of AI than ordinary car manufacturers. Believe that the support of Baidu AI for Polestar in all aspects shown at this AI Day will inspire other car manufacturers in the use of AI. It is believed that Chinese companies have a very high expectation for the acceptance and popularization speed of good technology, and also look forward to Chinese cars using AI technology to accelerate the process of intelligence. Perhaps one day in the not too distant future, Polestar will truly usher in the era of car robots.