Valued at $1 billion, Kai-Fu Lee’s LLM startup unveils an open source model

Kai-Fu Lee, the computer scientist known in the West for his bestseller AI Superpowers and in China for its bets on the unicorns of artificial intelligence, has a new adventure – and a great ambition.

At the end of March, Lee launched a company called 01.AI with the vision of developing a large local language model for the Chinese market. The venture puts him in competition with other prominent Chinese tech leaders, including Sogou founder Wang Xiaochuan, who quickly assembled talent and venture capital to establish China’s equivalent of OpenAI.

“I think necessity is the mother of innovation, and there is clearly a huge necessity in China,” Lee told TechCrunch in an interview, explaining the motivation behind launching 01.AI. “Unlike the rest of the world, China doesn’t have access to OpenAI and Google because those two companies haven’t made their products available in China. So I think a lot of people doing LLM are trying to contribute to the creating a solution for a market. this really needs that.

The growth of 01.AI perfectly reflects the rapid development of the generative AI field. Seven months after its creation, the startup released its first model, the open source Yi-34B. The decision to introduce an open LLM as its first product is a way to “give back” to society, Lee said. For people who felt LLaMA was a “godsend,” “we provided a compelling alternative,” he added.

At the time of writing, the Yi-34B, which is a bilingual (English and Chinese) base model trained with 34 billion parameters and significantly smaller than other open models like the Falcon-180B and Meta LlaMa2-70B , came first among pre-trained LLM models, according to a ranking by Hugging Face.

“We still believe that larger models, when trained well, on a large amount of high-quality data, will always outperform significantly smaller models of comparable quality and technology. So I think (Yi-34B) outperforming much larger models is something that we don’t usually see,” Lee said. “We’re confident that as we release 100 billion to 400 billion models over the next year, year and a half , these models will be significantly better than today’s model that we announced.”

The startup’s ability to quickly begin model training is undoubtedly a result of its smooth fundraising, which is essential for securing top-tier AI talent and processors. While refusing to disclose the amount raised by 01.AI, Lee said it was valued at $1 billion after receiving funding from Sinovation Ventures, Alibaba Cloud and other undisclosed investors.

01.AI already has more than 100 employees, more than half of whom are LLM experts from large multinational and Chinese technology companies. Its VP of Technology, for example, is an early member of Google’s Bard, and its chief architect was a founding member of TensorFlow and worked alongside renowned researchers like Jeff Dean and Samy Bengio at Google Brain. The key figures behind Yi-34B are Wenhao Huang, a veteran of Microsoft Research Asia, and Ethan Dai, who held senior AI positions at Huawei and Alibaba.

Having backed over ten unicorns and started seven companies through Sinovation Ventures, Lee is probably one of the most well-connected investors and entrepreneurs in China.

“It’s been, you know, over 25 years since the founding of Microsoft Research Asia, and everything I’ve done has been about recruiting really great talent,” said Lee, who launched Microsoft Research Asia, the largest research center of the American giant abroad. before heading to Google China. Over the years, Microsoft Research Asia has earned a reputation as the “West Point” for developing Chinese AI entrepreneurs.

“Now, of course, you want to pay people fairly, and you have to be competitive in terms of compensation, but I really think you also have to have people believing that they can make a difference and that the company can be successful ” Lee added.

It’s no secret that establishing LLMs is an expensive endeavor. To sustain its cash-intensive operations, 01.AI has monetization projects from the start. Although the company will continue to open source some of its models, its goal is to create a cutting-edge proprietary model that will serve as the basis for a diverse range of commercial products.

“We I can’t open-source everything,” Lee said. “We were very aware that these large language models are computationally intensive and therefore very expensive. When we raise a lot of money, most of it will be spent on GPU. Given this, we first needed to acquire as many GPUs as possible, which we did.

Like other LLM players in China, 01.AI proactively stockpiled GPUs in anticipation of US sanctions; she borrowed money to buy processors before she even got financing. Over the past year, the Biden administration has tightened restrictions on China’s access to high-end AI chips, prompting Chinese companies to pay inflated prices for the chips. The foresight has been rewarded: 01.AI now has sufficient supply to at least the next 12 to 18 months.

In addition to causing headaches for Chinese companies, U.S. sanctions have been a catalyst for innovation by encouraging them to optimize the use of computing power. “With a very high-quality infrastructure team, for 1,000 GPUs, we might be able to squeeze a 2,000 GPU workload out of it,” Lee said.

01.AI’s path to monetization depends largely on its ability to find market-fit products for its expensive AI models. Although top-notch LLM scientists are rare, there is no shortage of product talent in China.

“China is not ahead of the United States in LLM, but there is no doubt that China can create better applications than American developers, mainly thanks to the phenomenal mobile Internet ecosystem that has been built over the Last 12 years,” Lee said.

While the founder didn’t give any details about the services in the works, he hinted that the company was experimenting with concepts in the areas of productivity and social, and that he would be “disappointed” if 01.AI doesn’t did not publish an application in this calendar. year.

According to Lee, the startup’s ultimate goal is to become an ecosystem where external developers can easily build apps. “The duty is not only to come up with good research models, but, more importantly, to facilitate application development so that there can be compelling applications,” he said. “At the end of the day. It’s an ecosystem game. Time will tell if Lee’s AI efforts will bear fruit.