Kai-Fu Lee, the pc scientist recognized within the West for his bestseller AI Superpowers and in China for his bets on synthetic intelligence unicorns, has a brand new enterprise — and a terrific ambition.
In late March, Lee launched an organization referred to as 01.AI with the imaginative and prescient to develop a homegrown massive language mannequin for the Chinese language market. The enterprise places him in competitors with different outstanding Chinese language tech leaders, together with Sogou’s founder Wang Xiaochuan, who’ve been swiftly gathering expertise and enterprise capital to ascertain China’s equivalents of OpenAI.
“I feel necessity is the mom of innovation, and there’s clearly an enormous necessity in China,” Lee informed TechCrunch in an interview, explaining the motive behind beginning 01.AI. “Not like the remainder of the world, China doesn’t have entry to OpenAI and Google as a result of these two corporations didn’t make their merchandise obtainable in China, so I feel many doing LLM try to do their half in creating an answer for a market that basically wants this.”
01.AI’s development is a becoming reflection of the fast improvement within the generative AI area. Seven months after its founding, the startup has launched its first mannequin, the open-source Yi-34B. The choice to introduce an open LLM as its debut product is a option to “give again” to society, stated Lee. For individuals who have felt LLaMA is a “godsend” to them, “we’ve supplied a compelling different,” he added.
As of writing, Yi-34B, which is a bilingual (English and Chinese language) base mannequin skilled with 34 billion parameters and considerably smaller than different open fashions like Falcon-180B and Meta LlaMa2-70B, got here in first amongst pre-trained LLM fashions, according to a ranking by Hugging Face.
“We nonetheless consider that bigger fashions, when skilled effectively, on a considerable amount of high-quality information, will at all times outperform considerably smaller fashions of comparable high quality and comparable expertise, so I feel [Yi-34B] outperforming a lot bigger fashions is one thing that we don’t often see,” stated Lee. “We really feel fairly assured as we launched fashions which might be 100 billion to 400 billion over the subsequent coming yr, yr and a half, these fashions can be dramatically higher than right now’s mannequin that we introduced.”
The startup’s potential to begin mannequin coaching shortly is little question an consequence of its clean fundraising, which is vital to securing top-tier expertise and AI processors. Whereas declining to reveal how a lot 01.AI has raised, Lee stated it’s valued at $1 billion after receiving financing from Sinovation Ventures, Alibaba Cloud and different undisclosed traders.
01.AI has already grown to greater than 100 workers, over half of whom are LLM consultants from main multinational and Chinese language tech companies. Its vice chairman of expertise, for example, is an early member of Google’s Bard, and its chief architect was a founding member of TensorFlow and labored alongside famend researchers like Jeff Dean and Samy Bengio at Google Mind. The important thing figures behind Yi-34B are Wenhao Huang, a Microsoft Analysis Asia veteran, and Ethan Dai, who held senior AI positions at Huawei and Alibaba.
Having backed over ten unicorns and venture-built seven corporations by way of Sinovation Ventures, Lee is presumably one of the vital well-connected traders and entrepreneurs in China.
“It’s been, you recognize, over 25 years for the reason that founding of Microsoft Analysis Asia, and every part I’ve performed has been about getting tremendous nice expertise,” stated Lee, who launched Microsoft Analysis Asia, the U.S. large’s greatest analysis heart overseas, earlier than heading Google China. Through the years, Microsoft Analysis Asia has earned the fame because the “West Level” for nurturing China’s AI entrepreneurs.
“Now, in fact, you wish to pay folks pretty, and it’s worthwhile to be aggressive in pay, however I actually suppose that it’s additionally about folks believing they will make a distinction and believing the corporate can succeed,” Lee added.
It’s no secret that constructing LLMs is a expensive endeavor. To maintain its cash-intensive operations, 01.AI has plans for monetization proper from the beginning. Whereas the corporate will proceed to open supply a few of its fashions, its goal is to construct a state-of-the-art proprietary mannequin that serves as a basis for a various vary of business merchandise.
“We can’t open supply every part,” stated Lee. “We had been fairly cognizant of the truth that these massive language fashions require plenty of compute, and due to this fact, are very costly. After we increase some huge cash, most of it will likely be spent on the GPU. Provided that, we wanted to first purchase as a lot GPU as we might, which we did.”
Like different LLM gamers in China, 01.AI has proactively stockpiled GPUs in anticipation of U.S. sanctions; it borrowed cash to purchase processors even earlier than it landed funding. Over the previous yr, the Biden administration has heightened restrictions on China’s entry to high-end AI chips, prompting Chinese firms to pay inflated prices for chips. The foresight was rewarded — 01.AI now has a provide that may suffice for no less than the subsequent 12-18 months.
Except for inflicting complications for Chinese language companies, U.S. sanctions have been a catalyst for innovation by encouraging them to optimize using computing energy. “With a really high-quality infrastructure crew, for each 1000 GPUs, we would have the ability to squeeze 2000 GPUs workload out of them,” stated Lee.
01.AI’s path to monetization hinges largely on its potential to search out product-market match for its costly AI fashions. Whereas top-notch LLM scientists are scarce, there’s no scarcity of product expertise in China.
“China’s not forward of the U.S. in LLM, however there’s little question China can construct higher purposes than American builders largely due to the outstanding cellular web ecosystem that was constructed during the last 12 years or so,” argued Lee.
Whereas the founder gave no particulars on the providers within the pipeline, he hinted that the corporate is experimenting with ideas within the productiveness and social instructions, and he’d be “disillusioned” if 01.AI didn’t launch an app inside this calendar yr.
The startup’s final aim, in keeping with Lee, is to grow to be an ecosystem the place outdoors builders can construct purposes simply. “The responsibility is not only to push out good analysis fashions, however much more importantly to make software improvement straightforward in order that there might be compelling purposes,” he stated. “On the finish of the day. It’s an ecosystem play.” Time will inform if Lee’s AI endeavor will repay.