Meta, hell-bent on catching as much as rivals within the generative AI area, is spending billions by itself AI efforts. A portion of these billions goes towards recruiting AI researchers. However an excellent bigger chunk is being spent creating {hardware}, particularly chips to run and practice Meta’s AI fashions.
Meta unveiled the latest fruit of its chip dev efforts right now, conspicuously a day after Intel announced its newest AI accelerator {hardware}. Known as the “next-gen” Meta Coaching and Inference Accelerator (MTIA), the successor to final yr’s MTIA v1, the chip runs fashions together with for rating and recommending show advertisements on Meta’s properties (e.g. Fb).
In comparison with MTIA v1, which was constructed on a 7nm course of, the next-gen MTIA is 5nm. (In chip manufacturing, “course of” refers back to the dimension of the smallest element that may be constructed on the chip.) The subsequent-gen MTIA is a bodily bigger design, full of extra processing cores than its predecessor. And whereas it consumes extra energy — 90W versus 25W — it additionally boasts extra inner reminiscence (128MB versus 64MB) and runs at the next common clock velocity (1.35GHz up from 800MHz).
Meta says the next-gen MTIA is at present dwell in 16 of its information heart areas and delivering as much as 3x general higher efficiency in comparison with MTIA v1. If that “3x” declare sounds a bit imprecise, you’re not improper — we thought so too. however Meta would solely volunteer that the determine got here from testing the efficiency of “4 key fashions” throughout each chips.
“As a result of we management the entire stack, we are able to obtain higher effectivity in comparison with commercially obtainable GPUs,” Meta writes in a weblog submit shared with TechCrunch.
Meta’s {hardware} showcase — which comes a mere 24 hours after a press briefing on the corporate’s varied ongoing generative AI initiatives — is uncommon for a number of causes.
One, Meta reveals within the weblog submit that it’s not utilizing the next-gen MTIA for generative AI coaching workloads in the meanwhile, though the corporate claims it has “a number of applications underway” exploring this. Two, Meta admits that the next-gen MTIA received’t exchange GPUs for operating or coaching fashions — however as an alternative complement them.
Studying between the traces, Meta is shifting slowly — maybe extra slowly than it’d like.
Meta’s AI groups are nearly definitely underneath stress to chop prices. The corporate’s set to spend an estimated $18 billion by the top of 2024 on GPUs for coaching and operating generative AI fashions, and — with coaching prices for cutting-edge generative fashions ranging within the tens of thousands and thousands of {dollars} — in-house {hardware} presents a beautiful various.
And whereas Meta’s {hardware} drags, rivals are pulling forward, a lot to the consternation of Meta’s management, I’d suspect.
Google this week made its fifth-generation customized chip for coaching AI fashions, TPU v5p, typically obtainable to Google Cloud prospects, and revealed its first devoted chip for operating fashions, Axion. Amazon has a number of customized AI chip households underneath its belt. And Microsoft final yr jumped into the fray with the Azure Maia AI Accelerator and the Azure Cobalt 100 CPU.
Within the weblog submit, Meta says it took fewer than 9 months to “go from first silicon to manufacturing fashions” of the next-gen MTIA, which to be truthful is shorter than the everyday window between Google TPUs. However Meta has loads of catching as much as do if it hopes to attain a measure of independence from third-party GPUs — and match its stiff competitors.