This New Snapdragon Chipset Supports 220 Tokens Per Second - Here's Why That's A Big Deal

Sabrina Ortiz/ZDNET

Follow ZDNET: Add america arsenic a preferred source on Google.

ZDNET's cardinal takeaways

• Qualcomm launches Snapdragon 8 Elite Gen 5 chipset. 
• It features nan Qualcomm Oryon CPU and next-gen Qualcomm Adreno GPU.
• The results see nan fastest on-device AI processing. 

In nan AI-first world we unrecorded in, adjuvant AI features matter much and much to smartphone buyers. As a result, chipsets down these devices person to go progressively powerful to support these features, and Qualcomm's caller chipset conscionable upped nan ante. 

In a speech pinch Durga Malladi, SVP and GM of exertion planning, separator solutions, and information halfway astatine Qualcomm, he told maine that nan Snapdragon 8 Elite Gen 5 supports up to 220 tokens per 2nd erstwhile utilizing a 3B parameter mini connection model. These stats make nan Snapdragon 8 Elite Gen 5 nan fastest mobile SoC for moving reasoning models on-device, compared to different published numbers. 

Also: I sewage a glimpse of early Android smartphones - present are 3 awesome upgrades you tin expect

"If you conscionable deliberation astir it, we went from adjacent to 20 to this, it's for illustration a 10x summation successful nan tokens per 2nd that you get now," said Malladi. "I can't publication 200 words per second. None of america can." 

I said pinch Malladi connected Wednesday, aft Qualcomm unveiled its newest mobile platform, nan Snapdragon 8 Elite Gen 5 chipset, astatine its yearly Snapdragon Summit. The motorboat emphasized its capacity to substance moreover much precocious AI experiences that connection improvements successful photography, videography, audio, gaming, and, of course, AI-inferencing. 

Also: Your adjacent Android telephone could beryllium a monolithic upgrade - acknowledgment to Qualcomm's newest chipset

Tokens per 2nd mention to nan magnitude of accusation that AI models tin intake aliases process successful a fixed magnitude of time. The much tokens that a exemplary tin process astatine a time, nan faster users tin acquisition and execute much analyzable tasks on-device. On-device is peculiarly important arsenic it nary longer contributes to little latency but besides increases privateness arsenic it keeps accusation connected nan device, bypassing nan cloud. 

"It's thing that tin beryllium utilized to very quickly spell from [if] you conscionable had a PDF, possibly successful 1 language, and you conscionable wanted to construe it virtually instantaneously, to immoderate different language. That tin beryllium done successful a really accelerated way," Malladi said. 

Disclosure: The costs of Sabrina Ortiz's recreation to Maui, Hawaii, for nan Snapdragon Summit was covered by Qualcomm, a communal manufacture believe for long-distance trips. The judgments and opinions of ZDNET's writers and editors are ever independent of nan companies we cover.