Semiconductor Engineering

AI Inference Needs A Mix-And-Match Memory Strategy

Interactive LLMs (chat, copilots, agents) with strict latency targets Long‑context reasoning (codebases, research, video) with massive KV (key value) cache footprints Ranking and recommendation models ...
VentureBeat

IndexCache, a new sparse attention optimizer, delivers 1.82x faster inference on long-context AI models

Processing 200,000 tokens through a large language model is expensive and slow: the longer the context, the faster the costs spiral. Researchers at Tsinghua University and Z.ai have built a technique ...
The Eastern Herald

Google Supercharges Gemma 4 With Multi-Token Prediction, Delivering Up to 3× Faster AI Inference

Google’s Multi-Token Prediction upgrade for Gemma 4 dramatically improves AI speed and efficiency without sacrificing ...
SiliconANGLE

Nvidia GTC 2026: Jensen Huang’s Groq ‘Mellanox moment’ and the inference land grab

Ahead of Nvidia Corp.’s GTC 2026 this week, we reiterate our thesis that the center of gravity in artificial intelligence is shifting from “How fast can you train?” to “How well can you serve?” ...
Design-Reuse

ChiPy®: Bridge Neural Networks and C++ on Silicon — Full Inference Pipelines with Zero CPU Round-Trips

Every object detection deployment hits the same wall. The neural network backbone runs on an accelerator — GPU, NPU, DSP — and then the results get shipped back to a host CPU for postprocessing. Box ...