Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit

LCLMs compress LLM context before decode — 8.8x faster at 16x compression, beating every KV cache method tested. Open-sourced by NYU and Columbia.
EDN

MLPerf and the rise of latency-aware LLM benchmarking

Here is a sneak peek at the evolution of the MLPerf benchmark and how generative AI forced a radical shift in AI hardware ...
Tech Xplore

Making LLMs faster and more efficient across multiple languages

Large language models (LLMs), which are the artificial intelligence (AI) systems behind modern chatbots, translation tools, ...
EDN

The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking

Recent frontier LLM inference benchmarks have highlighted a recurring pattern. GPU-based systems deliver outstanding ...

MediaTek unveils Dimensity 8550 with LLM Booster and support for Gemini Nano V3

The chipset is built on TSMC's N4P node and has eight Cortex-A725 CPU cores, a Mali-G720 MC8 GPU and an NPU 880. Earlier this year, MediaTek unveiled ...
Forbes

Making Sense Of What’s Really Going On Inside AI By Using Newly Devised Natural Language Autoencoders

Forbes contributors publish independent expert analyses and insights. Dr. Lance B. Eliot is a world-renowned AI scientist and consultant. This voice experience is generated by AI. Learn more. This ...
marktechpost

IBM Releases Two Granite Speech 4.1 2B Models: Autoregressive ASR with Translation and Non-Autoregressive Editing for Fast Inference

IBM released two new open speech recognition models— Granite Speech 4.1 2B and Granite Speech 4.1 2B-NAR — and they make a compelling case for what a ~2B-parameter speech model can do. Both are ...
Semiconductor Engineering

Microarchitecture Tailored to 3D-Stacked Near-Memory Processing LLM Decoding (U. of Edinburgh, Peking U., Cambridge et al.)

A new technical paper, “Rethinking Compute Substrates for 3D-Stacked Near-Memory LLM Decoding: Microarchitecture-Scheduling Co-Design,” was published by researchers at University of Edinburgh, Peking ...
marktechpost

How to Build a Fully Searchable AI Knowledge Base with OpenKB, OpenRouter, and Llama

In this tutorial, we explore how to build and query a local knowledge base with OpenKB using a free, open model via OpenRouter. We securely retrieve the API key with getpass, set up the environment ...
The Verge

THE PEOPLE DO NOT YEARN FOR AUTOMATION

is editor-in-chief of The Verge, host of the Decoder podcast, and co-host of The Vergecast. Today on Decoder, I want to lay out an idea that’s been banging around my head for weeks now as we’ve been ...
Machine Design

Linear Encoder Showdown: Wired vs. Wireless Read Heads

In automation, precision and reliability are no longer optional; they are requirements. For a wide variety of machine types and processes, linear guides provide that accuracy and high-capacity travel.
GitHub

Asad-Ismail/ternary-models

Pre-quantized models produced with ternary-quant and published on Hugging Face. This repo is the model-release companion to ternary-quant. The library handles post-training ternary quantization and ...