A NEW COMPUTER FOR ALWAYS-ON INTELLIGENCE
Babylon develops computer architectures for efficient AI inference at low concurrency. We start with recurrent and hybrid language models, keeping their working state next to compute to cut energy and latency.
They pay off when thousands of requests share them. With one or a few active users, much of the energy goes into moving data rather than computing.
Our approach: design a model's compression, working memory and execution as one system, starting with recurrent and hybrid models whose working state can live next to compute. The target is 10× lower energy per token at matched quality and latency.
Hardwired chips specialize one base model into silicon. Babylon is building a programmable architecture for a model family, so it carries over to the next release.
Model architectures are moving faster than the hardware beneath them. Babylon is built for where they are going.
Fixed-size, per-step-updated working memory (Gated DeltaNet, Mamba-style and test-time-training layers) are becoming a large share of inference.
Qwen3.5 ships with Gated DeltaNet layers in production.
With one or a few active users, moving and updating recurrent state can become a significant share of inference cost.
Gupta et al. (USC), March 2026, find subquadratic decode more memory-bound than standard attention at batch 1, and propose holding Gated DeltaNet state on-chip.
Dedicated state storage and execution, targeting 10× lower energy per token at matched quality and latency.
Taking that proposal from synthesis estimates to measured hardware: one Qwen3.5-4B layer on an FPGA against a strong GPU implementation.
The first two shifts are already here. The third is what we're building.
Each part of our thesis rests on published work, from university labs to the companies that build today's models and chips.
3.0× vs 1.6×
Growth every two years: peak server compute vs DRAM bandwidth
Peak compute has grown far faster than memory bandwidth for 20 years. For decoder models at low batch sizes, the authors show memory bandwidth, not compute, becoming the dominant bottleneck.
Also: Pope et al., Google, MLSys 2023, where low-batch cost is dominated by loading weights.
AI and Memory WallGholami et al. · UC Berkeley · IEEE Micro 2024up to 75%
Less KV cache, with up to 6× decode throughput at 1M tokens
Moonshot AI's Kimi Linear runs three linear-attention layers for every full-attention layer, and beats a full-attention model trained the same way on every task evaluated.
Also: Gated DeltaNet, NVIDIA & MIT, ICLR 2025, the layer Kimi Linear extends and Qwen3.5 ships.
Kimi LinearKimi Team · Moonshot AI · Oct 202531–56%
More tokens per joule than the strongest GPU-only baseline, on a B200-based proxy
NVIDIA Research and Harvard find frontier models leaning harder on subquadratic attention, and give those layers their own hardware, matched to their arithmetic intensity and memory footprint.
Also: Gupta et al., USC, 2026: at batch 1, Gated DeltaNet decode is memory-bound on GPUs.
Subquadratic DisaggregationTschand et al. · NVIDIA & Harvard · Sep 2026Our first milestone takes published architecture work from synthesis estimates to measured hardware: one layer of a production model on an FPGA, recurrent state on-chip, against a strong GPU implementation with the same measurement boundaries.
Measured whole-board energy and latency for on-chip state, not estimates.
State precision, memory organization and a runtime that carry one layer to a full hybrid model.
Others shape the socket for datacenter scale or for a single base model. Babylon shapes it for a model family, at the size of one device.
General-purpose and most efficient at large batch.
On-chip SRAM at datacenter scale, now mapping hybrid and linear-attention models.
A base model specialized into silicon, with light adaptation such as LoRA.
One programmable architecture for recurrent and hybrid models: reloadable weights, configurable layer dimensions, a defined operator set.
Each tier rests on the one below it, and each one ends in a measured result.
Models that write to their working memory while they run, on hardware built for it.
A state-update cell taped out on a Tiny Tapeout shuttle and characterized on the bench, running the core operation.
Recurrent layers on FPGA with attention on GPU, or the complete hybrid on a larger board, counting every device, the host and their links. One audited number per token and per watt.
DeltaNet state update for one layer of Qwen3.5-4B on an FPGA, state in on-chip memory, measured against a GPU at batch 1.
A correct reference implementation, a full memory budget and strong GPU baselines for Qwen3.5-4B, so every later result has a fair comparison.
A shared datacenter can pool demand across users. Some products can't: they need the model local, dedicated, responsive and inside a fixed power budget.
Machines, vehicles and instruments that need a capable language model on the device, within a fixed power, memory and latency budget.
Always-available assistants and agents where price, cooling and responsiveness decide the product, and idle power matters as much as peak.
Tell us the model, the task, the latency you need and the hardware you use today. Real workloads shape what we build next.
Every number we publish states its scope, from a single kernel to a full system, so it is never read as more than it is.
Babylon is an early-stage computer architecture company building efficient inference for recurrent and hybrid AI models. We come from systems software: model representation, compilers, runtimes and benchmarking, the layer where architecture meets the model. Now we are building out the hardware team.