A Miami startup called Subquadratic emerged from stealth this week with an extraordinary claim: their new model, SubQ, can process 12 million tokens (roughly 9 million words or 120 books) while using nearly 1,000 times less compute than standard transformer architectures. If true, this would fundamentally change how we build and deploy AI applications.
The Quadratic Scaling Problem
Every AI practitioner knows the fundamental bottleneck of transformer architectures: attention scales quadratically with sequence length. Double your input size, and you need four times the compute. This is why most production models cap context windows at 128K tokens, and even frontier models like Claude Sonnet 4.7 and Gemini 3.1 Pro max out at around 1 million tokens.
The implications are significant. When I am building agentic systems or working with clients on document processing pipelines, context limitations force architectural compromises. We chunk documents, implement retrieval systems, or accept that some relationships between distant pieces of information will be missed. A 12 million token context window would eliminate many of these workarounds.
How SSA Claims to Break the Barrier
Subquadratic's approach, called Subquadratic Sparse Attention (SSA), targets what the company calls "wasted compute" in standard attention. Instead of comparing every token to every other token, SSA learns to identify which comparisons actually matter and computes attention only over those positions.
The key differentiator from previous sparse attention attempts is that SSA's selection is content-dependent. The model decides where to look based on meaning, not fixed positional patterns. This allows it to retrieve specific information from arbitrary positions across a very long context without paying the quadratic tax.
According to their published benchmarks, SSA achieves a 7.2x input processing speedup over standard attention with FlashAttention-2 on NVIDIA B200s at 128K tokens. At 256K tokens, that rises to 13.2x. At 512K, 23.0x. At 1 million tokens, 52.2x.
The Numbers That Raised Eyebrows
Subquadratic's benchmark claims are remarkable:
- RULER 128K benchmark: 97% accuracy for $8 versus approximately $2,600 on frontier models (a 300x cost reduction)
- MRCR v2: Score of 83, beating OpenAI by nine points
- Needle-in-a-haystack at 12M tokens: 92.1% retrieval accuracy
- SWE-Bench Verified: 81.8%, outperforming Claude Opus 4.6 (80.8%) and DeepSeek 4.0 Pro (80.0%)
The company has raised $29 million in seed funding at a $500 million valuation. Investors include Javier Villamizar (former SoftBank Vision Fund partner) and Justin Mateen (Tinder co-founder), along with early backers of Anthropic, OpenAI, Stripe, and Brex.
The Skepticism is Warranted
The AI research community has responded with a mixture of excitement and caution. As AI commentator Dan McAteer put it: "SubQ is either the biggest breakthrough since the Transformer or it's AI Theranos."
My concern is straightforward: Subquadratic has not released model weights or a full technical paper. For a claim this significant, independent verification is essential. We have seen impressive benchmark numbers before that did not translate to real-world performance.
Several questions remain unanswered:
- Training costs: How does training scale with this architecture?
- Quality at scale: Do the efficiency gains come with quality tradeoffs on general reasoning tasks?
- Reproducibility: Can other researchers validate these results?
- Production readiness: How does SSA perform under real-world load conditions?
What This Would Mean for AI Development
If SubQ's claims hold up under scrutiny, the implications are significant for several areas:
Agentic AI: Current agent architectures struggle with memory and context management. A 12 million token context window would allow agents to maintain comprehensive session histories without retrieval overhead.
Code generation: SubQ Code, their CLI coding agent, claims to load entire codebases into context. This would eliminate the context juggling that makes current coding assistants lose track of project structure.
Enterprise document processing: Legal contracts, medical records, financial filings: many enterprise use cases involve documents that exceed current context limits. Direct processing without chunking would simplify architectures considerably.
Research synthesis: Academic researchers could theoretically load hundreds of papers into a single context for comprehensive literature analysis.
What I am Watching
Subquadratic's launch comes at an interesting moment. The industry has been moving away from raw parameter scaling toward efficiency innovations. Google's TurboQuant (presented at ICLR 2026) demonstrated 6x KV cache compression with no accuracy loss. Mixture-of-experts architectures have become standard. The trend is clear: efficiency is the new frontier.
SubQ represents the most aggressive efficiency claim yet. The company plans to offer trainable versions for customer-specific use cases but will not open-source the model in the near term. They have launched SubQ Search as a free product, presumably to build a user base while they refine the API offering.
For now, I am treating SubQ as an interesting development to watch rather than a proven breakthrough. The benchmarks are impressive, the investor backing is credible, and the founding team (CEO Justin Dangel and CTO Alexander Whedon) appears technically serious. But extraordinary claims require extraordinary evidence, and we do not have that yet.
I will be following this closely. If independent researchers can validate even half of what Subquadratic claims, we are looking at a meaningful shift in what is architecturally possible with LLMs. If not, it becomes another cautionary tale about AI hype outpacing reality. Either outcome will be instructive.