Miami-based startup Subquadratic recently unveiled its AI model, SubQ, which claims to solve a longstanding computational bottleneck in large language models (LLMs). The company asserts that SubQ is significantly faster and more cost-effective than existing models, capable of processing twelve times more text simultaneously. This efficiency could position SubQ to perform complex tasks such as analyzing large databases or codebases efficiently.
Traditional LLMs, which use transformers and dense attention, require extensive computations by evaluating every word against all others, exponentially increasing costs as text length grows. Subquadratic’s approach relies on “sparse attention,” aiming to perform only necessary comparisons by selecting crucial word relationships, thus reducing computational demands. Previous attempts to use sparse attention have not succeeded at scale, but Subquadratic claims its model performs comparably to leading dense attention models.
The validity of these claims gained traction when independent testing by Appen revealed that SubQ operated 56 times faster than other models, with notable efficiency in specific tasks such as long-document retrieval. However, the model’s availability remains limited, and doubts persist because SubQ was developed using the weights from Qwen, an open-source Chinese model, rather than being entirely novel.
Subquadratic’s introduction of sparse attention in AI may significantly impact LLM efficiency, and the company’s goal is to pioneer a new era of efficient AI architectures. Yet, the lack of broader accessibility and reliance on existing model components keep skepticism alive despite promising initial test results. As Subquadratic prepares to distribute SubQ more widely, it aims to demonstrate its model’s practical utility in diverse scenarios, although further validation is required to substantiate its claims of a technological breakthrough.
Source: Technology Review
Read Original:
https://www.technologyreview.com/2026/06/19/1139313/a-startup-claims-it-broke-through-a-bottleneck-thats-holding-back-llms/
