The context window has been shattered: Subquadratic debuts a 12-million-token window
Subquadratic, a Miami-based startup, has introduced a groundbreaking model capable of processing a 12-million-token context window, surpassing existing limits. With its innovative Subquadratic Selective Attention architecture, the model claims remarkable performance in retrieval and efficiency, challenging the industry's reliance on traditional methods.