Boltzbit publishes working paper proposing models that adapt directly from live interactions

LONDON, Sept. 22, 2026 — Boltzbit, the AI research company developing General Learning Intelligence (GLI), today released the preview version of its latest paper titled “Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data”. The research outlines a new architectural foundation, Bayesian Self-learning Transformers (BAST), which Boltzbit's evaluations show can learn up to 1,000x faster than SOTA  training algorithms designed by human researchers, making BAST the first architecture to enable model learning and adaptation directly during live use, and marking a shift from cost-intensive, static-weight AI toward compute-efficient, dynamic-weight AI.

Current AI agents rely on harness frameworks that bolt context onto the model. As a result, any domain context, user instructions or conversational corrections must be re-entered at the start of each new session. Boltzbit's paper outlines an alternative where runtime evidence is absorbed directly into the model's weights, freeing up context capacity and keeping conversations coherent over long sessions.

“The industry has treated memory as a context-window engineering problem, but the real bottleneck is deep in the architecture of AI,” said Dr. Jinli Hu, CTO and co-founder of Boltzbit. “Scaling up frozen models or layering on more complex retrieval pipelines cannot overcome the limits of static parameters. To build systems that learn and evolve alongside users, models must possess the capacity to update their own weights on the fly.”

The limits of static models and the impending data wall

Over the past decade, building more advanced models has been achieved by training bigger ones on more data. However, current projections suggest the supply of pretraining text will run out before the decade concludes. In contrast, the fast-growing use of AI agents is producing a continuously expanding stream of conversational data. Under existing architectures, none of it is captured. It is processed within a single session and discarded once inference completes. 

To mitigate this, users have turned to fixes such as Retrieval-Augmented Generation (RAG), expanded context windows, and multi-agent frameworks. Boltzbit's benchmark evaluations show that these external fixes delay memory decay rather than resolving it. 

In tasks where the answer depends on earlier turns, retrieval-based methods degrade sharply as the conversation continues, collapsing to near-random routing accuracy (10-20% on ambiguous turns). Meanwhile, simply stuffing the full history into the prompt gives diminishing returns. Accuracy peaks early around turn 2 or 3, then steadily decays as query dilution sets in while per-turn inference costs scale linearly with history length.

Realising the value of that data means bringing self-learning AI agents to users.

Generating the expertise each task requires on the spot 

Building on Mixture-of-Experts (MoE) architectures, Boltzbit’s BAST takes a different approach to how models draw on specialised knowledge. Where MoE selects from a fixed bank of pre-trained sub-networks, BAST generates specialised weights a task requires in real time, drawing from an unbounded space of possible expertise:

• Bank-free expert generation A compact generator reads incoming user data and produces small, targeted weight updates to the model’s core network, generating the exact expert behaviour a task requires instead of drawing from a fixed library. 
• Continuous understanding: Instead of reading context once and freezing the adapter, the model maintains an active Bayesian belief across turns, allowing specialised expertise to sharpen as more of the conversation unfolds. 
• Flat computational cost: While prompt concatenation scales linearly in compute cost with conversation length, the belief update compute stays strictly flat, requiring only K inner products per layer independent of the turn index. Boltzbit's evaluations show this enables models to learn up to 1,000x faster than standard training algorithms.
• Bounded and fully reversible: Because dynamic weight updates are anchored to a frozen base model, adaptation remains low-dimensional, safe from catastrophic forgetting, and fully reversible once the user session concludes.

Validation across multi-turn dialogue and retrieval

The paper specifies the architecture and training methods, and reports evaluations across multi-hop reasoning, passage retrieval, and extended multi-turn conversations:

• Outperforming in-context learning as evidence grows longer and noisier: Prompt-stuffing holds an edge on short single-passage tasks (SQuAD: 85.3 vs. 51.8 F1), but compiling runtime evidence into weights overtakes it as inputs grow longer and noisier, achieving 48.0 vs. 33.6 F1 on MS MARCO v2.1 (+14.4 points), 45.3 vs. 40.9 F1 on MuSiQue hard multi-hop, and 58.1 vs. 55.5 F1 on 2WikiMultihopQA.
• Beating strong dense retrieval: A trained belief over candidate passages consistently outperforms state-of-the-art dense retrieval (bge-small) across benchmark datasets. 53.3 vs. 45.3 top-1 accuracy on MS MARCO (+8.0 points), 70.1 vs. 58.1 on 2WikiMultihopQA (+12.0 points), 62.1 vs. 52.2 on HotpotQA (+9.9 points), and 53.0 vs. 40.9 on MuSiQue (+12.1 points).
• Monotonic accuracy gains across dialogue: In multi-turn context-dependent dialogue, the accumulated Bayesian belief concentrates as the conversation becomes established, driving routing accuracy up monotonically from turn 1 through turn 6+ toward near-oracle levels (approaching 80–90%). This contrasts sharply with concatenated-history retrieval, which degrades due to query dilution.

“Frontier models today store hundreds of billions of static parameters, but they remain amnesic when deployed,” added Dr. Yichuan Zhang, CEO and co-founder of Boltzbit. “This paper demonstrates that runtime data can enter the weights cleanly, allowing models to learn once, and continuously improve through live interaction.”

Paper availability and ongoing research

The working paper, “Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data”, is authored by Dr. Jinli Hu, Dr. Ross M. Clarke, Dr. Yichuan Zhang, and Prof. José Miguel Hernández-Lobato. 

It is available for the research community on arXiv at https://arxiv.org/abs/2609.18842 and Boltzbit plans to release the first commercial offering that puts this architecture into practice over the coming weeks. 

About Boltzbit

Boltzbit is an AI research company headquartered in London, developing General Learning Intelligence (GLI). Founded in 2020 by Dr Yichuan Zhang and Dr Jinli Hu, the company builds on academic research into Boltzmann machines to develop generative models capable of live learning in production.

Its models are designed to address the limitations of traditional pre-trained transformers by continuously adapting to new data and environments. Boltzbit’s technology is already in production, powering applications across financial services, data, and high-growth technology sectors.

The company is part of a new wave of research focused on moving beyond static, model-centric AI toward adaptive, context-driven intelligence, an approach referred to as AGI 2.0. For more information, visit www.boltzbit.com.

‍