搜索“RAG”

t learning only trains small subnetworks of the original model,Reinforcement Learning Finetunes Small Subnetworks in Large Language Models (Mukherjee et al, 2025) and thus can be fragile when the network is further trained on a large amount of data. We study the extent to which it happens, and how the desired behavior can be recovered. In order to reduce such catastrophic forgetting, a common approach in mid-training is to mix in “background data” from the original model’s pretraining distributi

匹配字段正文

nt that LoRA underperforms in settings that resemble pre-training,LoRA Learns Less and Forgets Less (Biderman et al, 2024) namely those with very large datasets that exceed the storage limits of LoRA parameters. But for dataset sizes that are typical in post-training, LoRA has sufficient capacity to store the essential information. However, this fact makes no guarantees regarding sample efficiency and compute efficiency. The question is: can LoRA match the performance of full fine-tuning, and if

匹配字段正文

han can be produced instantaneously, the interaction model delegates to a background model that runs asynchronously.This approach builds upon prior work like Qwen-omni, KAME, MoshiRAG. The interaction model remains present throughout — answering follow-ups, taking new input, holding the thread — and integrates background results into the conversation as they arrive. This split lets the user benefit from both responsiveness as well as the full extent of intelligence: the planning, tool-use, and a

匹配字段正文

too small, forcing us to split along the reduction dimension (K). Despite having two “batch” dimensions, matmuls also require us to have much more “work” per core in order to leverage tensorcores effectively. For example, if you have a [1024, K] x [K, 1024] matmul and a standard 2D tile size of [128, 128], a data-parallel strategy would only be able to split this matmul into 64 cores, insufficient to saturate the GPU. Splitting along the reduction dimension in a matmul is known as a Split-K Mat

匹配字段正文

oth are defensive layers built by fine-tuning—and accessible fine-tuning is what lets defenders everywhere build their own; Give vetted safety researchers white-box access to encourage novel safety work. We understand how shallow refusal safeguards are — bypassable through adversarial token-level optimizationUniversal and Transferable Adversarial Attacks on Aligned Language Models (Zou et al, 2023), removable via a single direction in activation spaceRefusal in Language Models Is Mediated by a S

匹配字段正文

rom interactions generated more than a day earlier. Learning to reason efficiently We trained Beam with a controllable length penalty that rewards successful solutions while discouraging unnecessary tokens. Early in RL, performance improved even as completion lengths fell: the model learned to solve tasks more effectively with less reasoning. Later, as Beam developed stronger agentic capabilities, completion lengths grew again, but those additional tokens supported further gains in performance.

匹配字段正文

Fragmented data, disconnected tools, and unclear experimental records can get in the way of scientific work: the data may be in one place, the analysis in another, and the record of how a result was produced somewhere else. Following the evidence means keeping those pieces connected as you decide what to investigate next. That’s why we built Rosalind Workbench – to bring that work into one environment. It provides life science users with a central environment to leverage their favorite scientifi

匹配字段正文

rn context and slow work down. Pointing to some docs can still be helpful, however, so long as it is contextual. Be sure to keep your docs updated too! Previous models needed encouragement to run tests and check their work. GPT-6 Astra does that on its own, so the same instructions can lead to unnecessary testing. GPT-6 Astra is thorough, but it can be more tentative about how far to take a task. Sometimes it needs a little push to keep going. You can use AGENTS.md to give it permission for a sp

匹配字段正文

. Atmosphere changes with physical altitude, and clouds move over the planet’s geography. They help connect the orbital view to the landscape. The renderer only removes coarse coverage where finer terrain is ready. Mountain silhouettes can still change as finer geometry appears. Landing adds a stricter requirement. The visible ground, collision triangles, and liquid hazards for a contact patch must all come from the same generation and become ready together. Walking uses Rapier in a small local

匹配字段正文

rtyard. The living room, dining area, and kitchen formed the central pavilion. The wings added three bedrooms, an office, bathrooms, and a dressing room. A pantry, laundry, and storage spaces gave the kitchen and household their own supporting rooms. The concept floor plan we reviewed before rebuilding the house. Select the image to open it at full size. The plan also gave us specific things to check. Bedrooms didn’t need to become passageways. A route around the courtyard kept movement between

匹配字段正文

pan breakdown LLM observability records what an agent did at the level of individual model calls and tool executions. This reads the trace of a live agent run span by span from storage, shows what a hallucination and a failed tool call look like, covers where the OpenTelemetry GenAI conventions stand in mid-2026, and walks the four dials that control tracing cost in production. What are durable AI agents? How state persistence and checkpoints survive a restart A long-running agent is a durable r

匹配字段正文

durable, fault-tolerant execution. Your agents can now stream progress feedback during long-running tool calls. Make your Mastra application production-ready with observability storage that scales. Your agents can now work with remote filesystems like S3, GCS, and Azure. Introducing Channels for Mastra Agents Give users access to your agent from Slack, Discord, Telegram, and more. LSP Inspection for Mastra Workspaces Give agents semantic understanding of your codebase. Your agents can now browse

匹配字段正文

LongMemEval — with a completely stable context window. Announcing Mastra 1.0! APIs are locked, server adapters are the new default, and we've shipped thread cloning, composite storage, AI SDK v6 support, and more. Announcing our new logo and brand We're excited to announce the launch of our new logo and brand for Mastra. We believe developers deserve beautiful tools that match the sophistication and dynamism of the systems they're building. Announcing our second book: Patterns for Building AI A

匹配字段正文

ore sign-in and an order change made afterward are part of the same visit. The customer doesn’t have to start over, and the company sees the whole journey instead of disconnected fragments; Getting the job done. The company decides which channels to offer, based on what works best for its customers:Its website: the agent browses the company’s regular pages, and the company applies the customer’s permissions on each one; Its APIs: the agent connects through interfaces built on standards such as O

匹配字段正文

wareness also refuse harmful requests 3 to 18 percentage points more often than those that do not, across all four models (Figure 4). To estimate this, we use a within-question average: For each question, we calculate the difference in refusal rates between rollouts displaying verbalized eval awareness and those not doing so, and then average these rates, weighting the questions to reduce variance (Appendix C1). This is corroborated by a linear regression yielding nearly identical results (Appen

匹配字段正文

r training losses are an L2 reconstruction error and an L1 sparsity-promoting loss, but we also use other metrics to evaluate SAEs. The simplest scalar metrics we track are the average L0 of the SAE and the fidelity. Average L0 measures the mean number of nonzero features: where $||\cdot|| 0$ is the $L 0$ metric and $h(x i)$ is the hidden vector produced on input activation $x i$. We track this as an exponential moving average across batches in training. $L 0$ compares the relative sparsity of d

匹配字段正文

es to answer challenging problems that exercise its reasoning chops. At 671B parameters, the undistilled R1 model is an engineering challenge to run at scale. Our SAE training leveraged our custom inference engine and interpreter model training infrastructure, which enables interpretability at the half a trillion parameter scale. Feature map We used DataMapPlot to create an interactive UMAP visualization of our general reasoning SAE's features. Two initial insights into steering R1 Many of our i

匹配字段正文

et al. 2022, Marks & Tegmark 2023, Turner et al. 2024, Panickssery et al. 2024, Li et al. 2024. For example, to create a steering vector that represents truthfulness, we might average the embeddings from several true statements and subtract the embeddings from several similar-but-false statements. Linear models[17]The term "linear" is ambiguous and admits several principled definitions, some of which allow curved geometry. In this post, we use "linear" to refer to models of neural network repre

匹配字段正文

tions while reading short stories, both geometrically (in activation space) and temporally (changing from sentence to sentence)? Stories have shapes: a comedy rises toward joy; a tragedy falls into loss; a hero leaves home, descends into danger, and returns transformed. We usually mean this metaphorically—but inside a language model, the shape of a story may be visible more literally. As an LLM reads sentence by sentence, its internal activations move through conceptual space, tracing a wanderin

匹配字段正文

note on Orkut.com, saying that he was “building something new.” He did not offer further details. Two years later, he told WIRED he was working on a social network that would “leverage AI and machine learning” to bring people together and build communities. That platform has yet to launch. Now, Büyükkökten is asking people to register their support for Orkut’s return. He has, however, not clarified whether a new version of the platform is already in development or whether it will ever launch. An

匹配字段正文