搜索“RAG”

table, so this provides a simple way to find interesting heads quickly. We can find induction heads by using a proxy metric called the induction scoreThe induction score is the average attention pattern of a head on (query, key) token pairs that correspond to the literal copying case of induction. In a sequence that looks like [A][B]…[A]- [B], an induction head would attend from the second A to the first B. This is the same as the prefix score from , except the prefix score was on a synthetic da

匹配字段正文

we are limited to expressing interpretations as a weighted sum of atoms from a fixed vocabulary (tokens or dictionary features). Moreover, SAE features can have unpredictable coverage gaps , and require a sometimes-difficult interpretative step, either by a human or a model analyzing top-activating examples . Natural language explanations of activations. By contrast, some recent work trains language models to describe activations in free text. Off-the-shelf models have some capacity for this: L

匹配字段正文

+ TWERA-ranked downstream features; Top activating examples + top unembeds + TWERA-ranked downstream features For each group, we asked Opus to rank candidates multiple times to average across different orders of the features within each group. Specifically, we put the correct answer in each location twice, so the total number of samples was equal to 2× the number of features. We measure the average normalized rank of the correct feature as (predicted rank − 1) / (num candidates − 1). Lower is be

匹配字段正文

he Jacobian lens identifies a vector representation that encodes the potential for the model to verbalize that token in the future. Concretely, it computes, for each layer, the average linearized effect of an activation on the model's likelihood of producing a particular token (now or in the future), averaging over a large corpus of contexts (see Methods for details). The averaging step is key, as it distinguishes representations that are verbalizable—poised to be spoken about, should the occasi

匹配字段正文

The most natural global components of a network are its weights; they describe the direct effect of each unit on another in every context in which they appear. However, models leverage superposition in maximizing their capacity to represent different circuits , and this presents two primary challenges to directly reading the weights. First, the basis of neurons is polysemantic : each neuron activation contributes to multiple features of the computation and they do so in a context-dependent fashi

匹配字段正文

transferring knowledge learned from data-rich physical properties to data-scarce ones. While GEAR demonstrated the potential of a new form of geometric transfer learning that leverages curvature, Sim-GEAR built on this by developing it into a simpler and more efficient form. At the conference, in addition to research presentations, there were lively discussions on how to apply AI to materials development, the imbalance in scientific data, and the process of translating AI predictions into actua

匹配字段正文

h nothing to lose. Q: The Bulletin piece has these interesting examples of civic feedback forums where it looks like there are real people saying things, and instead you have a barrage of comments from AI. How did you find some of these other examples in society where things go off the rails in this way? A: After the release of ChatGPT, I started hearing a concern from people inside government who review public comments on proposed rules: a suspicion that not all of the comments were written by

匹配字段正文

Fresh context restores diversity in the search. The conversation history is lost, but the code, files, and saved results remain in the workspace. A 66 percent higher normalized average score on the same compute budget The researchers tested Fork-and-Flush on 13 long-running tasks , including polyomino packing, job-shop scheduling, and cryptanalysis. The tasks also included compressing a CIFAR classifier and optimizing a VLIW kernel. Individual runs lasted from several hours to over 100 hours. Th

匹配字段正文

ile scores drop slightly on Terminal-Bench 2.0 and SWE-bench Verified. The paper's appendix shows that completed MLR-Bench submissions rise from 31 to 50 out of 60, while their average grade falls from 3.07 to 2.88. The delegation tool goes unused in all 180 runs, leaving it unclear whether teamwork explains the gains. Those gains are no longer statistically significant when results are compared task by task. The authors describe mixed results across research tasks, with AstaBench showing the sa

匹配字段正文

the assumption that a function call corresponds to a single LLM call — so long as you respect the interface of f:X→Yf: \mathbb{X} \to \mathbb{Y}f:X→Y — enabling the ability to leverage complex inference-time optimizations (like the OpenAI o1 family of models). After you’ve used one or more of these techniques to generate a new policy, you’ll want to use historical data to backtest and find out if it’s good or not. These are evals, but have been studied for decades under the name offline policy e

匹配字段正文

engineers edited and approved the AI-generated changelogs, those human revisions were automatically used by TensorZero to optimize future inference requests. Mark and his team leveraged TensorZero’s built-in dynamic in-context learning feature to make the system get smarter over time — without fine-tuning, labeling datasets, or writing complex pipelines. It was a human-in-the-loop learning flywheel, out of the box. Dynamic In-Context Learning (DICL) is an inference-time optimization strategy tha

匹配字段正文

n for each environment at a given bin corresponds to the height of its colored segment. The increase in prompt length is primarily due to the demonstrations: either because the average number of turns in a successful episode increases (BabyAI PickUpSeqGoTo) or because environment observations and agent actions are more verbose (HoVer and τ-bench Retail). This observation motivates our experimentation with a larger model (GPT-4o instead of GPT-4o Mini) with larger context capacity for the τ-bench

匹配字段正文

m exhaustively evaluates every candidate subtree to guarantee the closest match. Each distance is normalized by the size of the LLM snippet, yielding a ratio in [0, 1]. We then average these ratios across all snippets in the inference, and compute the reward is then computed as 1 - average ratio. A perfect match receives a score of 1, while lower scores indicate poorer performance. Formally, say that an LLM inference outputs nnn code snippets with corresponding concrete syntax trees I={t1,…,tn}I

匹配字段正文

Fine-Tuning for o4-mini on terminal-benchThe script below demonstrates how to fine-tune o4-mini on terminal-bench using TensorZero’s programmatic API; Deploy with confidence: Leverage the experimentation features to gradually route traffic to your fine-tuned models, starting with a small allocation and scaling based on performance metrics So, is reinforcement fine-tuning worth it? For most use cases, not yet. At 100-700x the cost of supervised fine-tuning, we only observed RFT to deliver clear

匹配字段正文

while dynamically allocating LLM inference traffic for maximum efficiency; Across a diverse set of realistic and challenging environments, adaptive experimentation reduced the average time to correctly identify the best LLM variants (prompts, models, etc.) by 37% compared to simple A/B testing Developing LLM applications involves numerous design decisions: which model to use, how to structure prompts, what temperature and sampling parameters to set, etc. While offline evaluations and synthetic

匹配字段正文

ted value for production decisions that hinge on judging a single output (e.g. guardrails); However, even (very) noisy evaluators can reliably tell you which agent is better on average, meaning they can still help you pick the best variant to deploy and improve it over time It’s surprisingly hard to develop reliable LLM evaluators: they’re often noisy and poorly correlated with the metrics or outcomes practitioners actually care about. Sometimes the target is directly measurable but evaluators s

匹配字段正文

e entire reason this workflow can filter at all, rather than summarizing everything and calling it filtering. Step 4: Group by story, not by article One model release produces coverage in six places by lunchtime. A digest that lists six items for one event is padding, and readers notice. Material articles are grouped by topic before the brief is written, so multiple outlets covering the same development become a single item with multiple sources behind it. More sources on one story is a signal t

匹配字段正文

models lose at most 3 percentage points of accuracy with a shared NVFP4 checkpoint, while the separate checkpoint configurations stay within half a percentage point of FP32 on average The paper includes results for all three benchmarks and a long-input RULER test. ReplaySSM is in production. GLM-5.3-Flash serves with ReplaySSM, and Z.ai’s infrastructure report describes adding it to the KDA decode kernel. Kimi-K3 caches the SSM inputs of draft tokens for verification in its KDA layers and cites

匹配字段正文

tested consistently. For example, after identifying that responses to symptom questions could be overly reassuring, Flo created a “balanced causes” rule requiring appropriate coverage of both benign and potentially serious causes when relevant. Those rubrics were then turned into AI judges, each with a definition, grading instructions, and training cases. Flo validated the judges against synthetic cases and anonymized cases from real-user experiments. When an AI judge disagreed with an initial

匹配字段正文

owing teams to automate builds, tests, and deployments from the same development platform. It’s a natural fit for teams already centered on GitLab, but organizations using a more fragmented development stack may get less value from that tight integration. Terraform Terraform is an infrastructure-as-code (IaC) tool for defining and provisioning cloud and on-premises resources with declarative configuration. It’s a strong fit for teams managing infrastructure across multiple providers. The trade-o

匹配字段正文