公开论文雷达

公开 arXiv 研究简报 · 2026-09-16T00:53:49.387203+00:00

先看结论和关键数字,再决定要不要读原文。候选只在首次出现时展示;旧候选若后来通过深读门槛,仍会进入重点。

先别信通过率:八张卡的共同结论是给验收加第二道判据

这八篇的共同点是:单看"测试通过/跑分变好"不足以判定系统正确。综述直接说测试通过不等于行为等价;代码优化那篇发现9个模型对已最优代码100%编造加速说辞。建议先读综述和弃权护栏两篇,把评测口径校准了,再看隐藏状态探针、运行时覆盖率和硬件锁步这些具体的第二道判据怎么搭。

推荐阅读顺序

  1. 2609.12012:先校准判断口径:它对83篇做协议级抽取后指出,测试通过不等于行为等价、反馈有效或流程被遵守,读完再看别的卡才不会被单一通过率带偏。
  2. 2609.14839:紧接着看指标怎么被钻空子:9个模型面对已最优代码过度编辑率100%,加弃权护栏后正确弃权率从0%升到44.4%。
  3. 2609.13975:第一个可落地的第二道判据:静态隐藏状态探针先筛整段对错,再只对已确认有错的程序做行级Top-3定位。
  4. 2609.10590:看能力那一头能做到什么程度:需求直转代码Pass@1达89.2%,但64.5%的失败来自代码生成智能体自己幻觉。
  5. 2609.15387:把判据从代码搬到运行时:插桩采覆盖率引导探索,再把探索和打分分开,与人类偏好一致率85.3%。
  6. 2609.13311:看判据做到极致是什么样:黄金模型逐指令逐通道锁步加三层覆盖率,覆盖闭合98.1%并查出5个真实缺陷。
  7. 2609.15318:顺着硬件往上游看规格层:Given-When-Then场景同时驱动RTL和断言生成,正确性提升2.48倍、覆盖率2.54倍。
  8. 2609.15983:最后看长流程怎么组织:探索—就绪门—依赖分解—并行求解—全局验证,TCS-Bench 71.0%、Codeforces解出218/222。
共性方法
八张卡都在处理同一件事:别让"跑通了"直接当成"对了"。做法分三类——加一个独立参照(SimX黄金模型、Spike二次核对、Gherkin规格层),加一层运行时或内部证据(JS覆盖率插桩、隐藏状态探针),或者把单一分数拆成可分别核查的几项(测试可用性、oracle有效性、反馈利用、评测独立性,以及把弃权也计入分)。
关键分歧
分歧在有没有现成的外部判据可用。硬件那两篇能拿黄金模型和形式验证工具当真值,所以敢报覆盖率闭合和倍数提升;LLM软件那几篇没有真值,只能拿测试、探针、裁判模型做近似,于是限制写得很具体——探针的行级定位只在已知有错时才准,IWC-Bench的197个会话也不是独立随机采样。
选择准则
先问手上有没有能独立核对的真值。有(参考模型、形式工具、运行时覆盖率)就先接上,让失败能定位到具体指令或章节;没有就别报单一通过率,拆成几项分开报,并给系统留一个"不用改了"的弃权出口。

重点深读(8 / 8 篇)

形式化与程序验证(3 篇)

形式化与程序验证 9/30

SIMT-Aware Lockstep Verification and Functional-Coverage Closure Methodology for an Open-Source RISC-V GPGPU: A UVM 1.2 Environment

定向回归全绿不算签核:给开源RISC-V GPGPU补上逐指令锁步校验和覆盖率闭环:如果你的开源GPGPU验证还只靠定向内核回归,那么没有参考模型比对、没有功能覆盖率,签核结论就无法量化,缺陷很可能藏在回归全绿之后。这个方法用SimX黄金模型做逐指令逐通道锁步比对,加三层覆盖模型和故障注入自证,在Vortex上把覆盖率闭合到98.1% covergroup-bin/94.7%总覆盖,并查出5个真实缺陷。

两句看懂

开源GPGPU Vortex此前只靠定向内核回归,没有参考模型校验和功能覆盖率;该文用SimX黄金模型做逐指令逐通道锁步比对,并建三层覆盖模型补齐签核闭环。在11,076次架构写回和5,432次竞态retirement上残差为零,借此查出JALR LSB偏差等5个真实缺陷,覆盖率闭合98.1%/94.7%。

核心判断

该方法能否补齐开源GPGPU的验证闭环?能。用SimX黄金模型逐指令逐通道锁步,加三层覆盖率与故障注入证明,证据是11,076次写回零失配、98.1%/94.7%的覆盖率闭合和5个真实缺陷。

关键要点

1.旧缺口:Vortex等开源RISC-V GPGPU只有定向内核回归和CI,无参考模型校验、无功能覆盖率,签核结论无法量化,竞态无栏栅多核程序此前架构上不可验证。 2.方法加受控检查:SimX经DPI-C重建作黄金模型,Spike做标量二次核对(11,076次写回零失配),端态记分板加五条对齐规则逐指令逐通道锁步,两遍取值法使竞态程序可验证(5,432次retirement残差为零),每个checker经故障注入自证。 3.决定性结果与行动:覆盖率闭合98.1% covergroup-bin/94.7%总覆盖(主配置),查出5个真实缺陷;给加速器验证引入黄金模型锁步加覆盖率闭合,别再只看定向测试是否全绿。

证据与结果

DUT是Vortex开源SIMT GPGPU,黄金模型是SimX。Spike对同一ELF做标量二次核对,在warp0/lane0基础ISA上与DUT/SimX共11,076次架构写回全部一致、零失配(不覆盖SIMT轴)。锁步在2×2×3×2配置轴的5个配置、6个程序上验证到lane精确;两遍取值法处理的竞态无栏栅多核程序上5,432次retirement残差为零。覆盖率主配置闭合98.1% covergroup-bin、94.7%总覆盖,ISA层单独83.1% bins、89.3%加权(寄存器索引排除后)。5个缺陷:JALR LSB偏差、看门狗常量不缩放、复位X态窗口、AXI错误路径缺失、黄金模型取指bug。

打开论文原文
它要解决什么
Vortex这类开源RISC-V GPGPU只有定向内核回归和CI,缺参考模型校验、功能覆盖率和签核纪律;竞态的无栏栅多核程序此前在架构上甚至不可验证。怎么补齐验证闭环,并让置信度可量化?
研究路径
六步走:1)UVM1.2把DUT当AXI主控,agent反转为被动从设备;2)SimX按配置经DPI-C重建作黄金模型,Spike做标量二次核对;3)端态记分板加五条对齐规则做逐指令逐通道锁步比对;4)两遍取值法记录竞态load值,使无栏栅多核程序可验证;5)三层覆盖模型覆盖ISA、SIMT微架构和协议断言,排除项机器生成且可溯源;6)对每个checker做永久故障注入,确认它真的能被触发。
这对工程意味着什么
第一件事:给你的RTL GPU或加速器验证引入按指令粒度的黄金模型锁步和覆盖率闭合目标,而不是只看定向测试是否通过。要避开的捷径:定向回归全绿就当已验证——不对,checker本身还必须经故障注入证明会失效,否则高覆盖率可能是假的。
证据定位
Spike二次核对在warp0/lane0基础ISA上与DUT/SimX共11,076次架构写回全部一致、零失配;锁步在2×2×3×2配置轴的5个配置、6个程序上做到lane精确;两遍取值法处理的竞态无栏栅多核程序上5,432次retirement残差为零。主配置覆盖率闭合98.1% covergroup-bin、94.7%总覆盖;ISA层单独83.1% bins、89.3%加权(寄存器索引排除后)。(筛选维度:形式化验证、可复核评测、软件工程方法)
适用边界
标量二次核对(Spike)只覆盖warp0/lane0的基础指令,到第一个自定义SIMT操作为止,SIMT维度没有独立参考。声明的可靠性边界不含异步中断时序。D-extension意外综合只影响代码覆盖率总数,不影响功能bin。
方法与英文摘要

对象是Vortex开源SIMT GPGPU。UVM1.2环境把DUT当AXI总线主控,agent角色反转为被动从设备。SimX经DPI-C按配置重建,充当黄金模型;Spike做标量二次核对,只覆盖warp0/lane0的基础ISA。核心检查是端态记分板加五条对齐规则的逐指令逐通道锁步比对;两遍取值法记录竞态load的取值,让无栏栅竞态多核程序也能按指令粒度验证。覆盖模型分三层:ISA层、SIMT微架构层(分化深度×重收敛栈、bank冲突、合并访存)、协议断言层;排除项由机器生成且可RTL溯源,并设阻断式豁免完整性关卡。

Open-source RISC-V GPGPUs such as Vortex ship with directed-kernel regressions but no reference-model checking, functional-coverage model, or sign-off discipline. This paper presents a UVM 1.2 environment and methodology that closes that gap. The environment wraps a bus-master SIMT DUT with role-inverted agents, integrates Vortex's functional simulator (SimX) as a per-configuration golden model over DPI-C, and renders verdicts via two injection-qualified checkers: a bidirectional end-state scoreboard and a per-instruction, per-lane lockstep comparator under five SIMT alignment rules. A two-pass load-value feed makes racy fenceless multi-core programs instruction-granularity verifiable (residual zero over 5,432 retirements), with the interrupt-timing boundary stated. A three-layer coverage model adds, to our knowledge, the first published SIMT functional-coverage layer for RTL GPU verification (divergence depth, bank conflicts, coalescing classes), closing its own layers at 98.1% covergroup-bin / 94.7% total (ISA layer separately: 83.1% bins / 89.3% weighted) under machine-generated, RTL-cited exclusions and a blocking waiver-integrity gate; an unstimulated D-extension elaboration affects no functional bin, only lowering totals. The checking depth surfaced real defects on both sides of the comparison: a JALR LSB ISA deviation, a non-scaling watchdog constant (since fixed upstream), a reset-relay X window found by restoring a silenced assertion, a missing AXI error path proven by fault injection, and a reference-model fetch bug found by the lockstep itself. FuzzGPU (USENIX Security 2026), a concurrent RTL GPU fuzzer on the same DUT, is complementary: fuzzing finds bugs, this methodology quantifies sign-off; both independently found the JALR deviation. All findings ship in an evidence-cited register; every number carries provenance and the method's boundaries are stated rather than waived.

形式化与程序验证 7/30

Stellar Colosseum: A Many-Agent Harness for Long-Horizon Research in Mathematics and Theoretical Computer Science

分阶段多智能体流程能把长程数学证明做可靠:TCS-Bench 71.0%,Codef…:如果你在做自动化证明或长程研究类智能体,单次生成长证明不可靠是绕不开的问题。Colosseum用一条“探索—门控—分解—并行求解—全局验证”的五阶段流水线,加上保留批评意见的候选聚合,在TCS-Bench上取得71.0%准确率,在Codeforces 222题中解出218题。

两句看懂

单次生成的长证明常因阶段间依赖断裂而失效,Colosseum改用探索、就绪门控、依赖分解与全局验证的分阶段流程,并以定向证伪和保留批评的聚合处理候选。经TCS-Bench与Codeforces检验,分别取得71.0%准确率与218/222的解题数。

核心判断

长程数学/TCS证明可以可靠自动化:用分阶段就绪门控、依赖分解与保留批评的候选聚合来处理。证据是TCS-Bench 71.0%准确率、Codeforces 218/222解题,以及多个开放问题的新结果。

关键要点

1.旧假设是模型可一次性生成长证明,或用扁平候选投票筛答案;但候选“一致”可能共享同一错误,扁平投票对长程证明指导有限。2.Colosseum分五阶段(探索、就绪门、依赖分解、并行求解、全局验证),阶段内每个合成节点聚合候选子集及其批评并配证伪器,用重叠随机采样树聚合时保留未解决的批评,验证失败路由回具体章节。3.TCS-Bench达71.0%准确率(Gemini 3.1 Pro/3.7 Flash),Codeforces解出222题中218题;落地时应设就绪门并让失败精确回路由,不要靠一致性投票判对错。

证据与结果

TCS-Bench取自FOCS、STOC、SODA已发表论文的研究级定理证明任务,配Gemini 3.1 Pro与Gemini 3.7 Flash,准确率71.0%。另一评测用Codeforces竞赛题222题,带执行反馈的证明导向流程配Gemini 3.1 Pro解出218题。此外系统对FOCS、JMLR等顶会论文的公开问题产出若干新结果,作为开放式研究能力的补充验证。文中未给出与其他基线系统的直接对比数字。

打开论文原文
它要解决什么
长程数学与理论计算机科学证明,能否通过分阶段、多候选、可验证的智能体协作流程,系统性降低失败率?
研究路径
具体步骤:1)策略探索阶段并行生成候选研究路线并互相挑战;2)就绪门判断路线是否成熟到可分解;3)把证明计划拆成带显式依赖的分节子问题;4)各子问题并行求解,每个合成节点随机聚合候选子集与其批评,并配一个证伪器攻击;5)全局验证评估组装后的完整证明;6)验证发现路由回受影响章节触发局部修订,路线失效则重新进入探索。
这对工程意味着什么
第一个动作:在你的流程里显式加就绪门和依赖分解,让验证失败精确路由回具体章节,而不是整体重跑。要避开的捷径:不要只靠多候选一致性投票判定证明正确,一致不代表没有共同错误。
证据定位
TCS-Bench上,Colosseum配Gemini 3.1 Pro与Gemini 3.7 Flash取得71.0%准确率。独立的Codeforces评测中,带执行反馈的证明式流程(Gemini 3.1 Pro)解出222题中的218题。系统还针对FOCS、JMLR等顶会论文遗留的公开问题产出了若干新结果。(筛选维度:形式化验证、可复核评测)
适用边界
论文未给出与其他基线系统的直接对照数字;TCS-Bench与Codeforces结果依赖特定模型组合(Gemini 3.1 Pro/3.7 Flash);开放问题新结果是个案展示而非系统性统计,泛化性未知。
方法与英文摘要

数据用TCS-Bench(取自FOCS/STOC/SODA论文的研究级定理证明任务)和Codeforces竞赛题222题。流程分五步:策略探索→就绪门判断路线是否成熟→分解为带显式依赖的分节子问题→并行求解→全局验证。每个阶段并行生成候选并配定向证伪器,再用重叠随机采样树聚合候选与批评;验证发现的问题路由回对应章节,触发局部修补或重新探索。

Language models can produce plausible short proofs, but may still be unreliable on long-horizon research problems, where progress depends on a sequence of uncertain and interdependent decisions. We introduce Stellar Colosseum, a model-agnostic harness for allocating inference across research in mathematics and theoretical computer science. Colosseum explores alternative strategies before proof construction, uses a readiness gate to decide when a route is mature enough to decompose, represents the proof plan as interdependent section-level subproblems, and routes verifier findings back to the affected part of the argument. Across these stages, it generates candidates in parallel, attacks them with targeted falsification, and combines candidates and their critiques into a single research artifact through overlapping random-sample tree aggregation. The Colosseum workflow has also been integrated into Google Antigravity's Teamwork framework as the Long Proof pattern. We demonstrate the capabilities of Colosseum through open-ended research and evaluations on theorem-proving and competitive programming benchmarks. Using Colosseum with Gemini 3.1 Pro, we obtain several new results that address open problems arising from papers published at top venues such as FOCS and JMLR. On TCS-Bench, a benchmark of research-level theorem-proving tasks drawn from papers published at FOCS, STOC, and SODA, Colosseum achieves 71.0% accuracy using Gemini 3.1 Pro and Gemini 3.7 Flash. In a separate Codeforces evaluation using Gemini 3.1 Pro, the proof-oriented pipeline with execution feedback solves 218 of 222 problems.

形式化与程序验证 7/30

LLM-enabled Behavior Driven Development Workflow for Formally Verified Hardware Designs

用Gherkin场景做硬件规格:RTL正确率提升2.48倍,断言覆盖率提升2.54倍:如果你在用大模型生成RTL或验证断言,规格的写法直接决定输出质量。自由格式的自然语言需求有歧义,会同时拖累两条生成路径。这篇论文的办法是:先写一层Given-When-Then结构的受控自然语言场景,再用同一份规格分别驱动RTL生成和FPV断言生成。对比已有LLM方法,RTL功能正确性提升2.48倍,断言的形式覆盖率提升2.54倍。

两句看懂

自然语言规格模糊会拖累大模型生成的RTL和断言质量,论文用Gherkin式CNL场景统一驱动这两条生成路径。与已有LLM方法对比,RTL功能正确性提升2.48倍,断言的形式覆盖率提升2.54倍。

核心判断

统一的受控自然语言规格能同时提升RTL生成和断言生成的质量。采用Given-When-Then结构的FV Gherkin Scenarios作为规格层后,RTL正确性提升2.48倍,断言形式覆盖率提升2.54倍。

关键要点

1.以往LLM用于EDA多单独评测RTL生成或断言生成,缺少贯通规格到形式验证的工作流;自然语言规格的歧义直接损害LLM输出质量。2.方法是FV Gherkin Scenarios:用Given-When-Then结构写CNL规格(含Scenario Outline/Examples参数化),同一份规格驱动LLM分别生成RTL和FPV断言,接入形式验证工具做对照检查。3.结果:RTL功能正确性提升2.48倍,断言形式覆盖率提升2.54倍;工程上应先把需求固化成这层CNL规格再让LLM生成。

证据与结果

评测看两个维度:生成RTL的功能正确性,以及生成断言在FPV下的形式覆盖率。对比对象是其他已建立的LLM方法。结果是功能正确性提升2.48倍、形式覆盖率提升2.54倍。摘录未给出基准数据集名称、题目数量或划分方式,示例场景只有ALU加法。

打开论文原文
它要解决什么
老问题是:自然语言规格模糊,LLM生成的RTL和断言质量跟着下降。能不能用一种人能读、结构受控的规格语言,同时驱动RTL生成和断言生成,并接入形式验证工具?
研究路径
步骤分五步:1)用Given-When-Then句式写CNL场景,明确前置条件、触发条件和后置断言;2)用Scenario Outline加Examples表参数化多组输入取值;3)LLM依据该规格生成RTL设计;4)LLM依据同一份规格生成FPV断言;5)把RTL和断言一起接入形式验证工具,检查功能正确性和形式覆盖率,并与其他LLM方法对比。两条生成路径共用同一份规格,口径天然一致。
这对工程意味着什么
第一步行动:在让LLM生成RTL或断言之前,先固化一层Given-When-Then式CNL规格作为唯一真源,保证两条生成路径口径一致且人工可审阅。要避开的捷径:不要让LLM直接从自由格式的自然语言需求生成RTL或断言,规格歧义会同时拖累两者质量。
证据定位
在同一流程下与其他已建立的LLM方法对比:生成的RTL设计功能正确性提升2.48倍,生成的断言在FPV下的形式覆盖率提升2.54倍。摘录未给出具体基准数据集名称和题目规模。(筛选维度:形式化验证、可复核评测)
适用边界
摘录未提供基准数据集名称、题目数量、划分方式或失败案例细节。示例只有ALU加法一个场景,工作流在更复杂时序逻辑或多模块设计上的表现,原文未展示。
方法与英文摘要

论文定义了FV Gherkin Scenarios。做法是用Given-When-Then结构描述硬件行为:以ALU加法为例,Given设reset为低、给定操作数A/B和操作码,When在时钟上升沿触发,Then校验结果和零/溢出/负数标志位。再用Scenario Outline加Examples表批量覆盖多组输入取值。同一份CNL规格交给LLM,分别生成RTL设计和FPV断言,然后接入形式验证工具检查正确性和覆盖率,与其他已有LLM方法在同一流程下对比。

Recently, the use of Large Language Models (LLMs) for different tasks in the Electronic Design Automation (EDA) life-cycle has been studied extensively, but an integrated view is lacking. Specifications are the foundation of this life-cycle, but they suffer from ambiguity when written in natural language, which especially affects the quality of LLM output. Formal specifications mitigate these ambiguities, but they come with their own challenges. On the other hand, Controlled Natural Language (CNL) specifications can serve as a middle-ground, reducing ambiguity while retaining interpretability. In this work, we propose an integrated view on the use of LLMs for EDA and establish an LLM-enabled behavior driven hardware development workflow. We introduce and define Formal Verification Gherkin Scenarios (FV Gherkin Scenarios), unlocking CNL specifications as the foundation for formally verified hardware designs via Formal Property Verification (FPV). Experimental evaluation shows that our workflow is able to outperform other established LLM-based methods by 2.48x in functional correctness of generated Register Transfer Level (RTL) designs and by 2.54x in formal coverage of generated assertions for FPV.

软件工程与仓库智能(3 篇)

软件工程与仓库智能 10/30

Introspective Uncertainty Estimation for LLM-Based Code Generation

用隐藏状态筛查代码对错有效,行级定位只在已知有错时才可靠:如果你用LLM生成代码又没时间逐行人工审查,这个方法能帮你省成本:先用模型内部隐藏状态判断整段代码对不对,再定位错在哪一行。LiveCodeBench上静态探针做到0.90 AUROC、0.96 F1,超过以往静态基线;但行级定位只在确认程序有错误时才可靠,Top-3命中率达81%,混合场景下明显更弱。

两句看懂

以往判断LLM生成代码是否正确多依赖输出层面的黑盒信号,难以定位具体错误行;本研究改用隐藏状态做静态探针和行级Top-K定位。静态探针在LiveCodeBench上达0.90 AUROC、0.96 F1,在已知错误程序前提下Top-3行定位命中率达81%。

核心判断

隐藏状态确实能可靠反映代码正确性:response级用静态探针即可达0.90 AUROC、0.96 F1;但行级定位仅在已知程序有错误的前提下有效,Top-3命中率81%,混合场景中行级判断明显更弱。

关键要点

1.旧做法:对LLM代码生成的不确定性估计多基于输出层面的黑盒信号,较少利用内部隐藏状态,错误代码难以定位到具体行。2.方法与对照:在LCB和BCB上生成代码并按功能测试标注,从错误程序派生token级和行级标签,对比静态单token探针与动态token选择/序列建模,并测试跨任务、领域、token位置的泛化。3.决定性结果:静态探针在LCB最佳设置达0.90 AUROC、0.96 F1,超过静态基线,动态方法无稳定提升;混合场景行级定位明显更难,已知错误程序下Top-3命中率达81%。

证据与结果

数据源为LiveCodeBench(LCB)、BigCodeBench(BCB)两个代码生成基准。response级用AUROC、F1比较静态单token探针与动态token选择/序列建模方法,并测试跨任务、跨领域、跨token位置泛化;行级分混合正确错误程序与仅含已知错误程序两种设置,用Top-K命中率衡量定位。结果:静态探针在LCB最佳设置达0.90 AUROC、0.96 F1,超过此前静态基线,动态方法无稳定提升;真实项目场景跨设置退化明显;混合场景行级定位远难于response级,已知错误程序下Top-3命中率达81%。

打开论文原文
它要解决什么
大语言模型的隐藏状态能否可靠判断生成代码是否功能正确?能否进一步定位到具体的错误行?
研究路径
1.在LCB/BCB上生成代码并用功能测试标注正确或错误;2.构建增强流程从错误程序中派生token级和行级标签;3.从隐藏状态训练静态单token探针及动态token选择/序列建模分类器;4.用AUROC/F1比较并测试跨任务、领域、token位置泛化;5.在已知错误程序前提下用隐藏状态做Top-K行定位并统计命中率。
这对工程意味着什么
行动:先用静态单token探针筛查生成代码的正确性风险,再对确认有错误的程序做行级Top-K定位。要避开的捷径:不要直接对未经筛查的混合代码做行级定位,那种场景下准确率会明显下降。
证据定位
静态单token探针在LCB最佳设置达到0.90 AUROC、0.96 F1,超过此前静态探针基线。动态方法没有稳定提升。行级定位在混合正确错误程序的场景中明显更难;在已知程序有错误的前提下,Top-K定位的Top-3命中率达到81%。(筛选维度:置信度与不确定性、可复核评测、软件工程方法)
适用边界
仅在LiveCodeBench、BigCodeBench两个基准及其错误标注派生数据上验证;跨真实软件项目场景存在设置依赖的性能下降;行级定位仅在已知错误程序前提下验证有效,未充分覆盖混合真实代码库场景。
方法与英文摘要

在LiveCodeBench(LCB)和BigCodeBench(BCB)上生成代码,按功能测试标注对错。构建增强流程,从错误程序派生token级和行级标签。对比两类方法:静态单token探针,以及动态token选择/序列建模。评估跨任务、跨领域、跨token位置的泛化能力。行级定位分两种场景评估:混合正确与错误程序,以及仅含已知错误程序。

Large Language Models (LLMs) are increasingly used for code generation but can produce fluent yet functionally incorrect outputs, which limits trust in their usage for practical software engineering workflows. This thesis investigates whether Introspective Uncertainty Estimation (IUE), based on internal hidden-state representations of LLMs, can reliably indicate correctness at the response and line levels for code generation tasks. The objective is to determine the extent to which hidden states encode information about functional code correctness and how this can be leveraged for practical risk assessment and fault localization. Methodologically, this thesis combines response-level evaluation on LiveCodeBench (LCB) and BigCodeBench (BCB) with an augmentation pipeline that derives token- and line-level labels from incorrect programs. In this setup, it compares static and dynamic response-level features, evaluates generalization across tasks, programming domains, and token positions, and studies line-level fault localization. The results show that hidden states contain a strong response-level correctness signal. Static single-token probes perform best, while more elaborate dynamic strategies yield no consistent gains. While generalization across tasks, domains, and token positions is feasible, setting-dependent degradation largely remains for real-world software projects. At a fine granularity, line-level prediction is substantially harder than response-level estimation. However, in a conditional localization setup with known-incorrect programs, Top-K point-of-failure ranking remains effective. Overall, the findings suggest that hidden states are a robust and informative resource for estimating functional code correctness, supporting a two-stage workflow that combines response-level risk screening with targeted line-level prioritization.

软件工程与仓库智能 9/30

ReqEvolve: User-Oriented Software Self-Evolution through Automatic Requirement Interpretation

用户需求直转代码,幻觉仍是主要失败源:系统将用户自然语言请求直接转成可运行代码,跳过开发者先写技术规格的步骤。72个跨18个项目的演化任务中,Pass@1达89.2%,超过RE基线SpecFix18.8%。

两句看懂

以往自演化系统依赖开发者先把用户需求译成技术规格才能生成代码,拖慢早期验证;ReqEvolve改为直接接收用户自然语言请求,靠需求澄清与测试驱动开发合成功能。在72个跨18个项目的演化用例上,Pass@1达89.2%,比SpecFix基线高18.8%,比消融版本高32.6%。

核心判断

用户自然语言请求可以直接驱动可运行的软件自演化,不必先转成技术规格;72个用例上ReqEvolve Pass@1达89.2%,显著优于SpecFix与消融基线(大效应)。

关键要点

1. 以往自演化系统假设用户需求已被开发者转写为实现导向的技术规格(如指定函数名、数据文件、环境变量),系统本身不直接消化用户用日常语言表达的意图,导致早期功能验证要等开发者完成翻译才能进行。 2. 构建基准含72个软件演化用例,覆盖18个项目、代码量40-3100行;每用例给出用户自然语言请求,ReqEvolve按需求澄清→规格分解→测试用例生成→功能代码合成→运行时集成执行,对照RE专用静态代码生成基线SpecFix与去除部分模块的消融版本。 3. ReqEvolve整体Pass@1达89.2%,比SpecFix高18.8%(p<0.01,r=0.79,大效应),比消融基线高32.6%(p<0.001,r=0.88,大效应);根因分析显示64.5%的失败来自代码生成智能体本身产生幻觉,即便输入清晰、需求细化正确也会出错。

证据与结果

基准包含72个软件演化用例,分布在18个项目中,代码规模40-3100行。对比两个基线:SpecFix(仅做需求工程的静态代码生成)与消融版本(去除部分ReqEvolve模块)。指标为Pass@1。结果:ReqEvolve 89.2%,比SpecFix高18.8%(p<0.01,r=0.79),比消融基线高32.6%(p<0.001,r=0.88)。根因分析显示64.5%失败源于代码生成智能体在输入清晰、需求正确情况下仍产生幻觉。

打开论文原文
它要解决什么
自演化系统能否直接消化用户用日常语言表达的功能请求,而不必先由开发者转写为技术规格?
研究路径
ReqEvolve收到用户高层请求后依次执行:①需求澄清厘清模糊意图;②规格分解拆成可执行技术子需求;③测试驱动开发先生成测试用例;④代码生成智能体合成功能实现;⑤运行时集成把新功能接入现有代码库并执行验证。
这对工程意味着什么
在自动化代码生成流程中加入需求澄清与测试先行环节,可显著提升通过率;但即使需求澄清和规格都正确,代码生成模型仍可能幻觉出错,不能省略对生成代码的独立测试校验。
证据定位
ReqEvolve在72个用例上Pass@1达89.2%,比SpecFix高18.8%(p<0.01,r=0.79,大效应),比消融基线高32.6%(p<0.001,r=0.88,大效应)。(筛选维度:形式化验证、可复核评测、软件工程方法)
适用边界
评测仅覆盖72个用例、18个项目,代码规模限定在40-3100行,结果规模有限,未说明用例选取标准或跨语言、跨领域的代表性。
方法与英文摘要

基准含72个软件演化用例,覆盖18个项目(代码量40-3100行)。每个用例给出用户用日常语言表达的功能请求。ReqEvolve流程:需求澄清→规格分解→测试用例生成→功能代码合成→运行时集成执行。对照SpecFix(仅做需求工程的静态代码生成)与去掉部分模块的消融版本。

The paradigm of software self-evolution enables systems to autonomously extend and reconfigure their own capabilities during execution in response to technical specifications. Yet requests for new functionality often originate from end users and are rarely expressed in technical terms. As a result, developers must translate user needs into technical specifications before the system can evolve, delaying early validation of the requested functionality by preventing users from immediately observing the resulting behaviour. To address this gap, we present ReqEvolve, a runtime code generation system that enables user-driven self-evolution by accepting high-level user requests. The system integrates automatic requirements engineering (RE) and test-driven development (TDD) to transform these requests into executable functionality through clarification, specification decomposition, test generation, and runtime integration. We evaluate ReqEvolve on 72 software evolution cases across 18 projects against two baselines: SpecFix, an RE-focused code generation approach, and an ablation variant of our system. ReqEvolve achieves 89.2% Pass@1, outperforming SpecFix by 18.8% (p < 0.01, r = 0.79, large effect) and the ablation baseline by 32.6% (p < 0.001, r = 0.88, large effect). These results provide initial evidence that user-driven self-evolution is a viable paradigm for autonomously extending software capabilities from user requests, thereby accelerating requirements validation prior to developer verification.

软件工程与仓库智能 7/30

IWC-Bench: Evaluating Web Application Generation from a Software Testing Perspective

评测生成网页应用,要用覆盖率引导运行时探索,并把探索与打分分开:静态看源码或截图会把不可达功能误判成已实现;只做固定验收交互又分不清是应用坏了还是智能体没探到。IWC-Bench用JavaScript插桩采集运行时覆盖率,引导智能体像测试者一样探索,再按视觉、可用性、需求对齐独立打分;在197个内部竞技场会话上与人类偏好一致率85.3%,与Code Arena排名相关系数0.876。

两句看懂

静态评测会误判代码存在但运行时不可达的功能,交互式评测又难以区分应用缺陷与智能体探索失败,IWC-Bench用代码覆盖率引导探索并将探索与打分分离。在197个内部竞技场验证会话上核验,与人类偏好一致率85.3%,且分差越大一致率越高。

核心判断

判断:评测LLM生成的网页应用,核心不是再看代码或截图,而是用覆盖率引导运行时探索,并把探索与打分分离。支撑数字是197个会话人类偏好一致率85.3%,与Code Arena排名Spearman相关0.876。

关键要点

1. 旧做法失效:看源码或截图会把失效入口背后的不可达功能计为完整;固定验收交互失败时分不清是应用缺陷还是探索出错。2. 方法与受控核对:369条需求、5088条验收标准;插桩采覆盖率,覆盖率引导探索,生成状态转换图,三维独立打分且探索/打分分离。3. 结果与动作:人类偏好一致率85.3%、与Code Arena相关0.876;无模型三项全领先,应改用运行时插桩评测替代人工两两比较。

证据与结果

评测覆盖16个前沿LLM生成的网页应用;结果显示各模型在视觉美观、可用性、需求对齐三维度各有优势,没有模型三项全部领先。人类核对用内部竞技场197个验证会话,一致率85.3%,且分差越大越一致;与Code Arena共有模型排名相关0.876。覆盖率引导被消融验证有效,更换裁判模型后排名基本稳定。

打开论文原文
它要解决什么
如何在不依赖预定义验收标准的前提下,用可复现的运行时证据,评测LLM生成网页应用里真实可用的功能,并与人类偏好对齐?
研究路径
机制是测试式闭环:先自动插桩JavaScript,拿到运行时覆盖率;再让LLM智能体以测试者身份,用覆盖率找未执行路径并模拟用户交互;交互轨迹压成状态转换图;最后裁判按视觉美观、可用性、需求对齐分别给分。关键点在于探索不受验收标准约束,打分也不受探索过程限制。
这对工程意味着什么
第一步行动:给生成网页应用加JavaScript运行时插桩,用覆盖率引导智能体探索后再按视觉、可用性、需求对齐打分。要避开的捷径:只跑固定验收标准做交互测试,因为失败时会把应用缺陷和智能体探索失败混为一谈。
证据定位
在内部竞技场抽取的197个验证会话上,IWC-Bench与人类偏好一致率85.3%,且两应用分差越大一致率越高;16个共有模型排名与Code Arena的Spearman相关系数0.876。消融显示覆盖率引导提升探索覆盖率;更换裁判模型后排名基本不变。(筛选维度:可复核评测、软件工程方法)
适用边界
边界很清楚:人类偏好核对来自内部竞技场抽取的197个验证会话,不是独立随机采样;不同模型还使用不同执行环境和推理强度设置,例如Codex与Claude Code等harness差异,可能混淆模型间直接对比。
方法与英文摘要

数据是369条真实用户需求和5088条验收标准。流程四步:自动插桩生成应用的JavaScript代码并采集运行时覆盖率;LLM智能体作为软件测试者,按覆盖率引导做用户模拟交互探索;把交互轨迹抽象成状态转换图;按视觉美观、可用性、需求对齐三个维度独立打分,探索与打分分离。共评测16个前沿模型。

Human evaluation provides a direct measure of the quality of LLM-generated web applications. However, fitting human judgments through automated evaluation remains challenging. Static benchmarks can credit functionality that exists in source code but is unreachable at runtime. Interactive benchmarks exercise the application, yet incomplete exploration can cause them to miss implemented functionality and confound application defects with agent execution failures. To address these limitations, we propose IWC-Bench, an interactive benchmark for evaluating web application generation from a software testing perspective. IWC-Bench instruments each generated application and uses code coverage to guide an agent in exploring its functionality through user-simulated interactions. It then abstracts the interaction trace into a state-transition graph and evaluates the application along three dimensions: visual aesthetics, usability, and requirement alignment. By separating exploration from scoring, IWC-Bench collects runtime evidence without constraining exploration to predefined acceptance criteria. IWC-Bench comprises 369 real-world user requirements and 5,088 acceptance criteria. Evaluation of 16 frontier LLMs reveals distinct strengths across the three dimensions, with no model leading on every dimension. On 197 validated sessions sampled from an internal arena, IWC-Bench achieves 85.3\% agreement with human preferences, with agreement generally increasing as the score difference between paired applications grows. Further experiments show that coverage guidance improves exploration coverage and the model rankings remain stable when the judge model is replaced.

代码质量与优化(2 篇)

代码质量与优化 9/30

Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skills

测试通过不等于代码正确:87篇综述要求拆开评估LLM测试驱动研究:如果你在用通过率评估LLM代码生成或智能体系统,这条结论直接关系到你下判断的方式:一篇覆盖87篇研究的综述指出,测试通过既不能证明行为等价,也不能证明反馈有效或流程被遵守,聚合的通过率提升还可能掩盖模型和任务之间的真实差异。方法是对83篇文献做协议级抽取后交叉比较。

两句看懂

过去很多研究把测试通过当作代码正确、反馈有效、流程被遵守的证据。这篇综述抽取83篇协议级文献后发现,聚合的通过率提升可能掩盖不同模型、任务、统计分母下的真实差异,四项属性必须分开核查。

核心判断

研究问题:测试在LLM软件工程中到底改变了什么决策?结论:测试通过不代表行为等价、有效反馈或流程遵循。证据来自对83篇协议级文献的交叉比较——四项属性在现有研究中常被混淆评估。

关键要点

1. 旧假设:许多研究默认'测试通过'等于行为正确、反馈被有效利用、流程被遵守,综述指出这三者可分离,不能互相替代。 2. 方法与核查:对87篇文献中的83篇做协议级抽取,按5种阶段×9类任务×智能体工作流/技能交叉比较,受控核对测试可用性、oracle有效性、反馈利用、评测独立性四项。 3. 决定性结论与行动:聚合通过率会掩盖模型、任务、分母间的真实差异,评测时必须把四项属性拆成独立指标分别报告。

证据与结果

数据来源:87篇研究与支持性记录,其中83篇做方法/协议级抽取,另设5份独立实践资源。比较维度:5种测试驱动机制(红绿重构、测试条件生成、执行引导修正、测试中介分析、仅评测测试)×9类任务×智能体工作流与技能。综述没有给出统一数值指标,而是做定性交叉比较,核心发现是测试可用性、有效性、反馈利用、评测独立性四项常被混为一谈,聚合通过率可掩盖模型、任务、分母差异。

打开论文原文
它要解决什么
在LLM软件工程里,测试到底改变了哪个决策?测试通过是否就等于代码正确、反馈被有效利用、开发流程被遵守?
研究路径
综述的工作分五步:1)收集87篇LLM测试驱动软件工程研究及支持性文献;2)对83篇做方法/协议级信息抽取;3)另建5份实践资源库;4)按TDD阶段、9类软件工程任务、智能体工作流与技能三个维度分类;5)比较各文献的测试可用性、oracle有效性、反馈利用方式与评测独立性,归纳出跨任务的机制分类和协议敏感的证据分析。
这对工程意味着什么
第一步行动:审核任何LLM测试驱动代码系统的评测报告时,分别追问测试可见性、oracle是否正确、反馈是否被采纳、统计分母是否一致。要避开的捷径:只看'通过率提升'就断定系统更好。
证据定位
对83条协议级文献的交叉比较发现:测试通过不能证明行为等价、有效反馈或流程遵循。不同研究在测试可见性、oracle有效性和统计分母上各不相同,因此同样的'通过率提升'可能掩盖模型之间、任务之间的真实差异。(筛选维度:形式化验证、可复核评测、软件工程方法)
适用边界
这是范围性综述,基于87篇文献(83篇协议级抽取、5份实践资源)的定性归纳,未做统一定量实验或跨系统直接对照,结论受纳入文献覆盖面和分类标准影响。
方法与英文摘要

范围性综述。作者收集87篇LLM测试驱动软件工程文献,对其中83篇做方法/协议级抽取,另整理5份实践资源。然后按三条主线交叉比较:5种阶段(红绿重构、测试条件生成、执行引导修正、测试中介分析、仅评测测试)、9类任务(生成、修复、翻译、重构、克隆检测、代码搜索、定位、训练数据构建、规约验证)、智能体工作流与技能,重点核对测试可用性、有效性、反馈利用与评测独立性。

Tests increasingly participate in the decisions made by large language models and software engineering agents. They specify intended behavior, guide program construction and repair, select candidates, constrain transformations, and provide execution evidence for software analysis. These uses draw on test-driven development, yet differ substantially in test order, oracle availability, editable artifacts, and the role of execution. We present a structured scoping survey organized around the question of what decision a test changes. The review integrates 87 research and supporting records, with method- or protocol-level extraction for 83 records, alongside a separate collection of five practice resources. We distinguish the Red--Green--Refactor cycle from test-conditioned generation, execution-guided refinement, test-mediated analysis, and evaluation-only testing. We then compare code generation, repair, translation, refactoring, clone detection, code search, localization, training-data construction, and formal-specification validation. A dedicated analysis examines how agent workflows and reusable skills encode testing procedures and how their effects are evaluated. Across these tasks, the evidence supports treating test availability, test validity, feedback use, and evaluation independence as separate properties. Test passing alone does not establish behavioral equivalence, effective feedback, or process adherence; aggregate improvements can also conceal different outcomes across models, tasks, and denominators. We synthesize these distinctions into a mechanism taxonomy, a cross-task comparison, and a protocol-sensitive evidence analysis, and identify research directions in oracle validation, causal evaluation, long-horizon maintenance, and reusable test-driven agent capabilities

代码质量与优化 7/30

Efficiency Hallucination: Formalizing and Measuring Behavioral Calibration in LLM-Based Code Optimization

代码已最优,LLM仍100%编造加速说辞——加个弃权护栏就能治:如果你用LLM做代码性能优化,这条要先看:面对已最优的代码,9个主流模型在标准评测下全部乱编加速说辞,过度编辑率100%。原因是评测只看功能正确和运行时,不惩罚虚假说辞。研究提出一个无需训练的弃权护栏,让模型能诚实说'不用改了',正确弃权率从0%升到44.4%。

两句看懂

二元评测只奖励编辑、不奖励诚实承认已最优,导致9个模型在标准提示下对最优代码的过度编辑率高达100%。加入分类惩罚护栏后,正确弃权率从0%升到44.4%,对次优代码仍保持100%编辑率且零错误弃权。

核心判断

问题出在评测激励,不是模型能力上限:标准评测下LLM对已最优代码100%编造加速说辞;加入护栏后正确弃权率从0%升至44.4%,且不伤对次优代码的优化能力。

关键要点

①旧评测(如EffiBench、PIE)只按功能正确和运行时打分,不惩罚虚假加速说辞,'瞎编'成了对已最优代码的零风险策略,过度编辑率达100%。②方法:180次试验覆盖GPT、Claude、Gemini共9个模型,用Optimal Baseline协议区分四类结果,加基于IIV的分类惩罚护栏并做前后对照。③结果:正确弃权率0%→44.4%,次优代码编辑率仍100%、零错误弃权;做法是给优化流水线加可弃权护栏,并把弃权纳入评测指标。

证据与结果

数据用EffiBench基准;规模为9个模型(GPT、Claude、Gemini三家族)×180次优化试验。协议用Optimal Baseline法,区分过度编辑、错误弃权、正确编辑、正确弃权。结果:标准提示下最优代码过度编辑率100%;护栏后正确弃权率0%→44.4%,次优代码编辑率仍100%且零错误弃权。诊断:GPT-5.4 Mini弃权最准;简单代码比复杂代码更容易被正确识别为已最优。

打开论文原文
它要解决什么
面对已经最优的代码,LLM优化工具会不会编造并不存在的性能提升,而不是如实承认无法再优化?
研究路径
流程四步:①用EffiBench准备已最优和次优代码样本;②标准提示下运行9个模型共180次,记录编辑与弃权行为;③引入基于IIV框架的分类惩罚护栏,允许模型主动弃权;④对比护栏前后的过度编辑率、正确弃权率、错误弃权率和编辑率。机制上,护栏改变了激励结构:承认'已最优'不再丢分,瞎编反而受罚。
这对工程意味着什么
行动:给优化提示加可弃权选项并配惩罚机制,上线前先测过度编辑率。要避开的捷径:别以为通过功能测试和运行时基准就等于加速说辞可信——这两个指标都不惩罚虚假说辞。
证据定位
标准提示下,9个模型对已最优代码的过度编辑率达100%,即全部编造加速说辞。加入护栏后,正确弃权率从0%升至44.4%;同时对次优代码仍保持100%编辑率,且零错误弃权。模型间校准能力参差:GPT-5.4 Mini弃权最准;模型对简单代码的识别明显比复杂代码可靠。(筛选维度:置信度与不确定性、可复核评测)
适用边界
这只是180次试验的先导性研究,只覆盖9个模型和EffiBench一个基准;复杂代码上的判别可靠性明显低于简单代码,结论的泛化性有限。
方法与英文摘要

研究基于EffiBench数据集,对GPT、Claude、Gemini三家共9个模型做了180次代码优化试验。核心做法有两个:一是Optimal Baseline评测协议,把结果分成过度编辑、错误弃权、正确编辑、正确弃权四类;二是基于IIV理论的分类惩罚护栏,允许模型放弃修改,而不是强行编辑已最优的代码。

The integration of Large Language Models (LLMs) into automated code optimization introduces a critical reliability risk we term the Efficiency Hallucination: an LLM's tendency to issue non-functional mutations with unsubstantiated performance claims on already-optimized code. This is driven by the Evaluation Trap, wherein binary benchmarks incentivize unnecessary modifications over safely abstaining. We present a validation framework using classification penalty methods, evaluated across 180 optimization runs on nine models (GPT, Claude, Gemini) using EffiBench. Under standard prompts, models exhibit a 100% over-edit rate on optimal code. Our guardrail raises correct abstention from 0% to to 44.4%, preserving a 100% edit rate on sub-optimal code with zero false abstentions. Calibration is uneven: GPT-5.4 Mini approaches near-perfect abstention, and simple code is recognized more reliably than complex code. Our framework offers a training-free mechanism to mitigate LLM overconfidence before deployment in production.

UI 与 GUI Agent(0 篇)

本轮没有通过深读证据门的重点论文。

个人知识与本体(0 篇)

本轮没有通过深读证据门的重点论文。

人机协同与对齐(0 篇)

本轮没有通过深读证据门的重点论文。

本轮分类概览

同一论文只归入一个最先命中的赛道,避免重复计数;“新增候选”只统计首次展示的论文。

赛道新增候选重点
形式化与程序验证63
软件工程与仓库智能103
代码质量与优化42
UI 与 GUI Agent00
个人知识与本体30
人机协同与对齐00

近一个季度监测日历

北京时间。绿色表示有可阅读的新候选,灰蓝表示已监测但无新增,橙色表示部分降级;“无记录”不等于失败。

近 14 次监测窗口

仅展示公开源的聚合运行状态,不含提示词、全文或个人数据。

本轮新增候选(23 篇)

按赛道、评分和日期展开;中文标签用于导航,英文摘要用于核验。已展示过的旧候选不会每日重复。

形式化与程序验证(6 篇)

形式化与程序验证 · 7/30 · 2026-09-14多智能体长程数学研究框架为推理分配设计框架:并行候选、定向反驳、就绪门控,证据仅来自摘要Stellar Colosseum: A Many-Agent Harness for Long-Horizon Research in Mathematics and Theoretical Computer Science

Language models can produce plausible short proofs, but may still be unreliable on long-horizon research problems, where progress depends on a sequence of uncertain and interdependent decisions. We introduce Stellar Colosseum, a model-agnostic harness for allocating inference across research in mathematics and theoretical computer science. Colosseum explores alternative strategies before proof construction, uses a readiness gate to decide when a route is mature enough to decompose, represents the proof plan as interdependent section-level subproblems, and routes verifier findings back to the affected part of the argument. Across these stages, it generates candidates in parallel, attacks them with targeted falsification, and combines candidates and their critiques into a single research artifact through overlapping random-sample tree aggregation. The Colosseum workflow has also been integrated into Google Antigravity's Teamwork framework as the Long Proof pattern. We demonstrate the capabilities of Colosseum through open-ended research and evaluations on theorem-proving and competitive programming benchmarks. Using Colosseum with Gemini 3.1 Pro, we obtain several new results that address open problems arising from papers published at top venues such as FOCS and JMLR. On TCS-Bench, a benchmark of research-level theorem-proving tasks drawn from papers published at FOCS, STOC, and SODA, Colosseum achieves 71.0% accuracy using Gemini 3.1 Pro and Gemini 3.7 Flash. In a separate Codeforces evaluation using Gemini 3.1 Pro, the proof-oriented pipeline with execution feedback solves 218 of 222 problems.

阅读 arXiv 原文
形式化与程序验证 · 3/30 · 2026-09-14替代网络忠实性的形式验证用可达性分析认证忠实度差距上界;摘要称轻微扰动可翻转解释The Misery of Mechanistic Interpretability: A Formal Perspective

Mechanistic interpretability has become the dominant lens for understanding frontier language models, as their inner workings are complex and inherently black boxes. To gain insights into these models, interpretable replacement networks (IRNs) are trained at all layers, exposing interpretable features through sparsely activated neurons. However, the faithfulness of an IRN is usually evaluated only empirically on clean data, and we show that even semantically minor input perturbations flip the dominant IRN features-and thus the human-understandable interpretation-across five open-weight model families (GPT-2 small, Gemma 2 2B, Gemma 3 1B, Llama 3.2 1B, R1-Distill-Qwen 1.5B). We propose the first formal verification framework for the faithfulness of an IRN, where reachability analysis certifies a sound upper bound of the faithfulness gap in adversarial scenarios. Moreover, we show that verification-aware training of IRNs substantially tightens this certified bound, restoring a feature-level interpretation that safety auditors can act on. Together, these results give, to the best of our knowledge, the first formal guarantees for mechanistic interpretability of large language models.

阅读 arXiv 原文
形式化与程序验证 · 7/30 · 2026-09-14LLM驱动的硬件形式验证工作流提出FV Gherkin场景,以受控自然语言衔接规范与形式验证LLM-enabled Behavior Driven Development Workflow for Formally Verified Hardware Designs

Recently, the use of Large Language Models (LLMs) for different tasks in the Electronic Design Automation (EDA) life-cycle has been studied extensively, but an integrated view is lacking. Specifications are the foundation of this life-cycle, but they suffer from ambiguity when written in natural language, which especially affects the quality of LLM output. Formal specifications mitigate these ambiguities, but they come with their own challenges. On the other hand, Controlled Natural Language (CNL) specifications can serve as a middle-ground, reducing ambiguity while retaining interpretability. In this work, we propose an integrated view on the use of LLMs for EDA and establish an LLM-enabled behavior driven hardware development workflow. We introduce and define Formal Verification Gherkin Scenarios (FV Gherkin Scenarios), unlocking CNL specifications as the foundation for formally verified hardware designs via Formal Property Verification (FPV). Experimental evaluation shows that our workflow is able to outperform other established LLM-based methods by 2.48x in functional correctness of generated Register Transfer Level (RTL) designs and by 2.54x in formal coverage of generated assertions for FPV.

阅读 arXiv 原文
形式化与程序验证 · 3/30 · 2026-09-11梅德韦杰夫逻辑不可判定由周期密铺问题归约证明;摘要称解决长期开放问题,证据限于摘要Medvedev logic is undecidable

We show that Medvedev's logic of finite problems, a well-known superintuitionistic logic, is undecidable. The key method is a reduction from the periodic tiling problem to non-theoremhood in Medvedev's logic. This settles a longstanding open problem. Using similar techniques, but reducing instead to the ordinary tiling problem, we likewise obtain undecidability of Skvortsov's logic of infinite problems, and the fact that the two logics are distinct -- in fact, they are separated by any aperiodic tiling of the plane. Due to the fact that Medvedev's logic figures in so many different areas, these results have implications for several fields -- for example, the study of schematic fragments of logics such as propositional dependence logic, or the study of internal logics of toposes. The core idea and technical work of the undecidability proof were obtained using ChatGPT Sol 5.6, and formally verified in Lean by Claude Opus 5. A detailed methodology section outlines how such results were obtained.

阅读 arXiv 原文
形式化与程序验证 · 9/30 · 2026-09-10开源RISC-V GPGPU验证方法学搭建UVM环境,以SimX为黄金模型做逐指令逐通道锁步比对SIMT-Aware Lockstep Verification and Functional-Coverage Closure Methodology for an Open-Source RISC-V GPGPU: A UVM 1.2 Environment

Open-source RISC-V GPGPUs such as Vortex ship with directed-kernel regressions but no reference-model checking, functional-coverage model, or sign-off discipline. This paper presents a UVM 1.2 environment and methodology that closes that gap. The environment wraps a bus-master SIMT DUT with role-inverted agents, integrates Vortex's functional simulator (SimX) as a per-configuration golden model over DPI-C, and renders verdicts via two injection-qualified checkers: a bidirectional end-state scoreboard and a per-instruction, per-lane lockstep comparator under five SIMT alignment rules. A two-pass load-value feed makes racy fenceless multi-core programs instruction-granularity verifiable (residual zero over 5,432 retirements), with the interrupt-timing boundary stated. A three-layer coverage model adds, to our knowledge, the first published SIMT functional-coverage layer for RTL GPU verification (divergence depth, bank conflicts, coalescing classes), closing its own layers at 98.1% covergroup-bin / 94.7% total (ISA layer separately: 83.1% bins / 89.3% weighted) under machine-generated, RTL-cited exclusions and a blocking waiver-integrity gate; an unstimulated D-extension elaboration affects no functional bin, only lowering totals. The checking depth surfaced real defects on both sides of the comparison: a JALR LSB ISA deviation, a non-scaling watchdog constant (since fixed upstream), a reset-relay X window found by restoring a silenced assertion, a missing AXI error path proven by fault injection, and a reference-model fetch bug found by the lockstep itself. FuzzGPU (USENIX Security 2026), a concurrent RTL GPU fuzzer on the same DUT, is complementary: fuzzing finds bugs, this methodology quantifies sign-off; both independently found the JALR deviation. All findings ship in an evidence-cited register; every number carries provenance and the method's boundaries are stated rather than waived.

阅读 arXiv 原文
形式化与程序验证 · 3/30 · 2026-09-10衔接数学推理与Lean验证免训练流程将自然语言问题写成Lean陈述并构造可检证明Magenta: Closing the Loop Between Mathematical Reasoning and Lean Verification

Most of mathematical knowledge has been communicated through so-called informal use of mathematics and natural language. With large language models (LLMs) being highly adept in using natural language, they achieve strong performance, yet not perfect, in informal mathematical reasoning. Restraining LLMs to informal reasoning misses out on the opportunity to use the discrete verification abilities that machines offer through machine-checkable proofs. In this paper, we bridge the gap between informal and formal reasoning by integrating Lean signals into the informal reasoning process. We introduce Magenta, a training-free agentic pipeline that, given only a natural-language problem, produces an answer, expresses it as a Lean 4 statement, and constructs a machine-checked proof. A statement judge verifies whether the formalisation preserves the original problem, while an error-attribution judge routes failed attempts either to mathematical re-derivation or local Lean repair. Magenta achieves 100% accuracy across all evaluated olympiad benchmarks, including AIME 2025, AIME 2026, and HMMT February 2026. When paired with the open-weight K2-Horizon-7B reasoner, it solves all six IMO 2026 problems. Our analysis shows that statement adjudication is essential for preventing false certificates and that feedback-guided correction outperforms independent resampling on difficult problems.

阅读 arXiv 原文

软件工程与仓库智能(10 篇)

软件工程与仓库智能 · 6/30 · 2026-09-14自动修复智能体的对抗测试摘要称基于150个修复任务构造750条对抗问题,涉五类安全攻击Adversarial Testing of Automated Program Repair Agents for Security Vulnerabilities

Software agents with Large Language Models (LLMs) are designed for Automated Program Repair (APR) tasks, raising the possibility that, in the near future, APR agents will fix bugs automatically without much human intervention. Can we trust an APR agent to produce both functionally correct and secure code in such situations? What if attackers target production APR agents with adversarial issues that seem benign but may influence the agents to produce correct but insecure code? In this paper, we took a first step towards answering these questions by conducting an empirical study. First, we created SWEADV, a benchmark of 750 adversarial issue descriptions constructed from 150 repair tasks in SWE-bench Verified. For each repair task, we created five adversarial issue descriptions, one for each attack type: command execution, deserialization, path traversal, denial of service, and weak hashing. Second, we evaluated mini_swe APR agents from three LLM backends on SWEADV: GPT-5-Mini, MiniMax-M2.5, and DeepSeek-R. We found that on average, adversarial issue descriptions can induce malicious behaviors with successful repair in 51.7% of cases. Third, we investigated whether typical detection mechanisms are sufficient to prevent such malicious patches from being accepted. Pre-repair detection with LLM-as-judge on the adversarial issue descriptions resulted in an average detection accuracy of only 62.3%. Post-repair detection on adversarial APR patches using static analysis tools and LLM-as-judge achieved average detection accuracies of only 39.4% and 55.4%, respectively. We conclude that autonomous APR agents cannot be trusted yet in production deployment, given their susceptibility to adversarial attacks.

阅读 arXiv 原文
软件工程与仓库智能 · 7/30 · 2026-09-14从测试视角评测网页应用生成以代码覆盖率引导交互探索,抽象状态转移图后多维评估IWC-Bench: Evaluating Web Application Generation from a Software Testing Perspective

Human evaluation provides a direct measure of the quality of LLM-generated web applications. However, fitting human judgments through automated evaluation remains challenging. Static benchmarks can credit functionality that exists in source code but is unreachable at runtime. Interactive benchmarks exercise the application, yet incomplete exploration can cause them to miss implemented functionality and confound application defects with agent execution failures. To address these limitations, we propose IWC-Bench, an interactive benchmark for evaluating web application generation from a software testing perspective. IWC-Bench instruments each generated application and uses code coverage to guide an agent in exploring its functionality through user-simulated interactions. It then abstracts the interaction trace into a state-transition graph and evaluates the application along three dimensions: visual aesthetics, usability, and requirement alignment. By separating exploration from scoring, IWC-Bench collects runtime evidence without constraining exploration to predefined acceptance criteria. IWC-Bench comprises 369 real-world user requirements and 5,088 acceptance criteria. Evaluation of 16 frontier LLMs reveals distinct strengths across the three dimensions, with no model leading on every dimension. On 197 validated sessions sampled from an internal arena, IWC-Bench achieves 85.3\% agreement with human preferences, with agreement generally increasing as the score difference between paired applications grows. Further experiments show that coverage guidance improves exploration coverage and the model rankings remain stable when the judge model is replaced.

阅读 arXiv 原文
软件工程与仓库智能 · 4/30 · 2026-09-14需求到修复的可观测轨迹外化摘要称回顾300个SWE-bench Lite案例,外化修复理由与审计记录Externalizing Requirement-to-Repair Artifacts as Observable Traces for LLM-Based Program Repair

Repository-level repair requires not only correct patches but also inspectable records that explain how issue requirements are translated into code changes and post-edit evidence. We contribute THEMIS, a stage-aware repair workflow that externalizes this requirement-to-repair process through semantic interpretation, a runtime requirement-code graph, graph-derived Developer guidance, retained repair rationale and patches, and post-edit audit records. A retrospective audit of 300 SWE-bench Lite cases demonstrates that these artifacts provide broad support for cross-stage inspection: a complete Developer rationale is available for 288 cases, and 214 cases (71.3%) retain a complete audited field set connecting the selected stages. The retained records further enable systematic measurement of cross-stage correspondence: target symbols recur in 62.6% of Developer rationales and in 62.8% of patches, rising to 75.8% when related symbols are included. In a paired 100-case comparison, the relational workflow resolves 19 cases versus 9 for the direct same-input condition; because the two conditions also differ in Analyzer output, graph-derived distillation, and Judge records, we report this as preliminary, workflow-level evidence rather than a causal effect of the graph component. Together, these results show that THEMIS makes otherwise implicit requirement-to-repair transitions inspectable, enabling systematic examination of how repair decisions persist, align, and evolve across stages.

阅读 arXiv 原文
软件工程与仓库智能 · 10/30 · 2026-09-12代码生成的内省不确定性估计探究隐藏状态能否指示功能性正确,做响应级与行级风险评估Introspective Uncertainty Estimation for LLM-Based Code Generation

Large Language Models (LLMs) are increasingly used for code generation but can produce fluent yet functionally incorrect outputs, which limits trust in their usage for practical software engineering workflows. This thesis investigates whether Introspective Uncertainty Estimation (IUE), based on internal hidden-state representations of LLMs, can reliably indicate correctness at the response and line levels for code generation tasks. The objective is to determine the extent to which hidden states encode information about functional code correctness and how this can be leveraged for practical risk assessment and fault localization. Methodologically, this thesis combines response-level evaluation on LiveCodeBench (LCB) and BigCodeBench (BCB) with an augmentation pipeline that derives token- and line-level labels from incorrect programs. In this setup, it compares static and dynamic response-level features, evaluates generalization across tasks, programming domains, and token positions, and studies line-level fault localization. The results show that hidden states contain a strong response-level correctness signal. Static single-token probes perform best, while more elaborate dynamic strategies yield no consistent gains. While generalization across tasks, domains, and token positions is feasible, setting-dependent degradation largely remains for real-world software projects. At a fine granularity, line-level prediction is substantially harder than response-level estimation. However, in a conditional localization setup with known-incorrect programs, Top-K point-of-failure ranking remains effective. Overall, the findings suggest that hidden states are a robust and informative resource for estimating functional code correctness, supporting a two-stage workflow that combines response-level risk screening with targeted line-level prioritization.

阅读 arXiv 原文
软件工程与仓库智能 · 6/30 · 2026-09-11自认技术债是否被测试对八个Java项目实证,分析债务代码覆盖率、共变与缺陷可观测性Is Self-Admitted Technical Debt Tested? An Empirical Study of Coverage, Co-change, and Impact

When developers write a TODO or FIXME comment, they are explicitly admitting that the code is suboptimal: a built-in warning that this logic deserves extra scrutiny. Yet it is an open question whether Self-Admitted Technical Debt (SATD) actually receives that scrutiny in the form of software testing. We aim to characterize the relationship between SATD and testing across three dimensions: the extent to which SATD-affected code is covered by existing tests, whether developers synchronize test additions with debt resolution, and whether such testing affects the long-term observability of resulting defects. For that, we conducted an empirical study on eight open-source Java projects, analyzing test coverage of 784 SATD instances identified in the latest releases and performing a longitudinal examination of 5,175 SATD removal events. Our results show that while 60.7% of SATD-affected code is covered by existing test suites, developers rarely synchronize test modifications with debt resolution; manual inspection confirms that only 3.4% of SATD removal commits include new tests specifically targeting the resolved debt (vs. 12.5% that co-add tests in the same commit). Longitudinal analysis further suggests that SATD resolutions exhibit nearly identical localized bug induction rates within short-to-medium-term windows regardless of test modifications. However, over a longer, unrestricted observation window, a slight divergence emerges where the test-added group reaches a higher cumulative defect alignment probability (6.32% vs. 4.37%), a counterintuitive trend potentially driven by the selective testing of inherently complex components. Developers treat SATD repayment as an ordinary code change rather than as a high-risk maintenance activity: most debt removals proceed without targeted verification, despite the developer's own prior flag that the code is suboptimal.

阅读 arXiv 原文
软件工程与仓库智能 · 6/30 · 2026-09-11公共数据交换的使用控制挑战以X-Road为例,跨形式验证、需求工程与数字治理做分析Usage Control in Public Data Exchange Infrastructures for Future Digital Governance

Public data exchange infrastructures, such as X-Road, are essential to data-intensive digital public services and the EU digital economy. Their next generation requires advanced data usage control (DUC) mechanisms that address emerging requirements, including privacy by design, to mitigate privacy threats inherent in large-scale automated data exchange. Although research within the individual disciplines already addresses many of the emerging requirements, significant practical challenges arise at the intersection of the disciplines. This paper analytically examines the practical challenges and requirements of data usage control in public data exchange infrastructures. We examine the Estonian X-Road system and emerging use cases through an interdisciplinary lens that integrates formal verification and monitoring of usage control, requirements engineering, and digital governance research. We formulate key requirements and propose a conceptual vision for developing usage control in public data-exchange infrastructures to enable future digital-governance systems to comply with privacy requirements by design.

阅读 arXiv 原文
软件工程与仓库智能 · -3/30 · 2026-09-11视频教程的语义检索匹配用SBERT生成语义向量并重排,摘要称改善编程教程搜索相关性Intelligent Semantic Matching (ISM) for Video Tutorial Search using Transformer Models

The rise in the number and diversity of available software development video tutorials has enhanced digital learning for developers but also introduced challenges in locating relevant content efficiently. Existing video search methods, including keyword-based approaches and tools like CodeTube and TechTube, rely primarily on retrieval algorithms such as BM25, which fail to capture the semantic nuances and user intentions behind search queries. To address these limitations, we introduce ISM, an approach that uses SBERT to generate semantically rich vectors from video tutorial transcripts to improve the search for programming video tutorials. By segmenting transcripts and implementing a re-ranking process, ISM effectively preserves context and enhances the relevance of search results. Additionally, ISM generates informative video summaries using GPT-4, allowing developers to quickly assess the relevance of video content. To evaluate our approach, we first performed a quantitative study comparing ISM with the baseline TechTube. The results revealed that ISM performs better in both video retrieval and fragment identification, achieving a Hit@5 score of 0.95 and an average F1 score of 0.70 compared to the baseline's 0.58 and 0.52, respectively. We also performed a user study, which revealed that users strongly preferred the semantic matching capabilities and AI-generated summaries of our approach. This work advances the state-of-the-art in programming video tutorial search and summarization by offering more nuanced and user-aligned retrieval and summarization mechanisms.

阅读 arXiv 原文
软件工程与仓库智能 · 3/30 · 2026-09-11检索增强的自动化干系人模拟用于需求获取教学角色扮演,摘要称缓解幻觉与回答不一致A Retrieval-Augmented Automated Stakeholder for Requirements Elicitation Education: A Comparative Study

Developing the skills required for requirements engineering students to conduct effective requirements elicitation interviews is critical yet challenging, as it requires the development of soft skills in addition to technical knowledge. Role-playing is widely adopted in requirements engineering education to support the development of these skills but is often constrained by time and resource limitations. Although recent advances in large language models (LLMs) enable automated and interactive stakeholder simulations for role-playing, their application in requirements engineering education remains limited by hallucinations and inconsistent responses. To address these limitations, this study investigates the use of retrieval-augmented generation (RAG), implemented using the LangChain framework, to support requirements elicitation activities in a requirements engineering course. We conducted controlled experiments with 69 students, comparing cohorts who interacted with non-technical faculty role-players and those who engaged with RAG-based simulated stakeholders. The results indicate that while students perceived invited stakeholders as more realistic and engaging, the RAG-based automated stakeholder produced higher-quality elicitation questions and more complete feature identification.

阅读 arXiv 原文
软件工程与仓库智能 · 4/30 · 2026-09-10开源守护型社区与贡献门槛提出守护社区概念:核心保留实现权,社区以非编码方式参与Open Source Stewardship Communities: "We need you, but not your pull request"

Human-centric AI for software engineering means keeping humans responsible for work performed with AI. In Open Source Software (OSS), AI lowers the cost of implementing changes, but reviewing someone else's contribution remains comparatively expensive, so some projects now restrict who may contribute implementations while still welcoming other participation---not because the code is AI-generated, but because it no longer justifies the review cost. We call the resulting form a stewardship community: a small core retains implementation authority while a broader community continues to shape the software without writing code, and access to coding increasingly depends on approval rather than self-initiated contribution. This raises a broader question: what happens to the human community when coding agents let maintainers replace implementation work once supplied by external contributors? For human-centric software engineering, keeping humans in control of AI agents is not enough: AI can replace implementation labor while weakening how OSS communities renew themselves.

阅读 arXiv 原文
软件工程与仓库智能 · 9/30 · 2026-09-06用户请求驱动的软件自演化结合自动需求工程与测试驱动开发,把高层请求转为可执行功能ReqEvolve: User-Oriented Software Self-Evolution through Automatic Requirement Interpretation

The paradigm of software self-evolution enables systems to autonomously extend and reconfigure their own capabilities during execution in response to technical specifications. Yet requests for new functionality often originate from end users and are rarely expressed in technical terms. As a result, developers must translate user needs into technical specifications before the system can evolve, delaying early validation of the requested functionality by preventing users from immediately observing the resulting behaviour. To address this gap, we present ReqEvolve, a runtime code generation system that enables user-driven self-evolution by accepting high-level user requests. The system integrates automatic requirements engineering (RE) and test-driven development (TDD) to transform these requests into executable functionality through clarification, specification decomposition, test generation, and runtime integration. We evaluate ReqEvolve on 72 software evolution cases across 18 projects against two baselines: SpecFix, an RE-focused code generation approach, and an ablation variant of our system. ReqEvolve achieves 89.2% Pass@1, outperforming SpecFix by 18.8% (p < 0.01, r = 0.79, large effect) and the ablation baseline by 32.6% (p < 0.001, r = 0.88, large effect). These results provide initial evidence that user-driven self-evolution is a viable paradigm for autonomously extending software capabilities from user requests, thereby accelerating requirements validation prior to developer verification.

阅读 arXiv 原文

代码质量与优化(4 篇)

代码质量与优化 · 7/30 · 2026-09-13代码优化的效率幻觉与校准用分类惩罚框架测多模型优化;摘要称最优代码上过度修改普遍Efficiency Hallucination: Formalizing and Measuring Behavioral Calibration in LLM-Based Code Optimization

The integration of Large Language Models (LLMs) into automated code optimization introduces a critical reliability risk we term the Efficiency Hallucination: an LLM's tendency to issue non-functional mutations with unsubstantiated performance claims on already-optimized code. This is driven by the Evaluation Trap, wherein binary benchmarks incentivize unnecessary modifications over safely abstaining. We present a validation framework using classification penalty methods, evaluated across 180 optimization runs on nine models (GPT, Claude, Gemini) using EffiBench. Under standard prompts, models exhibit a 100% over-edit rate on optimal code. Our guardrail raises correct abstention from 0% to to 44.4%, preserving a 100% edit rate on sub-optimal code with zero false abstentions. Calibration is uneven: GPT-5.4 Mini approaches near-perfect abstention, and simple code is recognized more reliably than complex code. Our framework offers a training-free mechanism to mitigate LLM overconfidence before deployment in production.

阅读 arXiv 原文
代码质量与优化 · 3/30 · 2026-09-11面向智能体的行为商LoRA以局部行为商组织轨迹更新,缓解冗余更新与秩预算受限问题Behavior Quotient Learning for Low-Rank Adaptation of LLM Agents

LLM-based agents rely on heterogeneous interaction capabilities to accomplish complex tasks. Existing approaches often distribute these capabilities across multiple LoRA adapters, which increases adapter storage requirements and introduces routing overhead during inference. A single LoRA avoids this overhead, but learning from diverse agent trajectories under a fixed rank budget presents two challenges. First, trajectories with different interaction traces and parameter gradients can induce equivalent changes in decision distributions, causing repeated updates to overemphasize redundant behavioral changes. Second, an aggregated update may exceed the rank budget of the adapter, and approximating it in weight space can distort the decision changes that it is intended to produce. We propose BQ-LoRA, a low-rank adaptation framework that organizes trajectory updates through a local behavior quotient manifold. It contains two modules, i.e., behavior quotient balancing (BQB) and decision preserving compression (DPC). BQB constructs the quotient manifold from decision distributions and reweights trajectory update directions according to their local density in the quotient tangent space. DPC projects the balanced gradient onto the intrinsic fixed rank tangent space and refactorizes the resulting target by jointly controlling effective weight error and distortion of decision distributions. Experiments on AppWorld and BrowseComp-Plus compare BQ-LoRA with standard LoRA and recent low-rank adaptation methods, while separate ablations evaluate the complementary contributions of both components.

阅读 arXiv 原文
代码质量与优化 · 9/30 · 2026-09-10测试驱动LLM软件工程综述围绕测试改变何种决策组织综述,区分多种测试参与范式Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skills

Tests increasingly participate in the decisions made by large language models and software engineering agents. They specify intended behavior, guide program construction and repair, select candidates, constrain transformations, and provide execution evidence for software analysis. These uses draw on test-driven development, yet differ substantially in test order, oracle availability, editable artifacts, and the role of execution. We present a structured scoping survey organized around the question of what decision a test changes. The review integrates 87 research and supporting records, with method- or protocol-level extraction for 83 records, alongside a separate collection of five practice resources. We distinguish the Red--Green--Refactor cycle from test-conditioned generation, execution-guided refinement, test-mediated analysis, and evaluation-only testing. We then compare code generation, repair, translation, refactoring, clone detection, code search, localization, training-data construction, and formal-specification validation. A dedicated analysis examines how agent workflows and reusable skills encode testing procedures and how their effects are evaluated. Across these tasks, the evidence supports treating test availability, test validity, feedback use, and evaluation independence as separate properties. Test passing alone does not establish behavioral equivalence, effective feedback, or process adherence; aggregate improvements can also conceal different outcomes across models, tasks, and denominators. We synthesize these distinctions into a mechanism taxonomy, a cross-task comparison, and a protocol-sensitive evidence analysis, and identify research directions in oracle validation, causal evaluation, long-horizon maintenance, and reusable test-driven agent capabilities

阅读 arXiv 原文
代码质量与优化 · 0/30 · 2026-09-08代码质量与模型性能的关系摘要称分析26万余Kaggle笔记本,考察质量、流行度与性能关联On the Relation between Code Quality and Machine Learning Performance: A Large-scale Empirical Study

Context: Computational notebooks are the standard environment for machine learning (ML) development. Within the ML community, model performance is often the primary considered metric, and code quality is treated as a secondary concern. This prioritization relies on a largely untested assumption that code quality and ML performance are unrelated. Practitioners also reuse existing code that may come from notebooks selected through social signals (popularity, author expertise) whose reliability as quality proxies has never been assessed. Objective: We empirically investigated the relationship between code quality and ML performance in notebooks, and evaluated whether popularity and author expertise give indication on code quality or performance. Method: We conducted a large-scale empirical study of 265,363 Python notebooks submitted to Kaggle competitions. We assessed code quality with two static analysis tools: Pylint, capturing general Python code quality, and SonarQube, configured with a profile of 34 rules targeting data-science and ML-specific practices. Results: The relationship between code quality and performance depends on the notion of quality considered. General Python code quality is decoupled from ML performance, showing negligible or non-significant correlations across all observations. In contrast, ML-specific violations exhibit a consistent, small negative association with performance that persists across all observations. The popularity of a notebook does not give information on the code quality or performance. Code expertise provides no information on quality or performance, but competition expertise correlates with better performance, fewer ML-specific violations, and slightly more Python errors and refactoring violations.

阅读 arXiv 原文

UI 与 GUI Agent(0 篇)

本轮该赛道没有候选论文。

个人知识与本体(3 篇)

个人知识与本体 · 4/30 · 2026-09-11可溯源个人知识图谱属性推断把发帖史建成源链接图谱,使推断可审计并定位泄露触点GraphProfiler: Source-Linked Sensitive Attribute Inference via Personal Knowledge Graphs

Sensitive attributes such as age, income, and occupation can be inferred from user-generated content by aggregating indirect cues across many ordinary posts. LLM-based profilers can perform this aggregation automatically and with high accuracy, which makes large-scale personal attribute inference a major privacy threat. Existing LLM-based profilers, however, offer limited insight into which specific posts, concepts, and relationships made an inference possible, which is key to targeted privacy mitigation, i.e., redacting or rewriting only the few posts that actually leak an attribute, rather than perturbing entire histories. We introduce GraphProfiler, an auditable LLM-based profiler that represents each user's post history as a source-linked personal knowledge graph where nodes and edges trace back to the originating post and resolves attribute predictions to cited graph records and source texts. GraphProfiler reaches 86.7% attack success rate on the eight-attribute SynthPAI benchmark, within two points of strong text-only baselines, and 84.6% on PANDORA, while citing supporting evidence for over 98% of predictions. Our controlled ablation experiments provide evidence that the cited posts contribute to attack success, as removing them reduces the attack success rate substantially more than removing an equal number of random posts.

阅读 arXiv 原文
个人知识与本体 · 0/30 · 2026-09-11过程锚定的语言认知诊断用语言构建概念模式与认知过程图,以语义记忆检索历史作答Beyond ID Embeddings: Process-Grounded Language Modeling for Cognitive Diagnosis

Cognitive Diagnosis Models (CDMs) play a pivotal role in personalized online learning. Traditional CDMs rely on discrete, ID-based embeddings to represent students, exercises, and concepts. This paradigm diverges from the nature of learner cognition, where knowledge is not stored and retrieved as isolated symbols. As a result, CDMs suffer from semantic limitations when new exercises or concepts appear. In this paper, we propose a Process-aware Language Cognitive Diagnosis (PLCD) framework that uses language-derived structures as cognitive priors and response records to calibrate student posterior states. PLCD leverages large language models (LLMs) to construct concept schemas and cognitive process graphs, and uses target-conditioned semantic memory to retrieve historical responses that are relevant to each target exercise. A process-grounded Language-to-Cognition Mapper with DA-MoE experts and process-level contrastive learning then maps the textual evidence into a unified cognitive space. Experimental results show that PLCD not only outperforms traditional baselines in predicting student performance but also exhibits strong cognitive transfer capabilities. These results connect the computational power of LLMs with the psychometric goal of measuring latent knowledge states, suggesting that structured language priors calibrated by response records can improve cold-start robustness and cognitive grounding.

阅读 arXiv 原文
个人知识与本体 · 4/30 · 2026-09-11多用户多智能体隐私感知记忆动态区分私有与共享记忆,以索引级访问控制限制私有检索AIM: A Privacy-Aware Interoperable Memory Framework for Multi-Agent Multi-User LLM Systems

Traditional large language models (LLMs) are scoped to individual user sessions, limiting their knowledge to a single conversation and preventing them from learning user preferences that evolve over time. Existing agentic memory systems address this limitation but generally operate at the individual-user level, restricting the public knowledge that could be shared across users to improve downstream responses. We introduce AIM (Agentic Interoperable Memory), a unified, privacy-aware memory framework that enables multi-agent, multi-user LLM systems to persistently manage private and shared memory. AIM dynamically classifies information as private, scoped to one user and inaccessible to others, or public, accessible to all users. It enforces index-level access controls so that private memories are retrievable only by their owner, protecting sensitive data while allowing beneficial shared knowledge to improve coordination and consistency. We also introduce MUMBench (Multi-User Memory Benchmark), a dataset of multi-user interactions containing private and shareable information across four domains. To our knowledge, MUMBench is the first public dataset designed to evaluate multiple memory operations, including retrieval, creation, update, and deletion, in a multi-user environment. Across three independent runs on MUMBench, AIM achieves 96.0% visibility classification accuracy, 58.8% strict operation accuracy, and 70.5% state-aware operation accuracy.

阅读 arXiv 原文

人机协同与对齐(0 篇)

本轮该赛道没有候选论文。