Repository navigation
feat(context,benchmark): add default-off Stage-then-Swap EpochReset prefetch, Quiet-Turn, and KV/operator telemetry - #9213
loopx-agent wants to merge 1 commit into
Conversation
…refetch, Quiet-Turn, and KV/operator telemetry Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
loopx-agent
left a comment
There was a problem hiding this comment.
Reviewer: model_agent · gpt-6.2-sol · OpenAI (runtime_reported, xhigh)
Request changes conclusion (author-owned PR; GitHub blocks formal self-review)
评审固定提交 0ab05307f85aa87430e5f9d1e10a9d353d9f9a3b,merge base de61cfe6a35a22a236d88e5e8a930428929d10d2。与核对时 main 55bae0ac55c8858a09f05fdebd3a1b7b5dcc131f 三方整合无冲突。
[P2 · Blocking] 已提交 Turn 的回读会重复预取并产生不同的 receipt
File 和 SQLite 中,已提交 Turn 用相同参数回读时复用原结果,再换一个同源读取选择来恢复同一 Turn。CLI 仍返回 replayed=true 和 state_written=false,实际启动第二次预取,两个侧存储各产生第二条 receipt。回显的预取摘要与原 journal 不同,模型未再启动。在已提交 Turn 的 return_existing 路径只读取原 journal 的 receipt。变化的预取选项应拒绝或提示重入新 Turn,不能进入会运行读命令并落盘的 resolver。 候选位置
[P2 · Blocking] 尚有续页的 context-read 被当成完整工作集
真实 context-read 返回正常第一页,已读 1600/4485 字节,eof=false。两种后端 都产生 EpochReset,标记 truncated=false、preserves_raw_state=false,还记入 1 次 round trip 节省。复用 context-read 的页与来源校验。在同一输入预算内核对摘要和连续字节直至 EOF,或回退 KeepRaw 并按实际保留与调用记录计算收益。 候选位置
动机 / Motivation
启用 EpochReset 的受管 Turn 和 Codex compact hook 需要保留完整来源、失败事实与未完义务,并按原回执恢复。基线使用现有 context-read 分页及 KeepRaw 恢复。普通 Turn 回读已提交 journal 时复用原结果。候选新增可选预取、Quiet-Turn 和 benchmark telemetry,用退出、输出上限与净字节收益决定切换。
实际 turn run-once 与真实 context-read reader,分别走隔离 File 和 SQLite。用无敏感数据的合成工作记录测试已提交回读和有续页的成功读取。评审覆盖真实本地 CLI 和合成 Host、两种隔离存储。正式 Codex App、Windows 管道与生产策略切换未验收。命令启动、token 口径和相同结果的完整成本仍需验证。 现有交付规则
改动思路 / Approach
仅显式开启 EpochReset 或 Quiet-Turn 的入口使用新流程。默认关闭时沿普通 CLI 和 compact hook 的旧返回。现有 Turn journal 应继续作为完成事实与恢复选择的权威,来源由 context-read 校验。TS 负责资格与语义判断,Python 调用和持久化读取。候选已将 opt-in CLI 和 hook 接到共享 runtime。此次差异把调用、双份持久化、工作集选择与 benchmark 度量放在一起,修复两个确定的边界后可以分别验收。
具体改动 / Changes
完整 diff 共 20 个文件,新增 3549 行、删除 47 行。作者现有用例覆盖同配置重读和进程输出截断。独立反例补上配置改变后同一已提交 Turn 的恢复,以及读者自身的续页合同。
-
同一 Turn 原回执。回读已提交结果时保留原有预取 receipt,且不再执行。
-
完整披露和 KeepRaw。仅在完整来源页与既有未完义务核验后允许工作集切换。
-
功能关闭的行为一致。未开启选项时普通 CLI 和 compact hook 返回现有结果。
-
execute_turn_run_once。转发本次可选 prefetch 配置到共享执行器。 精确代码 -
run_loopx_turn_once。决定已提交 Turn 是否走 return_existing。 精确代码 -
prepare_turn_epoch_context。读取原 inline receipt,但未向 resolver 传递回读禁止执行的边界。 精确代码 -
resolve_or_execute_epoch_prefetch。把进程结果交给 TS 并写入两种侧存储。 精确代码 -
evaluateEpochPrefetchSwap。以进程状态和字节统计选择 EpochReset 或 KeepRaw。 精确代码 -
codex_context_hook。compact 恢复复用预取执行和原读者。 精确代码 -
summarizeContextPolicyTelemetry。按配置生成可选 operator、prefetch 与 KV 统计。 精确代码
对主干的风险 / Risks
精确提交已有功能套件 Python 14 项、TS 24 项通过。主干与候选的普通 CLI 和 compact hook 默认关闭对照各 2 项通过。
同配置回读和常规 KeepRaw 的正例通过。与当前 main 的整合树同样复现两类问题,共四例。git diff --check 另提示 epoch_prefetch.py 末尾新增空行。
我的整体评价 / Verdict
REQUEST_CHANGES。 请修复已提交回读和不完整来源的两个路径,再为 File 和 SQLite 独立复测。发布仍需用匹配的任务结果衡量真实成本,而非用单次 fixture 的字节比作为采用证据。
English verdict: REQUEST_CHANGES. A settled Turn can run a different prefetch and return a second receipt. An incomplete reader page can also commit an EpochReset and claim savings. Preserve the original receipt on replay and require verified EOF or KeepRaw.
loopx-agent
left a comment
There was a problem hiding this comment.
Request changes conclusion (author-owned PR; GitHub blocks formal self-review)
Reviewer: model_agent · Gemini · Google DeepMind · self_reported
English verdict: REQUEST_CHANGES
动机 / Motivation
针对 9213@0ab05307f85aa87430e5f9d1e10a9d353d9f9a3b 的独立 fresh audit(锚定规范 docs/architecture/rfcs/history-working-set-context-policy-v0.md 固定提交 94613abe6bdbd6931212d2bc551875f1b1c315a3,覆盖验收准则 A2、A7、A8、A10;注:PR 引用的 #9148 尚处于 OPEN 未合入状态):启用可选 Stage-then-Swap 预取或 Quiet-Turn 跳过的 Turn CLI/执行器(turn_run_once 与 executor)、Codex compact 恢复钩子(codex_context_hook)以及基准测试统计模块(benchmark_toolkit 与 session_observation)的使用者会依赖本次改动。
修改前基准关闭分支在无新参数时不产生任何侧存储或遥测字段;修改后显式启用参数可在单次捕获上完成按需预取并记录冷热 KV 与算子统计,但在真实局部边界反例下已实测存在 D8 允许 git branch 新建分支与 --output 写盘、Quiet-Turn 在 Turn 失败后误跳过下一次执行且不记入 telemetry、已提交 Turn 换参回读二次执行预取、以及未读到 EOF 的分页被当成完整 EpochReset 四个缺陷。
默认关闭隔离与正常同配置回读保持成立,TS 语义判定与基准冷热 KV 分离字段按约定缺省省略;但 D8 安全边界、失败后 Quiet-Turn、换参回读幂等及分页 EOF 完整度存在 4 处可复现 P1 漏洞,且新增的 epoch_prefetch.py 末尾多出空行导致 git diff --check 未过。本 PR 仍处于 Proposal 阶段下的 default-off 受控切片,不修改默认历史上下文投影策略、不接入自动 LLM 摘要模型调用、不迁移动态前端设置面板或触发付费基准全量重跑。
改动思路 / Approach
本 PR 将算子与安全判定放在 TypeScript 语义层(stage_then_swap.ts 与 turn_envelope.ts),由 Python epoch_prefetch.py 承担 selectors 超时/进程组强杀、UTF-8 安全截断以及 File + SQLite WAL 持久化,并通过 --epoch-prefetch-json 与 --quiet-turn 以严格 default-off 方式接入 executor.py 与 codex_context.py,同时在 trajectory_codex_usage.ts、token_capacity.py 与 session_observation.py 增加可选冷/热 prefix KV 与算子计数。
由 stage_then_swap.ts 与 turn_envelope.ts 统一掌握 D8 安全校验与算子决策语义、epoch_prefetch.py 仅承担受限子进程与 File+SQLite 持久化的架构分层符合仓库边界,但必须先修复本轮实测发现的 4 项 P1 正确性与安全缺陷再合入。以默认关闭为保护、包含 TS 语义校验、Python 受限预取与冷热 KV 遥测的边界切分合理,但存在 4 个已实证的 P1 缺陷与 1 个尾部空行问题,当前不应合入,需在同分支完成最小修复与回归验证。
具体改动 / Changes
本次共涉及 20 个文件(+3549/-47),我们在独立只读 worktree 对提交 0ab05307f85aa87430e5f9d1e10a9d353d9f9a3b 逐项核查并独立复现以下阻塞问题(同时独立复核确认此前已有同头评审指出的两条边界问题,并新发现 D8 绕过与 Quiet-Turn 致命缺陷):
- Blocking Finding 1(P1)— D8 只读白名单允许
git建分支与任意文件写盘(stage_then_swap.ts:1065-1088):- 实测直接调用
validatePrefetchCommandSafety对["git","branch","evil-branch-created"]、["git","branch","--track","evil","main"]、["git","log","--output=/tmp/evil-log.txt","-n","1"]、["git","diff","--output=/tmp/evil-diff.txt"]、["git","show","--output=/tmp/evil-show.txt","HEAD"]与["git","diff","--ext-diff"]全部返回{"allowed":true,"reason":null}。原因在于GIT_MUTATING_BRANCH_FLAGS仅黑名单禁用删改标志却放行位置参数建分支,且未禁止git读子命令的--output=与--ext-diff。 - 最小修复:将
git branch改为只读选项白名单(如--list/--show-current/-a/-r/-v且禁止位置参数创建分支),并拦截git的--output、--output=*与--ext-diff。
- 实测直接调用
- Blocking Finding 2(P1)— Quiet-Turn 在 Turn 执行前提前落盘 digest,导致失败 Turn 1 后 Turn 2 静默跳过且遥测归零(
executor.py:1819-1857、epoch_prefetch.py:2221-2286、session_observation.py:85-96):- 本地实跑反例复现三处断点:(a)Turn 1 在
host_runner抛RuntimeError失败(status='failed'),但prepare_turn_execution_epoch_context已预先写入quiet_turn状态,随后同 digest 的 Turn 2 直接返回status='skipped', reason='unchanged_state',静默跳过未完成工作;(b)if skip_host:在admit_start()预占配额与capture.publish_for_host()之后直接提前返回,不写turns/*.json,导致collect_context_policy_telemetry统计quiet_turn_skips=0;(c)resolve_quiet_turn_metadata在 File 与 SQLite 不一致时让 SQLite 静默覆盖 File,未 fail-closed。 - 最小修复:在
admit_start/publish_for_host之前判断 skip;仅当 Turn 结算成功后才写入新state_digest;File/SQLite 不一致时回退state_changed=True;为跳过记录可被collect_context_policy_telemetry读取的持久化计数/收据。
- 本地实跑反例复现三处断点:(a)Turn 1 在
- Blocking Finding 3(P1)— 已提交 Turn 换参回读时二次执行预取并覆盖 receipt(
executor.py:1773-1789、epoch_prefetch.py:2505):- 独立复现确认:Turn
t1提交后换--epoch-prefetch-json调用--resume-turn-key,return_existing虽传allow_execute=False,但if allow_execute or isinstance(existing_inline, Mapping):仍进入resolve_or_execute_epoch_prefetch并新跑一次run_bounded_prefetch_subprocess(replay calls count: 2),产生不同receipt_sha256(f73e363164be != 66ad8b03eeed)且返回replayed=True与ep_executed=True矛盾状态。 - 最小修复:
return_existing路径仅只读返回原journal记录,拒绝或阻止对已提交 Turn 执行新的prefetch_command。
- 独立复现确认:Turn
- Blocking Finding 4(P1)—
context-read分页未达 EOF(eof=false)被误判为完整EpochReset(stage_then_swap.ts:1254-1346):- 实测对
16,884字节捕获运行loopx quota context-read --select /big --page-bytes 1600返回exit_code=0, eof=false,送入resolve_or_execute_epoch_prefetch竟产出operator='EpochReset',truncated=False,preserves_raw_state=False,round_trips_saved=1,bytes_saved=13801,违反 RFC A2。 - 最小修复:识别结构化读取结果中的
eof === false/truncated === true并按未完整截断回退KeepRaw或强制翻页至 EOF。
- 实测对
- Non-blocking(P2):
epoch_prefetch.py:1217末尾存在多余空行,导致 committed exact head 上git diff --check origin/main...HEAD失败;另建议后续统一session_observation.py与trajectory_codex_usage.ts的遥测字段名,并在tokenCountDeltas分支补充 compaction 冷重基识别。
对主干的风险 / Risks
在独立只读 worktree 对 committed exact head 0ab05307f85aa87430e5f9d1e10a9d353d9f9a3b 运行完整本地校验(wait_for_ci: false,未等待远端 CI):
npm run typecheck:control-plane通过(0错误);- 定向 TS 测试(
stage_then_swap_telemetry.test.ts+content_digest_single_owner.test.ts):24 passed, 0 failed; - 全量
npm run test:control-plane(tests/control_plane_ts/*.test.ts):6714 passed, 0 failed, 93 skipped(共6807项,skip 项为 Postgres 等环境条件跳过); - Python
uv run --extra test ruff check(13 个改动文件)通过,uv run --extra test python -m mypy通过(Success: no issues found in 17 source files); - 13 个受影响 Python pytest 套件(
test_stage_then_swap_quiet_turn.py、4 个 architecture 套件、3 个 benchmark_toolkit 套件、3 个benchmark/tests/、test_codex_context_lifecycle.py与test_loopx_turn_driver.py):937 passed, 1 skipped(可选harbor跳过); loopx check --scan-path全数 20 个改动文件errors=0;loopx canary premerge --from-git-diff --git-diff-base origin/main:risk_profile_smokes8/8 passed、python_compile1/1 passed、10 个catalog_canaries中 9 项通过,第 3 项cli-output-budget-regression-smoke.py因系统python3无pytest失败后已在.venv/bin/python中独立复验通过(cli-output-budget-regression-smoke ok,test_cli_output_budget.py: 49 passed, 1 skipped);而direct_checks 1/4(git diff --check origin/main...HEAD)因epoch_prefetch.py:1217尾空行失败(direct_checks共3/4通过)。- 对主干剩余风险:由于整套能力保持严格 default-off(且
test_default_off_turn_and_hook_have_zero_behavior_drift通过),未启用该参数的主干普通 Turn/hook 链路未受破坏;但若不修复上述 4 项 P1 即合入,任何启用Stage-then-Swap或Quiet-Turn的运行与评测都会触发安全绕过、失败后吞掉工作、回读二次执行或残页误判风险。
我的整体评价 / Verdict
PR #9213 架构主线(TS 语义 + Python IO + 默认关闭隔离 + KV 冷热拆解)方向清晰且大部分基线测试稳健,但独立对抗测试在同一精确提交头实证了 4 处 P1 阻塞缺陷(D8 git branch/--output 逃逸、Quiet-Turn 失败后误跳且漏统计、换参回读二次执行、eof=false 残页提交 EpochReset)以及 1 处 git diff --check 尾空行失败,因此本次给出 REQUEST_CHANGES。
- Visual Review:Backend runtime, CLI options, and benchmark telemetry JSON schemas only; no visual presentation surface changed. Unverified Scope:Production App/SSH benchmark runs (
FFmpeg/BLC) measuring real LLM accuracy and cached KV token spend underEpochResetremain outside offline verification; all File/SQLite protocol boundaries, D8 safety checks, and failure counterfactuals were executed directly on exact head0ab05307f8. - 建议:按上述 4 项最小修复与 EOF 空行清理更新本 PR 后,再对 File/SQLite 重新运行对抗回归与
canary premerge。
Summary
Implements the first strictly default-off Level-1 (
BoundaryCLI) Stage-then-Swap (EpochResetprefetch_command) + Quiet-Turn runtime slice and benchmark context-policy telemetry, anchored todocs/architecture/rfcs/history-working-set-context-policy-v0.mdand #9148.What Changed
TypeScript Semantic & Safety Owners (
loopx/control_plane/):loopx/control_plane/quota/stage_then_swap.ts&turn_envelope.{ts,py}: policy validation, D8 read-only argv safety allowlist (rejects passivehistory_view_policyplugins, shell metacharacters,--execute, mutating commands, and quota debits), deterministic anti-whitewashing receipt header (reclaimed_turns,dropped_failures,exit_code,truncated,source_ref,failure_facts,open_obligations,trust_level="tool"), fail-closedKeepRawevaluation, andQuiet-Turnevaluation.loopx/control_plane/quota/native_context_lifecycle.ts: preservesepoch_prefetchandquiet_turnacross idempotentPreCompact -> PostCompact -> SessionStart(compact)transitions and clears them only on newcontext_epochrotation.loopx/control_plane/capabilities/trajectory_assessment_types.ts&trajectory_codex_usage.ts: adds optionalContextPolicyTelemetry(operator_counts,quiet_turn_skips,prefetch, andcold/warmprefix_kvtoken breakdown), omitted when absent for zero legacy schema drift.Bounded Execution, Dual File + SQLite Once-Only Persistence, & Fail-Closed Runtime (
loopx/control_plane/turn_driver/,loopx/cli_commands/):loopx/control_plane/turn_driver/epoch_prefetch.py: nonblocking selector-basedrun_bounded_prefetch_subprocess(explicittimeout_ms, process-group kill, UTF-8-safemax_output_bytescap), dual File + WAL SQLite once-only resolution (resolve_or_execute_epoch_prefetch/resolve_quiet_turn_metadata) with missing-store self-healing andstate_conflict -> KeepRaw, plus shared Turn/hook helpers.codex_context.py,executor.py,execution_readback.py,codex_cli.py,turn_registration.py,turn_run_once.py: opt-in--epoch-prefetch-jsonand--quiet-turnwiring (default-off),allow_execute=Falsereplay recovery,tool-trust prompt rendering, andskip_host_when_unchangedsupport.Benchmark Cost & Action Telemetry (
loopx/capabilities/benchmark_toolkit/,benchmark/runtime/):token_capacity.py,experiment_board.py,study_projection.py, andbenchmark/runtime/session_observation.py: record and aggregate{KeepRaw, FoldToReceipt, EpochReset}operator_counts,quiet_turn_skips,prefetch_latency_ms,prefetch_bytes,prefetch_bytes_saved, andcold/warmprefix_kvtoken counts (cached_prefix_kv_tokens,uncached_prefix_kv_tokens), omitted when absent.Measured Impact (Bounded
loopx quota context-readFixture Probe)Measured on the 25-turn exploratory capture fixture (
--capture-file ... --select /summary, 5,710 bytes baseline) usingresolve_or_execute_epoch_prefetch:operator:EpochReset(trust_level="tool")baseline_bytes:5710->hydrated_bytes:1264(retained_bytes:826,bytes_saved=4446/77.86%reduction)round_trips_saved:1,latency_ms:204replay_executed=false,replay_replayed=true, shared receipt digest across File + SQLite)KeepRawverified on non-zero exit (exit_code=2), OS FIFO timeout, policy rejection, strict truncation, File/SQLite conflict, and negative net byte savingsVerification
uv run --extra test ruff check ...(13 Python files): passed (All checks passed!)uv run --extra test python -m mypy(17 configured source files): passed (Success: no issues found in 17 source files)npm run typecheck:control-plane: passed (0 errors)stage_then_swap_telemetry.test.ts+content_digest_single_owner.test.ts): 24 passed, 0 failed (prior fullnpm run test:control-planebefore finalbenchmark_toolkit/refactor:6716 passed, 0 failed, 91 skipped)pytest:test_stage_then_swap_quiet_turn.py,tests/architecture/{test_control_plane_import_boundaries,test_content_digest_single_owner,test_wire_schema_version_owners}.py,tests/capabilities/test_benchmark_{experiment_board,token_capacity,study_projection}.py: 426 passedtests/architecture/test_semantic_vocabulary_drift.py(withnode_modules): 124 passedbenchmark/tests/{test_context_delivery,test_five_arm_causal_attribution,test_harbor_usage_completeness}.py: 4 passed, 1 skipped (harboroptional import)test_codex_context_lifecycle.py: 171 passed; full 16-file batch prior tonode_modulessemantic-drift re-run had 1118 passed, 15 skipped (with only the 17test_semantic_vocabulary_drift.pysetup cases failing withoutnode_modules, all 124 of which passed withnode_modules)loopx check --scan-path ...(all 20 files, committed0ab05307f8):errors=0,public boundary scan clean: 20 filesloopx change-quality verify --goal-id edgebench-research-20261004 --base-ref origin/main(committed0ab05307f8):ok: true,status: "disabled",changed_files: 20loopx canary premerge --from-git-diff --git-diff-base origin/main(run right before committing identical working tree):direct_checks: 6/6 passed (diff_check_committed,diff_check_staged,diff_check_unstaged,rfc_bilingual_diff,changed_python_py_compile,module_ceiling_colocation)catalog_canaries: 10/10 passed, failures=0risk_profile_smokes: 8/8 passed, failures=0manual_review_required(expected 1 manual hold:benchmark_sensitive)