fix: 修复 dedup 高重复率返回码被掩蔽的问题 (P1-1)

- run_dedup: 生产模式重复率仅作 WARNING 告警,不影响退出码(执行成功即 0)
- run_dedup: 新增 --strict 验收模式(重复率 > 5% 返回 1),保留 M3 验收门槛
- run_dedup: 新增统计快照 data/deduped/{date}/stats.json(原子写)
- run_dedup: 顺带修复空日场景 sources.json 写入 FileNotFoundError
- pipeline: 移除 dedup rc=1 特判,恢复'非 0 即失败'统一语义
- 新增 tests/test_run_dedup.py 7 个测试;全量 266 passed
This commit is contained in:
2026-08-22 19:50:14 +08:00
parent 85ba218f2f
commit 8fa27ad65b
4 changed files with 242 additions and 10 deletions
-5
View File
@@ -290,11 +290,6 @@ def run_step(name: str, date_str: str) -> StepResult:
elapsed = (datetime.now() - started).total_seconds()
ok = proc.returncode == 0
# dedup 返回 1 是"重复率过高"(无新文章的正常场景)
if name == "dedup" and proc.returncode == 1:
ok = True
logger.info("dedup 重复率超过阈值(无新数据场景,视为成功)")
tail_msg = f"rc={proc.returncode}" if not ok else ""
if ok: