fix: 修复 dedup 高重复率返回码被掩蔽的问题 (P1-1)
- run_dedup: 生产模式重复率仅作 WARNING 告警,不影响退出码(执行成功即 0)
- run_dedup: 新增 --strict 验收模式(重复率 > 5% 返回 1),保留 M3 验收门槛
- run_dedup: 新增统计快照 data/deduped/{date}/stats.json(原子写)
- run_dedup: 顺带修复空日场景 sources.json 写入 FileNotFoundError
- pipeline: 移除 dedup rc=1 特判,恢复'非 0 即失败'统一语义
- 新增 tests/test_run_dedup.py 7 个测试;全量 266 passed
This commit is contained in:
@@ -290,11 +290,6 @@ def run_step(name: str, date_str: str) -> StepResult:
|
||||
elapsed = (datetime.now() - started).total_seconds()
|
||||
ok = proc.returncode == 0
|
||||
|
||||
# dedup 返回 1 是"重复率过高"(无新文章的正常场景)
|
||||
if name == "dedup" and proc.returncode == 1:
|
||||
ok = True
|
||||
logger.info("dedup 重复率超过阈值(无新数据场景,视为成功)")
|
||||
|
||||
tail_msg = f"rc={proc.returncode}" if not ok else ""
|
||||
|
||||
if ok:
|
||||
|
||||
Reference in New Issue
Block a user