feat: 大模型使用场景化配置与去重多源记录

- 新增 configs/llm_models.yaml: 4 个场景(event_extraction/daily_report/stock_report/embedding)
  可独立配置 provider/model/api_key_env/base_url_env/temperature 等,含用途与模型要求说明
- 新增 configs/loader.py: YAML 场景加载器(优先级: CLI 参数 > YAML > .env > 内置默认)
- llm/client.py: load_llm_config 支持 scene 参数,LLMConfig 增加 max_attempts
- embedding/factory+remote+local: provider/model/batch_limit 支持场景覆盖
- scheduler/reporter+stock_reporter: 日报/个股摘要接入场景配置
- dedup: Fingerprint.source_ids 多源记录 + 旧库自动迁移 + DedupResult 多源字段
- scripts/run_dedup: uniques JSON 的 sources 字段 + data/deduped/{day}/sources.json 汇总
- scripts/run_event_extraction: 接入 event_extraction 场景
- 补充测试: 场景优先级/零值、多源合并、旧库迁移、embedding 场景覆盖
This commit is contained in:
2026-08-12 07:57:10 +08:00
parent 0c032196d2
commit 3c65701449
21 changed files with 886 additions and 80 deletions
+21
View File
@@ -167,6 +167,27 @@ def test_resolve_provider_type_env_override(monkeypatch: pytest.MonkeyPatch) ->
assert resolve_provider_type() == EmbeddingProviderType.LOCAL_BGE
def test_resolve_provider_type_scene_override(monkeypatch: pytest.MonkeyPatch) -> None:
"""configs/llm_models.yaml 的 scenes.embedding.provider 优先于 .env。"""
monkeypatch.delenv("EMBEDDING_PROVIDER", raising=False)
monkeypatch.setattr(
"embedding.factory.load_scene_config",
lambda scene: {"provider": "local-bge"} if scene == "embedding" else {},
)
assert resolve_provider_type() == EmbeddingProviderType.LOCAL_BGE
def test_resolve_provider_type_explicit_arg_beats_scene(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""显式参数优先级最高,覆盖 YAML 场景。"""
monkeypatch.setattr(
"embedding.factory.load_scene_config",
lambda scene: {"provider": "local-bge"} if scene == "embedding" else {},
)
assert resolve_provider_type("dashscope") == EmbeddingProviderType.DASHSCOPE
def test_resolve_provider_type_unknown_raises() -> None:
with pytest.raises(EmbeddingError):
resolve_provider_type("anthropic-emb")