feat: 大模型使用场景化配置与去重多源记录
- 新增 configs/llm_models.yaml: 4 个场景(event_extraction/daily_report/stock_report/embedding)
可独立配置 provider/model/api_key_env/base_url_env/temperature 等,含用途与模型要求说明
- 新增 configs/loader.py: YAML 场景加载器(优先级: CLI 参数 > YAML > .env > 内置默认)
- llm/client.py: load_llm_config 支持 scene 参数,LLMConfig 增加 max_attempts
- embedding/factory+remote+local: provider/model/batch_limit 支持场景覆盖
- scheduler/reporter+stock_reporter: 日报/个股摘要接入场景配置
- dedup: Fingerprint.source_ids 多源记录 + 旧库自动迁移 + DedupResult 多源字段
- scripts/run_dedup: uniques JSON 的 sources 字段 + data/deduped/{day}/sources.json 汇总
- scripts/run_event_extraction: 接入 event_extraction 场景
- 补充测试: 场景优先级/零值、多源合并、旧库迁移、embedding 场景覆盖
This commit is contained in:
+12
-4
@@ -187,11 +187,15 @@ def extract_event(
|
||||
article: Article,
|
||||
*,
|
||||
template: PromptTemplate | None = None,
|
||||
max_attempts: int = DEFAULT_MAX_ATTEMPTS,
|
||||
max_attempts: int | None = None,
|
||||
) -> ExtractedEvent:
|
||||
"""同步抽取单篇文章的事件(带重试)。"""
|
||||
"""同步抽取单篇文章的事件(带重试)。
|
||||
|
||||
max_attempts 为 None 时使用 config.max_attempts(来自 YAML/环境变量配置)。
|
||||
"""
|
||||
tpl = template or PromptTemplate()
|
||||
prompt = tpl.render(article)
|
||||
max_attempts = max_attempts or config.max_attempts
|
||||
|
||||
last_err: Exception | None = None
|
||||
for attempt in range(1, max_attempts + 1):
|
||||
@@ -241,12 +245,16 @@ async def extract_event_async(
|
||||
article: Article,
|
||||
*,
|
||||
template: PromptTemplate | None = None,
|
||||
max_attempts: int = DEFAULT_MAX_ATTEMPTS,
|
||||
max_attempts: int | None = None,
|
||||
semaphore: asyncio.Semaphore | None = None,
|
||||
) -> ExtractedEvent:
|
||||
"""异步抽取(批处理用),与同步版逻辑等价。"""
|
||||
"""异步抽取(批处理用),与同步版逻辑等价。
|
||||
|
||||
max_attempts 为 None 时使用 config.max_attempts。
|
||||
"""
|
||||
tpl = template or PromptTemplate()
|
||||
prompt = tpl.render(article)
|
||||
max_attempts = max_attempts or config.max_attempts
|
||||
|
||||
async def _run() -> ExtractedEvent:
|
||||
last_err: Exception | None = None
|
||||
|
||||
Reference in New Issue
Block a user