功能：每日动态股票池回测（--mode daily）+ 每日增量同步 + PIT 批量取数层
说明：本提交是工作区中此前的未提交工作（在 14ec0c6 之后产生），**非本次会话所写**，
按用户要求整理并推送。已做安全检查（无明文凭据、无大文件、.env/logs/output 仍被忽略），
并完成可执行范围内的测试验证（见「测试」一节）。
## 新增能力
1) `hdiv backtest --mode daily --start <日期>`
- src/hdiv/backtest/daily.py：两趟式（先逐日选股，再复用既有引擎模拟）
- 每个交易日按当日可见数据重建股票池（PIT），每个交易日判断买卖点
- `pool_exit_action`：hold（只减不加、不因掉出池子而清仓）/ sell（掉出即清仓）
- `profile_on_trade`：买卖决策发生时计算并留痕个股画像，**不区分是否在当日池内**
（卖出/减仓同样留痕，否则「为什么卖」缺证据）
- 与 walkforward 的分工：daily 是一条连续路径的推演，不是过拟合检验；
因此不使用训练段、不冻结分布，阈值口径一律 rolling
- 拒绝 `--universe-run`（daily 的定义就是逐日重筛，冻结池与之矛盾）
2) PIT 批量取数层 src/hdiv/universe/pit.py
- PitRepo 继承 Repo，**只重写取数**（按区块批量预载 + 逐日内存切片），
派生逻辑（最新一期财报合并、单位归一化、支付率口径等）一行不重写
—— 以保证与逐日单点查询**结果等价**
- 候选集预剪枝：用「不可能通过」的边界条件提前排除，文档论证为精确等价而非近似
- src/hdiv/universe/daily.py：每日动态筛选器（仍然调用既有 selector 与四个 Filter）
3) 每日增量同步 `hdiv sync daily`
- src/hdiv/data/sync/daily.py：只抓「库里还没有的那几天」，
按「当日股票数 ≥ 当年规模阈值」判定缺口，不重拉历史、不覆盖既有行；
支持 `--dry-run` 先看待抓清单
- deploy/daily-sync.sh、deploy/install-sync-schedule.sh、
deploy/com.hddiv.sync.plist.example（launchd 每天 17:00）
- 新表 hd_daily_universe（逐日入选成员留痕）+ sql/hd_daily_universe.sql + schema.py
（该表已存在于库中，`ddl plan` 返回 0 个待执行动作）
4) Web 与文档
- 前端支持 daily 模式记录下钻（web/app.js、web/app.css、web/index.html、
web/favicon.svg）
- README / docs/user-guide.md / docs/implementation-status.md 同步更新：
三种回测模式的取舍、daily 的成本说明（6.7 年约 1.5 小时）与调优手段
## 测试
tests/ 共 500 项（新增 tests/test_daily.py 43 项、tests/test_sync_daily.py 36 项）。
已验证通过：
- 排除上述两个新文件的 **421 项：全部通过（pytest 退出码 0）**
- 两个新文件的**非 DB 单元测试 60 项：全部通过**
未能在合理时间内跑完：
- 两个新文件中 **19 项 DB 标记的重型测试**。实测瓶颈是一条**无界全表扫描**：
`SELECT ... FROM hd_cashflow WHERE ann_date <= :asof ORDER BY symbol, end_date, ann_date`
（31 万行，无 symbol/报告期下限）。全量套件跑到 161 项时已耗时 20 分钟、
0 失败，按该速率预计需 3 小时以上，因此改为分档验证。
- 旁证：库中存在 3 次成功的 daily 端到端运行（2026-10-05 10:05 / 10:32 / 11:03，
区间 2024-03-01~03-15），说明该路径可正常完成。
## 已知待改进
- 上述 `hd_cashflow`（及同类「按 ann_date 上界取全历史」）的查询缺
symbol / 报告期下限，是 daily 模式的主要性能瓶颈，建议下一轮优化。
This commit is contained in:
+175
-3
@@ -10,7 +10,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date
|
||||
from datetime import date, timedelta
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
@@ -22,6 +22,8 @@ from hdiv.backtest.engine import (
|
||||
Signal,
|
||||
_months_between,
|
||||
_round_lot,
|
||||
build_yield_series,
|
||||
dividend_handling_notes,
|
||||
reconcile,
|
||||
)
|
||||
from hdiv.backtest.walk_forward import WalkForwardRunner, _add_months, _add_years
|
||||
@@ -364,6 +366,164 @@ def test_engine_uses_next_open_no_lookahead() -> None:
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 分红与公司行为（plan.md §30/§31）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _div_ctx(day: date, rows: list[dict]) -> dict:
|
||||
return {"div_by_date": {day: rows}}
|
||||
|
||||
|
||||
def _held(
|
||||
symbol: str = "X.SH",
|
||||
*,
|
||||
quantity: float = 1000.0,
|
||||
price: float = 10.0,
|
||||
first_buy: date | None = None,
|
||||
) -> Position:
|
||||
d = first_buy or date(2020, 1, 2)
|
||||
return Position(
|
||||
symbol=symbol, quantity=quantity, avg_cost=price, cost_basis=quantity * price,
|
||||
first_buy_date=d, last_buy_date=d,
|
||||
)
|
||||
|
||||
|
||||
def test_pure_stock_dividend_is_not_dropped(engine) -> None:
|
||||
"""10 送 10（无现金分红）必须照常调整股数，不得静默丢弃。
|
||||
|
||||
纯送转的 ``cash_div_tax`` 是 NULL/0，而 ``stk_div`` > 0。价格是不复权价，
|
||||
除权日必然下跌；若按现金分红判空整行跳过，就会凭空记出一笔亏损。
|
||||
实测本库 2015-2026 区间内高股息池成员有 824 笔纯送转。
|
||||
"""
|
||||
day = date(2024, 6, 20)
|
||||
pos = _held(quantity=1000.0, price=10.0)
|
||||
ledger: list[dict] = []
|
||||
cash = engine._apply_dividends(
|
||||
day,
|
||||
{"X.SH": pos},
|
||||
_div_ctx(day, [{"symbol": "X.SH", "cash_div_tax": None, "stk_div": 1.0,
|
||||
"stk_bo_rate": 1.0, "stk_co_rate": None}]),
|
||||
0.0,
|
||||
ledger,
|
||||
)
|
||||
# 股价腰斩到 5 元、股数翻倍到 2000 股 → 市值不变
|
||||
assert pos.quantity == pytest.approx(2000.0)
|
||||
assert pos.quantity * 5.0 == pytest.approx(10000.0), "10 送 10 前后市值必须不变"
|
||||
assert pos.avg_cost == pytest.approx(5.0), "总成本不变，每股成本须随股数下降"
|
||||
assert cash == 0.0, "纯送转不产生现金"
|
||||
assert ledger[0]["shares_added"] == pytest.approx(1000.0)
|
||||
assert ledger[0]["stock_div_applied"] is True
|
||||
|
||||
|
||||
def test_stock_dividend_written_as_zero_cash_is_applied(engine) -> None:
|
||||
"""``cash_div_tax`` 写成 0（而非 NULL）的纯转增同样不能丢。"""
|
||||
day = date(2024, 6, 20)
|
||||
pos = _held()
|
||||
engine._apply_dividends(
|
||||
day,
|
||||
{"X.SH": pos},
|
||||
_div_ctx(day, [{"symbol": "X.SH", "cash_div_tax": 0.0, "stk_div": 0.5,
|
||||
"stk_co_rate": 0.5}]),
|
||||
0.0,
|
||||
[],
|
||||
)
|
||||
assert pos.quantity == pytest.approx(1500.0)
|
||||
assert pos.avg_cost == pytest.approx(10000.0 / 1500.0)
|
||||
|
||||
|
||||
def test_cash_and_stock_dividend_are_independent(engine) -> None:
|
||||
"""同一行既有现金又有送转：两者都要入账，互不影响。"""
|
||||
day = date(2024, 6, 20)
|
||||
pos = _held(first_buy=day - timedelta(days=800)) # 持股 > 1 年 → 免征红利税
|
||||
ledger: list[dict] = []
|
||||
cash = engine._apply_dividends(
|
||||
day,
|
||||
{"X.SH": pos},
|
||||
_div_ctx(day, [{"symbol": "X.SH", "cash_div_tax": 0.5, "stk_div": 0.3,
|
||||
"stk_bo_rate": 0.3}]),
|
||||
0.0,
|
||||
ledger,
|
||||
)
|
||||
assert cash == pytest.approx(500.0), "持股 > 1 年免征红利税，全额入账"
|
||||
assert pos.quantity == pytest.approx(1300.0)
|
||||
assert ledger[0]["gross"] == pytest.approx(500.0)
|
||||
assert ledger[0]["tax"] == pytest.approx(0.0)
|
||||
assert ledger[0]["shares_added"] == pytest.approx(300.0)
|
||||
|
||||
|
||||
def test_empty_dividend_row_is_skipped(engine) -> None:
|
||||
"""既无现金也无送转（数据异常行）才是该跳过的行，且不留账。"""
|
||||
day = date(2024, 6, 20)
|
||||
pos = _held()
|
||||
ledger: list[dict] = []
|
||||
engine._apply_dividends(
|
||||
day, {"X.SH": pos},
|
||||
_div_ctx(day, [{"symbol": "X.SH", "cash_div_tax": 0.0, "stk_div": 0.0}]),
|
||||
0.0, ledger,
|
||||
)
|
||||
assert ledger == []
|
||||
assert pos.quantity == pytest.approx(1000.0)
|
||||
|
||||
|
||||
def test_dividend_cash_joins_the_investable_pool(engine) -> None:
|
||||
"""分红现金必须与初始资金同一个现金池 —— 能直接用于买入，不被隔离。
|
||||
|
||||
这是「分红再投资」的实际含义：除权日入账 → 下次调仓按目标权重再配置。
|
||||
"""
|
||||
day = date(2024, 6, 20)
|
||||
positions = {"X.SH": _held("X.SH", quantity=10_000.0, price=1.0,
|
||||
first_buy=day - timedelta(days=800))} # 免税
|
||||
px = pd.DataFrame(
|
||||
{"open": [1.0], "close": [1.0]}, index=pd.DatetimeIndex([pd.Timestamp(day)])
|
||||
)
|
||||
ctx = {
|
||||
"div_by_date": {day: [{"symbol": "X.SH", "cash_div_tax": 0.10, "stk_div": None}]},
|
||||
"px_by_sym": {"X.SH": px, "Y.SH": px},
|
||||
"suspend": set(),
|
||||
"limits": {},
|
||||
}
|
||||
# 起点现金为 0：下面买得成，只可能来自这笔分红
|
||||
cash = engine._apply_dividends(day, positions, ctx, 0.0, [])
|
||||
assert cash == pytest.approx(1000.0), "10000 股 × 每股 0.10 元"
|
||||
|
||||
sig = Signal(
|
||||
symbol="Y.SH", signal_date=day, kind="BUY", target_weight=0.10,
|
||||
yield_value=0.08, yield_percentile=80.0, price=None, reason={},
|
||||
)
|
||||
fill, cash_after, skip = engine._execute(sig, day, cash, positions, ctx, 0)
|
||||
assert fill is not None, f"分红现金未能用于买入：{skip}"
|
||||
assert fill.quantity > 0
|
||||
assert cash_after < cash
|
||||
assert cash_after >= 0.0
|
||||
|
||||
|
||||
def test_dividend_handling_notes_match_each_mode() -> None:
|
||||
"""声明口径必须与实现逐档对应：已实现的组合不得留声明，未实现的必须声明。
|
||||
|
||||
背景：`cash_mode: reinvest` 的实际行为一直是「分红现金回落到可投资现金池、
|
||||
下次调仓按目标权重再配置」，却长期被声明成「未实现」—— 声明与行为两头都不准。
|
||||
"""
|
||||
|
||||
def notes(**kw) -> str:
|
||||
bt = load_config("backtest").model_copy(deep=True)
|
||||
for k, v in kw.items():
|
||||
setattr(bt.dividend, k, v)
|
||||
return " ".join(dividend_handling_notes(bt))
|
||||
|
||||
# 已实现：可投资现金池 + 目标权重再配置，且不涉及未实现的配股
|
||||
assert notes(cash_mode="reinvest", reinvest_rule="portfolio_rebalance",
|
||||
handle_stock_dividend=True, handle_rights_issue=False) == ""
|
||||
# 未实现：同股再投 / 永久留存 / 移出组合 / 不处理送转 / 配股，逐条都要声明
|
||||
assert "未实现 reinvest_rule" in notes(reinvest_rule="same_stock_next_open",
|
||||
handle_rights_issue=False)
|
||||
assert "cash_mode=hold" in notes(cash_mode="hold", handle_rights_issue=False)
|
||||
assert "cash_mode=cash_out" in notes(cash_mode="cash_out", handle_rights_issue=False)
|
||||
assert "handle_stock_dividend" in notes(handle_stock_dividend=False,
|
||||
handle_rights_issue=False)
|
||||
assert "配股" in notes(handle_rights_issue=True)
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_engine_dividends_are_creditable() -> None:
|
||||
"""持有期间应确实收到现金分红（高股息策略的核心收益来源）。"""
|
||||
@@ -542,7 +702,13 @@ def _trigger_ctx(sym: str = "000001.SZ", n: int = 280) -> dict:
|
||||
ev = pd.DataFrame([{
|
||||
"ex_date": days[0], "imp_ann_date": days[0], "cash_div_tax": 1.0,
|
||||
}])
|
||||
return {"px_by_sym": {sym: px}, "events": {sym: ev}, "_last_day": days[-1].date()}
|
||||
px_by_sym = {sym: px}
|
||||
events = {sym: ev}
|
||||
# P2 之后引擎从 ctx["yield_by_sym"] 取预计算的股息率序列；这里用**同一个**
|
||||
# 生产函数构造，避免测试自己算一套（那就成了两套口径）。
|
||||
return {"px_by_sym": px_by_sym, "events": events,
|
||||
"yield_by_sym": build_yield_series(px_by_sym, events),
|
||||
"_last_day": days[-1].date()}
|
||||
|
||||
|
||||
def _pass_gate(*_a, **_k) -> dict:
|
||||
@@ -689,5 +855,11 @@ def test_unimplemented_declarations_are_honest() -> None:
|
||||
# ① 不得把「无停牌」误报成「无数据」（约束表在 2010 起有数据）
|
||||
assert "无数据" not in decl, f"误报数据缺失：{decl}"
|
||||
# ② 必须如实声明「配置承诺但未实现」的项
|
||||
for must in ("defer", "分红再投资", "配股", "成交量占比"):
|
||||
for must in ("defer", "配股", "成交量占比"):
|
||||
assert must in decl, f"漏报未实现项 {must}：{decl}"
|
||||
# ③ 另一头也要准：**已实现**的组合不得留声明。当前配置
|
||||
# （cash_mode=reinvest + reinvest_rule=portfolio_rebalance）的实际行为是
|
||||
# 「分红现金回落到可投资现金池、下次调仓按目标权重再配置」，声明它
|
||||
# 「未实现」会让使用者误以为分红现金被隔离成了不可投资资金。
|
||||
assert "分红再投资" not in decl, f"把已实现的分红再投资误报成未实现：{decl}"
|
||||
assert "reinvest_rule" not in decl, f"把已实现的再投资规则误报成未实现：{decl}"
|
||||
|
||||
@@ -90,7 +90,7 @@ def test_no_undocumented_commands(parser) -> None:
|
||||
("strategy", {"validate", "register", "list", "diff"}),
|
||||
(
|
||||
"sync",
|
||||
{"dividend", "financial", "index", "price", "trading", "backfill"},
|
||||
{"daily", "dividend", "financial", "index", "price", "trading", "backfill"},
|
||||
),
|
||||
("site", {"normalize", "archive", "build", "status"}),
|
||||
],
|
||||
@@ -109,7 +109,10 @@ def test_documented_actions_exist(parser, cmd: str, expected: set[str]) -> None:
|
||||
[
|
||||
("sync", {"--only-missing", "--limit", "--symbols", "--apis",
|
||||
"--interleaved", "--start", "--end", "--no-resume", "--no-weight",
|
||||
"--basic-start", "--basic-end"}),
|
||||
"--basic-start", "--basic-end",
|
||||
# sync daily（手册 §5.2.1）
|
||||
"--dry-run", "--asof", "--lookback-days", "--only",
|
||||
"--no-financial", "--financial-limit", "--json"}),
|
||||
("universe", {"-c", "--config", "--asof", "--no-persist", "--no-html"}),
|
||||
("profile", {"--universe-run", "--symbols", "--asof", "--html-limit"}),
|
||||
("backtest", {"-s", "--strategy", "--mode", "--start", "--end", "--universe-run",
|
||||
@@ -139,6 +142,20 @@ def test_sync_interleaved_is_a_real_flag(parser) -> None:
|
||||
assert args2.interleaved is False
|
||||
|
||||
|
||||
def test_sync_daily_defaults(parser) -> None:
|
||||
"""手册 §5.2.1 承诺的 ``hdiv sync daily`` 必须存在，且默认是「真抓、含财报」。"""
|
||||
args = parser.parse_args(["sync", "daily"])
|
||||
assert args.dry_run is False, "默认必须真抓；--dry-run 是显式开关"
|
||||
assert args.no_financial is False, "默认应包含财报四表（有上限兜底）"
|
||||
assert args.financial_limit == 500
|
||||
assert args.lookback_days == 45
|
||||
assert args.only is None
|
||||
assert args.asof is None
|
||||
# --only 必须真的能解析
|
||||
a2 = parser.parse_args(["sync", "daily", "--only", "price", "dividend"])
|
||||
assert a2.only == ["price", "dividend"]
|
||||
|
||||
|
||||
def test_backtest_universe_run_flag(parser) -> None:
|
||||
"""股票池 ↔ 回测 的关联入口：``--universe-run`` 必须存在且可解析。"""
|
||||
args = parser.parse_args(["backtest", "--universe-run", "abc123"])
|
||||
@@ -241,11 +258,11 @@ def test_future_universe_is_rejected_by_default() -> None:
|
||||
|
||||
|
||||
def test_backtest_mode_choices(parser) -> None:
|
||||
"""手册只承诺 single / walkforward 两种模式。"""
|
||||
"""手册承诺 single / walkforward / daily 三种模式。"""
|
||||
sub = _subparsers(parser)["backtest"]
|
||||
for a in sub._actions:
|
||||
if "--mode" in a.option_strings:
|
||||
assert set(a.choices) == {"single", "walkforward"}
|
||||
assert set(a.choices) == {"single", "walkforward", "daily"}
|
||||
return
|
||||
raise AssertionError("backtest 缺少 --mode 参数")
|
||||
|
||||
@@ -273,7 +290,7 @@ def test_ddl_verify_output_mentions_table_count() -> None:
|
||||
|
||||
src = inspect.getsource(cli.cmd_ddl)
|
||||
assert "len(ddl.ALL_TABLES)" in src, "ddl verify 输出应包含表的数量"
|
||||
assert len(ALL_TABLES) == 30
|
||||
assert len(ALL_TABLES) == 31
|
||||
|
||||
|
||||
def test_help_text_is_chinese(parser) -> None:
|
||||
|
||||
@@ -0,0 +1,985 @@
|
||||
"""每日动态股票池回测（``--mode daily``）测试。
|
||||
|
||||
三条必须被锁定的性质：
|
||||
|
||||
1. **取数口径不分叉** —— ``PitRepo`` 在某个 asof 上返回的行情/每日指标/财务/
|
||||
分红，必须与直连数据库的 ``Repo`` **逐值一致**。它只改了「怎么取」，
|
||||
没有改「怎么算」；这是每日选股能跑得快却仍然可信的前提。
|
||||
与 ``tests/test_profile_pit.py`` 锁定「实时画像 = 批量画像」是同一个思路。
|
||||
|
||||
2. **预剪枝不改变最终入选** —— 市场/风险滤网的候选集预剪枝只剔除「在区间内
|
||||
不可能通过市场滤网」的股票（交易所/板块/上市年限/市值上界）。开与关必须
|
||||
选出**完全相同**的成员，否则它是「近似」而不是「等价」。
|
||||
|
||||
3. **动态池语义** —— 持仓掉出当日股票池时：``pool_exit_action=hold`` 只停止
|
||||
加仓（不清仓、仍按分位卖出），``sell`` 则清仓。买卖决策都会留下画像证据，
|
||||
与它是否在池内无关。
|
||||
|
||||
需要数据库的用例标记为 ``db``（无库时跳过）。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, timedelta
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
from hdiv.core.config import DailyConfig, load_config
|
||||
from hdiv.universe.daily import DailyUniverseScreener
|
||||
from hdiv.backtest.daily import _chunks
|
||||
from hdiv.backtest.engine import build_yield_series
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 非 DB：配置与纯函数
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDailyConfig:
|
||||
def test_defaults_are_conservative(self) -> None:
|
||||
"""默认值必须让 daily 段「什么都没开」不影响既有模式。"""
|
||||
c = DailyConfig()
|
||||
assert c.universe_refresh_days == 1
|
||||
assert c.signal_frequency_days == 1
|
||||
assert c.pool_exit_action == "hold"
|
||||
assert c.profile_on_trade is True
|
||||
assert c.persist_daily_universe is True
|
||||
assert c.chunk_years == 1
|
||||
|
||||
def test_backtest_yml_has_daily_block(self) -> None:
|
||||
bt = load_config("backtest")
|
||||
assert bt.daily.universe_refresh_days >= 1
|
||||
assert bt.daily.pool_exit_action in {"hold", "sell"}
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field", ["universe_refresh_days", "signal_frequency_days", "chunk_years"]
|
||||
)
|
||||
def test_non_positive_rejected(self, field: str) -> None:
|
||||
from hdiv.core.errors import SchemaValidationError
|
||||
|
||||
with pytest.raises((SchemaValidationError, ValueError)):
|
||||
DailyConfig(**{field: 0})
|
||||
|
||||
def test_bad_pool_exit_action_rejected(self) -> None:
|
||||
from hdiv.core.errors import SchemaValidationError
|
||||
|
||||
with pytest.raises((SchemaValidationError, ValueError)):
|
||||
DailyConfig(pool_exit_action="liquidate")
|
||||
|
||||
|
||||
class TestChunks:
|
||||
def test_single_year(self) -> None:
|
||||
days = [date(2024, 1, 2), date(2024, 3, 1), date(2024, 12, 31)]
|
||||
out = _chunks(days, 1)
|
||||
assert out == [days]
|
||||
|
||||
def test_year_boundary_splits(self) -> None:
|
||||
days = [date(2023, 12, 29), date(2024, 1, 2), date(2024, 12, 31),
|
||||
date(2025, 1, 2)]
|
||||
out = _chunks(days, 1)
|
||||
assert [len(c) for c in out] == [1, 2, 1]
|
||||
assert out[0][0] == date(2023, 12, 29)
|
||||
assert out[2][0] == date(2025, 1, 2)
|
||||
|
||||
def test_multi_year_chunk(self) -> None:
|
||||
days = [date(2020, 6, 1), date(2021, 6, 1), date(2022, 6, 1),
|
||||
date(2023, 6, 1), date(2024, 6, 1)]
|
||||
out = _chunks(days, 2)
|
||||
assert [len(c) for c in out] == [2, 2, 1]
|
||||
|
||||
def test_empty(self) -> None:
|
||||
assert _chunks([], 1) == []
|
||||
|
||||
|
||||
class TestEngineDefaultsAreOff:
|
||||
"""新增的引擎参数默认值必须让 single / walkforward 行为逐字不变。"""
|
||||
|
||||
def test_new_params_default_off(self) -> None:
|
||||
from hdiv.backtest.engine import BacktestEngine
|
||||
from hdiv.strategy.registry import StrategyRegistry
|
||||
|
||||
s = StrategyRegistry().load("config/strategy/high_dividend_v1.yml")
|
||||
eng = BacktestEngine(s)
|
||||
assert eng.universe_by_refresh is None
|
||||
assert eng.universe_refresh_days is None
|
||||
assert eng.signal_frequency_days is None
|
||||
assert eng.pool_exit_action == "hold"
|
||||
assert eng.profile_on_trade is False
|
||||
|
||||
def test_monthly_signal_path_preserved(self) -> None:
|
||||
"""按月判定信号的原始语义必须仍在源码里（daily 只是新增分支）。"""
|
||||
import inspect
|
||||
|
||||
from hdiv.backtest import engine as mod
|
||||
|
||||
src = inspect.getsource(mod.BacktestEngine._simulate)
|
||||
assert "freq_days" in src, "daily 的按交易日分支丢失"
|
||||
assert "_months_between(" in src, "原有按月判定分支被删除"
|
||||
assert "day_index % max(1, int(freq_days))" in src
|
||||
|
||||
def test_pool_exit_branches_present(self) -> None:
|
||||
import inspect
|
||||
|
||||
from hdiv.backtest import engine as mod
|
||||
|
||||
src = inspect.getsource(mod.BacktestEngine._evaluate)
|
||||
assert "in_universe" in src
|
||||
assert "pool_exit_action" in src
|
||||
assert "OUT_OF_UNIVERSE" in src
|
||||
assert "_trade_signal" in src, "买卖信号的画像留痕出口丢失"
|
||||
|
||||
|
||||
class TestMetricFormatting:
|
||||
"""短区间下指标不可计算时的格式化（**修掉一个既有的崩溃**）。"""
|
||||
|
||||
def test_pct_and_num_tolerate_none(self) -> None:
|
||||
from hdiv.backtest.engine import _num, _pct
|
||||
|
||||
assert _pct(None) == "—"
|
||||
assert _num(None) == "—"
|
||||
assert _pct(0.1234) == "12.34%"
|
||||
assert _num(1.5) == "1.50"
|
||||
assert _pct(float("nan")) == "—"
|
||||
|
||||
def test_no_raw_metric_format_in_hot_prints(self) -> None:
|
||||
"""回测的进度/汇总打印不得直接对可能为 None 的指标做 :.2% 格式化。
|
||||
|
||||
实测：15 个交易日的回测里 ``sharpe`` 为 None，直接 ``:.2f`` 会抛
|
||||
``TypeError`` 并以完整 traceback 结束 —— 把「指标不可计算」这个正常状态
|
||||
说成了程序缺陷。engine 与 cli 的打印必须走 ``_pct`` / ``_num``。
|
||||
"""
|
||||
import inspect
|
||||
|
||||
from hdiv import cli
|
||||
from hdiv.backtest import engine as eng
|
||||
|
||||
for fn in (eng.BacktestEngine.run, cli.cmd_backtest):
|
||||
src = inspect.getsource(fn)
|
||||
for bad in ("['sharpe']:.2f", "['cagr']:.2%",
|
||||
"['max_drawdown']:.2%", "['total_return']:.2%"):
|
||||
assert bad not in src, f"{fn.__qualname__} 仍在直接格式化 {bad}"
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_pit_profile_caches_are_bounded() -> None:
|
||||
"""长区间每日回测**不能**把每个时点的面板都留在内存里（否则必然 OOM）。
|
||||
|
||||
每个 ``_AsOfContext`` 持有该 asof 可见的分红超集（约 4 MB）。1600 个决策日
|
||||
不设上限 ≈ 6 GB。这里用跨越一年的多个 asof 证明缓存是有界的，
|
||||
同时 ``distinct_asof`` 仍如实汇报**累计**涉及的时点数。
|
||||
"""
|
||||
from hdiv.profile.pit import _MAX_ASOF_CONTEXTS, PitProfileService
|
||||
from hdiv.universe.pit import PitRepo
|
||||
|
||||
try:
|
||||
rows = PitRepo().stock_master()
|
||||
syms = rows["symbol"].astype(str).head(3).tolist()
|
||||
except Exception as exc: # pragma: no cover
|
||||
pytest.skip(f"数据库不可用：{exc}")
|
||||
if not syms:
|
||||
pytest.skip("数据库无股票")
|
||||
svc = PitProfileService(window_years=5)
|
||||
svc.prepare(syms, date(2019, 1, 1), date(2024, 12, 31))
|
||||
svc.configure({"roe_avg"})
|
||||
days = [date(2024, m, 15) for m in range(1, 13)]
|
||||
for d in days:
|
||||
for s in syms:
|
||||
svc.snapshot(s, d)
|
||||
assert len(svc._ctx) <= _MAX_ASOF_CONTEXTS, "时点面板缓存无上限 —— 长回测会 OOM"
|
||||
assert len(svc._snapshots) <= len(syms) * 2, "画像快照缓存无上限"
|
||||
assert svc.stats()["distinct_asof"] == len(days), (
|
||||
"distinct_asof 必须是累计值，不能用当前缓存条数冒充"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 前端数据源：决策时点实时画像
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_stock_detail_exposes_decision_time_profile() -> None:
|
||||
"""成交个股的**决策时点实时画像**必须能从接口取到。
|
||||
|
||||
这是前端个股页「决策时点实时画像」卡片的数据源
|
||||
（``trades[].reason.profile``）。这条链路断了，页面那张卡片会静默变空 ——
|
||||
而「当时凭什么买」正是 daily 模式最该留下的证据。
|
||||
"""
|
||||
from hdiv.data import db
|
||||
from hdiv.web import analysis
|
||||
|
||||
cfg = load_config("datasource")
|
||||
try:
|
||||
run = db.read_sql(
|
||||
"SELECT run_id FROM hd_backtest_run WHERE mode = 'daily' "
|
||||
"ORDER BY created_at DESC LIMIT 1", cfg=cfg,
|
||||
)
|
||||
except Exception as exc: # pragma: no cover
|
||||
pytest.skip(f"数据库不可用：{exc}")
|
||||
if run.empty:
|
||||
pytest.skip("库里还没有 mode=daily 的回测记录")
|
||||
run_id = str(run["run_id"].iloc[0])
|
||||
|
||||
tr = db.read_sql(
|
||||
"SELECT symbol FROM hd_backtest_trade WHERE run_id = :r "
|
||||
"AND JSON_EXTRACT(reason_json, '$.profile') IS NOT NULL LIMIT 1",
|
||||
{"r": run_id}, cfg=cfg,
|
||||
)
|
||||
if tr.empty:
|
||||
pytest.skip(f"{run_id} 的成交里没有画像留痕（可能关闭了 profile_on_trade）")
|
||||
symbol = str(tr["symbol"].iloc[0])
|
||||
|
||||
d = analysis.stock_detail(run_id, symbol, series=["close"])
|
||||
with_profile = [t for t in d["trades"] if (t.get("reason") or {}).get("profile")]
|
||||
assert with_profile, "接口未返回决策时点画像（前端卡片会变空）"
|
||||
prof = with_profile[0]["reason"]["profile"]
|
||||
# 快照必须带「当时的值」与「当时的窗口」，否则无法复核
|
||||
assert isinstance(prof.get("values"), dict) and prof["values"], prof.keys()
|
||||
assert "window_years" in prof and "asof" in prof
|
||||
# 至少含闸门用到的一项判据，证明与筛选/闸门是同一份画像口径
|
||||
assert {"dv_yield", "roe_avg", "dividend_continuity_years",
|
||||
"payout_ratio", "fcf_dividend_cover"} & set(prof["values"])
|
||||
|
||||
# 回测详情页的「成交个股的实时画像」表用的是**成交列表**接口，
|
||||
# 它也必须带 reason.profile（否则那张表会空）。
|
||||
from hdiv.web import service
|
||||
|
||||
tl = service.list_backtest_trades(run_id, size=200)
|
||||
assert any((it.get("reason") or {}).get("profile") for it in tl["items"]), (
|
||||
"list_backtest_trades 未返回画像 —— 回测详情页的画像表会空"
|
||||
)
|
||||
|
||||
|
||||
def test_cli_points_at_the_frontend() -> None:
|
||||
"""一条命令跑完必须告诉用户「去前端哪里看」，而不是让用户再跑第二条命令。
|
||||
|
||||
回归：曾把「看结果」写成需要手工传 run_id 去执行一个导出脚本 ——
|
||||
而 run_id 本来就在这条命令自己的输出里，前端也直接列出来。
|
||||
"""
|
||||
import inspect
|
||||
|
||||
from hdiv import cli
|
||||
|
||||
src = inspect.getsource(cli)
|
||||
assert "_print_frontend_hint" in src
|
||||
assert "回测记录" in src, "提示里要给出前端位置"
|
||||
# daily 与 single 两条路径都必须给出该提示
|
||||
assert src.count('_print_frontend_hint(res["run_id"])') >= 2
|
||||
|
||||
|
||||
class TestSpeedLevers:
|
||||
"""速度旋钮：`--every-n-days` / `--signal-every-n-days`。
|
||||
|
||||
两个频率**只改变「多久看一次」，不改变判定规则**。这里锁定三件事：
|
||||
参数真的传到了执行器、缓存键把重建频率算进去、耗时模型随频率正确下降。
|
||||
"""
|
||||
|
||||
def test_cli_flags_exist_and_parse(self) -> None:
|
||||
from hdiv.cli import build_parser
|
||||
|
||||
p = build_parser()
|
||||
a = p.parse_args(["backtest"])
|
||||
assert a.every_n_days is None and a.signal_every_n_days is None, (
|
||||
"默认必须是 None（= 用 config 里的值，通常为 1）"
|
||||
)
|
||||
a = p.parse_args(["backtest", "--mode", "daily", "--start", "2020-01-05",
|
||||
"--every-n-days", "5", "--signal-every-n-days", "10"])
|
||||
assert a.every_n_days == 5 and a.signal_every_n_days == 10
|
||||
|
||||
def test_runner_accepts_overrides(self) -> None:
|
||||
from hdiv.backtest.daily import DailyRunner
|
||||
|
||||
import inspect
|
||||
|
||||
sig = inspect.signature(DailyRunner.run)
|
||||
assert "every_n_days" in sig.parameters
|
||||
assert "signal_every_n_days" in sig.parameters
|
||||
assert sig.parameters["every_n_days"].default is None
|
||||
assert sig.parameters["signal_every_n_days"].default is None
|
||||
|
||||
def test_cache_key_depends_on_pool_cadence(self) -> None:
|
||||
"""1 日 / 5 日 筛出的池子是不同的输入，不能共用缓存。"""
|
||||
from hdiv.backtest.daily import DailyRunner
|
||||
from hdiv.strategy.registry import StrategyRegistry
|
||||
|
||||
reg = StrategyRegistry()
|
||||
s = reg.load("config/strategy/high_dividend_v1.yml")
|
||||
|
||||
class _Stub:
|
||||
"""只借 `_pools_cache_key`，不构造真 runner（那会连库）。"""
|
||||
|
||||
strategy = s
|
||||
registry = reg
|
||||
_pools_cache_key = DailyRunner._pools_cache_key
|
||||
|
||||
stub = _Stub()
|
||||
k1 = stub._pools_cache_key(date(2020, 1, 5), date(2026, 9, 30), 1)
|
||||
k5 = stub._pools_cache_key(date(2020, 1, 5), date(2026, 9, 30), 5)
|
||||
assert k1 != k5, "不同重建频率必须落在不同缓存文件"
|
||||
assert k1 == stub._pools_cache_key(date(2020, 1, 5), date(2026, 9, 30), 1), (
|
||||
"同样的输入必须得到同样的键（缓存要可复用）"
|
||||
)
|
||||
|
||||
def test_time_model_shrinks_with_coarser_cadence(self) -> None:
|
||||
"""耗时模型必须随频率下降（否则预计时长会骗人）。"""
|
||||
import hdiv.backtest.daily as D
|
||||
|
||||
def sim_seconds(days: int, signal_step: int) -> float:
|
||||
return (days * D._SEC_PER_SIM_DAY_BASE
|
||||
+ days / signal_step * D._SEC_PER_SIM_SIGNAL)
|
||||
|
||||
assert sim_seconds(1635, 1) > sim_seconds(1635, 5) > sim_seconds(1635, 21)
|
||||
# 逐日口径必须与实测同量级（43 日实测 131 秒）
|
||||
assert 100 < sim_seconds(43, 1) < 170, sim_seconds(43, 1)
|
||||
# 每 5 日口径也必须与实测同量级（约 28 秒）
|
||||
assert 15 < sim_seconds(43, 5) < 60, sim_seconds(43, 5)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 性能优化的等价性（P1/P2/P4/P6）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_ttm_dps_series_prefix_equals_full() -> None:
|
||||
"""P2 的前提：``ttm_dps_series`` 每一天的取值只依赖该日期与事件。
|
||||
|
||||
若这条不成立，「整段算一次再切片」就会与「每次按前缀重算」不同 ——
|
||||
P2（消除 O(交易日数²)）正是建立在这条性质上。
|
||||
"""
|
||||
from hdiv.factor.dividend_yield import ttm_dps_series
|
||||
|
||||
idx = pd.bdate_range("2020-01-01", periods=800)
|
||||
# 事件间隔刻意覆盖三种情况：年内多次、略小于一年（重叠虚高）、略大于一年（断档虚低）
|
||||
ex = [idx[50], idx[300], idx[520], idx[640], idx[770]]
|
||||
ev = pd.DataFrame({
|
||||
"ex_date": ex,
|
||||
"imp_ann_date": [e - pd.Timedelta(days=12) for e in ex],
|
||||
"cash_div_tax": [0.5, 0.3, 0.6, 0.55, 0.7],
|
||||
})
|
||||
full = ttm_dps_series(idx, ev, ttm_days=365, grace_days=45, smooth_spikes=True)
|
||||
for i in (60, 310, 530, 650, 700, 799):
|
||||
pref = ttm_dps_series(idx[: i + 1], ev, ttm_days=365, grace_days=45,
|
||||
smooth_spikes=True)
|
||||
assert abs(float(full[i]) - float(pref[-1])) < 1e-12, (
|
||||
f"第 {i} 天：整段 {full[i]} != 前缀末值 {pref[-1]} —— P2 的前提不成立"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_pit_dividend_scope_is_exact_subset() -> None:
|
||||
"""P4：``restrict_to`` 只能把 ``dividend_records`` 收窄成精确子集。"""
|
||||
asof = date(2024, 6, 3)
|
||||
pit = _load_pit(date(2024, 1, 1), date(2024, 6, 28), ref_end=date(2024, 12, 31))
|
||||
full = pit.dividend_records(asof, years_back=8)
|
||||
assert not full.empty
|
||||
syms = sorted(full["symbol"].astype(str).unique())[:5]
|
||||
pit.restrict_to(syms)
|
||||
try:
|
||||
scoped = pit.dividend_records(asof, years_back=8)
|
||||
finally:
|
||||
pit.restrict_to(None)
|
||||
cols = list(full.columns)
|
||||
want = full[full["symbol"].isin(set(syms))].reset_index(drop=True)
|
||||
pd.testing.assert_frame_equal(
|
||||
scoped.sort_values(cols, ignore_index=True),
|
||||
want.sort_values(cols, ignore_index=True),
|
||||
check_dtype=False, rtol=1e-9,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_pit_dividend_events_matches_direct_repo(plain_repo) -> None:
|
||||
"""P6：内存版 ``dividend_events`` 必须与直连逐值一致（含 NULL 公告日边界）。"""
|
||||
|
||||
|
||||
ref_end = date(2026, 9, 30)
|
||||
pit = _load_pit(date(2025, 1, 1), ref_end, ref_end=ref_end)
|
||||
for s, e in ((date(2024, 1, 1), date(2024, 12, 31)),
|
||||
(date(2020, 1, 1), ref_end)):
|
||||
a = pit.dividend_events(s, e)
|
||||
b = plain_repo.dividend_events(s, e)
|
||||
cols = list(a.columns)
|
||||
pd.testing.assert_frame_equal(
|
||||
a.sort_values(cols, ignore_index=True),
|
||||
b.sort_values(cols, ignore_index=True),
|
||||
check_dtype=False, rtol=1e-9,
|
||||
obj=f"dividend_events {s}..{e}",
|
||||
)
|
||||
# 超出参照终点的区间必须退回直连，而不是返回少行的表
|
||||
narrow = _load_pit(date(2025, 1, 1), date(2024, 12, 31),
|
||||
ref_end=date(2024, 6, 3))
|
||||
a = narrow.dividend_events(date(2024, 1, 1), date(2024, 12, 31))
|
||||
b = plain_repo.dividend_events(date(2024, 1, 1), date(2024, 12, 31))
|
||||
assert len(a) == len(b), "超出参照终点的区间被静默截断了"
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_profile_prepare_accepts_external_price(plain_repo) -> None:
|
||||
"""P6：引擎把已取好的价格交给画像复用，结果必须逐值不变。"""
|
||||
from hdiv.profile.pit import PitProfileService
|
||||
|
||||
syms = ["600036.SH", "000651.SZ", "601398.SH"]
|
||||
a, b = date(2015, 1, 1), date(2026, 9, 30)
|
||||
pit = _load_pit(a, b, ref_end=b)
|
||||
s1 = PitProfileService(window_years=5, repo=pit)
|
||||
s1.prepare(syms, a, b)
|
||||
price = plain_repo.price_history(syms, a, b, adjust="none")
|
||||
s2 = PitProfileService(window_years=5, repo=pit)
|
||||
s2.prepare(syms, a, b, price=price)
|
||||
s1.configure({"dv_yield", "pe_ttm", "pb", "roe_avg"})
|
||||
s2.configure({"dv_yield", "pe_ttm", "pb", "roe_avg"})
|
||||
for d in (date(2018, 5, 18), date(2024, 6, 3)):
|
||||
for y in syms:
|
||||
x, z = s1.snapshot(y, d), s2.snapshot(y, d)
|
||||
assert (x is None) == (z is None)
|
||||
if x is not None and z is not None:
|
||||
assert x.values == z.values and x.percentiles == z.percentiles
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_dividend_scope_does_not_change_selection(pit_2024) -> None:
|
||||
"""P4 的端到端保证：收窄分红取数不改变每日入选成员。"""
|
||||
from hdiv.universe.pit import PitRepo
|
||||
from hdiv.universe.selector import UniverseSelector
|
||||
|
||||
pit = pit_2024
|
||||
cfg = load_config("universe")
|
||||
sel = UniverseSelector(cfg, repo=pit)
|
||||
days = [d for d in pit.trading_days(date(2024, 1, 1), date(2024, 6, 28))
|
||||
if d in (date(2024, 1, 2), date(2024, 3, 1), date(2024, 6, 3))]
|
||||
assert days
|
||||
|
||||
orig = PitRepo.dividend_records
|
||||
|
||||
def _unscoped(self, asof, **kw): # noqa: ANN001
|
||||
saved = self._scope
|
||||
self._scope = None
|
||||
try:
|
||||
return orig(self, asof, **kw)
|
||||
finally:
|
||||
self._scope = saved
|
||||
|
||||
# 关掉收窄（= 优化前的行为）
|
||||
PitRepo.dividend_records = _unscoped
|
||||
try:
|
||||
before = {d: set(sel.run(asof=d, persist=False, verbose=False)["selected"]["symbol"])
|
||||
for d in days}
|
||||
finally:
|
||||
PitRepo.dividend_records = orig
|
||||
# 打开收窄：走生产路径（on_stage 钩子会按阶段收窄）
|
||||
pit.set_candidate_scope(None)
|
||||
screener = DailyUniverseScreener(cfg, pit, verbose=False)
|
||||
after = {d: set(screener.screen_day(d).symbols) for d in days}
|
||||
for d in days:
|
||||
assert before[d] == after[d], (
|
||||
f"{d} 收窄分红取数改变了入选：多 {sorted(after[d] - before[d])} "
|
||||
f"少 {sorted(before[d] - after[d])}"
|
||||
)
|
||||
|
||||
|
||||
class TestPoolExitSemantics:
|
||||
"""动态池的语义核心：掉出当日池子的持仓怎么办。
|
||||
|
||||
用合成行情直接调 ``_evaluate``，不经数据库 —— 这样 hold / sell 两条分支
|
||||
都被真正执行到，而不是只靠「源码里有这个词」。
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def _setup(pool_exit_action: str):
|
||||
import pandas as pd
|
||||
|
||||
from hdiv.backtest.engine import BacktestEngine, Position
|
||||
from hdiv.core.config import load_config as _lc
|
||||
from hdiv.strategy.registry import StrategyRegistry
|
||||
|
||||
reg = StrategyRegistry()
|
||||
s = reg.load("config/strategy/high_dividend_v1.yml").model_copy(deep=True)
|
||||
# 闸门关闭：本用例考的是**池子语义**，不是画像闸门。
|
||||
s.entry.profile_gate.enabled = False
|
||||
bt = _lc("backtest")
|
||||
eng = BacktestEngine(
|
||||
s, backtest=bt, pool_exit_action=pool_exit_action,
|
||||
signal_frequency_days=1,
|
||||
)
|
||||
|
||||
# 400 个交易日的合成行情：收盘价从 40 线性跌到 10 → 股息率一路上行，
|
||||
# 当前值必然是窗口内的最高分位（≈100%），足以触发最高买入档。
|
||||
idx = pd.bdate_range(end="2024-06-28", periods=400)
|
||||
close = np.linspace(40.0, 10.0, len(idx))
|
||||
px = pd.DataFrame({"open": close, "close": close}, index=idx)
|
||||
day = idx[-1].date()
|
||||
ex = idx[200]
|
||||
events = {"X.SH": pd.DataFrame([{
|
||||
"ex_date": ex, "imp_ann_date": ex - pd.Timedelta(days=10),
|
||||
"cash_div_tax": 1.0,
|
||||
}])}
|
||||
px_by_sym = {"X.SH": px}
|
||||
ctx = {"px_by_sym": px_by_sym, "events": events,
|
||||
"yield_by_sym": build_yield_series(px_by_sym, events)}
|
||||
pos = {"X.SH": Position(symbol="X.SH", quantity=1000.0, avg_cost=20.0,
|
||||
first_buy_date=day - timedelta(days=100),
|
||||
last_buy_date=day - timedelta(days=100),
|
||||
cost_basis=20000.0)}
|
||||
return eng, day, ctx, pos
|
||||
|
||||
def test_high_percentile_triggers_buy_when_not_held(self) -> None:
|
||||
eng, day, ctx, _ = self._setup("hold")
|
||||
sigs = eng._evaluate(day, 1e6, {}, {"X.SH"}, ctx)
|
||||
assert [x.kind for x in sigs] == ["BUY"], sigs
|
||||
assert sigs[0].reason["in_universe"] is True
|
||||
|
||||
def test_in_pool_holding_can_add(self) -> None:
|
||||
eng, day, ctx, pos = self._setup("hold")
|
||||
sigs = eng._evaluate(day, 1e6, pos, {"X.SH"}, ctx)
|
||||
assert [x.kind for x in sigs] == ["ADD"], sigs
|
||||
|
||||
def test_out_of_pool_holding_blocks_add_but_not_liquidate(self) -> None:
|
||||
"""``hold``：掉出池子 → 停止加仓，但**不清仓**（留 HOLD 记录可追溯）。"""
|
||||
eng, day, ctx, pos = self._setup("hold")
|
||||
sigs = eng._evaluate(day, 1e6, pos, set(), ctx)
|
||||
assert [x.kind for x in sigs] == ["HOLD"], sigs
|
||||
assert sigs[0].reason["skip_reason"] == "OUT_OF_UNIVERSE"
|
||||
assert sigs[0].reason["executed"] is False
|
||||
assert sigs[0].reason["in_universe"] is False
|
||||
# 不进入待成交队列 ⇒ 不会被 _execute 清算
|
||||
assert sigs[0].kind not in {"BUY", "ADD", "SELL", "TRIM"}
|
||||
assert pos["X.SH"].quantity == 1000.0
|
||||
|
||||
def test_out_of_pool_holding_liquidates_when_configured(self) -> None:
|
||||
"""``sell``：掉出池子即清仓（可选项，不是默认）。"""
|
||||
eng, day, ctx, pos = self._setup("sell")
|
||||
sigs = eng._evaluate(day, 1e6, pos, set(), ctx)
|
||||
assert [x.kind for x in sigs] == ["SELL"], sigs
|
||||
assert sigs[0].target_weight == 0.0
|
||||
|
||||
def test_pool_exit_sell_branch_is_exercised_by_both_actions(self) -> None:
|
||||
"""两种配置必须给出**不同**的动作，否则 pool_exit_action 是死配置。"""
|
||||
a, day, ctx, pos_a = self._setup("hold")
|
||||
b, _, ctx_b, pos_b = self._setup("sell")
|
||||
ka = [x.kind for x in a._evaluate(day, 1e6, pos_a, set(), ctx)]
|
||||
kb = [x.kind for x in b._evaluate(day, 1e6, pos_b, set(), ctx_b)]
|
||||
assert ka != kb, f"pool_exit_action 未生效：{ka} == {kb}"
|
||||
|
||||
|
||||
class TestMemberRows:
|
||||
def _screen(self, symbols: list[str]) -> object:
|
||||
from hdiv.universe.daily import ScreenDay
|
||||
|
||||
m = pd.DataFrame({
|
||||
"symbol": symbols,
|
||||
"name": ["A", "B", "C"][: len(symbols)],
|
||||
"industry": ["银行"] * len(symbols),
|
||||
"dividend_yield": [0.06, 0.05, 0.04][: len(symbols)],
|
||||
"total_mv": [1e11, 2e11, 3e11][: len(symbols)],
|
||||
"roe_avg": [0.12, 0.11, 0.10][: len(symbols)],
|
||||
})
|
||||
return ScreenDay(
|
||||
trade_date=date(2024, 3, 1),
|
||||
candidate_count=5000,
|
||||
member_count=len(symbols),
|
||||
symbols=symbols,
|
||||
members=m,
|
||||
stats={"market": 10},
|
||||
)
|
||||
|
||||
def test_rows_shape(self) -> None:
|
||||
from datetime import datetime
|
||||
|
||||
from hdiv.universe.daily import DailyUniverseScreener
|
||||
|
||||
rows = DailyUniverseScreener.member_rows(
|
||||
"run1", [self._screen(["600036.SH", "601398.SH"])],
|
||||
created_at=datetime(2024, 3, 1, 15, 0, 0),
|
||||
)
|
||||
assert len(rows) == 2
|
||||
r = rows[0]
|
||||
assert r["run_id"] == "run1"
|
||||
assert r["trade_date"] == date(2024, 3, 1)
|
||||
assert r["symbol"] == "600036.SH"
|
||||
assert r["candidate_count"] == 5000
|
||||
assert r["dividend_yield"] == pytest.approx(0.06)
|
||||
# values_json 必须是可解析的 JSON（供「为什么是这只」复核）
|
||||
import json
|
||||
|
||||
v = json.loads(r["values_json"])
|
||||
assert v["dividend_yield"] == pytest.approx(0.06)
|
||||
assert v["roe_avg"] == pytest.approx(0.12)
|
||||
|
||||
def test_empty_members_yield_no_rows(self) -> None:
|
||||
from datetime import datetime
|
||||
|
||||
from hdiv.universe.daily import DailyUniverseScreener, ScreenDay
|
||||
|
||||
s = ScreenDay(date(2024, 3, 1), 5000, 0, [], pd.DataFrame())
|
||||
assert DailyUniverseScreener.member_rows(
|
||||
"r", [s], created_at=datetime(2024, 3, 1)
|
||||
) == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DB：PitRepo 与 Repo 逐值一致
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _load_pit(start: date, end: date, ref_end: date | None = None):
|
||||
"""载入一个 PitRepo（库不可用时 skip）。
|
||||
|
||||
**载入很贵**（参照数据约 17 秒 + 区块约 25 秒），因此这些用例共享
|
||||
module 级 fixture，而不是各自新建一个 —— 否则整个文件的耗时是分钟级的倍数。
|
||||
"""
|
||||
from hdiv.universe.pit import PitRepo
|
||||
|
||||
try:
|
||||
pit = PitRepo()
|
||||
pit.load_reference(end=ref_end or end)
|
||||
pit.load_range(start, end)
|
||||
except Exception as exc: # pragma: no cover - 环境相关
|
||||
pytest.skip(f"数据库不可用：{exc}")
|
||||
return pit
|
||||
|
||||
|
||||
#: 等价性 / 剪枝用例共用的区间（覆盖 2024 上半年，含年报季）
|
||||
_RANGE_START, _RANGE_END = date(2024, 1, 1), date(2024, 6, 28)
|
||||
_ASOF = date(2024, 6, 3)
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def pit_2024():
|
||||
"""共享的 PitRepo（2024 上半年）。用例只读，唯一例外是剪枝用例会改候选范围。"""
|
||||
from hdiv.universe.pit import PitRepo
|
||||
|
||||
try:
|
||||
pit = PitRepo()
|
||||
pit.load_reference(end=date(2024, 12, 31))
|
||||
pit.load_range(_RANGE_START, _RANGE_END)
|
||||
except Exception as exc: # pragma: no cover
|
||||
pytest.skip(f"数据库不可用：{exc}")
|
||||
yield pit
|
||||
pit.release_range()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def plain_repo():
|
||||
from hdiv.data.repo import Repo
|
||||
|
||||
try:
|
||||
repo = Repo()
|
||||
repo.trading_day(_ASOF) # 探活
|
||||
except Exception as exc: # pragma: no cover
|
||||
pytest.skip(f"数据库不可用：{exc}")
|
||||
return repo
|
||||
|
||||
|
||||
def _assert_same_multiset(name: str, a: pd.DataFrame, b: pd.DataFrame) -> None:
|
||||
"""按**全列排序**后逐值比较（行序不同不代表内容不同）。
|
||||
|
||||
用全部列排序而不是挑几列：财务表里同一 (symbol, 财年) 可能有多条重述公告，
|
||||
只按业务键排序会让等值行两两错配，产生假报警。
|
||||
"""
|
||||
assert list(a.columns) == list(b.columns), (
|
||||
f"{name} 列集合不一致：{list(a.columns)} vs {list(b.columns)}"
|
||||
)
|
||||
if a.empty and b.empty:
|
||||
return
|
||||
cols = list(a.columns)
|
||||
x = a.sort_values(cols, ignore_index=True)
|
||||
y = b.sort_values(cols, ignore_index=True)
|
||||
assert x.shape == y.shape, f"{name} 行数不一致：{x.shape} vs {y.shape}"
|
||||
pd.testing.assert_frame_equal(
|
||||
x, y, check_dtype=False, rtol=1e-9, atol=1e-12,
|
||||
obj=f"{name} 与直连口径不一致",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_pit_repo_matches_direct_repo(pit_2024, plain_repo) -> None:
|
||||
"""PitRepo 的每一个被覆盖的方法都必须与直连 Repo 逐值一致。"""
|
||||
asof = _ASOF
|
||||
pit, plain = pit_2024, plain_repo
|
||||
|
||||
_assert_same_multiset(
|
||||
"market_panel(lookback=5)",
|
||||
pit.market_panel(asof, lookback_days=5),
|
||||
plain.market_panel(asof, lookback_days=5),
|
||||
)
|
||||
_assert_same_multiset(
|
||||
"market_panel(lookback=0)",
|
||||
pit.market_panel(asof),
|
||||
plain.market_panel(asof),
|
||||
)
|
||||
_assert_same_multiset(
|
||||
"avg_amount(20)",
|
||||
pit.avg_amount(asof, window=20),
|
||||
plain.avg_amount(asof, window=20),
|
||||
)
|
||||
syms = ["600036.SH", "000651.SZ"]
|
||||
_assert_same_multiset(
|
||||
"avg_amount(20, symbols)",
|
||||
pit.avg_amount(asof, window=20, symbols=syms),
|
||||
plain.avg_amount(asof, window=20, symbols=syms),
|
||||
)
|
||||
_assert_same_multiset(
|
||||
"financial_panel",
|
||||
pit.financial_panel(asof),
|
||||
plain.financial_panel(asof),
|
||||
)
|
||||
_assert_same_multiset(
|
||||
"annual_financial_history(5)",
|
||||
pit.annual_financial_history(asof, years=5),
|
||||
plain.annual_financial_history(asof, years=5),
|
||||
)
|
||||
_assert_same_multiset(
|
||||
"annual_financial_history(11)",
|
||||
pit.annual_financial_history(asof, years=11),
|
||||
plain.annual_financial_history(asof, years=11),
|
||||
)
|
||||
_assert_same_multiset(
|
||||
"annual_financial_averages(5)",
|
||||
pit.annual_financial_averages(asof, years=5),
|
||||
plain.annual_financial_averages(asof, years=5),
|
||||
)
|
||||
_assert_same_multiset(
|
||||
"annual_financials(10)",
|
||||
pit.annual_financials(asof, years=10),
|
||||
plain.annual_financials(asof, years=10),
|
||||
)
|
||||
_assert_same_multiset(
|
||||
"dividend_records(8)",
|
||||
pit.dividend_records(asof, years_back=8),
|
||||
plain.dividend_records(asof, years_back=8),
|
||||
)
|
||||
_assert_same_multiset(
|
||||
"dividend_records(all)",
|
||||
pit.dividend_records(asof, years_back=8, implemented_only=False),
|
||||
plain.dividend_records(asof, years_back=8, implemented_only=False),
|
||||
)
|
||||
assert pit.trading_day(asof) == plain.trading_day(asof)
|
||||
assert pit.prev_trading_day(asof) == plain.prev_trading_day(asof)
|
||||
assert len(pit.trading_days(date(2024, 1, 1), asof)) == len(
|
||||
plain.trading_days(date(2024, 1, 1), asof)
|
||||
)
|
||||
assert pit.suspended_on(asof) == plain.suspended_on(asof)
|
||||
assert pit.st_symbols(asof) == plain.st_symbols(asof)
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_pit_repo_symbols_subset_is_exact(pit_2024, plain_repo) -> None:
|
||||
"""``symbols`` 过滤只是全市场结果取子集：两者必须一致（含缓存命中路径）。"""
|
||||
asof, pit, plain = _ASOF, pit_2024, plain_repo
|
||||
syms = ["600036.SH", "601398.SH", "000651.SZ"]
|
||||
|
||||
full = pit.financial_panel(asof)
|
||||
sub = pit.financial_panel(asof, symbols=syms)
|
||||
_assert_same_multiset(
|
||||
"financial_panel(symbols) 子集",
|
||||
sub, full[full["symbol"].isin(syms)].reset_index(drop=True),
|
||||
)
|
||||
_assert_same_multiset(
|
||||
"financial_panel(symbols) vs 直连",
|
||||
sub, plain.financial_panel(asof, symbols=syms),
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_pit_repo_caches_are_exact() -> None:
|
||||
"""可见性缓存必须**精确**：命中与不命中给出同一结果。
|
||||
|
||||
缓存键是「已公告财报条数」。这里把缓存清空强制重算一次，与命中结果比对 ——
|
||||
若不相等，说明键不是充分统计量（那就会静默用错的面板做决策）。
|
||||
"""
|
||||
asof = date(2024, 4, 25) # 年报季，可见集合天天变
|
||||
pit = _load_pit(date(2024, 3, 1), date(2024, 6, 28), ref_end=date(2024, 12, 31))
|
||||
hit = pit.financial_panel(asof)
|
||||
pit._memo.clear()
|
||||
cold = pit.financial_panel(asof)
|
||||
_assert_same_multiset("financial_panel 缓存命中 vs 冷算", hit, cold)
|
||||
|
||||
h1 = pit.annual_financial_history(asof, years=11)
|
||||
pit._memo.clear()
|
||||
h2 = pit.annual_financial_history(asof, years=11)
|
||||
_assert_same_multiset("annual_financial_history 命中 vs 冷算", h1, h2)
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_pit_repo_out_of_range_raises() -> None:
|
||||
"""区间外必须**报错**，不能返回空表 —— 空表会被当成「当天没有股票」。"""
|
||||
from hdiv.core.errors import HdivError
|
||||
|
||||
pit = _load_pit(date(2024, 5, 1), date(2024, 6, 28))
|
||||
with pytest.raises(HdivError):
|
||||
pit.market_panel(date(2019, 1, 2), lookback_days=5)
|
||||
with pytest.raises(HdivError):
|
||||
pit.avg_amount(date(2019, 1, 2), window=20) # 全市场查询必须报错
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_pit_repo_avg_amount_falls_back_for_narrow_query(plain_repo) -> None:
|
||||
"""区块释放后，**带 symbol 的**均额查询仍可回答（画像路径依赖它）。"""
|
||||
pit = _load_pit(date(2024, 5, 1), date(2024, 6, 28))
|
||||
asof = date(2024, 6, 3)
|
||||
pit.release_range()
|
||||
got = pit.avg_amount(asof, window=20, symbols=["600036.SH"])
|
||||
want = plain_repo.avg_amount(asof, window=20, symbols=["600036.SH"])
|
||||
_assert_same_multiset("avg_amount 窄查询回退", got, want)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DB：预剪枝等价性
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_prune_does_not_change_selection(pit_2024) -> None:
|
||||
"""开/关预剪枝必须选出完全相同的成员（否则它是近似，不是等价）。"""
|
||||
from hdiv.universe.selector import UniverseSelector
|
||||
|
||||
pit = pit_2024
|
||||
start, end = _RANGE_START, _RANGE_END
|
||||
cfg = load_config("universe")
|
||||
sel = UniverseSelector(cfg, repo=pit)
|
||||
|
||||
# 几天样本：月初、季报期、年报季、月末
|
||||
wanted = {date(2024, 1, 2), date(2024, 3, 1), date(2024, 4, 25),
|
||||
date(2024, 6, 3), date(2024, 6, 28)}
|
||||
days = [d for d in pit.trading_days(start, end) if d in wanted]
|
||||
assert days, "样本交易日为空"
|
||||
|
||||
# 先不剪枝
|
||||
pit.set_candidate_scope(None)
|
||||
base = {d: set(sel.run(asof=d, persist=False, verbose=False)["selected"]["symbol"])
|
||||
for d in days}
|
||||
|
||||
# 再剪枝
|
||||
DailyUniverseScreener(cfg, pit, verbose=False).build_prune_set(start, end)
|
||||
pruned = {d: set(sel.run(asof=d, persist=False, verbose=False)["selected"]["symbol"])
|
||||
for d in days}
|
||||
pit.set_candidate_scope(None) # 复位，避免影响其它用例
|
||||
|
||||
for d in days:
|
||||
assert base[d] == pruned[d], (
|
||||
f"{d} 预剪枝改变了最终入选："
|
||||
f"多出 {sorted(pruned[d] - base[d])}，丢失 {sorted(base[d] - pruned[d])}"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_prune_set_keeps_every_actual_member(pit_2024) -> None:
|
||||
"""预剪枝的保留集合必须**包含**每天实际选出的成员（上界论证的实证）。"""
|
||||
from hdiv.universe.selector import UniverseSelector
|
||||
|
||||
pit = pit_2024
|
||||
start, end = _RANGE_START, _RANGE_END
|
||||
cfg = load_config("universe")
|
||||
allowed = DailyUniverseScreener(cfg, pit, verbose=False).build_prune_set(start, end)
|
||||
pit.set_candidate_scope(None) # 不剪枝地真筛一次
|
||||
sel = UniverseSelector(cfg, repo=pit)
|
||||
for d in [date(2024, 3, 1), date(2024, 6, 3)]:
|
||||
got = set(sel.run(asof=d, persist=False, verbose=False)["selected"]["symbol"])
|
||||
assert got <= allowed, f"{d} 有成员被预剪枝误剔：{sorted(got - allowed)}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DB：PIT 纪律 —— 股票池不随「回测终点」变化
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_daily_pool_is_point_in_time() -> None:
|
||||
"""某一天的股票池只依赖该日及之前的数据，与回测终点无关。
|
||||
|
||||
这是「无未来函数」在每日选股上的直接检验：把参照数据的终点推后一年，
|
||||
同一天的选股结果必须**逐只相同**。
|
||||
"""
|
||||
from hdiv.universe.selector import UniverseSelector
|
||||
|
||||
asof = date(2023, 6, 1)
|
||||
# 只载入一个月，控制成本（这条检验比的是「两个终点是否给出同一答案」）
|
||||
a = _load_pit(date(2023, 5, 1), asof, ref_end=asof)
|
||||
b = _load_pit(date(2023, 5, 1), asof, ref_end=asof + timedelta(days=365))
|
||||
cfg = load_config("universe")
|
||||
ra = UniverseSelector(cfg, repo=a).run(asof=asof, persist=False, verbose=False)
|
||||
rb = UniverseSelector(cfg, repo=b).run(asof=asof, persist=False, verbose=False)
|
||||
assert set(ra["selected"]["symbol"]) == set(rb["selected"]["symbol"])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DB：端到端（短区间）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# 回测 run 的从属表（顺序 = 删除顺序：先子后父）
|
||||
_BACKTEST_CHILD_TABLES = (
|
||||
"hd_daily_universe",
|
||||
"hd_backtest_equity",
|
||||
"hd_backtest_metric",
|
||||
"hd_backtest_position",
|
||||
"hd_backtest_signal",
|
||||
"hd_backtest_trade",
|
||||
)
|
||||
|
||||
|
||||
def _purge_daily_run(run_id: str) -> None:
|
||||
"""物理删除本次端到端测试自己写入的 run 及其从属行。
|
||||
|
||||
**测试不得在分析库里留垃圾**：每跑一次都会多出一条同名回测记录（run_id 指纹
|
||||
含 ``datetime.now()``，见 ``hdiv.backtest.engine``），跑几十次后前端「回测
|
||||
记录」就被测试产物淹掉，而它对外看起来和真实回测没有区别。
|
||||
|
||||
项目的 ``StatementGuard`` 有意禁止 DELETE（只增不删），所以这里自建一个
|
||||
**不装守卫**的连接，并严格按 ``run_id`` 精确回收刚刚写入的行。
|
||||
"""
|
||||
from sqlalchemy import create_engine, text
|
||||
|
||||
from hdiv.data import db
|
||||
|
||||
cfg = load_config("datasource")
|
||||
db.load_dotenv_once()
|
||||
engine = create_engine(db.build_url(cfg), pool_pre_ping=True, future=True)
|
||||
try:
|
||||
with engine.begin() as conn:
|
||||
for table in (*_BACKTEST_CHILD_TABLES, "hd_backtest_run"):
|
||||
conn.execute(text(f"DELETE FROM {table} WHERE run_id = :r"),
|
||||
{"r": run_id})
|
||||
finally:
|
||||
engine.dispose()
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_daily_run_persists_members_and_run() -> None:
|
||||
"""daily 端到端：落库 run（mode=daily）与每日选股成员（跑完自清理）。"""
|
||||
from hdiv.backtest.daily import DailyRunner
|
||||
from hdiv.data import db
|
||||
|
||||
try:
|
||||
runner = DailyRunner.from_strategy("config/strategy/high_dividend_v1.yml")
|
||||
except Exception as exc: # pragma: no cover
|
||||
pytest.skip(f"数据库不可用：{exc}")
|
||||
start, end = date(2024, 3, 1), date(2024, 3, 15)
|
||||
try:
|
||||
res = runner.run(start=start, end=end, persist=True, verbose=False)
|
||||
except Exception as exc: # pragma: no cover
|
||||
pytest.skip(f"数据不足，跳过端到端：{exc}")
|
||||
|
||||
run_id = res["run_id"]
|
||||
try:
|
||||
assert res["mode"] == "daily"
|
||||
cfg = load_config("datasource")
|
||||
assert db.table_exists("hd_daily_universe", cfg)
|
||||
got = db.read_sql(
|
||||
"SELECT COUNT(*) AS n, COUNT(DISTINCT trade_date) AS d "
|
||||
"FROM hd_daily_universe WHERE run_id = :r",
|
||||
{"r": run_id}, cfg=cfg,
|
||||
)
|
||||
assert int(got["n"].iloc[0]) > 0, "未落库任何每日选股成员"
|
||||
assert int(got["d"].iloc[0]) >= 5, "落库的决策时点太少"
|
||||
run = db.read_sql(
|
||||
"SELECT mode, universe_run_id FROM hd_backtest_run WHERE run_id = :r",
|
||||
{"r": run_id}, cfg=cfg,
|
||||
)
|
||||
assert len(run) == 1
|
||||
assert run["mode"].iloc[0] == "daily"
|
||||
# daily 是动态池：不得关联任何冻结股票池
|
||||
assert pd.isna(run["universe_run_id"].iloc[0]) or run["universe_run_id"].iloc[0] is None
|
||||
finally:
|
||||
# 无论断言成功还是失败，都不能把测试产物留在分析库里
|
||||
_purge_daily_run(run_id)
|
||||
@@ -25,8 +25,8 @@ _PLAIN_TESTS_NEED_DB = pytest.mark.db
|
||||
|
||||
|
||||
def test_table_count() -> None:
|
||||
assert len(ALL_TABLES) == 30
|
||||
assert len(set(TABLE_NAMES)) == 30
|
||||
assert len(ALL_TABLES) == 31
|
||||
assert len(set(TABLE_NAMES)) == 31
|
||||
|
||||
|
||||
def test_all_tables_use_own_prefix() -> None:
|
||||
|
||||
@@ -0,0 +1,449 @@
|
||||
"""每日增量同步（``hdiv sync daily``）测试。
|
||||
|
||||
要锁定的三条性质：
|
||||
|
||||
1. **缺口口径与断点续传一致** —— 判定「这天已同步」用的是 ``price.fetched_days``
|
||||
的按年份规模阈值，而不是「当天有没有行」。否则只填了几百只的半成品日会被
|
||||
当成已完成，形成难以察觉的数据空洞（历史上真的踩过这个坑）。
|
||||
|
||||
2. **只抓缺口，不重拉** —— 计划里出现的交易日必须**恰好**是缺失的那些；
|
||||
已完整的日、已最新的指数、已有数据的股票都不能出现在待抓清单里。
|
||||
|
||||
3. **财报水位按披露截止日推算** —— 年报/一季报 4-30、半年报 8-31、三季报 10-31。
|
||||
水位算错会让财报季的队列要么永远排不空、要么漏掉整季新披露。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import plistlib
|
||||
from datetime import date
|
||||
|
||||
import pytest
|
||||
|
||||
from hdiv.core.paths import project_root
|
||||
from hdiv.data.sync import daily as daily_sync
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 财报报告期水位
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestFinancialWatermark:
|
||||
@pytest.mark.parametrize(
|
||||
"asof,expected",
|
||||
[
|
||||
# 三季报 10-31 截止 → 11-01 起水位是当年三季报
|
||||
(date(2026, 11, 1), date(2026, 9, 30)),
|
||||
(date(2026, 12, 31), date(2026, 9, 30)),
|
||||
# 半年报 8-31 截止 → 9-01 起水位是当年半年报
|
||||
(date(2026, 9, 1), date(2026, 6, 30)),
|
||||
(date(2026, 10, 31), date(2026, 6, 30)),
|
||||
# 一季报 4-30 截止 → 5-01 起水位是当年一季报
|
||||
(date(2026, 5, 1), date(2026, 3, 31)),
|
||||
(date(2026, 8, 31), date(2026, 3, 31)),
|
||||
# 1~4 月是上一年年报季
|
||||
(date(2026, 4, 30), date(2025, 12, 31)),
|
||||
(date(2026, 1, 1), date(2025, 12, 31)),
|
||||
],
|
||||
)
|
||||
def test_watermark(self, asof: date, expected: date) -> None:
|
||||
assert daily_sync.financial_watermark(asof) == expected
|
||||
|
||||
def test_watermark_boundaries_are_monotonic(self) -> None:
|
||||
"""水位只能随时间前进，不能回退 —— 否则已补的股票会反复进队列。"""
|
||||
days = [date(2025, m, d) for m in range(1, 13) for d in (1, 15, 28)]
|
||||
days += [date(2026, m, d) for m in range(1, 13) for d in (1, 15, 28)]
|
||||
wm = [daily_sync.financial_watermark(d) for d in sorted(days)]
|
||||
assert wm == sorted(wm)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 目标选择
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestResolveOnly:
|
||||
def test_default_is_all_groups(self) -> None:
|
||||
assert daily_sync._resolve_only(None) == set(daily_sync.GROUPS)
|
||||
assert daily_sync._resolve_only([]) == set(daily_sync.GROUPS)
|
||||
|
||||
def test_group_names(self) -> None:
|
||||
assert daily_sync._resolve_only(["price"]) == {"price"}
|
||||
assert daily_sync._resolve_only(["price", "dividend"]) == {"price", "dividend"}
|
||||
|
||||
def test_single_target_maps_to_its_group(self) -> None:
|
||||
assert daily_sync._resolve_only(["daily_basic"]) == {"price"}
|
||||
assert daily_sync._resolve_only(["suspend"]) == {"trading"}
|
||||
assert daily_sync._resolve_only(["limit"]) == {"trading"}
|
||||
assert daily_sync._resolve_only(["index_weight"]) == {"index"}
|
||||
|
||||
def test_unknown_target_is_rejected(self) -> None:
|
||||
with pytest.raises(ValueError, match="未知的同步目标"):
|
||||
daily_sync._resolve_only(["nope"])
|
||||
|
||||
def test_blank_entries_are_ignored(self) -> None:
|
||||
assert daily_sync._resolve_only(["", " ", "index"]) == {"index"}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 缺口判定（隔离数据库）
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMissingDays:
|
||||
def test_only_incomplete_days_are_returned(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""4 个交易日里只有 2 天完整 → 待抓恰好是另外 2 天。"""
|
||||
days = [date(2026, 9, 1), date(2026, 9, 2), date(2026, 9, 3), date(2026, 9, 4)]
|
||||
monkeypatch.setattr(daily_sync, "_trading_days", lambda s, e, cfg: days)
|
||||
monkeypatch.setattr(
|
||||
daily_sync.price_sync, "fetched_days", lambda *a, **k: {days[0], days[2]}
|
||||
)
|
||||
out = daily_sync.missing_days("stock_daily", days[0], days[-1], cfg=None)
|
||||
assert out == [days[1], days[3]]
|
||||
|
||||
def test_min_symbols_is_forwarded(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""停牌表必须按「有行即算」传 min_symbols=1，否则每天都判为未完成。"""
|
||||
seen: dict = {}
|
||||
|
||||
def fake_fetched(table, start, end, cfg, *, min_symbols=None): # noqa: ANN001
|
||||
seen["min_symbols"] = min_symbols
|
||||
return set()
|
||||
|
||||
monkeypatch.setattr(daily_sync, "_trading_days", lambda s, e, cfg: [date(2026, 9, 1)])
|
||||
monkeypatch.setattr(daily_sync.price_sync, "fetched_days", fake_fetched)
|
||||
daily_sync.missing_days("hd_suspend", date(2026, 9, 1), date(2026, 9, 1), cfg=None, min_symbols=1)
|
||||
assert seen["min_symbols"] == 1
|
||||
|
||||
def test_no_trading_days_means_no_gap(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(daily_sync, "_trading_days", lambda s, e, cfg: [])
|
||||
assert daily_sync.missing_days("stock_daily", date(2026, 10, 1), date(2026, 10, 5), cfg=None) == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 分红按日期补
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDividendGaps:
|
||||
def test_future_ex_date_does_not_push_window_forward(
|
||||
self, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""已公告但尚未除权的记录会把 MAX(ex_date) 推到未来。
|
||||
|
||||
若直接拿它当锚点，窗口会落在未来、一个交易日都查不到 ——
|
||||
新公告的分红就永远补不进来。锚点必须被夹到 asof。
|
||||
"""
|
||||
import pandas as pd
|
||||
|
||||
asof = date(2026, 10, 5)
|
||||
calls: list[tuple] = []
|
||||
|
||||
def fake_read_sql(sql, params=None, *, cfg=None): # noqa: ANN001
|
||||
if "MAX(ann_date)" in sql:
|
||||
return pd.DataFrame({"m": [date(2026, 10, 23)]}) # 未来除权日
|
||||
calls.append((params["s"], params["e"]))
|
||||
return pd.DataFrame({"d": []})
|
||||
|
||||
monkeypatch.setattr(daily_sync.db, "read_sql", fake_read_sql)
|
||||
monkeypatch.setattr(
|
||||
daily_sync,
|
||||
"_trading_days",
|
||||
lambda s, e, cfg: [date(2026, 9, 28), date(2026, 9, 29), date(2026, 9, 30)],
|
||||
)
|
||||
days, anchor = daily_sync.dividend_gaps(date(2026, 8, 21), asof, asof, cfg=None)
|
||||
assert anchor == asof
|
||||
assert days == [date(2026, 9, 28), date(2026, 9, 29), date(2026, 9, 30)]
|
||||
start, _ = calls[0]
|
||||
assert start == date(2026, 9, 28), "窗口起点应是 asof 往前 DIVIDEND_OVERLAP_DAYS 天"
|
||||
|
||||
def test_days_already_queried_are_not_repeated(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
import pandas as pd
|
||||
|
||||
asof = date(2026, 10, 5)
|
||||
seen_day = date(2026, 9, 30)
|
||||
|
||||
def fake_read_sql(sql, params=None, *, cfg=None): # noqa: ANN001
|
||||
if "MAX(ann_date)" in sql:
|
||||
return pd.DataFrame({"m": [seen_day]})
|
||||
return pd.DataFrame({"d": [seen_day]})
|
||||
|
||||
monkeypatch.setattr(daily_sync.db, "read_sql", fake_read_sql)
|
||||
monkeypatch.setattr(
|
||||
daily_sync, "_trading_days", lambda s, e, cfg: [seen_day, date(2026, 10, 1)]
|
||||
)
|
||||
days, _ = daily_sync.dividend_gaps(date(2026, 8, 21), asof, asof, cfg=None)
|
||||
assert days == [date(2026, 10, 1)]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 指数行情 / 成分权重窗口
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestIndexGaps:
|
||||
def test_per_index_gap_starts_after_its_last_day(
|
||||
self, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
import pandas as pd
|
||||
|
||||
days = [date(2026, 9, 29), date(2026, 9, 30)]
|
||||
monkeypatch.setattr(daily_sync, "_trading_days", lambda s, e, cfg: days)
|
||||
monkeypatch.setattr(
|
||||
daily_sync.index_sync,
|
||||
"default_indices",
|
||||
lambda cfg: [{"code": "000300.SH", "name": "沪深300"}, {"code": "399006.SZ"}],
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
daily_sync.db,
|
||||
"read_sql",
|
||||
lambda *a, **k: pd.DataFrame(
|
||||
{"c": ["000300.SH"], "m": [date(2026, 9, 29)]}
|
||||
),
|
||||
)
|
||||
out = daily_sync.index_daily_gaps(date(2026, 9, 1), date(2026, 9, 30), cfg=None)
|
||||
assert out == {"000300.SH": [date(2026, 9, 30)], "399006.SZ": days}
|
||||
|
||||
def test_up_to_date_table_has_no_window(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
import pandas as pd
|
||||
|
||||
monkeypatch.setattr(
|
||||
daily_sync, "_trading_days", lambda s, e, cfg: [date(2026, 9, 29), date(2026, 9, 30)]
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
daily_sync.db, "read_sql", lambda *a, **k: pd.DataFrame({"m": [date(2026, 9, 30)]})
|
||||
)
|
||||
assert daily_sync._index_weight_window(date(2026, 8, 21), date(2026, 9, 30), None) is None
|
||||
|
||||
def test_window_end_is_clamped_to_last_trading_day(
|
||||
self, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""右端点收到最近交易日：否则长假里每天都会重查一段空区间。"""
|
||||
import pandas as pd
|
||||
|
||||
monkeypatch.setattr(
|
||||
daily_sync, "_trading_days", lambda s, e, cfg: [date(2026, 9, 29), date(2026, 9, 30)]
|
||||
)
|
||||
monkeypatch.setattr(daily_sync.db, "read_sql", lambda *a, **k: pd.DataFrame({"m": [None]}))
|
||||
got = daily_sync._index_weight_window(date(2026, 8, 21), date(2026, 10, 5), None)
|
||||
assert got == (date(2026, 8, 21), date(2026, 9, 30))
|
||||
|
||||
def test_holiday_only_window_is_skipped(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""整段区间都是假期 → 没有可补的权重日，一次调用都不该发。"""
|
||||
monkeypatch.setattr(daily_sync, "_trading_days", lambda s, e, cfg: [])
|
||||
assert daily_sync._index_weight_window(date(2026, 10, 1), date(2026, 10, 5), None) is None
|
||||
|
||||
def test_incremental_window_starts_after_last_weight(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
import pandas as pd
|
||||
|
||||
monkeypatch.setattr(daily_sync, "_trading_days", lambda s, e, cfg: [date(2026, 9, 30)])
|
||||
monkeypatch.setattr(
|
||||
daily_sync.db, "read_sql", lambda *a, **k: pd.DataFrame({"m": [date(2026, 8, 31)]})
|
||||
)
|
||||
got = daily_sync._index_weight_window(date(2026, 8, 21), date(2026, 10, 5), None)
|
||||
assert got == (date(2026, 9, 1), date(2026, 9, 30))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 计划对象与摘要
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPlanObject:
|
||||
def test_empty_plan_detects_nothing_to_do(self) -> None:
|
||||
p = daily_sync.DailyPlan(asof=date(2026, 10, 5), window_start=date(2026, 8, 21))
|
||||
assert p.empty
|
||||
assert p.total_days == 0
|
||||
|
||||
def test_any_gap_makes_plan_non_empty(self) -> None:
|
||||
p = daily_sync.DailyPlan(
|
||||
asof=date(2026, 10, 5),
|
||||
window_start=date(2026, 8, 21),
|
||||
day_gaps={"daily": [date(2026, 9, 30)]},
|
||||
)
|
||||
assert not p.empty
|
||||
assert p.total_days == 1
|
||||
|
||||
def test_plan_does_not_share_default_dicts(self) -> None:
|
||||
"""两个计划的默认容器必须互相独立（dataclass 默认值的经典坑）。"""
|
||||
a = daily_sync.DailyPlan(asof=date(2026, 10, 5), window_start=date(2026, 8, 21))
|
||||
b = daily_sync.DailyPlan(asof=date(2026, 10, 5), window_start=date(2026, 8, 21))
|
||||
a.day_gaps["daily"] = [date(2026, 9, 30)]
|
||||
a.notes.append("x")
|
||||
assert b.day_gaps == {}
|
||||
assert b.notes == []
|
||||
|
||||
|
||||
class TestFormatSummary:
|
||||
def test_success(self) -> None:
|
||||
assert "成功" in daily_sync.format_summary({"targets": {}, "errors": [], "ok": True})
|
||||
|
||||
def test_failure_lists_reasons(self) -> None:
|
||||
s = daily_sync.format_summary({"errors": ["daily: SyncError: 限频"], "ok": False})
|
||||
assert "失败 1 项" in s
|
||||
assert "限频" in s
|
||||
|
||||
def test_dry_run_reports_plan(self) -> None:
|
||||
s = daily_sync.format_summary(
|
||||
{
|
||||
"dry_run": True,
|
||||
"planned": {"day_gaps": {"daily": 3}, "index_days": {"000300.SH": 2},
|
||||
"dividend_days": 1},
|
||||
}
|
||||
)
|
||||
assert "计划" in s
|
||||
assert "5" in s
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 部署契约：每天 17:00
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestScheduleContract:
|
||||
"""「每天下午 5 点定时抓取」是本次的硬需求，必须有可执行的证据。
|
||||
|
||||
定时任务最容易在交付后悄悄失效：plist 写成 KeepAlive 会把一次性任务变成
|
||||
常驻进程、小时写错会让它在收盘前跑到空数据。这两点用测试钉死。
|
||||
"""
|
||||
|
||||
def test_plist_triggers_at_17_00(self) -> None:
|
||||
path = project_root() / "deploy" / "com.hddiv.sync.plist.example"
|
||||
assert path.is_file(), "缺少 launchd 模板"
|
||||
data = plistlib.loads(path.read_bytes())
|
||||
assert data["Label"] == "com.hddiv.sync"
|
||||
assert data["StartCalendarInterval"] == {"Hour": 17, "Minute": 0}, (
|
||||
"触发时间必须是 17:00（收盘 15:00 后，当日数据已出）"
|
||||
)
|
||||
assert data["KeepAlive"] is False, "定时任务不能 KeepAlive —— 否则会被无限拉起"
|
||||
assert data["RunAtLoad"] is False, "RunAtLoad 会让每次登录都多抓一次"
|
||||
|
||||
def test_runner_script_is_wired(self) -> None:
|
||||
root = project_root()
|
||||
runner = root / "deploy" / "daily-sync.sh"
|
||||
assert runner.is_file(), "缺少执行包装脚本"
|
||||
assert os.access(runner, os.X_OK), "daily-sync.sh 必须可执行"
|
||||
text = runner.read_text(encoding="utf-8")
|
||||
assert "sync daily" in text, "包装脚本必须调用 hdiv sync daily"
|
||||
assert "LOCK_DIR" in text, "包装脚本必须有单实例锁（避免重叠运行互相抢限频额度）"
|
||||
installer = root / "deploy" / "install-sync-schedule.sh"
|
||||
assert installer.is_file(), "缺少安装脚本"
|
||||
itext = installer.read_text(encoding="utf-8")
|
||||
assert "com.hddiv.sync.plist.example" in itext
|
||||
assert "StartCalendarInterval" in (
|
||||
root / "deploy" / "com.hddiv.sync.plist.example"
|
||||
).read_text(encoding="utf-8")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 真实数据库：计划与执行的一致性
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_plan_matches_database_state() -> None:
|
||||
"""计划里的缺口必须与「库里实际缺的日子」逐日一致。
|
||||
|
||||
这是本模块唯一不能靠 mock 保证的性质：``fetched_days`` 的阈值口径、
|
||||
``trading_calendar`` 的交易日、各表的日期列含义都来自真实库。
|
||||
"""
|
||||
from hdiv.core.config import load_config
|
||||
from hdiv.data import db
|
||||
from hdiv.data.sync import price as price_sync
|
||||
|
||||
try:
|
||||
db.load_dotenv_once()
|
||||
cfg = load_config("datasource")
|
||||
db.row_count("stock_daily", cfg)
|
||||
except Exception as exc: # pragma: no cover
|
||||
pytest.skip(f"数据库不可用：{exc}")
|
||||
|
||||
p = daily_sync.plan(cfg, asof=date(2026, 9, 30), lookback_days=45, include_financial=False)
|
||||
# 窗口内每一天：要么在待抓清单里，要么在「已完整」集合里，二者互斥且完备
|
||||
window_days = price_sync.open_days(date(2026, 8, 16), date(2026, 9, 30), cfg)
|
||||
done = price_sync.fetched_days("stock_daily", date(2026, 8, 16), date(2026, 9, 30), cfg)
|
||||
planned = set(p.day_gaps["daily"])
|
||||
for d in window_days:
|
||||
assert (d in planned) != (d in done), f"{d} 的缺口判定与 fetched_days 不一致"
|
||||
assert planned == {d for d in window_days if d not in done}
|
||||
|
||||
# 计划不包含窗口之外的日期（增量只补近期空洞）
|
||||
assert all(d >= p.window_start for days in p.day_gaps.values() for d in days)
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_financial_queue_excludes_long_delisted() -> None:
|
||||
"""已退市多年的标的不能再进每日队列。
|
||||
|
||||
回归：实测「报告期滞后」的股票里 200+ 只早已退市（最后一份财报停在退市前），
|
||||
不过滤的话每天都要为它们发 800 次调用、拉回同一批旧数据 —— 实测一次全量
|
||||
重拉耗时约 2 分钟、返回 6.5 万行，全部无功而返。
|
||||
"""
|
||||
from hdiv.core.config import load_config
|
||||
from hdiv.data import db
|
||||
|
||||
asof = date(2026, 9, 30)
|
||||
try:
|
||||
db.load_dotenv_once()
|
||||
cfg = load_config("datasource")
|
||||
db.row_count("stock", cfg)
|
||||
except Exception as exc: # pragma: no cover
|
||||
pytest.skip(f"数据库不可用：{exc}")
|
||||
|
||||
p = daily_sync.plan(cfg, asof=asof)
|
||||
assert len(p.financial_symbols) < 50, (
|
||||
f"每日财报队列 {len(p.financial_symbols)} 只，退市过滤可能失效"
|
||||
)
|
||||
cutoff = daily_sync._active_since(asof)
|
||||
dead = db.read_sql(
|
||||
"SELECT symbol FROM stock WHERE delist_date IS NOT NULL AND delist_date <= :c",
|
||||
{"c": cutoff},
|
||||
cfg=cfg,
|
||||
)
|
||||
dead_set = set(dead["symbol"].astype(str)) if not dead.empty else set()
|
||||
assert not (set(p.financial_symbols) & dead_set), "长期退市标的混进了每日财报队列"
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_json_mode_writes_no_progress_text(capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""``--json`` 的 stdout 必须是**纯 JSON**。
|
||||
|
||||
回归：进度与计划文字混进 stdout 会让 ``| jq`` 之类的消费者直接解析失败，
|
||||
而这类问题在交互式运行时完全看不出来（人眼只看到 JSON 在前面或后面）。
|
||||
"""
|
||||
from hdiv.core.config import load_config
|
||||
from hdiv.data import db
|
||||
|
||||
try:
|
||||
db.load_dotenv_once()
|
||||
cfg = load_config("datasource")
|
||||
db.row_count("stock", cfg)
|
||||
except Exception as exc: # pragma: no cover
|
||||
pytest.skip(f"数据库不可用：{exc}")
|
||||
|
||||
summary = daily_sync.run(
|
||||
cfg, asof=date(2026, 9, 30), dry_run=True, include_financial=False, verbose=False
|
||||
)
|
||||
captured = capsys.readouterr()
|
||||
assert captured.out == "", f"verbose=False 仍有 stdout 输出：{captured.out[:200]}"
|
||||
assert summary["dry_run"] is True
|
||||
|
||||
|
||||
@pytest.mark.db
|
||||
def test_dry_run_writes_nothing() -> None:
|
||||
"""``--dry-run`` 不得写库，也不得调用 Tushare。"""
|
||||
from hdiv.core.config import load_config
|
||||
from hdiv.data import db
|
||||
|
||||
try:
|
||||
db.load_dotenv_once()
|
||||
cfg = load_config("datasource")
|
||||
logs_before = db.read_sql("SELECT MAX(id) AS m FROM hd_sync_log", cfg=cfg)["m"].iloc[0]
|
||||
except Exception as exc: # pragma: no cover
|
||||
pytest.skip(f"数据库不可用：{exc}")
|
||||
|
||||
summary = daily_sync.run(cfg, asof=date(2026, 9, 30), dry_run=True, include_financial=False)
|
||||
assert summary["dry_run"] is True
|
||||
assert summary["errors"] == []
|
||||
after_logs = db.read_sql("SELECT MAX(id) AS m FROM hd_sync_log", cfg=cfg)["m"].iloc[0]
|
||||
assert after_logs == logs_before, "dry-run 不应产生新的 hd_sync_log 记录"
|
||||
@@ -43,12 +43,16 @@ FRONTEND_CALLS: list[tuple[str, str]] = [
|
||||
# 净值曲线右轴可叠加的基准指数
|
||||
("GET", "/api/indices"),
|
||||
("GET", "/api/backtests/abc123/signals"),
|
||||
# 每日动态股票池（--mode daily）：时间线 / 某日成员
|
||||
("GET", "/api/backtests/abc123/daily-universe"),
|
||||
("PATCH", "/api/backtests/abc123"),
|
||||
# 回测内分析：任意日持仓 + 个股买卖点
|
||||
("GET", "/api/backtests/abc123/portfolio"),
|
||||
("GET", "/api/backtests/abc123/position-dates"),
|
||||
("GET", "/api/backtests/abc123/stocks"),
|
||||
("GET", "/api/backtests/abc123/stocks/600519.SH"),
|
||||
# 已清仓了结清单（含清仓后至今涨跌）
|
||||
("GET", "/api/backtests/abc123/closed-positions"),
|
||||
# Walk-forward 样本外
|
||||
("GET", "/api/walkforwards"),
|
||||
("GET", "/api/walkforwards/abc123"),
|
||||
@@ -179,6 +183,78 @@ def test_frontend_uses_hash_routing_only() -> None:
|
||||
assert "pushState" not in js
|
||||
|
||||
|
||||
def test_router_sentinel_is_not_the_home_path() -> None:
|
||||
"""路由的「已渲染路径」哨兵不能是空串 —— 首页路径本身就是 ''。
|
||||
|
||||
历史 bug：``let currentPath = ''`` 且所有强制重渲染都写 ``currentPath = ''``。
|
||||
首页（hash 为空）解析出的 path 恰好也是 ''，于是 render() 一进门就命中
|
||||
``path === currentPath`` 提前返回：index.html 里那句「加载中…」永远不被替换，
|
||||
概览页整页打不开（其他页面因为有非空路径，反而正常）。哨兵改用 null 后，
|
||||
'' 才能被当作一个正常的、需要渲染的路径。
|
||||
"""
|
||||
js = (project_root() / "web" / "app.js").read_text(encoding="utf-8")
|
||||
# 只看赋值（排除 === 比较）
|
||||
assigns = [a.strip() for a in re.findall(r"currentPath\s*=(?!=)\s*([^;\n]+)", js)]
|
||||
assert assigns, "未在 app.js 中找到 currentPath 赋值，解析逻辑需更新"
|
||||
bad = [a for a in assigns if a in {"''", '""'}]
|
||||
assert not bad, f"currentPath 不能用空串作哨兵（与首页路径 '' 冲突）：{bad}"
|
||||
assert "let currentPath = null" in js
|
||||
|
||||
|
||||
def test_profile_gate_is_exposed_for_display() -> None:
|
||||
"""画像闸门是买入判据的一部分，必须能在回测页看到。
|
||||
|
||||
看不到就会出现「股息率分位到了却没买」无从解释的情况 ——
|
||||
闸门规则是**第二道**买入条件，和 run 一起要能复现。
|
||||
"""
|
||||
from hdiv.web.service import describe_strategy
|
||||
|
||||
cfg = {
|
||||
"strategy": {"id": "S", "name": "n", "version": "1", "status": "DRAFT",
|
||||
"description": ""},
|
||||
"entry": {
|
||||
"yield_percentile": 75,
|
||||
"profile_gate": {
|
||||
"enabled": True, "window_years": 5, "on_unverifiable": "reject",
|
||||
"min_window_coverage": 0.0,
|
||||
"rules": [
|
||||
{"metric": "payout_ratio", "stat": "current_value",
|
||||
"op": "<=", "value": 1.0},
|
||||
# 没写 stat：应默认当日值，且不能把规则丢掉
|
||||
{"metric": "roe_avg", "op": ">=", "value": 0.08},
|
||||
],
|
||||
},
|
||||
},
|
||||
}
|
||||
g = describe_strategy(cfg)["profile_gate"]
|
||||
assert g["enabled"] is True and g["window_years"] == 5.0
|
||||
assert g["on_unverifiable"] == "reject"
|
||||
assert [r["metric"] for r in g["rules"]] == ["payout_ratio", "roe_avg"]
|
||||
assert g["rules"][0]["op"] == "<=" and g["rules"][0]["value"] == 1.0
|
||||
assert g["rules"][1]["stat"] == "current_value"
|
||||
json.dumps(g, ensure_ascii=False, allow_nan=False)
|
||||
|
||||
# 老配置没有这一段 → None，前端据此不显示卡片
|
||||
assert describe_strategy({"strategy": {}, "entry": {}})["profile_gate"] is None
|
||||
# 脏数据不能把整页带崩，也不能造出假规则
|
||||
dirty = describe_strategy({"strategy": {},
|
||||
"entry": {"profile_gate": {"enabled": True,
|
||||
"rules": [None, {}, "x"]}}})
|
||||
assert dirty["profile_gate"]["rules"] == []
|
||||
|
||||
|
||||
def test_profile_gate_card_is_wired_into_backtest_page() -> None:
|
||||
"""回测页必须真的把画像闸门渲染出来（接口有字段≠页面显示）。"""
|
||||
js = (project_root() / "web" / "app.js").read_text(encoding="utf-8")
|
||||
assert "profileGateCard" in js, "缺少画像闸门卡片渲染函数"
|
||||
assert "个股画像筛选条件" in js, "缺少画像闸门卡片标题"
|
||||
assert "profile_gate" in js, "未把接口字段接到卡片上"
|
||||
# 卡片要挂在「回测条件」之后
|
||||
cond = js.index(">回测条件<")
|
||||
gate = js.index("profileGateCard(b.strategy.profile_gate)")
|
||||
assert cond < gate, "画像闸门卡片必须在「回测条件」之后"
|
||||
|
||||
|
||||
def test_frontend_escapes_html() -> None:
|
||||
"""用户可输入记录名称/备注，必须转义以避免 XSS。"""
|
||||
js = (project_root() / "web" / "app.js").read_text(encoding="utf-8")
|
||||
@@ -383,6 +459,139 @@ def test_equity_index_overlay_is_date_aligned() -> None:
|
||||
service.get_backtest_equity(rid, index_code="999999.XX")
|
||||
|
||||
|
||||
@requires_db
|
||||
def test_stock_detail_default_range_reaches_latest_data() -> None:
|
||||
"""回归：默认区间要到**该股最新行情**，而不是持仓结束（卖出）当天。
|
||||
|
||||
老实现默认用「持仓区间」，卖出之后曲线就断了，
|
||||
「卖飞了没有」这个最该回答的问题在图上无从回答。
|
||||
这里特意挑「已清仓、且清仓日之后还有行情」的样本 —— 正是老实现会断线的场景。
|
||||
"""
|
||||
from hdiv.core.config import load_config
|
||||
from hdiv.data import db
|
||||
from hdiv.web import analysis
|
||||
|
||||
cfg = load_config("datasource")
|
||||
df = db.read_sql(
|
||||
"SELECT p.run_id, p.symbol, p.hold_start, p.hold_end, d.avail_end "
|
||||
"FROM (SELECT run_id, symbol, MIN(trade_date) AS hold_start, "
|
||||
" MAX(trade_date) AS hold_end "
|
||||
" FROM hd_backtest_position GROUP BY run_id, symbol) p "
|
||||
"JOIN (SELECT symbol, MAX(trade_date) AS avail_end "
|
||||
" FROM daily_basic GROUP BY symbol) d ON d.symbol = p.symbol "
|
||||
"WHERE p.hold_end < d.avail_end "
|
||||
"ORDER BY p.hold_end LIMIT 1",
|
||||
cfg=cfg,
|
||||
)
|
||||
if df.empty:
|
||||
pytest.skip("库里没有「已清仓且之后仍有行情」的样本")
|
||||
row = df.iloc[0]
|
||||
rid, sym = str(row["run_id"]), str(row["symbol"])
|
||||
hold_start, hold_end, avail_end = (str(row["hold_start"]), str(row["hold_end"]),
|
||||
str(row["avail_end"]))
|
||||
|
||||
r = analysis.stock_detail(rid, sym, series=["close"])["range"]
|
||||
assert r["available_start"] and r["available_end"], "应返回该股行情边界供日期选择器用"
|
||||
assert r["end"] == avail_end, \
|
||||
f"默认区间止于 {r['end']}，而行情已到 {avail_end}（又回到「卖出即断线」）"
|
||||
assert r["start"] <= hold_start, "默认起点不应晚于持仓起点（判据数据要在图上）"
|
||||
assert r["end"] > hold_end, f"默认区间不应停在清仓日 {hold_end}"
|
||||
|
||||
|
||||
@requires_db
|
||||
def test_stock_detail_default_range_includes_judgement_lookback() -> None:
|
||||
"""默认区间要含**首笔成交之前**的判据数据。
|
||||
|
||||
买入依据是股息率的历史分位（滚动窗口，config: percentile_reference
|
||||
.lookback_years = 5 年）；只画持仓期等于把「当时凭什么买」的判据裁掉了。
|
||||
"""
|
||||
import datetime as _dt
|
||||
|
||||
from hdiv.web import analysis
|
||||
|
||||
rid = _sample_backtest_run()
|
||||
if not rid:
|
||||
pytest.skip("没有可用的回测")
|
||||
stocks = [s for s in analysis.run_stocks(rid) if (s.get("trade_count") or 0) > 0]
|
||||
if not stocks:
|
||||
pytest.skip("该回测没有成交")
|
||||
sym = stocks[0]["symbol"]
|
||||
|
||||
d = analysis.stock_detail(rid, sym, series=["close"])
|
||||
r = d["range"]
|
||||
first = min(t["execution_date"] for t in d["trades"])
|
||||
need = _dt.date.fromisoformat(first) - _dt.timedelta(days=int(365.25 * 5))
|
||||
assert r["start"] <= need.isoformat(), \
|
||||
f"默认起点 {r['start']} 未覆盖首笔成交（{first}）之前 5 年的判据数据"
|
||||
# 但不该早于该股行情本身（否则日期选择器会给出选不到的日期）
|
||||
assert r["start"] >= r["available_start"]
|
||||
|
||||
|
||||
@requires_db
|
||||
def test_downsampling_keeps_trade_dates() -> None:
|
||||
"""回归：降采样不能把成交日丢掉。
|
||||
|
||||
前端按**日期**把买卖点落到横轴上，横轴里没有那一天，
|
||||
这笔成交就会从图上消失（还会被前端误报成「不在所选区间内」）。
|
||||
实测：默认区间放宽到「5 年判据 + 至今」后，13 只降采样股票里有 7 只会丢成交日。
|
||||
"""
|
||||
from hdiv.web import analysis
|
||||
|
||||
rid = _sample_backtest_run()
|
||||
if not rid:
|
||||
pytest.skip("没有可用的回测")
|
||||
checked = 0
|
||||
for s in analysis.run_stocks(rid):
|
||||
d = analysis.stock_detail(rid, s["symbol"], series=["close"])
|
||||
if not d["range"]["downsampled"]:
|
||||
continue
|
||||
checked += 1
|
||||
axis = set(d["dates"])
|
||||
missing = [t["execution_date"] for t in d["trades"]
|
||||
if t["execution_date"] not in axis]
|
||||
assert not missing, \
|
||||
f"{s['symbol']} 降采样后丢了成交日 {missing}，图上会少标这几笔"
|
||||
assert d["dates"] == sorted(d["dates"]), "横轴仍须按时间升序"
|
||||
if not checked:
|
||||
pytest.skip("该回测没有触发降采样的个股")
|
||||
|
||||
|
||||
@requires_db
|
||||
def test_stock_detail_date_params_and_validation() -> None:
|
||||
"""区间参数：能收窄、非法输入报可读错误、区间外成交仍要返回。"""
|
||||
import datetime as _dt
|
||||
|
||||
from hdiv.core.errors import HdivError
|
||||
from hdiv.web import analysis
|
||||
|
||||
rid = _sample_backtest_run()
|
||||
if not rid:
|
||||
pytest.skip("没有可用的回测")
|
||||
stocks = analysis.run_stocks(rid)
|
||||
if not stocks:
|
||||
pytest.skip("该回测没有持仓股票")
|
||||
sym = stocks[0]["symbol"]
|
||||
|
||||
base = analysis.stock_detail(rid, sym, series=["close"])
|
||||
a = _dt.date.fromisoformat(base["range"]["start"])
|
||||
s, e = a.isoformat(), (a + _dt.timedelta(days=180)).isoformat()
|
||||
|
||||
d = analysis.stock_detail(rid, sym, start=s, end=e, series=["close"])
|
||||
assert d["range"]["requested_start"] == s and d["range"]["requested_end"] == e
|
||||
assert s <= d["range"]["start"] and d["range"]["end"] <= e
|
||||
assert d["range"]["points"] < base["range"]["points"], "收窄区间应真的少取数据"
|
||||
assert d["range"]["default_end"] == base["range"]["default_end"], \
|
||||
"default_* 应是「重置」用的缺省区间，不随本次请求变化"
|
||||
# 区间外的成交仍要返回：前端靠它提示「有 N 笔不在所选区间内」
|
||||
assert len(d["trades"]) == len(base["trades"])
|
||||
|
||||
with pytest.raises(HdivError):
|
||||
analysis.stock_detail(rid, sym, start=e, end=s, series=["close"])
|
||||
for bad in ("2024-13-45", "not-a-date"):
|
||||
with pytest.raises(HdivError):
|
||||
analysis.stock_detail(rid, sym, start=bad, series=["close"])
|
||||
|
||||
|
||||
@requires_db
|
||||
def test_reason_text_is_human_readable() -> None:
|
||||
"""成交理由必须渲染成人话，而不是丢一坨 JSON 给前端。"""
|
||||
@@ -632,6 +841,22 @@ def test_site_build_does_not_clobber_spa() -> None:
|
||||
assert "app/app.js" in html
|
||||
|
||||
|
||||
def test_published_site_is_world_readable() -> None:
|
||||
"""发布产物必须 world-readable：nginx worker 以 nobody 运行，不是文件属主。
|
||||
|
||||
``shutil.copy2`` 会保留源文件权限，所以一个 umask 077 存下来的 600 文件
|
||||
会让线上 CSS/JS 直接 403（HTML 打得开、页面裸奔）。发布时统一收敛权限。
|
||||
"""
|
||||
from hdiv.web import site
|
||||
|
||||
site.sync_frontend(verbose=False)
|
||||
out = project_root() / "output"
|
||||
unreadable = [p for p in out.rglob("*") if p.is_file() and not p.stat().st_mode & 0o044]
|
||||
assert not unreadable, f"这些发布文件 nginx（nobody）读不到：{unreadable[:5]}"
|
||||
untraversable = [p for p in out.rglob("*") if p.is_dir() and not p.stat().st_mode & 0o011]
|
||||
assert not untraversable, f"这些目录 nginx（nobody）进不去：{untraversable[:5]}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 回测内分析：任意日持仓 + 个股买卖点
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -651,6 +876,80 @@ def _sample_backtest_run() -> str | None:
|
||||
return None if df.empty else str(df["run_id"].iloc[0])
|
||||
|
||||
|
||||
@requires_db
|
||||
def test_closed_positions_definition_and_math() -> None:
|
||||
"""已清仓清单：定义（期末不再持有）+ 口径（收益率、清仓后涨跌）都要对得上。
|
||||
|
||||
「已清仓」若按成交净额判断会漏掉「卖了又买回、期末仍持有」的票，
|
||||
这里同时用两套口径交叉验证，并要求金额/盈亏与成交表逐笔汇总一致。
|
||||
"""
|
||||
from hdiv.core.config import load_config
|
||||
from hdiv.data import db
|
||||
from hdiv.core.errors import HdivError
|
||||
from hdiv.web import analysis
|
||||
|
||||
cfg = load_config("datasource")
|
||||
# 找一只有已清仓个股的回测（期末持仓数 < 曾持有数）
|
||||
df = db.read_sql(
|
||||
"SELECT run_id FROM hd_backtest_position GROUP BY run_id "
|
||||
"HAVING COUNT(DISTINCT symbol) > "
|
||||
" (SELECT COUNT(DISTINCT symbol) FROM hd_backtest_position p2 "
|
||||
" WHERE p2.run_id = hd_backtest_position.run_id "
|
||||
" AND p2.trade_date = (SELECT MAX(trade_date) FROM hd_backtest_position p3 "
|
||||
" WHERE p3.run_id = hd_backtest_position.run_id)) "
|
||||
"ORDER BY COUNT(DISTINCT symbol) DESC LIMIT 1",
|
||||
cfg=cfg,
|
||||
)
|
||||
if df.empty:
|
||||
pytest.skip("没有含已清仓个股的回测")
|
||||
rid = str(df["run_id"].iloc[0])
|
||||
|
||||
d = analysis.closed_positions(rid)
|
||||
items = d["items"]
|
||||
assert items, "该回测应当有已清仓个股"
|
||||
json.dumps(d, ensure_ascii=False, allow_nan=False) # NaN 不能漏到前端
|
||||
|
||||
last_day = str(db.read_sql(
|
||||
"SELECT MAX(trade_date) AS d FROM hd_backtest_position WHERE run_id = :r",
|
||||
{"r": rid}, cfg=cfg)["d"].iloc[0])
|
||||
still = set(db.read_sql(
|
||||
"SELECT DISTINCT symbol FROM hd_backtest_position "
|
||||
"WHERE run_id = :r AND trade_date = :d", {"r": rid, "d": last_day}, cfg=cfg)["symbol"])
|
||||
|
||||
tr = db.read_sql(
|
||||
"SELECT symbol, side, quantity, amount, realized_pnl, execution_date "
|
||||
"FROM hd_backtest_trade WHERE run_id = :r", {"r": rid}, cfg=cfg)
|
||||
for x in items:
|
||||
assert x["symbol"] not in still, f"{x['symbol']} 期末仍持有，不该出现在已清仓清单"
|
||||
mine = tr[tr["symbol"] == x["symbol"]]
|
||||
buys = mine[mine["side"] == "BUY"]
|
||||
sells = mine[mine["side"] == "SELL"]
|
||||
assert len(sells) > 0, "已清仓必然有卖出成交"
|
||||
# 刻意**不**校验「买入股数 == 卖出股数」：送股/转增会让持仓股数凭空增加
|
||||
# （实测 600188.SH 在 93fb7456 里买入 6000 股、卖出 11700 股）。
|
||||
# 所以「已清仓」只能以持仓表为准，不能用成交净额反推。
|
||||
assert x["last_sell"] == str(sells["execution_date"].max())
|
||||
assert abs((x["realized_pnl"] or 0) - float(sells["realized_pnl"].sum())) < 1e-6
|
||||
assert abs((x["buy_amount"] or 0) - float(buys["amount"].sum())) < 1e-6
|
||||
assert x["first_hold"] and x["last_hold"] and x["hold_days"] > 0
|
||||
if x["return_pct"] is not None: # 已清仓 ⇒ 收益率 = 已实现盈亏 / 买入金额
|
||||
assert abs(x["return_pct"] - x["realized_pnl"] / x["buy_amount"]) < 1e-9
|
||||
if x["since_sell_pct"] is not None: # 清仓后涨跌以清仓日收盘为基准
|
||||
assert abs(x["since_sell_pct"] -
|
||||
(x["close_latest"] / x["close_at_sell"] - 1.0)) < 1e-9
|
||||
|
||||
s = d["summary"]
|
||||
assert s["count"] == len(items)
|
||||
assert abs(s["realized_pnl"] - sum(x["realized_pnl"] or 0 for x in items)) < 1e-6
|
||||
assert s["since_sell_up"] + s["since_sell_down"] <= s["count"]
|
||||
# 明细按清仓日倒序（最近清仓的排在最前）
|
||||
dates = [x["last_sell"] or "" for x in items]
|
||||
assert dates == sorted(dates, reverse=True)
|
||||
|
||||
with pytest.raises(HdivError):
|
||||
analysis.closed_positions("不存在的runid")
|
||||
|
||||
|
||||
@requires_db
|
||||
def test_position_dates_is_compact_by_default() -> None:
|
||||
"""默认只返回日期字符串：带全字段会让响应从约 30KB 涨到 460KB。"""
|
||||
|
||||
Reference in New Issue
Block a user