feat: 股息率案例口径 + 策略库与图表统一 + 回测存档完整化
汇总三轮未提交的开发(每轮均在本机 MariaDB + 真实浏览器上验证):
1) 股息率案例(全市场股息率最高 n 只,默认 20,每 m 月择股)
- 新增日频估值表 daily_basic + 迁移;股息率因子(dv_ratio / dividend_yield / TTM)
- 名称历史表 stock_name_history:剔除 ST 按**择股日当时名称**判定,消除
「曾高股息后 ST」的股息陷阱(实测 3.70pp 偏差)
- 区间择股/调仓双周期(m 择股 / y 调仓)、指数成分与白名单、停牌近似剔除
- 复权因子口径核对(4,164,742 行、缺失 0.0%)、收盘价成交与涨跌停拦单
- 案例实测:2020-01-01~2026-09-04 总收益 +24.86%(年化 3.52%、回撤 -28.58%)
2) 策略库与前端统一
- strategy 表 + CRUD/PUT 原地更新 + `describe_strategy` 按 spec 真实推导
「一句话说明 + 计算公式 + 执行步骤 + 注意事项」(与引擎实执行规则同源)
- 任何出现股票代码处都成对显示名称且可点击进个股页
- 全站图表基座统一 TradingView Lightweight Charts(ECharts 依赖、
锁文件、组件与文档标注一并清除),买卖点标记只落在真实交易日上
3) 回测存档完整化(可往复查看)
- 同步端点(POST /api/backtests、/api/factor-tests)此前完全不落库 → 现在同样归档,
归档 id 经响应头 X-Experiment-Id 返回(不破坏 response_model)
- data_version 首次真实写入(数据快照指纹:最新交易日 + 各表规模)
- 个股收益曲线默认**全量保存**(此前硬截断 60 只);超出体积预算才裁剪,
并写 archive_meta(机器可读)+ unimplemented(人可读)如实标注
- 列表 kind/q 过滤 + X-Total-Count(此前 limit=50 静默截断)、DELETE 归档
- 只读归档页 /experiments/{id}(Server Component,SSR 直出**选股条件**与
**交易执行依据**);结果视图按 kind 分发(backtest/factor_test/selection),
非回测归档不套用回测口径
- 新增 CLI:prune_experiments(保留策略,默认 dry-run)、
restore_experiment_from_job(从 Job 副本按原 id 重建被删的历史归档,默认 dry-run)
门禁:pytest 388 passed、ruff All checks passed、tsc 0 错误、图表单测 7 passed、
next build 成功、契约脚本 verify_strategy_workspace 59/59(含按 kind 逐类验证归档页)。
This commit is contained in:
@@ -1,10 +1,17 @@
|
||||
"""LocalEngine —— 默认研究引擎(纯 pandas,AGENT.md §40 简单可替换优先)。
|
||||
|
||||
无未来函数纪律:
|
||||
- 调仓日 t 的选股只使用 <=t 的因子值与收盘价
|
||||
- 成交发生在 t 收盘(价格 = close[t] ± 滑点);t 当日组合收益用 t-1 收盘持仓结算,
|
||||
- 择股日 s 的选股只使用 <= s 的因子值、条件字段与收盘价
|
||||
- 成交发生在调仓日 t 收盘(价格 = close[t] ± 滑点);t 当日组合收益用 t-1 收盘持仓结算,
|
||||
调仓在 t 收盘生效、自 t+1 起计收益 —— 不存在「当日买入当日计收益」的未来函数
|
||||
- 顺延买入(defer_buy)只在**之后的交易日**补成交,绝不回溯到择股日之前
|
||||
- 涨跌停 / 停牌约束按可达信息近似建模,未建模部分显式写入结果 unimplemented
|
||||
|
||||
周期模型(本次扩展,见 ResearchSpec):
|
||||
- 择股日集合 S:每 m 个月(selection_interval_months),锚定回测起始月
|
||||
- 调仓日集合 R:每 y 个月(rebalance_interval_months,缺省 = m)
|
||||
- m 未给 → S = R(每次调仓都重新择股,与历史行为一致)
|
||||
- 候选池 = S 日按因子分排序的 Top n(top_n);实际持仓 = 池内前 x(hold_top_x)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -25,6 +32,7 @@ from app.domain.entities.research import (
|
||||
Position,
|
||||
RankedPick,
|
||||
ResearchSpec,
|
||||
SymbolCurve,
|
||||
Trade,
|
||||
YearlyReturn,
|
||||
)
|
||||
@@ -41,9 +49,41 @@ from app.quant.portfolio import (
|
||||
)
|
||||
|
||||
TRADING_DAYS = 252
|
||||
|
||||
# 个股收益曲线数量上限:
|
||||
# None(默认)= 不截断,期内持有的每只都输出(「完整存档」;体积由归档侧的
|
||||
# 字节预算兜底,见 app/application/services/experiment_archive.py);
|
||||
# 数字 = 按 |期末收益| 降序截断,且如实写入 unimplemented 说明。
|
||||
# 取值优先级:本模块变量(测试 monkeypatch 用)> config research.archive_curve_limit。
|
||||
_MAX_SYMBOL_CURVES: int | None = None
|
||||
|
||||
|
||||
def _resolved_curve_limit() -> int | None:
|
||||
"""当前生效的曲线数量上限(None = 完整输出)。"""
|
||||
if _MAX_SYMBOL_CURVES is not None:
|
||||
return _MAX_SYMBOL_CURVES
|
||||
from app.core.config import get_settings
|
||||
|
||||
return get_settings().research_archive_curve_limit
|
||||
|
||||
# 恒定的未建模说明(AGENT.md §24:未实现项必须在结果中显式标注)
|
||||
_DEFAULT_UNIMPLEMENTED = [
|
||||
"涨跌停按收盘价相对上一有效收盘近似判定(未建模开盘一字 / 集合竞价路径)",
|
||||
"成交假设发生在调仓日收盘(未建模盘中价格路径与流动性冲击)",
|
||||
(
|
||||
"调仓为「全部卖出 → 按目标等权重新买入」,未做权重漂移微调:"
|
||||
"保留在目标名单中的股票也会产生一次完整买卖往返,交易成本估计偏保守"
|
||||
),
|
||||
(
|
||||
"股票池来自本地行情表(已含退市股:stock.status='D' 且带 delist_date,"
|
||||
"退市日之后自动退出池子)。残余偏差:库里仅有 2019-12 之后退市的标的,"
|
||||
"更早退市者无行情数据"
|
||||
),
|
||||
(
|
||||
"exclude_st 的名称口径见 config_snapshot.price_basis.name_basis:时点口径依赖 "
|
||||
"stock_name_history(sync namechange),未同步时回退最新名称快照,"
|
||||
"会漏掉「曾是高股息、后来才变 ST」的股息陷阱样本"
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
@@ -57,16 +97,92 @@ def _limit_up_ratio(symbol: str) -> float:
|
||||
return 1.099
|
||||
|
||||
|
||||
def rebalance_dates(index: pd.Index, rebalance: str, start: date) -> list[pd.Timestamp]:
|
||||
"""按频率取首个交易日(>= start)。"""
|
||||
periods = index.to_period("M" if rebalance == "monthly" else "W")
|
||||
def _month_firsts(index: pd.Index) -> list[pd.Timestamp]:
|
||||
"""每个自然月的首个交易日(按 index 顺序)。"""
|
||||
periods = index.to_period("M")
|
||||
seen: dict = {}
|
||||
order: list[pd.Timestamp] = []
|
||||
for ts, per in zip(index, periods, strict=True):
|
||||
if per not in seen:
|
||||
seen[per] = ts
|
||||
order.append(ts)
|
||||
return [ts for ts in order if ts.date() >= start]
|
||||
return order
|
||||
|
||||
|
||||
def _week_firsts(index: pd.Index) -> list[pd.Timestamp]:
|
||||
"""每个自然周的首个交易日。"""
|
||||
periods = index.to_period("W")
|
||||
seen: dict = {}
|
||||
order: list[pd.Timestamp] = []
|
||||
for ts, per in zip(index, periods, strict=True):
|
||||
if per not in seen:
|
||||
seen[per] = ts
|
||||
order.append(ts)
|
||||
return order
|
||||
|
||||
|
||||
def _month_seq(ts: pd.Timestamp) -> int:
|
||||
"""月序号(year*12+month),用于「每 m 个月」的锚定计算。"""
|
||||
return int(ts.year) * 12 + int(ts.month)
|
||||
|
||||
|
||||
def rebalance_dates(
|
||||
index: pd.Index,
|
||||
rebalance: str,
|
||||
start: date,
|
||||
end: date | None = None,
|
||||
every_months: int | None = None,
|
||||
) -> list[pd.Timestamp]:
|
||||
"""调仓/择股日集合(按频率取首个交易日,>= start)。
|
||||
|
||||
every_months=n(n>0):忽略 rebalance 频率,改用「每 n 个月」——
|
||||
锚定 **首个 >= start 的交易日所在月**(锚点月 t0),取月序号满足
|
||||
`(t - t0) % n == 0` 的月份的首个交易日。
|
||||
|
||||
这样 2020-01-01 起、n=6 → 2020-01、2020-07、2021-01 …;
|
||||
起始日改为 2020-03-15(该月首个交易日 03-02 早于 start)→ **2020-03-16**
|
||||
(03 月内首个 >= start 的交易日)、2020-09、2021-03 …。
|
||||
⚠️ 刻意**不丢弃锚点月**:若把锚点月整体过滤掉,m=y=6 且起始日非月初时
|
||||
会白等 6 个月才首次建仓(净值在前期恒等于初始资金,指标明显失真)。
|
||||
|
||||
every_months=None:沿用 weekly / monthly 频率(原行为,保持向后兼容)。
|
||||
"""
|
||||
firsts = _week_firsts(index) if rebalance == "weekly" else _month_firsts(index)
|
||||
if every_months and every_months > 0:
|
||||
# 锚点 = start 所在月内首个 >= start 的交易日(可能不是该月首个交易日)
|
||||
days = pd.DatetimeIndex(index)
|
||||
after_start = days[days >= pd.Timestamp(start)]
|
||||
if len(after_start) == 0:
|
||||
return []
|
||||
anchor_ts = after_start[0]
|
||||
anchor = _month_seq(anchor_ts)
|
||||
# 后续月份:月序号与锚点月相差整数倍 m,取该月首个交易日
|
||||
out = [anchor_ts] + [
|
||||
ts
|
||||
for ts in firsts
|
||||
if _month_seq(ts) > anchor
|
||||
and (_month_seq(ts) - anchor) % every_months == 0
|
||||
and ts.date() >= start
|
||||
and ts != anchor_ts
|
||||
]
|
||||
else:
|
||||
out = [ts for ts in firsts if ts.date() >= start]
|
||||
if end is not None:
|
||||
out = [ts for ts in out if ts.date() <= end]
|
||||
return out
|
||||
|
||||
|
||||
@dataclass
|
||||
class PendingBuy:
|
||||
"""顺延买单:调仓日买不进(涨停/停牌)时挂起,之后逐日重试。
|
||||
|
||||
仅当 SelectionSpec.defer_buy=True 时产生;到下一次调仓日仍未成交则作废。
|
||||
`budget` 是调仓日按等权/上限为该标的预留的资金,成交时按 min(budget, 可用现金) 执行。
|
||||
"""
|
||||
|
||||
symbol: str
|
||||
budget: float
|
||||
since: date
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -78,9 +194,15 @@ class EngineResult:
|
||||
|
||||
|
||||
class TopKBacktestRunner:
|
||||
"""TopK 等权、固定调仓频率的低频回测。"""
|
||||
"""TopK 等权、固定调仓频率的低频回测(支持择股/调仓双周期与顺延买入)。"""
|
||||
|
||||
def __init__(self, spec: ResearchSpec, score: pd.DataFrame, close: pd.DataFrame) -> None:
|
||||
def __init__(
|
||||
self,
|
||||
spec: ResearchSpec,
|
||||
score: pd.DataFrame,
|
||||
close: pd.DataFrame,
|
||||
eligibility_fn=None,
|
||||
) -> None:
|
||||
self.spec = spec
|
||||
close = close.copy()
|
||||
close.index = pd.to_datetime(close.index)
|
||||
@@ -89,18 +211,49 @@ class TopKBacktestRunner:
|
||||
self.costs = spec.costs
|
||||
# 上一有效收盘(用于涨跌停与收益结算,处理停牌日)
|
||||
self.prev_close = self.close.ffill().shift(1)
|
||||
# 条件过滤(可选):(as_of: date) -> set[symbol] | None
|
||||
# 由 Service 注入(复用 selection.eligible_symbols),保证回测与选股同一套求值逻辑
|
||||
self.eligibility_fn = eligibility_fn
|
||||
# M9-2:调仓意图与信号/成交记录(v3 §20.3/§22.3)
|
||||
self.selection_history: list[RankedPick] = []
|
||||
self.signal_history: list[ActionRecord] = []
|
||||
# 当前候选池(择股日刷新):current_ranked 为全市场可评分排序,current_pool = 前 n
|
||||
self.current_ranked: list[str] = []
|
||||
self.current_pool: list[str] = []
|
||||
# 本次回测期内被持有过的股票(用于个股收益曲线)
|
||||
self.traded_symbols: list[str] = []
|
||||
self._traded: set[str] = set()
|
||||
# 无前收导致涨停无法判定、按可买处理并**实际成交**的标的集合(结果中如实标注)
|
||||
self._no_prev_close_symbols: set[str] = set()
|
||||
|
||||
# ---- 主流程 ----
|
||||
|
||||
def run(self) -> BacktestResult:
|
||||
end_date = self.spec.period[1]
|
||||
dates = [d for d in self.close.index if self.spec.period[0] <= d.date() <= end_date]
|
||||
rebal = {
|
||||
d
|
||||
for d in rebalance_dates(self.close.index, self.spec.rebalance, self.spec.period[0])
|
||||
if d.date() <= end_date
|
||||
}
|
||||
if not dates:
|
||||
raise ValueError(
|
||||
f"回测区间 {self.spec.period[0]}~{end_date} 内没有任何行情数据,无法回测"
|
||||
)
|
||||
|
||||
m = self.spec.effective_selection_months
|
||||
y = self.spec.effective_rebalance_months
|
||||
rebal = set(
|
||||
rebalance_dates(
|
||||
self.close.index, self.spec.rebalance, self.spec.period[0], end_date, every_months=y
|
||||
)
|
||||
)
|
||||
if m is None:
|
||||
# 未给 m:每次调仓都重新择股(与历史行为一致)
|
||||
select = set(rebal)
|
||||
else:
|
||||
select = set(
|
||||
rebalance_dates(
|
||||
self.close.index, self.spec.rebalance, self.spec.period[0], end_date,
|
||||
every_months=m,
|
||||
)
|
||||
)
|
||||
|
||||
cash = float(self.spec.initial_capital)
|
||||
shares: dict[str, float] = {}
|
||||
entry_date: dict[str, date] = {}
|
||||
@@ -109,6 +262,10 @@ class TopKBacktestRunner:
|
||||
trades: list[Trade] = []
|
||||
positions: list[Position] = []
|
||||
notional: list[float] = []
|
||||
pending: list[PendingBuy] = []
|
||||
# 个股收益曲线:cum = 该股「持仓期间」的累计净值(1.0 = 未涨未跌)
|
||||
cum: dict[str, float] = {}
|
||||
curve_rows: dict[str, list[CurvePoint]] = {}
|
||||
|
||||
def _value(d: pd.Timestamp) -> float:
|
||||
total = cash
|
||||
@@ -122,21 +279,56 @@ class TopKBacktestRunner:
|
||||
return total
|
||||
|
||||
for d in dates:
|
||||
# 1) 先用「上一交易日收盘持仓」结算当日个股收益(与组合净值同一时序口径:
|
||||
# 当日收益来自昨日持仓)→ 建仓当日不计收益、卖出当日仍有收益
|
||||
self._accrue_symbol_returns(d, shares, cum, curve_rows)
|
||||
# 2) 择股 / 调仓(成交发生在当日收盘)
|
||||
if d in select:
|
||||
self.current_ranked, self.current_pool = self._select(d)
|
||||
if d in rebal:
|
||||
# 上一次调仓挂起的顺延单作废(只在两次调仓之间有效)
|
||||
pending = []
|
||||
cash = self._rebalance(
|
||||
d, cash, shares, entry_date, entry_price, trades, positions, notional
|
||||
d, cash, shares, entry_date, entry_price, trades, positions, notional,
|
||||
pending,
|
||||
)
|
||||
elif pending:
|
||||
cash = self._fill_pending(d, cash, shares, entry_date, entry_price, pending, notional)
|
||||
equity_rows[d] = _value(d)
|
||||
# 3) 建仓当日补「基准点」:成交在当日收盘、收益自次日起计;该点使 BUY 标注
|
||||
# 能精确落在曲线上,也让多段持仓的分段起点可见(见 _mark_curve_dates)
|
||||
self._mark_curve_dates(d, shares, cum, curve_rows)
|
||||
|
||||
equity = pd.Series(equity_rows).sort_index()
|
||||
return self._to_result(equity, trades, positions, notional)
|
||||
return self._to_result(equity, trades, positions, notional, cum, curve_rows)
|
||||
|
||||
# ---- 择股(择股日 s:只用 <= s 的数据) ----
|
||||
|
||||
def _select(self, d: pd.Timestamp) -> tuple[list[str], list[str]]:
|
||||
"""返回 (全市场可评分排序, 候选池 top n),并记录 selection_history。"""
|
||||
score_d = self.score.loc[d].dropna()
|
||||
eligible = None
|
||||
if self.eligibility_fn is not None:
|
||||
eligible = self.eligibility_fn(d.date())
|
||||
if eligible is not None:
|
||||
score_d = score_d[score_d.index.isin(eligible)]
|
||||
ranked = score_d.sort_values(ascending=False).index.tolist()
|
||||
n = self.spec.selection.top_n
|
||||
pool = ranked[:n]
|
||||
day = d.date()
|
||||
for rank, sym in enumerate(pool, start=1):
|
||||
self.selection_history.append(
|
||||
RankedPick(date=day, symbol=sym, rank=rank, score=round(float(score_d[sym]), 6))
|
||||
)
|
||||
return ranked, pool
|
||||
|
||||
# ---- 调仓(t 收盘执行,自 t+1 生效) ----
|
||||
|
||||
def _rebalance(self, d, cash, shares, entry_date, entry_price, trades, positions, notional):
|
||||
def _rebalance(
|
||||
self, d, cash, shares, entry_date, entry_price, trades, positions, notional, pending
|
||||
):
|
||||
close_d = self.close.loc[d]
|
||||
prev_d = self.prev_close.loc[d]
|
||||
sold_notional = 0.0
|
||||
day = d.date()
|
||||
|
||||
# 1) 卖出:逐持仓记录 SELL 意图与实际成交(跌停/无价则保留并说明)
|
||||
@@ -156,12 +348,11 @@ class TopKBacktestRunner:
|
||||
continue # 跌停无法卖出:保留到下一调仓
|
||||
qty = shares[s]
|
||||
proceeds = qty * float(c) * (1 - self.costs.slippage_rate)
|
||||
fee = proceeds * (self.costs.commission_rate + self.costs.stamp_tax_rate)
|
||||
commission = max(proceeds * self.costs.commission_rate, self.costs.min_commission)
|
||||
fee = commission + proceeds * self.costs.stamp_tax_rate
|
||||
cash += proceeds - fee
|
||||
sold_notional += proceeds
|
||||
self.signal_history.append(
|
||||
ActionRecord(date=day, symbol=s, signal="SELL", filled=True,
|
||||
price=float(c))
|
||||
ActionRecord(date=day, symbol=s, signal="SELL", filled=True, price=float(c))
|
||||
)
|
||||
trades.append(
|
||||
Trade(
|
||||
@@ -177,43 +368,57 @@ class TopKBacktestRunner:
|
||||
entry_date.pop(s, None)
|
||||
entry_price.pop(s, None)
|
||||
|
||||
# 2) 买入:先记录「选股意图」(= select(as_of) 前 top_n,v3 §22.3)
|
||||
score_d = self.score.loc[d].dropna()
|
||||
top = score_d.sort_values(ascending=False).index.tolist()
|
||||
top_n = self.spec.selection.top_n
|
||||
picks = top[:top_n]
|
||||
for rank, sym in enumerate(picks, start=1):
|
||||
self.selection_history.append(
|
||||
RankedPick(date=day, symbol=sym, rank=rank,
|
||||
score=round(float(score_d[sym]), 6))
|
||||
# 2) 买入意图:候选池(= selection_history 记录的那批)
|
||||
picks = list(self.current_pool)
|
||||
x = min(self.spec.selection.x, len(picks))
|
||||
sel = self.spec.selection
|
||||
if x < self.spec.selection.x:
|
||||
self.signal_history.append(
|
||||
ActionRecord(
|
||||
date=day,
|
||||
symbol="",
|
||||
signal="BUY",
|
||||
filled=False,
|
||||
reject_reason=(
|
||||
f"候选池仅 {len(picks)} 只(< 目标持仓 x={self.spec.selection.x}),"
|
||||
"按池内数量持仓"
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
# 执行:顺序寻找可买(涨停/无价剔除;替补仅在意图被拒时进入)
|
||||
def _buyable(sym) -> tuple[bool, str | None]:
|
||||
c, p = close_d[sym], prev_d[sym]
|
||||
if _nan(c) or _nan(p) or p <= 0:
|
||||
if _nan(c):
|
||||
return False, "无行情(停牌),无法买入"
|
||||
if _nan(p) or p <= 0:
|
||||
# 无有效前收(数据窗口起点 / 长期停牌后复牌):无法判定涨停 → 按可买处理。
|
||||
# 这里不计数:_buyable 是纯探测函数(替补扫描会重复调用同一标的),
|
||||
# 计数放在真实成交路径 `_execute_buy`,避免把探测次数报成买入次数。
|
||||
return True, None
|
||||
if c / p >= _limit_up_ratio(sym):
|
||||
return False, "涨停,无法追买"
|
||||
return True, None
|
||||
|
||||
# 目标名单:默认 = 池内前 x;allow_substitute=True 时从全市场排序继续往下找
|
||||
targets: list[str] = []
|
||||
for sym in top:
|
||||
if len(targets) >= top_n:
|
||||
break
|
||||
ok, _ = _buyable(sym)
|
||||
if ok:
|
||||
targets.append(sym)
|
||||
target_set = set(targets)
|
||||
if sel.allow_substitute:
|
||||
for sym in self.current_ranked:
|
||||
if len(targets) >= self.spec.selection.x:
|
||||
break
|
||||
ok, _ = _buyable(sym)
|
||||
if ok:
|
||||
targets.append(sym)
|
||||
else:
|
||||
targets = picks[:x]
|
||||
|
||||
# BUY 信号/成交记录:意图入选(filled)或意图被拒(原因);替补成交同样如实记录
|
||||
pending_specs: list[tuple[str, str | None]] = []
|
||||
spends: dict[str, float] = {}
|
||||
if targets:
|
||||
cap = self.spec.portfolio.max_position_pct
|
||||
if cap is None:
|
||||
# 默认等权(与原实现位级一致,保持回归数值不变)
|
||||
# 默认等权:按「目标持仓数」均分可用现金(顺延未成交的部分留作现金)
|
||||
budget = equal_weight_budget(cash, len(targets))
|
||||
spends = {s: budget for s in targets}
|
||||
total_spend = budget * len(targets)
|
||||
else:
|
||||
# Portfolio v1.1:按单股上限(相对当日组合市值)分配,超出部分留现金
|
||||
equity_now = cash + sum(
|
||||
@@ -222,31 +427,61 @@ class TopKBacktestRunner:
|
||||
if qty > 0 and not _nan(self.close.at[d, s])
|
||||
)
|
||||
spends = allocate_with_max_position(cash, targets, equity_now, cap)
|
||||
total_spend = sum(spends.values())
|
||||
|
||||
for s in targets:
|
||||
budget = spends[s]
|
||||
if budget <= 1e-9:
|
||||
# 分配额过小(可用现金≈0 或上限约束):不成交且无额度可顺延,如实留痕
|
||||
self.signal_history.append(
|
||||
ActionRecord(
|
||||
date=day, symbol=s, signal="BUY", filled=False,
|
||||
reject_reason="分配额不足(可用现金≈0),未成交",
|
||||
)
|
||||
)
|
||||
continue
|
||||
c = float(close_d[s])
|
||||
price_in = c * (1 + self.costs.slippage_rate)
|
||||
invest = budget * (1 - self.costs.commission_rate)
|
||||
shares[s] = invest / price_in
|
||||
entry_date[s] = day
|
||||
entry_price[s] = price_in
|
||||
notional.append(budget)
|
||||
ok, reason = _buyable(s)
|
||||
if not ok:
|
||||
if sel.defer_buy:
|
||||
# 顺延:挂单到之后首个可成交交易日(本次不成交,资金留现金)
|
||||
pending_specs.append((s, reason))
|
||||
self.signal_history.append(
|
||||
ActionRecord(
|
||||
date=day, symbol=s, signal="BUY", filled=False,
|
||||
reject_reason=f"{reason},顺延到之后首个可成交日买入",
|
||||
)
|
||||
)
|
||||
else:
|
||||
self.signal_history.append(
|
||||
ActionRecord(
|
||||
date=day, symbol=s, signal="BUY", filled=False,
|
||||
reject_reason=reason or "不可买入",
|
||||
)
|
||||
)
|
||||
continue
|
||||
if not self._execute_buy(
|
||||
s, budget, d, close_d[s], shares, entry_date, entry_price, notional
|
||||
):
|
||||
self.signal_history.append(
|
||||
ActionRecord(
|
||||
date=day, symbol=s, signal="BUY", filled=False,
|
||||
reject_reason="预算不足以覆盖最低佣金,未成交",
|
||||
)
|
||||
)
|
||||
continue
|
||||
cash -= budget
|
||||
|
||||
# 替补模式下目标名单取自 n 名之外,池内被跳过的标的也要记录意图,
|
||||
# 否则「信号有了却没买」无法解释(v3 §20.3 Signal↔Fill 透明化)。
|
||||
# 非替补模式下 targets == picks[:x],池内标的都已在上面留痕,无需再遍历。
|
||||
if sel.allow_substitute:
|
||||
for sym in picks:
|
||||
if sym in set(targets):
|
||||
continue
|
||||
_ok, reason = _buyable(sym)
|
||||
self.signal_history.append(
|
||||
ActionRecord(date=day, symbol=s, signal="BUY", filled=True,
|
||||
price=round(price_in, 4))
|
||||
ActionRecord(date=day, symbol=sym, signal="BUY", filled=False,
|
||||
reject_reason=reason or "资金不足(未成交)")
|
||||
)
|
||||
cash -= total_spend
|
||||
for sym in picks:
|
||||
if sym in target_set:
|
||||
continue
|
||||
_ok, reason = _buyable(sym)
|
||||
self.signal_history.append(
|
||||
ActionRecord(date=day, symbol=sym, signal="BUY", filled=False,
|
||||
reject_reason=reason or "资金不足(未成交)")
|
||||
)
|
||||
|
||||
# 3) 记录调仓后仓位
|
||||
total = cash + sum(
|
||||
@@ -262,11 +497,120 @@ class TopKBacktestRunner:
|
||||
date=day, symbol=s, weight=float(qty * self.close.at[d, s] / total)
|
||||
)
|
||||
)
|
||||
# 顺延单登记:预留额度 = 调仓日的等权/上限分配额(不因后续价格变化而变)
|
||||
for sym, _reason in pending_specs:
|
||||
pending.append(PendingBuy(symbol=sym, budget=spends.get(sym, 0.0), since=day))
|
||||
return cash
|
||||
|
||||
def _execute_buy(
|
||||
self, s, budget, d, close_value, shares, entry_date, entry_price, notional
|
||||
) -> bool:
|
||||
"""按收盘价 + 滑点买入;佣金(含最低佣金)从投入资金中扣除。
|
||||
|
||||
现金支出恒为 budget:shares = (budget - 佣金) / (收盘价 × (1 + 滑点))。
|
||||
返回是否成交(预算不足以覆盖最低佣金时不成交,调用方不得扣减现金)。
|
||||
"""
|
||||
c = float(close_value)
|
||||
price_in = c * (1 + self.costs.slippage_rate)
|
||||
commission = max(budget * self.costs.commission_rate, self.costs.min_commission)
|
||||
invest = budget - commission
|
||||
if invest <= 0:
|
||||
return False
|
||||
# 累加而非覆盖:避免「跌停/停牌未卖出而保留的旧仓位」被静默清零
|
||||
shares[s] = shares.get(s, 0.0) + invest / price_in
|
||||
entry_date[s] = d.date()
|
||||
entry_price[s] = price_in
|
||||
prev = self.prev_close.at[d, s] if d in self.prev_close.index else float("nan")
|
||||
if _nan(prev) or prev <= 0:
|
||||
self._no_prev_close_symbols.add(s) # 无前收→涨停不可判定,如实记入标注
|
||||
notional.append(budget)
|
||||
self.signal_history.append(
|
||||
ActionRecord(date=d.date(), symbol=s, signal="BUY", filled=True,
|
||||
price=round(price_in, 4))
|
||||
)
|
||||
if s not in self._traded:
|
||||
self._traded.add(s)
|
||||
self.traded_symbols.append(s)
|
||||
return True
|
||||
|
||||
# ---- 顺延买入(defer_buy):之后逐日重试 ----
|
||||
|
||||
def _fill_pending(self, d, cash, shares, entry_date, entry_price, pending, notional):
|
||||
close_d = self.close.loc[d]
|
||||
prev_d = self.prev_close.loc[d]
|
||||
remaining: list[PendingBuy] = []
|
||||
for order in pending:
|
||||
if order.symbol in shares and shares[order.symbol] > 0:
|
||||
continue # 期间已通过其他路径持有 → 撤销该顺延单
|
||||
c, p = close_d.get(order.symbol), prev_d.get(order.symbol)
|
||||
if _nan(c) or _nan(p) or p <= 0:
|
||||
remaining.append(order)
|
||||
continue
|
||||
if c / p >= _limit_up_ratio(order.symbol):
|
||||
remaining.append(order) # 仍涨停 → 继续顺延
|
||||
continue
|
||||
budget = min(order.budget, cash)
|
||||
if budget <= 1e-9:
|
||||
remaining.append(order) # 无可用现金(理论上不会发生)
|
||||
continue
|
||||
if not self._execute_buy(
|
||||
order.symbol, budget, d, c, shares, entry_date, entry_price, notional
|
||||
):
|
||||
remaining.append(order) # 预算不足:保留挂单(下日现金可能已变化)
|
||||
continue
|
||||
cash -= budget
|
||||
pending[:] = remaining
|
||||
return cash
|
||||
|
||||
# ---- 个股收益曲线 ----
|
||||
|
||||
def _accrue_symbol_returns(self, d, shares, cum, curve_rows) -> None:
|
||||
"""逐日累计各持仓股的「持仓期收益」(以建仓日收盘为 0% 基准)。
|
||||
|
||||
口径:cum 以 1.0 起算,仅在该股**持有期间**按日复利(close/prev_close)。
|
||||
本方法在当日调仓**之前**调用,因此:
|
||||
- 建仓当日不计收益(成交发生在当日收盘)→ 不存在当日买入当日计收益的未来函数
|
||||
- 卖出当日仍计收益(当日收益来自昨日持仓)
|
||||
未持有期间不产生数据点(曲线不落点),多段持仓则以 cum 连乘衔接;
|
||||
前端以买卖点标注区分各段持仓区间。
|
||||
"""
|
||||
prev_d = self.prev_close.loc[d]
|
||||
close_d = self.close.loc[d]
|
||||
for s, qty in shares.items():
|
||||
if qty <= 0:
|
||||
continue
|
||||
c, p = close_d.get(s), prev_d.get(s)
|
||||
if _nan(c) or _nan(p) or p <= 0:
|
||||
continue # 停牌/无前收:无有效收益
|
||||
cum[s] = cum.get(s, 1.0) * (float(c) / float(p))
|
||||
# 只为「当日持有」的股票落点(未持有期间不落点,显著压缩结果体积)
|
||||
for s, qty in shares.items():
|
||||
if qty <= 0:
|
||||
continue
|
||||
curve_rows.setdefault(s, []).append(
|
||||
CurvePoint(date=d.date(), value=round((cum.get(s, 1.0) - 1.0) * 100, 4))
|
||||
)
|
||||
|
||||
def _mark_curve_dates(self, d, shares, cum, curve_rows) -> None:
|
||||
"""为当日持有但尚未落点的股票补一个基准点(建仓当日 / 顺延成交当日)。
|
||||
|
||||
值为该股当前的 `cum`(新标的为 1.0 → 0%,复买标的延续上一段的累计值),
|
||||
因此曲线总能在买卖点当日取到数值,前端标注不会因缺数据点而被丢弃。
|
||||
"""
|
||||
day = d.date()
|
||||
for s, qty in shares.items():
|
||||
if qty <= 0:
|
||||
continue
|
||||
points = curve_rows.setdefault(s, [])
|
||||
if points and points[-1].date == day:
|
||||
continue
|
||||
points.append(
|
||||
CurvePoint(date=day, value=round((cum.get(s, 1.0) - 1.0) * 100, 4))
|
||||
)
|
||||
|
||||
# ---- 指标 ----
|
||||
|
||||
def _to_result(self, equity, trades, positions, notional) -> BacktestResult:
|
||||
def _to_result(self, equity, trades, positions, notional, cum, curve_rows) -> BacktestResult:
|
||||
start, end = equity.index[0].date(), equity.index[-1].date()
|
||||
init = float(self.spec.initial_capital)
|
||||
final = float(equity.iloc[-1])
|
||||
@@ -322,6 +666,7 @@ class TopKBacktestRunner:
|
||||
total_trades=len(trades),
|
||||
avg_turnover_pct=round(avg_turn, 2),
|
||||
)
|
||||
curves, curve_note = self._symbol_curves(curve_rows, cum)
|
||||
return BacktestResult(
|
||||
summary=summary,
|
||||
equity_curve=eq_pts,
|
||||
@@ -333,11 +678,78 @@ class TopKBacktestRunner:
|
||||
selection_history=self.selection_history,
|
||||
signal_history=self.signal_history,
|
||||
fills=[a for a in self.signal_history if a.filled],
|
||||
symbol_curves=curves,
|
||||
turnover_pct=round(sum(notional) / max(init, 1) * 100, 2),
|
||||
unimplemented=list(_DEFAULT_UNIMPLEMENTED) + unimplemented_notes(self.spec.portfolio),
|
||||
unimplemented=self._unimplemented(curve_note),
|
||||
config_snapshot=self.spec.model_dump(mode="json"),
|
||||
)
|
||||
|
||||
def _symbol_curves(self, curve_rows, cum) -> tuple[list[SymbolCurve], str | None]:
|
||||
"""按「期末收益绝对值」降序输出个股曲线(前端默认展示前若干只)。
|
||||
|
||||
返回 (曲线列表, 截断说明)。默认**不截断**(`config research.archive_curve_limit`
|
||||
为 null):期内持有的每只都输出,保证归档完整;体积由归档侧的字节预算兜底
|
||||
(见 experiment_archive)。仅当配置了数字上限时才截断,并如实标注哪一部分
|
||||
被丢弃、为什么(AGENT §24:不静默降级,绝不假装完整)。
|
||||
"""
|
||||
marks: dict[str, list[ActionRecord]] = {}
|
||||
for a in self.signal_history:
|
||||
if a.filled and a.symbol:
|
||||
marks.setdefault(a.symbol, []).append(a)
|
||||
out: list[SymbolCurve] = []
|
||||
for s, points in curve_rows.items():
|
||||
if not points:
|
||||
continue
|
||||
out.append(
|
||||
SymbolCurve(
|
||||
symbol=s,
|
||||
points=points,
|
||||
marks=marks.get(s, []),
|
||||
final_return_pct=round((cum.get(s, 1.0) - 1.0) * 100, 4),
|
||||
)
|
||||
)
|
||||
out.sort(key=lambda c: abs(c.final_return_pct), reverse=True)
|
||||
note = None
|
||||
limit = _resolved_curve_limit()
|
||||
if limit is not None and len(out) > limit:
|
||||
note = (
|
||||
f"个股收益曲线仅输出收益绝对值最大的 {limit} 只"
|
||||
f"(期内共持有 {len(out)} 只):完整明细见 trades / signal_history"
|
||||
)
|
||||
out = out[:limit]
|
||||
return out, note
|
||||
|
||||
def _unimplemented(self, curve_note: str | None = None) -> list[str]:
|
||||
notes = list(_DEFAULT_UNIMPLEMENTED) + unimplemented_notes(self.spec.portfolio)
|
||||
if self._no_prev_close_symbols:
|
||||
notes.append(
|
||||
f"有 {len(self._no_prev_close_symbols)} 只标的成交时缺少上一有效收盘价,"
|
||||
"无法判定涨停(数据窗口起点或长期停牌后复牌),按可买处理"
|
||||
)
|
||||
if curve_note:
|
||||
notes.append(curve_note)
|
||||
sel = self.spec.selection
|
||||
m = self.spec.effective_selection_months
|
||||
y = self.spec.effective_rebalance_months
|
||||
if m is not None and y is not None and y < m:
|
||||
notes.append(
|
||||
f"调仓间隔 y={y} 个月 < 择股间隔 m={m} 个月:两次择股之间会复用同一候选池"
|
||||
"(池子陈旧),并非每次调仓都重新择股"
|
||||
)
|
||||
if sel.defer_buy:
|
||||
notes.append(
|
||||
"顺延买入:调仓日涨停/停牌无法买入的标的挂单至之后首个可成交交易日,"
|
||||
"按该日收盘价成交;到下一次调仓仍未成交则作废并留作现金"
|
||||
)
|
||||
if self.spec.price_adjustment == "none":
|
||||
notes.append(
|
||||
"行情口径为不复权:现金分红未计入收益,除权日的价格下移会被计为亏损。"
|
||||
"股息类策略建议使用 price_adjustment=hfq(后复权)"
|
||||
)
|
||||
if not self.spec.conditions:
|
||||
notes.append("未配置选股过滤条件(conditions),候选池仅由 universe + 因子排序决定")
|
||||
return notes
|
||||
|
||||
|
||||
def run_spec_factor_test(
|
||||
daily: pd.DataFrame,
|
||||
@@ -357,7 +769,13 @@ def run_spec_factor_test(
|
||||
|
||||
|
||||
def _nan(v) -> bool:
|
||||
"""缺失判定:None / NaN / 不可转 float 一律视为「无有效值」。
|
||||
|
||||
注意 Series.get(key) 对不存在的键返回 None(而非 NaN),故必须把 None 判为缺失。
|
||||
"""
|
||||
if v is None:
|
||||
return True
|
||||
try:
|
||||
return bool(math.isnan(float(v)))
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
return True
|
||||
Reference in New Issue
Block a user