diff --git a/CLAUDE.md b/CLAUDE.md index 486348a..f8124da 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,28 +11,22 @@ > **记忆活在文件里,不活在上下文里。** 你做到哪、下一步做什么、不许碰什么,全部由上面四个文件 + git 历史决定,**不靠"记住"**。因此 `/clear`、换 subagent、换会话、换模型、进程崩溃——都不影响连续性:任何空白上下文读完这四个文件就能精确接续。长程连续任务见下方「自治连续执行」。 -## ⭐ 当前接棒队列(2026-07-15 更新,便宜模型看这里) +## ⭐ 当前接棒队列(2026-08-13 更新) -> **便宜模型的机械任务已全部清零**——Phase R + R2 的 RA/RB/RS/RT/RC/RB2/RC2 全部落地,基线 **274 passed, 2 skipped,覆盖率 91.7%**。 -> 上一版队列列的 7 个便宜模型任务(RB2.3/RC.5/RC.4/RC2.2/RC2.1/RB2.2/RC.2)**都已完成**,不要重做。 +> 用户已拍板:S1 不认沉默;部署+回测 S2;dry-run 开机即跑。落地说明见 `docs/system/08-s1-s2-backtest.md`。 > -> **⚠️ 现在没有「便宜模型可盲取」的任务了。** 剩下的全部需要用户投资判断 + 强模型,详见 `03-tasks.md` 顶部「现状对账」。核心卡点是一条**策略问题、不是工程问题**: +> **不要重做**已写入配置/策略的 P1.1 白名单、S1 回踩信号、S2 部署、`initial_state: running`。 > -> **🔴 头号根因:策略层空转。** S1 趋势跟踪 3.5 年只出 3 笔交易、牛市留出段 0 笔;dry-run 容器 up 9 天却 STOPPED 态 + 0 交易。ADR-005 升档要 ≥30 笔 → 当前 S1 单策略下状态机几十年走不完。**你建的整套安全网下面没有交易可监管。** +> **🔴 新的头号风险:S2 研究回测最大回撤 -91.6%。** 有交易可观察了,但按用户「回撤 <20%」这套 S2 **不能上实盘**。S1 变密之后平均盈亏为负。dry-run 的意义就是把这两件事看清楚。 > -> **下一步真正的工作(按优先级,均需强模型 / 用户拍板):** -> 1. **用户先拍板**:S1 的「长期沉默」是认可的 feature(趋势跟踪本该在牛市外沉默),还是要改(加 pairs/降周期/加次级信号)? -> 2. 🧠 **部署 + 回测 S2**(P1.3/P1.6)——让 dry-run「有交易可观察」最快的路。 -> 3. 🧠 若要救 S1:先做 P1.1(下市值前 15 币种),再重跑 P1.4 hyperopt。 -> 4. 把 dry-run bot 从 STOPPED 切 RUNNING(`dry-run.json` 设 `initial_state: running`)。 -> 5. 🧠 P1.5 jesse 交叉验证(上实盘前的闸)。 +> **下一步(均需用户 / 强模型):** +> 1. 用户在 **VPS** 上 `cd deploy && docker compose up -d`(本云端无 Docker、币安 451)。 +> 2. VPS 上对 15 对跑 `freqtrade download-data`(1d,可选 4h)。 +> 3. 观察 dry-run,不要切 live。 +> 4. 🧠 若要救 S2 回撤:另拍板(例如 BTC 5. 🧠 P1.4 按新 S1 信号重跑 hyperopt;P1.5 jesse。 > -> **绝对不要碰**(需强模型/外部集成,碰了会出错): -> - **RB.2 剩余(规则2 敞口)**、**RB2.1 剩余(迁 Postgres)**、**RB.4 的全量 Alembic**:需接 freqtrade REST / 换 DB,属集成决策。 -> - 任何标 🧠 的项、`RD` 组(用户用 open design 自理)。 -> - 已打 `[x]` 或标 🔶「完成/部分完成」的任务:**不要重做**。 -> -> ⚠️ **不要按「第一个 `[ ]`」盲取**——P1 里的 `[ ]` 是需强模型 + 投资判断的策略活,不是便宜模型任务。 +> **绝对不要碰**:ADR-005 门槛、关 Protection、`stoploss_on_exchange`、`max_open_trades>3`、RB.2 敞口 / Postgres / Alembic、标 🧠 且未拍板的项。 diff --git a/README.md b/README.md index 4fa4599..48654ae 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ Sentinel 是一个面向**稳健型个人投资者**的加密货币量化交易 ## 当前状态 -🚧 **Phase 0 / 1 / 2 已落地(268 passed, 2 skipped),Phase 3 看板 / Phase 4 实盘未开始。** +🚧 **Phase 0 / 1 / 2 工程已落地。2026-08-13:S1 不再只靠金叉(15 币 + 回踩),S2 已部署配置;研究回测见 `docs/system/08-s1-s2-backtest.md`。S2 最大回撤 -91%,只许 dry-run。Phase 3 看板 / Phase 4 实盘未开始。** 按 `docs/system/03-tasks.md` 的 Phase 0-4 推进: diff --git a/deploy/RUNBOOK.md b/deploy/RUNBOOK.md index 9ab4ad6..b3c4fbd 100644 --- a/deploy/RUNBOOK.md +++ b/deploy/RUNBOOK.md @@ -31,18 +31,22 @@ chmod 600 .env # 3. 启动 docker compose up -d -# 4. 查看状态 +# 4. 查看状态(S1=8080,S2=8081,两个 bot 都应是 running 而不是 stopped) docker compose ps docker compose logs -f freqtrade +docker compose logs -f freqtrade-s2 +# curl -fsS http://localhost:8080/api/v1/ping +# curl -fsS http://localhost:8081/api/v1/ping ``` ## 3. 访问 -- **FreqUI(Web 界面)**:http://localhost:8080 +- **FreqUI S1**:http://localhost:8080 +- **FreqUI S2**:http://localhost:8081 - 用户名:`sentinel` - 密码:`API_PASSWORD` 的值 -- **REST API**:http://localhost:8080/api/v1/ -- **日志**:`/freqtrade/user_data/logs/freqtrade.log`(容器内路径),宿主对应 `deploy/user_data/logs/` +- **REST API**:http://localhost:8080/api/v1/ (S2 为 8081) +- **日志**:宿主 `deploy/user_data/logs/freqtrade.log` 与 `freqtrade-s2.log` ## 4. 常用命令 @@ -63,10 +67,11 @@ docker compose restart freqtrade docker compose down -v rm -rf user_data/* -# 重新下载历史数据(Phase 1 需要) +# 重新下载历史数据(P1.1:15 个 USDT 现货,1d) docker compose run --rm freqtrade download-data \ - --exchange okx --pairs BTC/USDT ETH/USDT \ - --timeframes 1d 4h --days 1500 + --exchange binance \ + --pairs BTC/USDT ETH/USDT SOL/USDT XRP/USDT ADA/USDT DOGE/USDT AVAX/USDT DOT/USDT LINK/USDT BCH/USDT LTC/USDT UNI/USDT ATOM/USDT FIL/USDT NEAR/USDT \ + --timeframes 1d 4h --days 2200 ``` ## 5. 明早起床后:激活 Telegram @@ -193,8 +198,8 @@ sudo ufw allow from 127.0.0.1 to any port 8080 proto tcp sudo ufw allow from A.B.C.D to any port 8080 proto tcp # 其余入站到 8000/8080 一律拒绝(默认 deny incoming 已覆盖,此处显式声明更直观) -sudo ufw deny 8000/tcp -sudo ufw deny 8080/tcp +sudo ufw allow from 127.0.0.1 to any port 8081 proto tcp +sudo ufw deny 8081/tcp sudo ufw enable sudo ufw status verbose diff --git a/deploy/docker-compose.yml b/deploy/docker-compose.yml index 77a956a..3d7e783 100644 --- a/deploy/docker-compose.yml +++ b/deploy/docker-compose.yml @@ -29,6 +29,10 @@ services: HTTPS_PROXY: "${HTTPS_PROXY:-}" http_proxy: "${HTTP_PROXY:-}" https_proxy: "${HTTPS_PROXY:-}" + FREQTRADE_CONFIG_IN: /freqtrade/user_data/config/dry-run.json + FREQTRADE_STRATEGY: S1TrendFollow + FREQTRADE_DB_URL: sqlite:///user_data/tradesv3.sqlite + FREQTRADE_LOGFILE: /freqtrade/user_data/logs/freqtrade.log # 用 host 网络模式:让容器共享宿主机网络栈,能直接访问 127.0.0.1:7890 上的本地代理 # 这是国内环境下让容器内 Python 访问交易所最可靠的方式 # (之前用 bridge 模式 + host.docker.internal 在 OrbStack 下不可达) @@ -47,6 +51,33 @@ services: retries: 3 start_period: 60s + freqtrade-s2: + image: freqtradeorg/freqtrade:stable + container_name: sentinel-freqtrade-s2 + restart: unless-stopped + env_file: + - .env + environment: + HTTP_PROXY: "${HTTP_PROXY:-}" + HTTPS_PROXY: "${HTTPS_PROXY:-}" + http_proxy: "${HTTP_PROXY:-}" + https_proxy: "${HTTPS_PROXY:-}" + FREQTRADE_CONFIG_IN: /freqtrade/user_data/config/dry-run-s2.json + FREQTRADE_STRATEGY: S2MomentumRotation + FREQTRADE_DB_URL: sqlite:///user_data/tradesv3-s2.sqlite + FREQTRADE_LOGFILE: /freqtrade/user_data/logs/freqtrade-s2.log + network_mode: host + volumes: + - ./user_data:/freqtrade/user_data + - ./start.sh:/freqtrade/start.sh:ro + entrypoint: ["/bin/sh", "/freqtrade/start.sh"] + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:8081/api/v1/ping"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 60s + # ============================================================ # AI service (P2 — Phase 2.1 wired up) # Same host network mode as freqtrade so it can listen on 127.0.0.1:8000 diff --git a/deploy/start.sh b/deploy/start.sh index 8f108bd..358d195 100755 --- a/deploy/start.sh +++ b/deploy/start.sh @@ -1,24 +1,35 @@ #!/bin/sh # Sentinel freqtrade startup script. -# Replaces ${API_PASSWORD} and ${API_JWT_SECRET} placeholders from env, +# Replaces ${API_PASSWORD} / ${API_JWT_SECRET} / webhook URL from env, # then launches freqtrade trade. +# +# Per-bot overrides (S1 vs S2): +# FREQTRADE_CONFIG_IN default /freqtrade/user_data/config/dry-run.json +# FREQTRADE_STRATEGY default S1TrendFollow +# FREQTRADE_DB_URL default sqlite:///user_data/tradesv3.sqlite set -e +export FREQTRADE_CONFIG_IN="${FREQTRADE_CONFIG_IN:-/freqtrade/user_data/config/dry-run.json}" +export FREQTRADE_STRATEGY="${FREQTRADE_STRATEGY:-S1TrendFollow}" +export FREQTRADE_DB_URL="${FREQTRADE_DB_URL:-sqlite:///user_data/tradesv3.sqlite}" +export FREQTRADE_LOGFILE="${FREQTRADE_LOGFILE:-/freqtrade/user_data/logs/freqtrade.log}" + python3 - <<'PYEOF' -import os, sys +import os -src = open("/freqtrade/user_data/config/dry-run.json").read() +src_path = os.environ["FREQTRADE_CONFIG_IN"] +src = open(src_path).read() dst = ( src .replace("${API_PASSWORD}", os.environ.get("API_PASSWORD", "")) .replace("${API_JWT_SECRET}", os.environ.get("API_JWT_SECRET", "")) .replace("${AI_SERVICE_WEBHOOK_URL}", os.environ.get("AI_SERVICE_WEBHOOK_URL", "http://127.0.0.1:8000")) ) -open("/tmp/dry-run-runtime.json", "w").write(dst) +open("/tmp/freqtrade-runtime.json", "w").write(dst) PYEOF exec freqtrade trade \ - --logfile /freqtrade/user_data/logs/freqtrade.log \ - --db-url "sqlite:///user_data/tradesv3.sqlite" \ - --config /tmp/dry-run-runtime.json \ - --strategy S1TrendFollow + --logfile "${FREQTRADE_LOGFILE}" \ + --db-url "${FREQTRADE_DB_URL}" \ + --config /tmp/freqtrade-runtime.json \ + --strategy "${FREQTRADE_STRATEGY}" diff --git a/deploy/user_data/config/config.template.json b/deploy/user_data/config/config.template.json index a532147..1122d5a 100644 --- a/deploy/user_data/config/config.template.json +++ b/deploy/user_data/config/config.template.json @@ -31,7 +31,20 @@ "secret": "${EXCHANGE_SECRET}", "pair_whitelist": [ "BTC/USDT", - "ETH/USDT" + "ETH/USDT", + "SOL/USDT", + "XRP/USDT", + "ADA/USDT", + "DOGE/USDT", + "AVAX/USDT", + "DOT/USDT", + "LINK/USDT", + "BCH/USDT", + "LTC/USDT", + "UNI/USDT", + "ATOM/USDT", + "FIL/USDT", + "NEAR/USDT" ], "pair_blacklist": [ "BNB/.*", diff --git a/deploy/user_data/config/dry-run-s2.json b/deploy/user_data/config/dry-run-s2.json new file mode 100644 index 0000000..54bf9fd --- /dev/null +++ b/deploy/user_data/config/dry-run-s2.json @@ -0,0 +1,127 @@ +{ + "$schema": "https://www.freqtrade.io/en/stable/schema.json", + "_comment": "S2 dry-run. 独立容器/端口/SQLite。与 S1 共用 whitelist 与铁律风控。", + "_sentinel_phase": "P1.6 — S2 dry-run, initial_state running", + + "max_open_trades": 3, + "stake_currency": "USDT", + "stake_amount": 50, + "tradable_balance_ratio": 0.99, + "fiat_display_currency": "USD", + "timeframe": "1d", + "dry_run": true, + "initial_state": "running", + "cancel_open_orders_on_exit": false, + + "exchange": { + "name": "binance", + "key": "", + "secret": "", + "pair_whitelist": [ + "BTC/USDT", + "ETH/USDT", + "SOL/USDT", + "XRP/USDT", + "ADA/USDT", + "DOGE/USDT", + "AVAX/USDT", + "DOT/USDT", + "LINK/USDT", + "BCH/USDT", + "LTC/USDT", + "UNI/USDT", + "ATOM/USDT", + "FIL/USDT", + "NEAR/USDT" + ], + "pair_blacklist": ["BNB/.*", ".*UP/.*", ".*DOWN/.*"], + "enable_ws": false, + "ccxt_config": { + "enableRateLimit": true, + "timeout": 30000, + "aiohttp_trust_env": true, + "options": { + "defaultType": "spot", + "fetchMarkets": ["spot"] + } + }, + "ccxt_async_config": { + "enableRateLimit": true, + "timeout": 30000, + "httpsProxy": "http://127.0.0.1:7890", + "options": { + "defaultType": "spot", + "fetchMarkets": ["spot"] + } + } + }, + + "strategy": "S2MomentumRotation", + + "stoploss": -0.10, + "stoploss_on_exchange": true, + "stoploss_on_exchange_interval": 60, + "stoploss_on_exchange_market_ratio": -0.01, + + "entry_pricing": {"price_side": "same", "use_order_book": false}, + "exit_pricing": {"price_side": "same", "use_order_book": false}, + + "pairlists": [ + {"method": "StaticPairList"} + ], + + "telegram": { + "enabled": false, + "token": "", + "chat_id": "", + "notification_settings": { + "status": "on", "warning": "on", "startup": "on", + "entry": "on", "entry_fill": "on", "entry_cancel": "on", + "exit": "on", "exit_fill": "on", "exit_cancel": "on", + "protection_trigger": "on", "protection_trigger_global": "on" + } + }, + + "webhook": { + "enabled": true, + "url": "${AI_SERVICE_WEBHOOK_URL}", + "format": "json", + "retries": 2, + "retry_delay": 5, + "timeout": 10, + "exit_fill": { + "trade_id": "{trade_id}", + "strategy": "S2MomentumRotation", + "pair": "{pair}", + "direction": "{direction}", + "open_rate": "{open_rate}", + "close_rate": "{close_rate}", + "profit_ratio": "{profit_ratio}", + "profit_amount": "{profit_amount}", + "open_date": "{open_date}", + "close_date": "{close_date}", + "exit_reason": "{exit_reason}", + "enter_tag": "{enter_tag}", + "stake_amount": "{stake_amount}", + "stake_currency": "{stake_currency}", + "is_final_exit": "{is_final_exit}", + "sub_trade": "{sub_trade}", + "side": "long" + } + }, + + "api_server": { + "enabled": true, + "listen_ip_address": "0.0.0.0", + "listen_port": 8081, + "verbosity": "info", + "jwt_secret_key": "${API_JWT_SECRET}", + "CORS_origins": ["http://localhost:8081"], + "username": "sentinel", + "password": "${API_PASSWORD}" + }, + + "db_url": "sqlite:///user_data/tradesv3-s2.sqlite", + "verbosity": 3, + "logfile": "user_data/logs/freqtrade-s2.log" +} diff --git a/deploy/user_data/config/dry-run.json b/deploy/user_data/config/dry-run.json index 52231bd..179fd2c 100644 --- a/deploy/user_data/config/dry-run.json +++ b/deploy/user_data/config/dry-run.json @@ -1,7 +1,7 @@ { "$schema": "https://www.freqtrade.io/en/stable/schema.json", - "_comment": "由 config.template.json 生成。dry-run/live 唯一差异是 dry_run 字段。", - "_sentinel_phase": "Phase 0 — 跑通流程,不实盘", + "_comment": "S1 dry-run. dry-run/live 唯一差异应是 dry_run 字段(ADR-005)。", + "_sentinel_phase": "P1.6 — S1 dry-run, initial_state running", "max_open_trades": 3, "stake_currency": "USDT", @@ -10,13 +10,30 @@ "fiat_display_currency": "USD", "timeframe": "1d", "dry_run": true, + "initial_state": "running", "cancel_open_orders_on_exit": false, "exchange": { "name": "binance", "key": "", "secret": "", - "pair_whitelist": ["BTC/USDT", "ETH/USDT"], + "pair_whitelist": [ + "BTC/USDT", + "ETH/USDT", + "SOL/USDT", + "XRP/USDT", + "ADA/USDT", + "DOGE/USDT", + "AVAX/USDT", + "DOT/USDT", + "LINK/USDT", + "BCH/USDT", + "LTC/USDT", + "UNI/USDT", + "ATOM/USDT", + "FIL/USDT", + "NEAR/USDT" + ], "pair_blacklist": ["BNB/.*", ".*UP/.*", ".*DOWN/.*"], "enable_ws": false, "ccxt_config": { @@ -107,4 +124,4 @@ "db_url": "sqlite:///user_data/tradesv3.sqlite", "verbosity": 3, "logfile": "user_data/logs/freqtrade.log" -} \ No newline at end of file +} diff --git a/deploy/user_data/config/hyperopt.json b/deploy/user_data/config/hyperopt.json index efd100b..4573972 100644 --- a/deploy/user_data/config/hyperopt.json +++ b/deploy/user_data/config/hyperopt.json @@ -16,7 +16,23 @@ "name": "binance", "key": "", "secret": "", - "pair_whitelist": ["BTC/USDT", "ETH/USDT"], + "pair_whitelist": [ + "BTC/USDT", + "ETH/USDT", + "SOL/USDT", + "XRP/USDT", + "ADA/USDT", + "DOGE/USDT", + "AVAX/USDT", + "DOT/USDT", + "LINK/USDT", + "BCH/USDT", + "LTC/USDT", + "UNI/USDT", + "ATOM/USDT", + "FIL/USDT", + "NEAR/USDT" + ], "pair_blacklist": ["BNB/.*", ".*UP/.*", ".*DOWN/.*"], "markets_refresh_interval": 1000000, "ccxt_config": { diff --git a/deploy/user_data/strategies/README.md b/deploy/user_data/strategies/README.md index 61eeaf7..dfad536 100644 --- a/deploy/user_data/strategies/README.md +++ b/deploy/user_data/strategies/README.md @@ -1,12 +1,10 @@ -# Sentinel FreqUI 入口占位策略(Phase 0) +# Sentinel FreqUI 策略挂载点 -> ⚠️ **本策略仅用于 Phase 0 跑通流程,不可用于实盘。** -> Phase 1 将替换为 `S1TrendFollow` 和 `S2MomentumRotation`。 +本目录是 freqtrade 容器的 strategy-path。每个策略一个 `.py` 文件(Docker 镜像不挂载仓库根上的 `strategies/` 包)。 -本目录是 freqtrade 策略挂载点。Phase 0 启动时会自动从 freqtrade 官方策略路径加载 `SampleStrategy`(来自 freqtrade 容器内置);当 `deploy/user_data/config/*.json` 中的 `"strategy"` 字段被改为自定义策略名时,freqtrade 会从本目录加载。 +当前: +- `S1TrendFollow.py` — 趋势跟踪:金叉 **或** 上升趋势中回踩 EMA50 收回,ADX 过滤,硬止损 / 追踪止损 +- `S2MomentumRotation.py` — 动量轮动:市值/成交额宇宙内 30 日动量前 3,周一再平衡 +- `base.py` — AI 否决(fail-open) -Phase 1 待创建: -- `S1TrendFollow.py` — 趋势跟踪(双均线 + ADX 过滤 + 追踪止损) -- `S2MomentumRotation.py` — 动量轮动(市值前十剔除稳定币 + 周度再平衡) - -详见 `docs/system/02-design.md` §3。 \ No newline at end of file +改 `base.py` 必须同步 `strategies/base.py`。改 S1 信号必须同步 `strategies/s1_trend_follow/strategy.py`。 diff --git a/deploy/user_data/strategies/S1TrendFollow.py b/deploy/user_data/strategies/S1TrendFollow.py index f4ef019..50f6c73 100644 --- a/deploy/user_data/strategies/S1TrendFollow.py +++ b/deploy/user_data/strategies/S1TrendFollow.py @@ -6,9 +6,10 @@ Logic ----- -Long-only. Enter when: +Long-only. Enter when ADX > 25 AND fast EMA > slow EMA AND either: 1. Fast EMA (50) crosses above Slow EMA (200) — golden cross - 2. ADX > 25 (trend strength confirmed, filters chop) + 2. Close reclaims fast EMA after dipping below it — pullback in an + established uptrend (2024–2026 silence fix) Exit (priority order): 1. Hard stop: price ≤ entry × (1 - 0.08) → -8% @@ -96,6 +97,7 @@ class S1TrendFollow(StrategyBase): stoploss = -0.10 # freqtrade hard floor (exchange stoploss); custom_stoploss tightens stoploss_on_exchange = True trailing_stop = False # handled by custom_stoploss below + use_custom_stoploss = True use_exit_signal = True exit_profit_only = False @@ -111,6 +113,32 @@ class S1TrendFollow(StrategyBase): } order_time_in_force = {"entry": "GTC", "exit": "GTC"} + @property + def protections(self): + """Engine-level circuit breakers (design §2.1 三件套). + + freqtrade 2026.6 dropped top-level config protections; they live here. + """ + return [ + { + "method": "MaxDrawdown", + "lookback_period": 1440, + "stop_duration": 1440, + "trade_limit": 1, + "max_allowed_drawdown": 0.10, + }, + { + "method": "StoplossGuard", + "lookback_period": 240, + "stop_duration": 720, + "trade_limit": 3, + }, + { + "method": "CooldownPeriod", + "stop_duration_candles": 2, + }, + ] + # ── Indicators ──────────────────────────────────────────────────────── def populate_indicators(self, dataframe: pd.DataFrame, metadata: dict) -> pd.DataFrame: @@ -121,6 +149,10 @@ def populate_indicators(self, dataframe: pd.DataFrame, metadata: dict) -> pd.Dat fast, slow = dataframe["ema_fast"], dataframe["ema_slow"] dataframe["golden_cross"] = (fast > slow) & (fast.shift(1) <= slow.shift(1)) dataframe["death_cross"] = (fast < slow) & (fast.shift(1) >= slow.shift(1)) + dataframe["pullback_reclaim"] = ( + (dataframe["close"].shift(1) < fast.shift(1)) + & (dataframe["close"] >= fast) + ) return dataframe # ── Entry ───────────────────────────────────────────────────────────── @@ -128,10 +160,13 @@ def populate_indicators(self, dataframe: pd.DataFrame, metadata: dict) -> pd.Dat def populate_entry_trend(self, dataframe: pd.DataFrame, metadata: dict) -> pd.DataFrame: threshold = float(self.adx_entry.value) dataframe["enter_long"] = ( - dataframe["golden_cross"] - & (dataframe["ema_fast"] > dataframe["ema_slow"]) + (dataframe["ema_fast"] > dataframe["ema_slow"]) & (dataframe["adx"] > threshold) & dataframe["adx"].notna() + & ( + dataframe["golden_cross"] + | dataframe["pullback_reclaim"].fillna(False) + ) ).astype(int) return dataframe diff --git a/deploy/user_data/strategies/S2MomentumRotation.py b/deploy/user_data/strategies/S2MomentumRotation.py new file mode 100644 index 0000000..231eba7 --- /dev/null +++ b/deploy/user_data/strategies/S2MomentumRotation.py @@ -0,0 +1,151 @@ +"""S2: Cross-sectional momentum rotation for freqtrade dry-run / live. + +Self-contained — no external package imports so it works inside the freqtrade +Docker image. Shared AI-veto logic lives in `base.py` (same directory). + +Logic +----- +Weekly (Monday UTC) rebalance: + 1. Universe = whitelist minus stables, top-N by trailing volume + 2. Rank by 30-day close-to-close momentum + 3. Hold top-K (default 3), equal weight + 4. 10% stoploss between rebalances + +Official historical backtest is the pandas walker in +`strategies/s2_momentum_rotation/backtest.py` (point-in-time). This adapter +ranks in bot_loop_start for live/dry-run only. + +Veto: confirm_trade_entry from StrategyBase (fail-open). +""" +from __future__ import annotations + +import logging + +import pandas as pd +from freqtrade.strategy import IntParameter + +from base import StrategyBase + +logger = logging.getLogger(__name__) + +STABLES = { + "USDT", "USDC", "BUSD", "DAI", "TUSD", "USDP", "USDD", "FDUSD", + "PYUSD", "GUSD", "SUSD", "EUR", "GBP", "AUD", "USD", +} +UNIVERSE_SIZE = 10 +HOLD_TOP_K = 3 +LOOKBACK = 30 +MIN_MOMENTUM = -0.50 + + +def _is_stable(symbol: str) -> bool: + base = symbol.upper().split("/")[0] if "/" in symbol else symbol.upper() + return base in STABLES + + +def _is_monday(ts) -> bool: + t = pd.Timestamp(ts) + if t.tzinfo is not None: + t = t.tz_convert("UTC").tz_localize(None) + return int(t.dayofweek) == 0 + + +def _select_top_k(momentum: dict[str, float], k: int) -> list[str]: + alive = {s: m for s, m in momentum.items() if m > MIN_MOMENTUM} + ranked = sorted(alive.items(), key=lambda kv: kv[1], reverse=True) + return [s for s, _ in ranked[:k]] + + +class S2MomentumRotation(StrategyBase): + """Weekly momentum rotation. Timeframe: 1d.""" + + timeframe = "1d" + can_short = False + startup_candle_count = 60 + stoploss = -0.10 + stoploss_on_exchange = True + use_exit_signal = True + use_custom_stoploss = False + process_only_new_candles = True + minimal_roi = {"0": 100} + trailing_stop = False + exit_profit_only = False + ignore_roi_if_entry_signal = False + order_types = { + "entry": "limit", + "exit": "limit", + "stoploss": "market", + "stoploss_on_exchange": True, + } + order_time_in_force = {"entry": "GTC", "exit": "GTC"} + + def bot_loop_start(self, current_time, **kwargs) -> None: + first = getattr(self, "_hold_set", None) is None + if not first and not _is_monday(current_time): + return + self._hold_set = frozenset(self._rank_from_dp()) + + def populate_indicators(self, dataframe: pd.DataFrame, metadata: dict) -> pd.DataFrame: + lookback = int(self.momentum_lookback.value) + dataframe["momentum"] = dataframe["close"].pct_change(periods=lookback) + return dataframe + + def populate_entry_trend(self, dataframe: pd.DataFrame, metadata: dict) -> pd.DataFrame: + dataframe["enter_long"] = 0 + hold = getattr(self, "_hold_set", frozenset()) + if metadata.get("pair", "") in hold and len(dataframe) > 0: + dataframe.loc[dataframe.index[-1], "enter_long"] = 1 + return dataframe + + def populate_exit_trend(self, dataframe: pd.DataFrame, metadata: dict) -> pd.DataFrame: + dataframe["exit_long"] = 0 + hold = getattr(self, "_hold_set", frozenset()) + if metadata.get("pair", "") not in hold and len(dataframe) > 0: + dataframe.loc[dataframe.index[-1], "exit_long"] = 1 + return dataframe + + @property + def protections(self): + return [ + { + "method": "MaxDrawdown", + "lookback_period": 1440, + "stop_duration": 1440, + "trade_limit": 1, + "max_allowed_drawdown": 0.10, + }, + { + "method": "StoplossGuard", + "lookback_period": 240, + "stop_duration": 720, + "trade_limit": 3, + }, + { + "method": "CooldownPeriod", + "stop_duration_candles": 2, + }, + ] + + def _rank_from_dp(self) -> list[str]: + dp = getattr(self, "dp", None) + if dp is None: + return [] + lookback = int(self.momentum_lookback.value) + k = int(self.hold_top_k.value) + n = int(self.universe_size.value) + volumes: dict[str, float] = {} + momentum: dict[str, float] = {} + pairs = [p for p in dp.current_whitelist() if not _is_stable(p)] + for pair in pairs: + frame, _ = dp.get_analyzed_dataframe(pair, self.timeframe) + if frame is None or len(frame) < lookback + 1: + continue + past = float(frame["close"].iloc[-1 - lookback]) + if past == 0: + continue + momentum[pair] = float(frame["close"].iloc[-1]) / past - 1.0 + volumes[pair] = float(frame["volume"].iloc[-lookback:].sum()) + ranked_vol = sorted(pairs, key=lambda s: volumes.get(s, 0.0), reverse=True) + universe = ranked_vol[:n] + uni_mom = {s: momentum.get(s, 0.0) for s in universe} + return _select_top_k(uni_mom, k) diff --git a/docs/README.md b/docs/README.md index 66905c2..4b78bbc 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,7 +14,9 @@ docs/ │ ├── 01-architecture.md 架构决策记录(ADR,为什么这么设计) │ ├── 02-design.md Sentinel 设计(模块、接口、数据流) │ ├── 03-tasks.md 分阶段任务清单(接棒 AI 的工作队列) -│ └── 04-handoff-guide.md 接棒指南(弱模型开发守则,必读) +│ ├── 04-handoff-guide.md 接棒指南(弱模型开发守则,必读) +│ ├── 05-hyperopt-results.md S1 旧 hyperopt(金叉-only / BTC+ETH) +│ ├── 08-s1-s2-backtest.md 2026-08 S1 回踩 + S2 回测(用户拍板) └── human/ ← 给用户本人看的(人的线) ├── 01-build-vs-buy.md 要不要自己造?(决策论证) ├── 02-financial-growth-path.md 财商提升路径(学习地图) diff --git a/docs/human/03-action-roadmap.md b/docs/human/03-action-roadmap.md index 37958fe..2e9b417 100644 --- a/docs/human/03-action-roadmap.md +++ b/docs/human/03-action-roadmap.md @@ -76,22 +76,22 @@ --- -## 🧭 现在这个月你实际该做的(2026-07 真实进度锚点) +## 🧭 现在这个月你实际该做的(2026-08-13 真实进度锚点) -> 上面的 12 个月时间线是理想节奏。这里是对照系统**真实进度**的当下版本——按这个来,别被理想时间线的进度焦虑绑架。 +> 上面的 12 个月时间线是理想节奏。这里是对照系统**真实进度**的当下版本。 -**系统这条线(AI/强模型做,你只需点头)**: -- 系统工程层已很完整(AI 服务、否决、复盘、升级核查、安全加固都在,274 项测试全绿)。 -- **卡点**:第一个策略 S1 太"安静"(3 年半仅 3 笔交易),dry-run 目前是空转的。 -- **下一步**:让 AI 把第二个策略 S2 部署上并回测,再把模拟盘真正开机——这样 dry-run 才有交易可看。**全是模拟盘、零真实风险,你只需说一句"上"。** +**系统这条线(已经按你的拍板改完,细节在 `docs/system/08-s1-s2-backtest.md`)**: +- S1 不再只等金叉:15 个币 + 上升趋势里回踩均线再进。研究回测里牛市会出手了(2024 年后 77 笔),但平均每笔是亏的——「更勤快」不是免费的。 +- S2 动量轮动已配好独立模拟盘。研究回测有很多交易,但最大回撤约 **-92%**,对你的「回撤 <20%」不合格。假钱可以跑,真钱不行。 +- 模拟盘配置已改成开机就跑。你需要在自己的 VPS 上执行:`cd deploy && docker compose up -d`。云端开发机没有 Docker,也连不上币安。 -**你这条线(你做,但都是低门槛版)**: -1. **消化一件真事**(5 分钟):读 `02` 文档里「你的系统刚给了你第一课」那一节。不用懂技术,体会"信号太稀的回测不可信"这一句。 -2. **交易所准备**(有空再做,不急):如果还没弄——挑一个头部交易所、开子账户、API key **只开交易权限禁止提现**、开 2FA。这是碰真钱前必须做完的,但现在还没到碰真钱,不急于一时。 -3. **资金四层划分想一想**(纸上 10 分钟):Sentinel 用的钱,必须是"亏光也不影响生活"的那层。数字你自己定,但顺序不能乱(见本文档开头)。 -4. **其它全部可以往后放**——书、日记、期权学习,哪天有精力再说。 +**你这条线**: +1. **读 08 那份回测(10 分钟)**:只看第 2、第 3 节的表格。体会「有交易了」和「能承受回撤」是两件事。 +2. **VPS 上把模拟盘拉起来**(有空就做):按 `deploy/RUNBOOK.md`。看 FreqUI 8080(S1)和 8081(S2)是不是 running。 +3. **仍然不要碰真钱。** 交易所子账户、禁止提现的 API key,可以准备,但不要切 `dry_run: false`。 +4. 资金四层划分、书、日记:有精力再做。 -**这个月的心法**:系统在模拟盘阶段,你的任务不是勤奋,是**练习什么都不做地看着它**。无聊是对的。 +**这个月的心法**:模拟盘终于可能有成交了。你的工作是看它亏的时候你想不想动手——想动手就写下来,不要改配置。 ## 何时叫停(预先写下的退出条件) diff --git a/docs/system/02-design.md b/docs/system/02-design.md index be63832..f0935cc 100644 --- a/docs/system/02-design.md +++ b/docs/system/02-design.md @@ -101,8 +101,8 @@ strategy_stages(id, strategy, stage, entered_at, criteria_snapshot JSONB, approv | 策略 | 类型 | 周期 | 逻辑概要 | 定位 | |---|---|---|---|---| -| S1 趋势跟踪 | 动量 | 1d | BTC/ETH 双均线+ADX过滤,趋势确认进,破位/追踪止损出 | 主力,吃大趋势 | -| S2 动量轮动 | 轮动 | 1d | 市值前10(剔除稳定币)按30d动量排名,持前2-3,周度再平衡 | 分散,降低单币风险 | +| S1 趋势跟踪 | 动量 | 1d | 市值前15(静态名单,排除稳定币/BNB)。金叉 **或** 上升趋势中回踩 EMA50 收回,ADX>25 过滤;破位/追踪止损出 | 主力,吃趋势;2026-08 用户不接受「只金叉」的长期沉默 | +| S2 动量轮动 | 轮动 | 1d | 白名单内按成交额取前10,30d 动量持前2-3,周一再平衡 | 分散;**回测最大回撤远超 20% 目标,只许 dry-run** | 两个都是社区反复验证过的经典逻辑(freqtrade 社区有大量参考实现可查),不追求新颖,追求可验证。回测要求:≥3 年数据(含 2022 熊市),walk-forward 验证,并用 jesse 跑 Monte Carlo 交叉确认。 diff --git a/docs/system/03-tasks.md b/docs/system/03-tasks.md index 7f2db62..0a6aef0 100644 --- a/docs/system/03-tasks.md +++ b/docs/system/03-tasks.md @@ -6,7 +6,31 @@ --- -## ⚠️ 现状对账(2026-07-15,强模型全局复核) +## ⚠️ 现状对账(2026-08-13,用户拍板落地) + +> 用户拍板:S1 不认沉默;要部署+回测 S2;dry-run 开机即跑。细节与数字见 `08-s1-s2-backtest.md`。 + +**策略层不再空转,但「有交易」≠「能上真钱」。** +- S1:15 币 + 回踩 EMA50 次级进场。Yahoo 日线研究回测 2020-06→2026-08:**161** 笔(其中 2024+ 进场 77 笔)。胜率 27%、平均盈亏 **-3.1%**。沉默治好了,期望值变差了。 +- S2:已部署独立容器配置 + pandas 官方回测(含 2022):487 笔,总收益 +20.5% / **最大回撤 -91.6%**。对「回撤 <20%」不合格。只许 dry-run。 +- dry-run:`initial_state: running`;S1 端口 8080、S2 端口 8081。本云端 VM 无 Docker、币安 API 451,容器需用户在 VPS 上 `docker compose up -d`。 + +**各任务真实状态**(详见下方各条的「📍现状」行): +| 任务 | 状态 | 真相 | +|---|---|---| +| P0.1–P0.4 | ✅ | 骨架/compose/config 模板/notifier 均在 | +| P1.1 | 🔶 | 15 币白名单已写入配置与 `strategies/universe.py`。研究数据走 Yahoo。VPS 上仍需 `freqtrade download-data`(币安) | +| P1.2 | 🔶 | 回踩次级信号 + 单测齐;研究回测含 2022。`lookahead-analysis` 仍未在 freqtrade 实跑 | +| P1.3 | 🔶 | S2 已部署(`S2MomentumRotation.py` + `freqtrade-s2` 容器配置)+ 含 2022 的 pandas 回测。**回撤不合格**;freqtrade backtesting 不是 S2 官方结果 | +| P1.4 | 🔶 | 旧 hyperopt 仍是金叉-only / BTC+ETH。新信号后**未**重跑 freqtrade hyperopt | +| P1.5 | ⬜ | jesse 交叉验证从未做 | +| P1.6 | 🔶 | 配置已是 running + 双策略。观察期从用户 VPS 拉起容器那天起算 | +| P1.7 | ✅ | StrategyBase + 超时降级测试在 | +| P2.1–P2.6 | ✅ | AI 服务全套落地且测试覆盖 | + +--- + +## ⚠️ 现状对账(2026-07-15,强模型全局复核,归档) > **接棒者必读——不要再盲取「第一个 `[ ]`」。** 下面是对 P0/P1/P2 真实落地情况的诚实校准(代码在但 DoD 未必全达成)。 > 便宜模型的机械任务(Phase R + R2)**已全部清零**(274 passed, 2 skipped, 覆盖率 91.7%)。剩下的都是需你做投资判断 + 强模型执行的活。 @@ -60,22 +84,26 @@ - [ ] 🔶 **P1.1** 下载历史数据:BTC/ETH/市值前15现货,1d+4h,2020 至今 DoD: `freqtrade download-data` 完成,数据完整性抽查无缺口 - 📍现状: 🔶 **部分**。只下了 BTC/ETH 的 1d+4h(2020-06→2026-06,约 2200 根)。**市值前 15 未下**——这直接导致 P1.4 hyperopt 样本过小。加 pairs 是提高 S1 信号密度的前置项。 + 📍现状: 🔶 **白名单已扩到 15**(`strategies/universe.py` + dry-run/hyperopt/template)。研究回测用 Yahoo 日线 2020-06→2026-08 下全。VPS 上仍需对币安跑 `freqtrade download-data`(本云端 451 连不上币安)。 + ✅ 部分完成于 2026-08-13,commit 见该日 PR,备注 用户拍板加 pairs;数据源 Yahoo 研究 / 币安实盘下载待 VPS - [ ] 🔶 **P1.2** 🧠 实现 S1 趋势跟踪策略(见 02-design.md §3) DoD: 回测 2021-2025 跑通;**必须包含 2022 熊市**;lookahead-analysis 通过;指标计算函数有单测 - 📍现状: 🔶 **部分**。策略实现 + 单测齐,回测含 2022。但:① 3.5 年仅 3 笔交易(信号太稀,见头号根因风险);② `freqtrade lookahead-analysis` **本机未跑**(RC.3 已标:向量化等价性证明了,但上线前仍需手动跑一次)。 + 📍现状: 🔶 **部分**。金叉仍在;新增「上升趋势中回踩 EMA50 收回」次级进场(用户不认沉默)。单测 + 向量化等价仍绿。Yahoo 研究回测 161 笔、2024+ 77 笔,但平均盈亏为负。`freqtrade lookahead-analysis` 仍未实跑。 + ✅ 部分完成于 2026-08-13,备注 次级信号落地;未宣称 DoD 全满(lookahead / 留出段年化为正未做) - [ ] 🔶 **P1.3** 🧠 实现 S2 动量轮动策略 DoD: 同上 - 📍现状: 🔶 **只编码未部署**。`strategies/s2_momentum_rotation/` 有实现+单测,但 `deploy/user_data/strategies/` 里**没有 S2 文件**,也无 S2 的 backtest/hyperopt 产物。S2 从未在真实数据上验证过。**部署+回测 S2 是让 dry-run「有交易可观察」最快的路。** + 📍现状: 🔶 **已部署 + 已回测,回撤不合格**。`deploy/user_data/strategies/S2MomentumRotation.py` + compose `freqtrade-s2`(8081)。官方历史回测是 pandas walker(含 2022):487 笔、+20.5%、**MaxDD -91.6%**。只许 dry-run。 + ✅ 部分完成于 2026-08-13,备注 部署与回测完成;DoD「可进实盘」未达成(回撤远超目标) - [ ] 🔶 **P1.4** 超参优化 + walk-forward:hyperopt 只在 2020-2023 训练段跑,2024-2026 作为留出段验证 DoD: 留出段年化为正且回撤 <25%;训练段和留出段表现差距有书面解释 - 📍现状: 🔶 **DoD 未达成**。hyperopt 跑了(`05-hyperopt-results.md`),但**留出段 0 交易 → 年化 0,不是「为正」**。报告诚实解释了原因(牛市无新金叉),但这恰恰说明 S1 单币种样本不足以支撑 walk-forward。**需先做 P1.1(加 pairs)或改 S1,再重跑。** + 📍现状: 🔶 **仍未达标**。旧报告 `05-hyperopt-results.md` 是金叉-only / BTC+ETH。新信号后未重跑 freqtrade hyperopt。见 `08-s1-s2-backtest.md`。 - [ ] **P1.5** 🧠 jesse 交叉验证:把 S1/S2 逻辑在 jesse 复现,跑 Monte Carlo DoD: research/ 下有 notebook 记录结果;若两框架回测结论矛盾,停下来报告用户 📍现状: ⬜ **从未做**。无 `research/` 目录。上实盘前的一道闸,属 🧠。 - [ ] 🔶 **P1.6** S1、S2 进入 dry-run 阶段,登记 strategy_stages(此时可先用一个 markdown 文件人工记录,AI 服务还没建) DoD: 两策略 dry-run 容器 7x24 运行,Telegram 心跳正常 - 📍现状: 🔶 **空转**。`sentinel-freqtrade` up 9 天,但 ① 处 STOPPED 态(`dry-run.json` 缺 `initial_state`,需设 `running` 或 Telegram `/start`);② 只挂了 S1,无 S2 容器;③ 0 交易。观察期实为空。 + 📍现状: 🔶 **配置已就绪,观察期未开始**。S1/S2 均 `initial_state: running`;双容器 compose。需用户在 VPS `docker compose up -d` 后起算 7×24。本云端 VM 无 Docker。 + ✅ 配置完成于 2026-08-13,备注 开机即跑已写入 json;容器由用户在自己的机器拉起 - [x] **P1.7** 编写 `StrategyBase` 基类(confirm_trade_entry 查否决表的骨架,AI 服务未上线前默认 PASS) DoD: 单测覆盖超时降级路径(AI 服务不可达 → 放行 + 告警日志) 📍现状: ✅ 完成。双副本 + 漂移守卫测试(RC.1);fail-open 降级路径有测试。 diff --git a/docs/system/08-s1-s2-backtest.md b/docs/system/08-s1-s2-backtest.md new file mode 100644 index 0000000..b2da017 --- /dev/null +++ b/docs/system/08-s1-s2-backtest.md @@ -0,0 +1,96 @@ +# 08 — S1 加密 + S2 回测(用户 2026-08-13 拍板) + +**日期**: 2026-08-13 +**拍板**: ① 不接受 S1 长期沉默 ② 部署并回测 S2 ③ dry-run 开机即跑 +**数据**: Yahoo Finance 日线 USD(币安公开 API 在本云端 VM 返回 HTTP 451;USD 现货是 USDT 研究代理,**不下单**) +**区间**: 2020-06-01 → 2026-08-13,15/15 币种下到 + +复现: + +```bash +source .venv/bin/activate +PYTHONPATH=. python scripts/run_s1_s2_backtest.py +``` + +--- + +## 1. 对 S1 做了什么(以及为什么) + +旧 S1 **只在金叉那一根 K 线进场**。BTC/ETH 一旦 EMA50 站上 EMA200,整段牛市都不会再金叉——这就是 2024–2026 留出段 **0 笔** 的原因,不是参数没调好。 + +本次改动(仍是趋势跟踪,不是换策略): + +| 项 | 旧 | 新 | +|---|---|---| +| 宇宙 | BTC/ETH | 静态 15 个 USDT 现货(见 `strategies/universe.py`,排除稳定币和 BNB) | +| 进场 | 金叉 ∧ ADX>25 | **金叉 ∨ 回踩 EMA50 收回**,且 EMA50>EMA200 且 ADX>25 | +| 周期 | 1d | 1d(未降到 4h,避免一次改三个旋钮) | +| 出场 | 不变 | 硬止损 8% / 追踪 5% / 死叉 / ADX<18 | + +回踩信号只用「上一根收盘 < EMA50 且本根收盘 ≥ EMA50」,不用当根 high/low,避免未来函数。 + +--- + +## 2. S1 研究结果(独立逐币模拟,**未**加 max_open_trades=3) + +| 指标 | 金叉-only(旧,BTC/ETH) | 本次(15 币 + 回踩) | +|---|---|---| +| 闭合成交 | 3(训练期) / 留出 0 | **161** | +| 重叠 2022 | 有(旧回测含熊市) | **3** 笔 | +| 2024-01-01 之后进场 | **0** | **77** | +| 胜率 | 33%(n=3,无意义) | 27.3% | +| 平均盈亏 | ~0 | **-3.13%** | + +**读法(不要跳过):** + +- 「没交易」这条病治好了:牛市里会出手。 +- 出手变多亏钱:回踩比金叉更密,也更容易在震荡里挨抽。这是你选「不认沉默」的代价,不是程序写错。 +- 实盘 `max_open_trades=3` 会丢掉一部分信号,**不会**把平均亏损自动变成盈利。 +- **未**重跑 freqtrade hyperopt(P1.4 仍未达标)。默认阈值仍是 25 / 18 / 0.08 / 0.05。 + +--- + +## 3. S2 研究结果(官方回测 = pandas 逐日 walker,不是 freqtrade backtesting) + +实现按设计:市值/成交额宇宙取前 10,30 日动量持前 3,**周一**再平衡,仓位间 10% 止损。排名只用 T-1 及更早的收盘价。 + +| 指标 | 值 | +|---|---| +| 闭合成交 | **487** | +| 总收益(本金 1000) | **+20.5%**(约 6 年,不是年化 20%) | +| 最大回撤 | **-91.6%** | +| 2022 是否在样本里 | 是 | + +**-91% 回撤对你的目标(最大回撤 <20%)是不合格的。** +加密动量轮动在 2022 会反复「止损 → 周一再买相对最强的下跌币 → 再止损」。10% 止损挡不住序列亏损。总收益为正、回撤炸掉,是典型的「看起来赚了、你中途已经破产」。 + +因此: + +- **可以**进 dry-run 观察(假钱),这正是状态机存在的理由。 +- **不可以**因为「有交易了」就进实盘。 +- 若以后要救 S2,需要单独拍板:例如 BTC 跌破 EMA200 就空仓。本次**没有**偷加这个过滤器(加了回测会变好看,但是过拟合)。 + +freqtrade 的 S2 适配器在 `bot_loop_start` 里做截面排名,只服务 live/dry-run。用 `freqtrade backtesting --strategy S2` **不能**当官方结果(`populate_*` 看不到历史截面)。 + +--- + +## 4. 部署侧(你 VPS 上要做的) + +配置已改好,本云端 VM **没有 Docker、也连不上币安**,所以容器没有在这里拉起。 + +1. `cd deploy && docker compose up -d` + - S1:`sentinel-freqtrade` FreqUI **8080**,`initial_state: running` + - S2:`sentinel-freqtrade-s2` FreqUI **8081**,独立 SQLite + - ai-service:8000 +2. 下载 15 对 1d(及可选 4h)历史,命令见 `deploy/RUNBOOK.md`。 +3. 看 FreqUI / Telegram:bot 应是 running,不是 stopped。 +4. 铁律未动:`max_open_trades=3`、`stoploss_on_exchange=true`、Protections 三件套写在策略类里(freqtrade 2026.6 不再读配置顶层 protections)。 + +--- + +## 5. 还没做(不要假装做完) + +- P1.4 freqtrade hyperopt + 留出段年化为正 +- P1.5 jesse Monte Carlo +- `freqtrade lookahead-analysis` 实跑 +- S2 回撤降到 <20% 的设计改动(需你再拍板) diff --git a/scripts/run_s1_s2_backtest.py b/scripts/run_s1_s2_backtest.py new file mode 100755 index 0000000..e9da9b5 --- /dev/null +++ b/scripts/run_s1_s2_backtest.py @@ -0,0 +1,75 @@ +#!/usr/bin/env python3 +"""Run S1 + S2 research backtests on Yahoo daily data and print a markdown report. + +Not used for live orders. Binance public API is blocked in some clouds (HTTP 451); +Yahoo USD daily is a close proxy for USDT spot research. +""" +from __future__ import annotations + +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +if str(ROOT) not in sys.path: + sys.path.insert(0, str(ROOT)) + +from strategies.data_yahoo import fetch_universe +from strategies.s1_trend_follow.backtest import simulate_universe +from strategies.s1_trend_follow.strategy import StrategyParams +from strategies.s2_momentum_rotation.backtest import run_rotation_backtest +from strategies.s2_momentum_rotation.strategy import RotationParams + + +def _in_window(trades, start, end) -> int: + n = 0 + for t in trades: + d = t.entry_date + if start <= str(d)[:10] <= end: + n += 1 + return n + + +def main() -> int: + prices = fetch_universe("2020-06-01") + if len(prices) < 8: + print(f"download failed: only {len(prices)} pairs", file=sys.stderr) + return 1 + + s1_trades = simulate_universe(prices, StrategyParams()) + s1_2022 = [t for t in s1_trades if str(t.entry_date)[:4] == "2022" or str(t.exit_date)[:4] == "2022"] + s1_holdout = [t for t in s1_trades if str(t.entry_date)[:10] >= "2024-01-01"] + + s2 = run_rotation_backtest( + prices, + RotationParams(universe_size=10, hold_top_k=3, momentum_lookback_days=30), + initial_capital=1000.0, + ) + s2_2022 = any(str(t.entry_date)[:4] == "2022" or str(t.exit_date)[:4] == "2022" for t in s2.trades) + + print("# S1 / S2 research backtest (Yahoo daily USD proxy)") + print() + print(f"- pairs downloaded: {len(prices)} / 15") + print(f"- coverage: {min(f.index.min() for f in prices.values()).date()} → {max(f.index.max() for f in prices.values()).date()}") + print() + print("## S1 trend-follow (golden cross OR EMA50 pullback reclaim, ADX>25)") + print(f"- closed trades (unconstrained per pair): **{len(s1_trades)}**") + print(f"- trades overlapping 2022: **{len(s1_2022)}**") + print(f"- entries in 2024-01-01+: **{len(s1_holdout)}**") + if s1_trades: + wins = sum(1 for t in s1_trades if t.pnl_pct > 0) + print(f"- win rate: {wins / len(s1_trades):.1%}") + print(f"- mean pnl: {sum(t.pnl_pct for t in s1_trades) / len(s1_trades):.2%}") + print() + print("## S2 weekly momentum rotation (top 3 of volume-10, 10% stop)") + print(f"- closed trades: **{s2.n_trades}**") + print(f"- total return: **{s2.total_return_pct:.1%}**") + print(f"- max drawdown: **{s2.max_drawdown_pct:.1%}**") + print(f"- 2022 in sample: **{s2_2022}**") + print() + print("Notes: S1 sim is per-pair independent (live `max_open_trades=3` will skip extras).") + print("S2 ranking uses T-1 closes (no current-bar lookahead). Not freqtrade hyperopt.") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/strategies/data_yahoo.py b/strategies/data_yahoo.py new file mode 100644 index 0000000..6c81000 --- /dev/null +++ b/strategies/data_yahoo.py @@ -0,0 +1,61 @@ +"""Download Yahoo daily OHLCV for the frozen USDT universe (research only).""" +from __future__ import annotations + +import json +import time +import urllib.request +from datetime import datetime, timezone + +import pandas as pd + +from strategies.universe import USDT_SPOT_UNIVERSE, yahoo_symbol + +YAHOO_UA = "Mozilla/5.0 (compatible; SentinelResearch/1.0)" + + +def fetch_yahoo_daily(symbol: str, start_ts: int, end_ts: int) -> pd.DataFrame: + """Fetch daily OHLCV. Raises on HTTP/parse failure.""" + url = ( + f"https://query1.finance.yahoo.com/v8/finance/chart/{symbol}" + f"?interval=1d&period1={start_ts}&period2={end_ts}" + ) + req = urllib.request.Request(url, headers={"User-Agent": YAHOO_UA}) + with urllib.request.urlopen(req, timeout=30) as resp: + payload = json.loads(resp.read()) + result = payload["chart"]["result"][0] + ts = result["timestamp"] + quote = result["indicators"]["quote"][0] + frame = pd.DataFrame( + { + "open": quote["open"], + "high": quote["high"], + "low": quote["low"], + "close": quote["close"], + "volume": quote["volume"], + }, + index=pd.to_datetime(ts, unit="s", utc=True).tz_convert(None).normalize(), + ) + return frame.dropna(subset=["close"]) + + +def fetch_universe( + start: str = "2020-06-01", + end: str | None = None, + *, + pause_s: float = 0.25, +) -> dict[str, pd.DataFrame]: + """Download all pairs. Missing symbols are skipped (delist / ticker change).""" + start_ts = int(datetime.fromisoformat(start).replace(tzinfo=timezone.utc).timestamp()) + if end is None: + end_ts = int(datetime.now(timezone.utc).timestamp()) + else: + end_ts = int(datetime.fromisoformat(end).replace(tzinfo=timezone.utc).timestamp()) + out: dict[str, pd.DataFrame] = {} + for pair in USDT_SPOT_UNIVERSE: + symbol = yahoo_symbol(pair) + try: + out[pair] = fetch_yahoo_daily(symbol, start_ts, end_ts) + except Exception: + continue + time.sleep(pause_s) + return out diff --git a/strategies/s1_trend_follow/backtest.py b/strategies/s1_trend_follow/backtest.py new file mode 100644 index 0000000..1dde9d4 --- /dev/null +++ b/strategies/s1_trend_follow/backtest.py @@ -0,0 +1,87 @@ +"""Per-pair S1 trade simulator using the tested entry/exit functions.""" +from __future__ import annotations + +from dataclasses import dataclass + +import pandas as pd + +from strategies.s1_trend_follow.strategy import ( + StrategyParams, + compute_indicators, + entry_signal, + exit_signal, +) + + +@dataclass(frozen=True) +class ClosedTrade: + pair: str + entry_date: object + exit_date: object + entry_price: float + exit_price: float + pnl_pct: float + reason: str + + +def simulate_pair( + df: pd.DataFrame, + params: StrategyParams, + pair: str = "PAIR", +) -> list[ClosedTrade]: + """Walk one OHLCV frame; enter/exit at close. No lookahead beyond row t.""" + out = compute_indicators(df, params) + trades: list[ClosedTrade] = [] + in_pos = False + entry = peak = 0.0 + entry_date = None + for ts, row in out.iterrows(): + price = float(row["close"]) + if not in_pos: + if entry_signal(row, params): + in_pos = True + entry = peak = price + entry_date = ts + continue + peak = max(peak, price) + should, reason = exit_signal(row, entry, peak, params) + if not should: + continue + trades.append( + ClosedTrade( + pair=pair, + entry_date=entry_date, + exit_date=ts, + entry_price=entry, + exit_price=price, + pnl_pct=price / entry - 1.0, + reason=reason, + ) + ) + in_pos = False + if in_pos: + last_ts = out.index[-1] + last_px = float(out.iloc[-1]["close"]) + trades.append( + ClosedTrade( + pair=pair, + entry_date=entry_date, + exit_date=last_ts, + entry_price=entry, + exit_price=last_px, + pnl_pct=last_px / entry - 1.0, + reason="end_of_data", + ) + ) + return trades + + +def simulate_universe( + prices: dict[str, pd.DataFrame], + params: StrategyParams, +) -> list[ClosedTrade]: + """Independent per-pair simulation (live max_open_trades=3 is NOT applied).""" + all_trades: list[ClosedTrade] = [] + for pair, frame in prices.items(): + all_trades.extend(simulate_pair(frame, params, pair=pair)) + return all_trades diff --git a/strategies/s1_trend_follow/strategy.py b/strategies/s1_trend_follow/strategy.py index 07e0d2a..8a2e8ad 100644 --- a/strategies/s1_trend_follow/strategy.py +++ b/strategies/s1_trend_follow/strategy.py @@ -1,10 +1,12 @@ -"""S1: Trend-following strategy on BTC/ETH daily. +"""S1: Trend-following strategy on large-cap USDT daily. Logic ----- -Long-only. Enter when: - 1. Fast EMA (50) crosses above Slow EMA (200) — golden cross - 2. ADX > 25 (confirms trend strength, filters out choppy markets) +Long-only. Enter when ADX > 25 AND fast EMA > slow EMA AND either: + 1. Fast EMA (50) crosses above Slow EMA (200) — golden cross (regime change) + 2. Close reclaims fast EMA after dipping below it — pullback in an + already-established uptrend (the 2024–2026 silence fix: no new golden + cross is required once the bull is underway) Exit (in priority order): 1. custom_stoploss below -8% from entry → hard stop @@ -48,39 +50,41 @@ def compute_indicators( df: pd.DataFrame, params: StrategyParams, ) -> pd.DataFrame: - """Return df with added columns: ema_fast, ema_slow, adx, golden_cross. + """Return df with added columns: ema_fast, ema_slow, adx, golden_cross, + death_cross, pullback_reclaim. - `golden_cross` is a boolean Series: True on the bar where fast EMA first - crosses above slow EMA. + Cross flags use only current/prior bars (shift(1)) — no future leakage. + `pullback_reclaim` is close crossing back above the fast EMA. """ out = df.copy() out["ema_fast"] = ema(out["close"], params.fast_ema_period) out["ema_slow"] = ema(out["close"], params.slow_ema_period) out["adx"] = adx(out, period=14) - # Cross detection: fast > slow now AND fast <= slow on the prior bar fast = out["ema_fast"] slow = out["ema_slow"] out["golden_cross"] = (fast > slow) & (fast.shift(1) <= slow.shift(1)) out["death_cross"] = (fast < slow) & (fast.shift(1) >= slow.shift(1)) + # Prior close below fast EMA, this close back above — no current-bar high/low. + out["pullback_reclaim"] = (out["close"].shift(1) < fast.shift(1)) & (out["close"] >= fast) return out def entry_signal(row: pd.Series, params: StrategyParams) -> bool: """True on the bar where we should enter long. - Conservative: requires all three conditions to be true simultaneously. - - Golden cross just happened (row-level) - - Fast EMA > Slow EMA (sustained) - - ADX > entry threshold (trend is real, not noise) + Shared regime filter: Fast EMA > Slow EMA and ADX > threshold. + Trigger is either a golden cross OR a pullback reclaim of the fast EMA. """ if pd.isna(row["adx"]) or pd.isna(row["ema_fast"]) or pd.isna(row["ema_slow"]): return False - if not row["golden_cross"]: - return False if not (row["ema_fast"] > row["ema_slow"]): return False - return row["adx"] > params.adx_entry_threshold + if not (row["adx"] > params.adx_entry_threshold): + return False + golden = bool(row["golden_cross"]) if not pd.isna(row["golden_cross"]) else False + pullback = bool(row["pullback_reclaim"]) if "pullback_reclaim" in row.index and not pd.isna(row["pullback_reclaim"]) else False + return golden or pullback def exit_signal( @@ -143,11 +147,13 @@ def build_entry_signals(df: pd.DataFrame, params: StrategyParams) -> pd.Series: # golden_cross is built from boolean ops in compute_indicators (no NaN), # but fillna(False) keeps this robust if a NaN ever appears. golden = df["golden_cross"].fillna(False).astype(bool) - # NaN comparisons evaluate to False, so warmup rows are already excluded; - # `valid` makes the intent explicit and pins us to the loop's behavior. + if "pullback_reclaim" in df.columns: + pullback = df["pullback_reclaim"].fillna(False).astype(bool) + else: + pullback = False ema_above = df["ema_fast"] > df["ema_slow"] adx_ok = df["adx"] > params.adx_entry_threshold - signal = valid & golden & ema_above & adx_ok + signal = valid & ema_above & adx_ok & (golden | pullback) return signal.astype(int) @@ -191,6 +197,7 @@ class S1TrendFollow(StrategyBase): can_short = False startup_candle_count = 220 # warmup for 200 EMA stoploss = -0.10 # freqtrade hard floor (custom_stoploss tightens this) + use_custom_stoploss = True # Parameter spaces for hyperopt (P1.4 — keep narrow to avoid overfit) adx_entry = DecimalParameter(20.0, 35.0, default=25.0, space="buy") diff --git a/strategies/s1_trend_follow/tests/test_backtest.py b/strategies/s1_trend_follow/tests/test_backtest.py new file mode 100644 index 0000000..2877eba --- /dev/null +++ b/strategies/s1_trend_follow/tests/test_backtest.py @@ -0,0 +1,55 @@ +"""S1 simulator: pullback entries produce trades after a golden-cross regime.""" +from __future__ import annotations + +import numpy as np +import pandas as pd + +from strategies.s1_trend_follow.backtest import simulate_pair +from strategies.s1_trend_follow.strategy import StrategyParams, compute_indicators, entry_signal + + +def _ohlcv(close: np.ndarray) -> pd.DataFrame: + return pd.DataFrame( + { + "open": close, + "high": close + 1.0, + "low": close - 1.0, + "close": close, + "volume": np.ones(len(close)) * 1000, + } + ) + + +def test_simulator_records_exit_reason(): + down = np.linspace(200, 100, 200) + up = np.linspace(100, 250, 300) + df = _ohlcv(np.concatenate([down, up])) + trades = simulate_pair(df, StrategyParams(), pair="BTC/USDT") + assert len(trades) >= 1 + assert trades[0].pair == "BTC/USDT" + assert trades[0].reason in { + "hard_stop", + "trailing_stop", + "death_cross", + "adx_collapse", + "end_of_data", + } + + +def test_established_uptrend_without_new_golden_cross_can_still_enter(): + """The 2024–2026 failure mode: already in uptrend, no new golden cross.""" + rng = np.random.default_rng(3) + up = np.linspace(100, 220, 280) + rng.normal(0, 0.2, 280) + # Force a dip below the fast EMA then reclaim — pullback path. + close = np.concatenate([up, np.array([up[-1] * 0.92, up[-1] * 0.90, up[-1] * 0.97])]) + df = _ohlcv(close) + params = StrategyParams() + out = compute_indicators(df, params) + tail = out.iloc[-1] + # Either the last bar or some post-warmup bar that is NOT a golden cross enters. + non_cross_entries = [] + for _, row in out.iloc[220:].iterrows(): + if entry_signal(row, params) and not bool(row["golden_cross"]): + non_cross_entries.append(True) + assert non_cross_entries, "pullback path must allow entries without a new golden cross" + assert bool(tail["pullback_reclaim"]) or len(non_cross_entries) >= 1 diff --git a/strategies/s1_trend_follow/tests/test_strategy.py b/strategies/s1_trend_follow/tests/test_strategy.py index 53653a2..9ab5692 100644 --- a/strategies/s1_trend_follow/tests/test_strategy.py +++ b/strategies/s1_trend_follow/tests/test_strategy.py @@ -63,9 +63,34 @@ class TestComputeIndicators: def test_adds_expected_columns(self, uptrend_with_cross_df): params = StrategyParams() out = compute_indicators(uptrend_with_cross_df, params) - for col in ["ema_fast", "ema_slow", "adx", "golden_cross", "death_cross"]: + for col in [ + "ema_fast", + "ema_slow", + "adx", + "golden_cross", + "death_cross", + "pullback_reclaim", + ]: assert col in out.columns + def test_pullback_reclaim_fires_after_dip_below_fast_ema(self): + """Close below EMA50 then back above it must set pullback_reclaim.""" + n = 260 + up = np.linspace(100, 200, n) + close = np.concatenate([up, np.array([199.0, 185.0, 205.0])]) + df = pd.DataFrame( + { + "open": close, + "high": close + 1.0, + "low": close - 1.0, + "close": close, + "volume": np.ones(len(close)) * 1000, + } + ) + out = compute_indicators(df, StrategyParams()) + assert out["pullback_reclaim"].iloc[-1] == True + assert out["golden_cross"].iloc[-1] == False + def test_slow_ema_warmup_is_nan(self, uptrend_with_cross_df): params = StrategyParams() out = compute_indicators(uptrend_with_cross_df, params) @@ -118,10 +143,64 @@ def test_no_entry_when_adx_below_threshold(self, uptrend_with_cross_df): def test_no_entry_on_nan_row(self): params = StrategyParams() nan_row = pd.Series( - {"close": 100, "ema_fast": np.nan, "ema_slow": np.nan, "adx": np.nan, "golden_cross": True} + { + "close": 100, + "ema_fast": np.nan, + "ema_slow": np.nan, + "adx": np.nan, + "golden_cross": True, + "pullback_reclaim": False, + } ) assert entry_signal(nan_row, params) is False + def test_entry_on_pullback_reclaim_in_uptrend(self): + """Secondary signal (user 2026-08-13): buy EMA50 reclaim in an + established uptrend — does NOT require a new golden cross.""" + params = StrategyParams() + row = pd.Series( + { + "close": 110.0, + "ema_fast": 108.0, + "ema_slow": 100.0, + "adx": 30.0, + "golden_cross": False, + "pullback_reclaim": True, + "death_cross": False, + } + ) + assert entry_signal(row, params) is True + + def test_no_pullback_entry_when_adx_weak(self): + params = StrategyParams() + row = pd.Series( + { + "close": 110.0, + "ema_fast": 108.0, + "ema_slow": 100.0, + "adx": 10.0, + "golden_cross": False, + "pullback_reclaim": True, + "death_cross": False, + } + ) + assert entry_signal(row, params) is False + + def test_no_pullback_entry_when_fast_below_slow(self): + params = StrategyParams() + row = pd.Series( + { + "close": 95.0, + "ema_fast": 90.0, + "ema_slow": 100.0, + "adx": 30.0, + "golden_cross": False, + "pullback_reclaim": True, + "death_cross": False, + } + ) + assert entry_signal(row, params) is False + class TestExitSignal: def test_hard_stop_fires_on_eight_percent_loss(self): diff --git a/strategies/s2_momentum_rotation/__init__.py b/strategies/s2_momentum_rotation/__init__.py index 14a65ad..a28cfb2 100644 --- a/strategies/s2_momentum_rotation/__init__.py +++ b/strategies/s2_momentum_rotation/__init__.py @@ -1,4 +1,22 @@ """Public API for S2 momentum rotation.""" -from .strategy import RotationParams, rotation_plan, select_top_k, compute_universe, equal_weights +from .strategy import ( + RotationParams, + compute_universe, + equal_weights, + is_rebalance_timestamp, + rank_holdings, + rotation_plan, + select_top_k, + snapshot_momentum_and_volume, +) -__all__ = ["RotationParams", "rotation_plan", "select_top_k", "compute_universe", "equal_weights"] \ No newline at end of file +__all__ = [ + "RotationParams", + "compute_universe", + "equal_weights", + "is_rebalance_timestamp", + "rank_holdings", + "rotation_plan", + "select_top_k", + "snapshot_momentum_and_volume", +] \ No newline at end of file diff --git a/strategies/s2_momentum_rotation/backtest.py b/strategies/s2_momentum_rotation/backtest.py new file mode 100644 index 0000000..8448974 --- /dev/null +++ b/strategies/s2_momentum_rotation/backtest.py @@ -0,0 +1,247 @@ +"""Point-in-time weekly rotation backtest (official S2 research result). + +Ranking on date T uses only closes through T-1 (no current-bar lookahead). +Fills at T's close. Intraday 10% stop uses that day's low. +""" +from __future__ import annotations + +from dataclasses import dataclass + +import pandas as pd + +from strategies.s2_momentum_rotation.strategy import ( + RotationParams, + is_rebalance_timestamp, + rank_holdings, +) + +STOPLOSS_PCT = 0.10 + + +@dataclass(frozen=True) +class ClosedTrade: + pair: str + entry_date: pd.Timestamp + exit_date: pd.Timestamp + entry_price: float + exit_price: float + pnl_pct: float + reason: str + + +@dataclass(frozen=True) +class BacktestReport: + trades: tuple[ClosedTrade, ...] + equity_curve: pd.Series + max_drawdown_pct: float + total_return_pct: float + n_trades: int + + +def _asof(frame: pd.DataFrame, asof: pd.Timestamp) -> pd.DataFrame: + return frame.loc[frame.index <= asof] + + +def _union_index(prices: dict[str, pd.DataFrame]) -> pd.DatetimeIndex: + idx = pd.DatetimeIndex([]) + for frame in prices.values(): + idx = idx.union(frame.index) + return pd.DatetimeIndex(idx).sort_values().unique() + + +def _snapshots_asof( + prices: dict[str, pd.DataFrame], + asof: pd.Timestamp, +) -> dict[str, pd.DataFrame]: + out: dict[str, pd.DataFrame] = {} + for pair, frame in prices.items(): + sliced = _asof(frame, asof) + if not sliced.empty: + out[pair] = sliced + return out + + +def _close_on(frame: pd.DataFrame, dt: pd.Timestamp) -> float | None: + if dt not in frame.index: + return None + return float(frame.loc[dt, "close"]) + + +def _low_on(frame: pd.DataFrame, dt: pd.Timestamp) -> float | None: + if dt not in frame.index: + return None + col = "low" if "low" in frame.columns else "close" + return float(frame.loc[dt, col]) + + +def _max_drawdown_pct(equity: pd.Series) -> float: + if equity.empty: + return 0.0 + peak = equity.cummax() + dd = (equity / peak) - 1.0 + return float(dd.min()) + + +def _close_trade( + pair: str, + pos: dict, + exit_date: pd.Timestamp, + exit_price: float, + reason: str, +) -> ClosedTrade: + entry = float(pos["entry_price"]) + pnl = exit_price / entry - 1.0 + return ClosedTrade( + pair=pair, + entry_date=pos["entry_date"], + exit_date=exit_date, + entry_price=entry, + exit_price=exit_price, + pnl_pct=pnl, + reason=reason, + ) + + +def _apply_stops( + positions: dict[str, dict], + prices: dict[str, pd.DataFrame], + dt: pd.Timestamp, + stoploss_pct: float, +) -> tuple[dict[str, dict], list[ClosedTrade], float]: + """Hit stop at the stop price; proceeds return to cash.""" + remaining: dict[str, dict] = {} + closed: list[ClosedTrade] = [] + cash_in = 0.0 + stop_level = 1.0 - stoploss_pct + for pair, pos in positions.items(): + frame = prices.get(pair) + if frame is None: + remaining[pair] = pos + continue + low = _low_on(frame, dt) + if low is None: + remaining[pair] = pos + continue + stop_px = float(pos["entry_price"]) * stop_level + if low <= stop_px: + closed.append(_close_trade(pair, pos, dt, stop_px, "stoploss")) + cash_in += stop_px * float(pos["qty"]) + else: + remaining[pair] = pos + return remaining, closed, cash_in + + +def _mark_to_market( + cash: float, + positions: dict[str, dict], + prices: dict[str, pd.DataFrame], + dt: pd.Timestamp, +) -> float: + total = cash + for pair, pos in positions.items(): + frame = prices[pair] + px = _close_on(frame, dt) + if px is None: + px = float(pos["entry_price"]) + total += px * float(pos["qty"]) + return total + + +def _rebalance( + cash: float, + positions: dict[str, dict], + prices: dict[str, pd.DataFrame], + dt: pd.Timestamp, + target: list[str], +) -> tuple[float, dict[str, dict], list[ClosedTrade]]: + """Liquidate names not in target; deploy equal weight into missing targets.""" + closed: list[ClosedTrade] = [] + kept: dict[str, dict] = {} + for pair, pos in positions.items(): + if pair in target: + kept[pair] = pos + continue + frame = prices[pair] + px = _close_on(frame, dt) or float(pos["entry_price"]) + closed.append(_close_trade(pair, pos, dt, px, "rebalance")) + cash += px * float(pos["qty"]) + missing = [p for p in target if p not in kept] + if not missing: + return cash, kept, closed + marked = cash + for pair, pos in kept.items(): + px = _close_on(prices[pair], dt) or float(pos["entry_price"]) + marked += px * float(pos["qty"]) + n = len(target) + slot = marked / n if n else 0.0 + cash, kept = _open_missing(cash, kept, prices, dt, missing, slot) + return cash, kept, closed + + +def _open_missing( + cash: float, + kept: dict[str, dict], + prices: dict[str, pd.DataFrame], + dt: pd.Timestamp, + missing: list[str], + slot: float, +) -> tuple[float, dict[str, dict]]: + for pair in missing: + px = _close_on(prices[pair], dt) + if px is None or px <= 0 or slot <= 0: + continue + qty = slot / px + if qty * px > cash: + qty = cash / px if px else 0.0 + if qty <= 0: + continue + cash -= qty * px + kept[pair] = { + "pair": pair, + "entry_date": dt, + "entry_price": px, + "qty": qty, + } + return cash, kept + + +def run_rotation_backtest( + prices: dict[str, pd.DataFrame], + params: RotationParams, + *, + stoploss_pct: float = STOPLOSS_PCT, + initial_capital: float = 1000.0, +) -> BacktestReport: + """Walk daily. Rank on T-1. Rebalance Mondays. 10% stop between weeks.""" + calendar = _union_index(prices) + cash = float(initial_capital) + positions: dict[str, dict] = {} + trades: list[ClosedTrade] = [] + equity_pts: list[tuple[pd.Timestamp, float]] = [] + lookback = params.momentum_lookback_days + + for i, dt in enumerate(calendar): + if i < lookback + 1: + equity_pts.append((dt, cash)) + continue + positions, stopped, proceeds = _apply_stops(positions, prices, dt, stoploss_pct) + cash += proceeds + trades.extend(stopped) + if is_rebalance_timestamp(dt): + asof = calendar[i - 1] + snaps = _snapshots_asof(prices, asof) + target = rank_holdings(snaps, params) + cash, positions, rotated = _rebalance(cash, positions, prices, dt, target) + trades.extend(rotated) + equity_pts.append((dt, _mark_to_market(cash, positions, prices, dt))) + + equity = pd.Series({t: v for t, v in equity_pts}).sort_index() + start = float(initial_capital) + end = float(equity.iloc[-1]) if not equity.empty else start + return BacktestReport( + trades=tuple(trades), + equity_curve=equity, + max_drawdown_pct=_max_drawdown_pct(equity), + total_return_pct=(end / start - 1.0) if start else 0.0, + n_trades=len(trades), + ) diff --git a/strategies/s2_momentum_rotation/strategy.py b/strategies/s2_momentum_rotation/strategy.py index 5e571cb..6b4a729 100644 --- a/strategies/s2_momentum_rotation/strategy.py +++ b/strategies/s2_momentum_rotation/strategy.py @@ -131,32 +131,149 @@ def rotation_plan( return equal_weights(top_k, total_stake, params) +def is_rebalance_timestamp(ts) -> bool: + """Weekly rebalance = Monday UTC. Naive timestamps treated as UTC.""" + t = pd.Timestamp(ts) + if t.tzinfo is not None: + t = t.tz_convert("UTC").tz_localize(None) + return int(t.dayofweek) == 0 + + +def snapshot_momentum_and_volume( + df: pd.DataFrame, + lookback: int, +) -> tuple[float, float] | None: + """Point-in-time momentum + volume using only rows already in ``df``.""" + if df is None or len(df) < lookback + 1: + return None + close = df["close"] + last = float(close.iloc[-1]) + past = float(close.iloc[-1 - lookback]) + if past == 0: + return None + mom = last / past - 1.0 + vol = float(df["volume"].iloc[-lookback:].sum()) if "volume" in df.columns else 0.0 + return mom, vol + + +def rank_holdings( + snapshots: dict[str, pd.DataFrame], + params: RotationParams, +) -> list[str]: + """Cross-sectional top-K from a dict of as-of OHLCV frames.""" + volumes: dict[str, float] = {} + momentum: dict[str, float] = {} + for pair, frame in snapshots.items(): + scored = snapshot_momentum_and_volume(frame, params.momentum_lookback_days) + if scored is None: + continue + momentum[pair], volumes[pair] = scored + if not momentum: + return [] + plan = rotation_plan(list(snapshots), volumes, momentum, total_stake=1.0, params=params) + return list(plan.keys()) + + # ---- freqtrade adapter ---- try: - from freqtrade.strategy import IntParameter, DecimalParameter + from freqtrade.strategy import IntParameter from strategies.base import StrategyBase class S2MomentumRotation(StrategyBase): - """freqtrade-compatible rotation strategy. Wires pure functions above. + """freqtrade adapter for live/dry-run. - Inherits confirm_trade_entry → AI veto from StrategyBase. + Cross-sectional rank is computed in ``bot_loop_start`` from the + dataprovider (current candle only). Historical freqtrade backtesting + of this adapter is NOT the official S2 result — use + ``strategies.s2_momentum_rotation.backtest.run_rotation_backtest``. """ + timeframe = "1d" can_short = False - startup_candle_count = 60 # need 30d momentum + warmup + startup_candle_count = 60 stoploss = -0.10 + use_exit_signal = True + use_custom_stoploss = False + process_only_new_candles = True + minimal_roi = {"0": 100} + stoploss_on_exchange = True - # Hyperopt ranges — narrow to prevent overfit universe_size = IntParameter(5, 20, default=10, space="buy") hold_top_k = IntParameter(2, 5, default=3, space="buy") momentum_lookback = IntParameter(14, 60, default=30, space="buy") + def bot_loop_start(self, current_time, **kwargs) -> None: + self._maybe_refresh_holdings(current_time) + def populate_indicators(self, dataframe: pd.DataFrame, metadata: dict) -> pd.DataFrame: - params = RotationParams(momentum_lookback_days=int(self.momentum_lookback.value)) - dataframe["momentum"] = momentum_score(dataframe, params.momentum_lookback_days) + lookback = int(self.momentum_lookback.value) + dataframe["momentum"] = momentum_score(dataframe, lookback) return dataframe + def populate_entry_trend(self, dataframe: pd.DataFrame, metadata: dict) -> pd.DataFrame: + dataframe["enter_long"] = 0 + hold = getattr(self, "_hold_set", frozenset()) + pair = metadata.get("pair", "") + if pair in hold and len(dataframe) > 0: + dataframe.loc[dataframe.index[-1], "enter_long"] = 1 + return dataframe + + def populate_exit_trend(self, dataframe: pd.DataFrame, metadata: dict) -> pd.DataFrame: + dataframe["exit_long"] = 0 + hold = getattr(self, "_hold_set", frozenset()) + pair = metadata.get("pair", "") + if pair not in hold and len(dataframe) > 0: + dataframe.loc[dataframe.index[-1], "exit_long"] = 1 + return dataframe + + @property + def protections(self): + return [ + { + "method": "MaxDrawdown", + "lookback_period": 1440, + "stop_duration": 1440, + "trade_limit": 1, + "max_allowed_drawdown": 0.10, + }, + { + "method": "StoplossGuard", + "lookback_period": 240, + "stop_duration": 720, + "trade_limit": 3, + }, + { + "method": "CooldownPeriod", + "stop_duration_candles": 2, + }, + ] + + def _rotation_params(self) -> RotationParams: + return RotationParams( + universe_size=int(self.universe_size.value), + hold_top_k=int(self.hold_top_k.value), + momentum_lookback_days=int(self.momentum_lookback.value), + ) + + def _maybe_refresh_holdings(self, current_time) -> None: + first = getattr(self, "_hold_set", None) is None + if not first and not is_rebalance_timestamp(current_time): + return + self._hold_set = frozenset(self._rank_from_dataprovider()) + + def _rank_from_dataprovider(self) -> list[str]: + dp = getattr(self, "dp", None) + if dp is None: + return [] + snapshots: dict[str, pd.DataFrame] = {} + for pair in dp.current_whitelist(): + frame, _ = dp.get_analyzed_dataframe(pair, self.timeframe) + if frame is None or frame.empty: + continue + snapshots[pair] = frame + return rank_holdings(snapshots, self._rotation_params()) + except ImportError: S2MomentumRotation = None # type: ignore[assignment] \ No newline at end of file diff --git a/strategies/s2_momentum_rotation/tests/test_backtest.py b/strategies/s2_momentum_rotation/tests/test_backtest.py new file mode 100644 index 0000000..21111bd --- /dev/null +++ b/strategies/s2_momentum_rotation/tests/test_backtest.py @@ -0,0 +1,71 @@ +"""S2 ranking + weekly rotation backtest (synthetic, no network).""" +from __future__ import annotations + +import numpy as np +import pandas as pd + +from strategies.s2_momentum_rotation import ( + RotationParams, + is_rebalance_timestamp, + rank_holdings, +) +from strategies.s2_momentum_rotation.backtest import run_rotation_backtest + + +def _ohlcv(close: np.ndarray, start: str = "2021-01-04") -> pd.DataFrame: + idx = pd.bdate_range(start, periods=len(close), freq="C") + close = close.astype(float) + return pd.DataFrame( + { + "open": close, + "high": close * 1.01, + "low": close * 0.99, + "close": close, + "volume": np.full(len(close), 1_000_000.0), + }, + index=idx, + ) + + +def test_monday_is_rebalance_and_tuesday_is_not(): + assert is_rebalance_timestamp(pd.Timestamp("2022-01-03")) is True # Monday + assert is_rebalance_timestamp(pd.Timestamp("2022-01-04")) is False + + +def test_rank_holdings_picks_highest_momentum(): + n = 40 + weak = _ohlcv(np.linspace(100, 90, n)) + strong = _ohlcv(np.linspace(100, 160, n)) + mid = _ohlcv(np.linspace(100, 120, n)) + params = RotationParams(universe_size=3, hold_top_k=2, momentum_lookback_days=30) + held = rank_holdings( + {"WEAK/USDT": weak, "STRONG/USDT": strong, "MID/USDT": mid}, + params, + ) + assert held[0] == "STRONG/USDT" + assert "WEAK/USDT" not in held + assert len(held) == 2 + + +def test_rotation_backtest_produces_trades_and_survives_crash(): + """2021 rally then 2022-like crash: must include the drawdown in sample.""" + rng = np.random.default_rng(0) + n = 520 # ~2 years of weekdays + dates_start = "2021-01-04" + bull = np.linspace(100, 200, 260) + crash = np.linspace(200, 80, 130) + recover = np.linspace(80, 110, 130) + a = np.concatenate([bull, crash, recover]) + rng.normal(0, 0.4, n) + b = np.concatenate([bull * 0.9, crash * 1.1, recover * 1.2]) + rng.normal(0, 0.4, n) + c = np.concatenate([bull * 1.1, crash * 0.8, recover]) + rng.normal(0, 0.4, n) + prices = { + "AAA/USDT": _ohlcv(a, dates_start), + "BBB/USDT": _ohlcv(b, dates_start), + "CCC/USDT": _ohlcv(c, dates_start), + } + params = RotationParams(universe_size=3, hold_top_k=2, momentum_lookback_days=30) + report = run_rotation_backtest(prices, params, initial_capital=1000.0) + assert report.n_trades >= 5 + assert report.equity_curve.index.min() <= pd.Timestamp("2022-01-03") + assert report.max_drawdown_pct < 0 # crash must show up + assert report.equity_curve.iloc[-1] > 0 diff --git a/strategies/tests/test_data_yahoo.py b/strategies/tests/test_data_yahoo.py new file mode 100644 index 0000000..96bb503 --- /dev/null +++ b/strategies/tests/test_data_yahoo.py @@ -0,0 +1,69 @@ +"""Yahoo fetch parses chart JSON (no live network).""" +from __future__ import annotations + +import json +from io import BytesIO +from unittest.mock import patch + +from strategies.data_yahoo import fetch_universe, fetch_yahoo_daily + + +class _Resp: + def __enter__(self): + payload = { + "chart": { + "result": [ + { + "timestamp": [1609459200, 1609545600], + "indicators": { + "quote": [ + { + "open": [100.0, 101.0], + "high": [102.0, 103.0], + "low": [99.0, 100.0], + "close": [101.0, 102.0], + "volume": [1.0, 2.0], + } + ] + }, + } + ] + } + } + self._buf = BytesIO(json.dumps(payload).encode()) + return self + + def __exit__(self, *args): + return False + + def read(self): + return self._buf.read() + + +def test_fetch_yahoo_daily_parses_ohlcv(): + with patch("strategies.data_yahoo.urllib.request.urlopen", return_value=_Resp()): + df = fetch_yahoo_daily("BTC-USD", 1609459200, 1609545600) + assert list(df.columns) == ["open", "high", "low", "close", "volume"] + assert len(df) == 2 + assert float(df["close"].iloc[-1]) == 102.0 + + +def test_fetch_universe_skips_failures_and_maps_pairs(): + import pandas as pd + + frame = pd.DataFrame( + {"open": [1], "high": [1], "low": [1], "close": [1], "volume": [1]}, + index=pd.to_datetime(["2021-01-01"]), + ) + + def _fake(symbol, start_ts, end_ts): + if symbol == "ETH-USD": + raise RuntimeError("skip") + return frame + + with patch("strategies.data_yahoo.fetch_yahoo_daily", side_effect=_fake): + with patch("strategies.data_yahoo.time.sleep"): + out = fetch_universe("2021-01-01", "2021-01-03", pause_s=0) + assert "BTC/USDT" in out + assert "ETH/USDT" not in out + assert len(out) == 14 diff --git a/strategies/tests/test_dry_run_config.py b/strategies/tests/test_dry_run_config.py new file mode 100644 index 0000000..b2062c7 --- /dev/null +++ b/strategies/tests/test_dry_run_config.py @@ -0,0 +1,64 @@ +"""Config contracts for S1/S2 dry-run (iron rules + user 2026-08-13 拍板).""" +from __future__ import annotations + +import json +from pathlib import Path + +from strategies.universe import MAX_OPEN_TRADES_CAP, USDT_SPOT_UNIVERSE + +_REPO = Path(__file__).resolve().parents[2] +_CONFIG = _REPO / "deploy" / "user_data" / "config" +_STRAT = _REPO / "deploy" / "user_data" / "strategies" +_COMPOSE = _REPO / "deploy" / "docker-compose.yml" + + +def _load(name: str) -> dict: + return json.loads((_CONFIG / name).read_text(encoding="utf-8")) + + +def test_s1_dry_run_starts_running_with_iron_rules(): + cfg = _load("dry-run.json") + assert cfg["initial_state"] == "running" + assert cfg["dry_run"] is True + assert cfg["max_open_trades"] <= MAX_OPEN_TRADES_CAP + assert cfg["stoploss_on_exchange"] is True + assert cfg["strategy"] == "S1TrendFollow" + assert cfg["timeframe"] == "1d" + assert tuple(cfg["exchange"]["pair_whitelist"]) == USDT_SPOT_UNIVERSE + + +def test_s2_dry_run_is_separate_bot(): + cfg = _load("dry-run-s2.json") + assert cfg["initial_state"] == "running" + assert cfg["dry_run"] is True + assert cfg["max_open_trades"] <= MAX_OPEN_TRADES_CAP + assert cfg["stoploss_on_exchange"] is True + assert cfg["strategy"] == "S2MomentumRotation" + assert cfg["api_server"]["listen_port"] == 8081 + assert "s2" in cfg["db_url"] + assert cfg["webhook"]["exit_fill"]["strategy"] == "S2MomentumRotation" + assert tuple(cfg["exchange"]["pair_whitelist"]) == USDT_SPOT_UNIVERSE + + +def test_s2_strategy_file_exists_and_keeps_protections(): + text = (_STRAT / "S2MomentumRotation.py").read_text(encoding="utf-8") + assert "class S2MomentumRotation" in text + assert "MaxDrawdown" in text + assert "StoplossGuard" in text + assert "CooldownPeriod" in text + assert "stoploss_on_exchange = True" in text + + +def test_s1_strategy_has_pullback_and_custom_stoploss_flag(): + text = (_STRAT / "S1TrendFollow.py").read_text(encoding="utf-8") + assert "pullback_reclaim" in text + assert "use_custom_stoploss = True" in text + assert "MaxDrawdown" in text + + +def test_compose_has_s2_service_on_8081(): + text = _COMPOSE.read_text(encoding="utf-8") + assert "sentinel-freqtrade-s2" in text + assert "S2MomentumRotation" in text + assert "8081" in text + assert "initial_state" not in text # lives in json, not compose diff --git a/strategies/tests/test_universe.py b/strategies/tests/test_universe.py new file mode 100644 index 0000000..3dae31d --- /dev/null +++ b/strategies/tests/test_universe.py @@ -0,0 +1,44 @@ +"""P1.1 universe contract: 15 USDT pairs, no stables, reproducible.""" +from __future__ import annotations + +import pytest + +from strategies.indicators import is_stablecoin +from strategies.universe import ( + MAX_OPEN_TRADES_CAP, + UNIVERSE_SIZE, + USDT_SPOT_UNIVERSE, + yahoo_symbol, +) + + +def test_universe_has_exactly_fifteen_pairs(): + assert UNIVERSE_SIZE == 15 + assert len(USDT_SPOT_UNIVERSE) == 15 + assert len(set(USDT_SPOT_UNIVERSE)) == 15 + + +def test_universe_excludes_stables_and_bnb(): + for pair in USDT_SPOT_UNIVERSE: + assert not is_stablecoin(pair), pair + assert not pair.upper().startswith("BNB/") + assert pair.endswith("/USDT") + + +def test_core_majors_are_present(): + for pair in ("BTC/USDT", "ETH/USDT", "SOL/USDT"): + assert pair in USDT_SPOT_UNIVERSE + + +def test_yahoo_symbol_maps_usdt_to_usd(): + assert yahoo_symbol("BTC/USDT") == "BTC-USD" + assert yahoo_symbol("ETH/USDT") == "ETH-USD" + + +def test_yahoo_symbol_rejects_unknown_pair(): + with pytest.raises(KeyError): + yahoo_symbol("FAKE/USDT") + + +def test_open_trades_cap_is_iron_rule(): + assert MAX_OPEN_TRADES_CAP == 3 diff --git a/strategies/universe.py b/strategies/universe.py new file mode 100644 index 0000000..f68eecc --- /dev/null +++ b/strategies/universe.py @@ -0,0 +1,46 @@ +"""Frozen USDT spot universe for S1/S2 (P1.1). + +Snapshot date: 2026-08-13. This is a static list so backtests are +reproducible — not a live CoinGecko scrape. + +Excludes: + - stables (USDT/USDC/DAI/…) + - BNB (existing config blacklist: exchange token concentration) + +Fourteen of fifteen names traded through the 2022 bear; NEAR listed +2022-04 so its 2022 sample is partial. S2 still ranks whatever pairs +have enough lookback on each rebalance date. +""" +from __future__ import annotations + +USDT_SPOT_UNIVERSE: tuple[str, ...] = ( + "BTC/USDT", + "ETH/USDT", + "SOL/USDT", + "XRP/USDT", + "ADA/USDT", + "DOGE/USDT", + "AVAX/USDT", + "DOT/USDT", + "LINK/USDT", + "BCH/USDT", + "LTC/USDT", + "UNI/USDT", + "ATOM/USDT", + "FIL/USDT", + "NEAR/USDT", +) + +# Yahoo Finance uses hyphenated USD pairs; prices track USDT spot closely +# enough for a research backtest (we are not placing orders). +YAHOO_SYMBOLS: dict[str, str] = {pair: pair.split("/")[0] + "-USD" for pair in USDT_SPOT_UNIVERSE} + +UNIVERSE_SIZE = len(USDT_SPOT_UNIVERSE) +MAX_OPEN_TRADES_CAP = 3 # iron rule 4 / design §2.1 + + +def yahoo_symbol(pair: str) -> str: + """Return the Yahoo chart symbol for a USDT pair.""" + if pair not in YAHOO_SYMBOLS: + raise KeyError(f"pair {pair!r} is not in USDT_SPOT_UNIVERSE") + return YAHOO_SYMBOLS[pair]