in ticker + timeframe (1m/5m/15m/30m/1h/4h/1d/1w) + date range
out OHLCV candles + provenance header (provider, source_mode, quality_flags,
is_synthetic=false, served_from, fresh, age_seconds, execution_venue,
cache_policy, quality_policy, fallback_policy)
fail ticker not found → search suggestions
cache hit → cached candles, tagged served_from=cache + age_seconds
realtime strict failure → error / blocked envelope, never hidden fallback
fail timeframe not supported → supported list
fail TuShare-only A-share no token → setup instructions; Phase 1 indices need no token
Built-in sources: tushare_pro, tencent_kline, sina_index,
treasury_official_csv, treasury_official_csv_derived, yahoo_finance,
yahoo_finance_index, yahoo_finance_etf, yahoo_finance_futures,
binance_spot_public, binance_usdm_futures, binance_usdm_futures_research,
hyperliquid_perpetual_public, and the FRED macro/flow/event adapters.
Tiger OpenAPI and OANDA v20 are credential-backed config adapters; see
configs/adapters.example.json.
datafeed 现在按 ports-and-adapters 组织:
API / consumers
│
▼
MarketDataPort
│
├─ SourceManifest: source id, asset class, market type, quality flags, execution venue, aliases
├─ fetch_candles(): REST/historical pull
├─ stream_candles(): realtime stream
├─ canonical_ticker(): adapter-owned symbol normalization
└─ last_raw_response: raw payload for audit/debug
│
├─ Binance USD-M Futures adapter
├─ Binance Spot adapter
├─ Yahoo adapter
├─ TuShare adapter
├─ Tencent A-share index adapter
├─ Sina A-share index fallback adapter
├─ Official U.S. Treasury yield adapter
├─ FRED macro / flow / event adapters
├─ Tiger quote + market-session adapter
├─ OANDA v20 pricing adapter
└─ any new broker adapter
新增 broker / exchange 的接入面:
- 实现
MarketDataPort,或用ProviderBackedMarketDataAdapter包一层旧 provider。 - 提供一个
SourceManifest,声明:source_idasset_classmarket_typeexecution_venuerealtime_supportedquality_flagsticker_aliases
- 通过 Python entry point
kline.market_data_adapters安装,或在 JSON 配置中声明 factory。
API 主流程不需要为新 broker 改分支,也不需要修改 registry。配置文件可通过
KLINE_ADAPTER_CONFIG_PATH 指定,敏感值使用 ${ENV_VAR},启动时缺失会 fail closed。
{
"adapters": [
{
"factory": "my_broker.datafeed:create_adapter",
"config": {"account": "paper", "token": "${MY_BROKER_TOKEN}"}
}
]
}请求只需要:
curl "localhost:8100/api/candles/crypto/BTC?source=fake_broker_feed&cache_policy=bypass"datafeed 本体只负责数据。它不判断“研究”或“交易”,而是把每次请求的 source + cache policy + quality policy + fallback policy 写进响应,让消费者自己决定是否可用。
| 概念 | 参数 | 含义 |
|---|---|---|
| Source | source=auto 或具体 source id | 选择上游,如 binance_usdm_futures |
| Cache | `cache_policy=allow | bypass |
| Quality | `quality=standard | strict` |
| Fallback | `fallback_policy=none | explicit` |
| Execution venue | `require_execution_venue=true | false` |
| Asset group | Canonical symbols | Daily / weekly source | 4H source | Explicit fallback |
|---|---|---|---|---|
| Shanghai Composite, STAR 50, Shanghai Dividend | sh000001, sh000688, sh000015 | Tencent tencent_kline | — | Sina sina_index |
| DXY, S&P 500, Nasdaq, VIX, Nikkei, KOSPI | DX-Y.NYB, ^GSPC, ^IXIC, ^VIX, ^N225, ^KS11 | Yahoo yahoo_finance_index | DXY only | none |
| U.S. 2Y, U.S. 10Y, 2s10s | DGS2, DGS10, T10Y2Y | Official Treasury CSV | — | none |
| U.S. dividend | SCHD | Yahoo yahoo_finance_etf | — | none |
| Bitcoin | BTC / provider BTCUSDT | Binance Spot | native Binance 4H | none |
| WTI, Gold, Silver | CL=F, GC=F, SI=F | Yahoo futures | Yahoo 1H → 4H | none |
For the three A-share indices, fallback is never implicit. A caller must send
fallback_policy=explicit&fallback_sources=sina_index; the response then
records attempted_sources, selected_source, selection_reason, provider
symbol, endpoint and any primary-source failure. Every other Phase 1 asset
keeps fallback_policy=none.
- Native 4H data is passed through as
raw_timeframe=4handtimeframe_origin=native; it is never aggregated a second time. - Yahoo 1H → 4H responses retain
raw_timeframe=1h,timeframe_origin=aggregated, the fixed bucket rule, UTC anchor and dropped incomplete bucket count. - Daily → weekly responses retain
raw_timeframe=1d, completed-week rule, bucket timezone and input source identity.
The current local service is launchd-managed as com.wendy.datafeed:
runtime root: /Users/wendy/datafeed-runtime
listen: http://127.0.0.1:8100
build: 5cd7472036dec95e4eaf5e8f1e0b71b7b4c65eb0
health: http://127.0.0.1:8100/api/health
docs: http://127.0.0.1:8100/docs
The health envelope exposes runtime_root, module_root, build_sha,
registry_version, database_path and identity_status. Provider
available=false with availability_basis=not_live_probed means capability
registration has not itself performed a live probe; an actual request remains
the source of truth for readiness.
常用 profile:
| Profile | 等价策略 | 用途 |
|---|---|---|
profile=historical | cache_policy=allow, quality=standard | 历史分析、回放、离线图表 |
profile=realtime | cache_policy=bypass, quality=strict, fallback_policy=none | 任意实时数据源的高标准拉取 |
profile=execution_live | source=binance_usdm_futures, cache_policy=bypass, quality=strict, fallback_policy=none, require_execution_venue=true | XAUUSDT 执行场所实时 K 线 |
兼容参数仍然可用:
mode=research等价默认 historical 行为。mode=live或strict=true是profile=execution_live的旧快捷方式。refresh=true等价cache_policy=bypass。
硬约束:
is_synthetic恒为false,没有合成/占位数据路径。cache_policy=bypass/profile=realtime/profile=execution_live不读 cache。- Phase 1
1d/1w/4hcache rows without a persisted timeframe receipt are blocked rather than relabeled; usecache_policy=bypassto obtain a fresh, source-bound response. - Phase 1 fresh
1d/1w/4hupstream results are not written to the legacy candle cache until the transformation receipt has a storage schema;allowmay fetch upstream, whilerequirereturnscache_miss. quality=strict下,上游失败、空数据、陈旧、gap、乱序都返回错误/blocked envelope。fallback_policy=explicit只尝试调用方明确列出的备用 source;没有 silent source switch。响应中的requested_source、attempted_sources、selected_source和selection_reason必须能还原选择过程。require_execution_venue=true会拒绝 Yahoo、TuShare、Binance Spot 等非执行场所 source。- normalized storage 以
source_id + ticker + asset_class + timeframe + timestamp隔离;同一标的多源不会互相覆盖。
# 历史/回放:允许 cache
curl "localhost:8100/api/candles/commodity/GOLD?timeframe=1d&profile=historical"# 任意实时源高标准:跳过 cache + strict quality
curl "localhost:8100/api/candles/crypto/BTC?timeframe=1m&source=binance_spot_public&profile=realtime"# 执行场所实时:Binance USD-M Futures XAUUSDT
curl "localhost:8100/api/candles/commodity/XAUUSDT?timeframe=1m&profile=execution_live&limit=200"# 显式 fallback:主源失败后只尝试调用方点名的源,响应记录 selected/attempted sources
curl "localhost:8100/api/candles/commodity/GOLD?timeframe=1m&source=binance_usdm_futures&cache_policy=bypass&fallback_policy=explicit&fallback_sources=yahoo_finance_futures"# Weekly Macro A-share index:Tencent 主源,Sina 显式 fallback
curl "localhost:8100/api/candles/index/sh000001?timeframe=1w&source=tencent_kline&cache_policy=bypass&quality=strict&fallback_policy=explicit&fallback_sources=sina_index&limit=3"# WebSocket 实时 candle update
ws://localhost:8100/api/ws/candles/commodity/XAUUSDT?timeframe=1m&source=binance_usdm_futures
# 浏览器可见的来源健康、最近请求和 source-scoped storage coverage
http://localhost:8100/health-ui$ curl "localhost:8100/api/candles/us_stock/AAPL?timeframe=1d&limit=3"{
"ticker": "AAPL",
"asset_class": "us_stock",
"timeframe": "1d",
"count": 3,
"schema_version": "kline-candles-v1",
"provider": "yahoo_finance",
"source_mode": "yahoo_finance",
"requested_source": "auto",
"cache_policy": "allow",
"quality_policy": "standard",
"fallback_policy": "none",
"require_execution_venue": false,
"quality_flags": ["delayed_possible", "market_hours", "research_only"],
"is_synthetic": false,
"served_from": "upstream",
"fresh": null,
"latest_timestamp": "2026-03-28",
"age_seconds": 172800.0,
"max_age_seconds": null,
"execution_venue": false,
"reject_reason": null,
"access_issues": [],
"candles": [
{"timestamp": "2026-03-26", "open": 178.5, "high": 182.3, "low": 177.8, "close": 181.2, "volume": 52340000, "provider": "yahoo_finance", "quality_flags": ["delayed_possible", "market_hours", "research_only"]},
{"timestamp": "2026-03-27", "open": 181.2, "high": 183.1, "low": 179.5, "close": 180.8, "volume": 48120000, "provider": "yahoo_finance", "quality_flags": ["delayed_possible", "market_hours", "research_only"]},
{"timestamp": "2026-03-28", "open": 180.8, "high": 185.0, "low": 180.2, "close": 184.5, "volume": 55670000, "provider": "yahoo_finance", "quality_flags": ["delayed_possible", "market_hours", "research_only"]}
]
}每个响应都带一层 provenance / 信任头,让下游(图表、回测、agent)不用猜数据来自哪、有多新、是不是真价格:
| 字段 | 含义 |
|---|---|
provider / source_mode | 具体上游与取数路径(binance_spot / yahoo_finance / tushare / binance_usdm_futures) |
requested_source | 调用方请求的 source。auto 会解析成 asset class 默认 source |
raw_timeframe / timeframe_origin | 上游原始周期,以及 native / aggregated 语义 |
aggregation | 聚合规则、输入周期、bucket timezone/anchor 和丢弃的不完整 bucket |
source_identity | provider symbol、source、周期转换和请求级 provenance receipt |
cache_policy | allow 可读 cache,bypass 跳过 cache,require 只读 cache |
quality_policy | standard 只标记质量事实;strict 会对 empty/stale/gap/duplicate/out-of-order blocked |
fallback_policy | 当前默认 none,不做静默 fallback |
require_execution_venue | 是否要求 source 必须是执行场所 |
quality_flags | 数据性质标记;research 源带 research_only / not_execution_venue,live futures 源带 live / execution_venue |
is_synthetic | 恒为 false。kline 没有任何合成/占位数据路径,返回的要么是真实数据、要么直接报错 |
served_from | "cache"(本地库)、"upstream"(刚从上游拉的)或 "websocket"(实时推送) |
latest_timestamp / age_seconds | 最新一根 bar 的时间与年龄(永远是诚实事实) |
fresh / max_age_seconds | 仅对 7×24 连续市场(crypto)给出新鲜度判定;行情有休市的源(美股/商品/A股)恒为 null,由消费方按自己的交易日历判断 |
execution_venue | 是否来自可作为交易页实时真源的执行场所。research 源为 false,Binance USD-M Futures live 为 true |
reject_reason / access_issues | strict quality blocked 时的直接原因,如 upstream_error、empty_data、stale、gap、out_of_order |
cache_policy=allow命中 cache 时不会自动回源刷新,但served_from+age_seconds+fresh让陈旧数据可见。实时路径应使用cache_policy=bypass或profile=realtime。
# Phase 1 A股指数日线(无需 TuShare token;Tencent 主源 + Sina 显式 fallback)
$ curl "localhost:8100/api/candles/index/sh000001?timeframe=1d&source=tencent_kline&cache_policy=bypass&quality=strict&fallback_policy=explicit&fallback_sources=sina_index"# 加密货币 1 小时线
$ curl "localhost:8100/api/candles/crypto/BTC?timeframe=1h&limit=100"# 商品 — 黄金日线
$ curl "localhost:8100/api/candles/commodity/GOLD?timeframe=1d"Request ▶ Resolve source manifest + cache/quality/fallback policy
│
├─ cache_policy=require ▶ cache hit? yes: return served_from=cache
│ no: error cache_miss
│
├─ cache_policy=allow ▶ cache hit? yes: return served_from=cache
│ no: fetch upstream
│
└─ cache_policy=bypass ▶ fetch upstream
MarketDataPort adapter ▶ raw response capture ▶ normalize candles
│
├─ quality=standard ▶ return envelope with visible quality flags
└─ quality=strict ▶ empty/stale/gap/duplicate/out-of-order => blocked
Adapters: TuShare / Yahoo / Binance Spot / Binance USD-M Futures / registered brokers
# 1. 克隆仓库
git clone https://github.com/zinan92/datafeed.git
cd datafeed
# 2. 安装依赖
python -m venv .venv &&source .venv/bin/activate
pip install -e .# 3. 可选:配置 TuShare(仅用于非 Phase 1 的 A-share equity adapter)
cp .env.example .env
# 编辑 .env 填入 KLINE_TUSHARE_TOKEN(如果确实使用 TuShare)# Phase 1 的三个指数 sh000001/sh000688/sh000015 不需要 TuShare token。# 4. 启动服务
python -m kline
# 访问 http://localhost:8100/docs 查看交互式 API 文档| 方法 | 路径 | 说明 |
|---|---|---|
GET | /api/candles/{asset_class}/{ticker} | 获取 K 线蜡烛图数据;支持 source/cache/quality policy |
WS | /api/ws/candles/{asset_class}/{ticker} | 实时 candle update;当前支持 Binance USD-M Futures |
GET | /api/tickers | 列出本地已缓存的 ticker |
GET | /api/health | 健康检查 + provider availability |
GET | /api/sessions/{asset_class}/{ticker} | 获取 adapter-owned market sessions |
| 参数 | 类型 | 默认值 | 说明 |
|---|---|---|---|
asset_class | path | — | a_share / us_stock / index / etf / crypto / commodity |
ticker | path | — | 代码: 000001, AAPL, BTC, GOLD |
timeframe | query | 1d | 1m / 5m / 15m / 30m / 1h / 4h / 1d / 1w |
start | query | — | 起始日期 YYYY-MM-DD |
end | query | — | 结束日期 YYYY-MM-DD |
limit | query | 500 | 返回蜡烛数量上限 (1-2000) |
source | query | auto | auto / tushare_pro / tencent_kline / sina_index / treasury_official_csv / treasury_official_csv_derived / Yahoo / Binance / FRED source ids |
cache_policy | query | allow | allow / bypass / require |
quality | query | standard | standard / strict |
fallback_policy | query | none | none / explicit;explicit 时必须同时传 fallback_sources |
fallback_sources | query | — | 可重复 query 参数;例如 fallback_sources=sina_index |
require_execution_venue | query | false | true 会拒绝非执行场所 source |
profile | query | — | historical / realtime / execution_live |
refresh | query | false | 兼容参数;true 等价 cache_policy=bypass |
mode | query | research | 兼容参数;live 等价 profile=execution_live |
strict | query | false | 兼容参数;true 等价 profile=execution_live |
| 别名 | Yahoo Finance 代码 |
|---|---|
GOLD, XAUUSD | GC=F |
SILVER, XAGUSD | SI=F |
OIL, CRUDE, WTI | CL=F |
BRENT | BZ=F |
NATGAS | NG=F |
COPPER | HG=F |
CORN, WHEAT, SOYBEAN | ZC=F, ZW=F, ZS=F |
| Source | Asset class | Provider | Market type | Realtime | Execution venue | 支持 Timeframe |
|---|---|---|---|---|---|---|
tushare_pro | a_share | TuShare Pro | equity | false | false | 1d, 1w |
tencent_kline | index | Tencent Finance | A-share index | false | false | sh000001/sh000688/sh000015: 1d, 1w |
sina_index | index | Sina Finance | A-share index fallback | false | false | sh000001/sh000688/sh000015: 1d, 1w |
treasury_official_csv | macro | U.S. Treasury | yield level | false | false | DGS2/DGS10: 1d, 1w |
treasury_official_csv_derived | macro | U.S. Treasury | 2s10s derived spread | false | false | T10Y2Y: 1d, 1w |
yahoo_finance | us_stock | Yahoo Finance | equity | false | false | 1m, 5m, 15m, 30m, 1h, 1d, 1w |
yahoo_finance_index | index | Yahoo Finance | index | false | false | DX-Y.NYB: 1d, 1w, 4h; ^GSPC/^IXIC/^VIX/^N225/^KS11: 1d, 1w |
yahoo_finance_etf | etf | Yahoo Finance | ETF | false | false | SPY/QQQ/SCHD: 1d, 1w; UUP: 4h, 1d, 1w |
yahoo_finance_futures | commodity | Yahoo Finance | continuous futures contract | false | false | CL=F/GC=F/SI=F: 1m, 5m, 15m, 30m, 1h, 4h, 1d, 1w; other aliases: 1d, 1w |
binance_spot_public | crypto | Binance Spot API | spot | true | false | BTC/BTCUSDT: 1m, 5m, 15m, 30m, 1h, 4h, 1d, 1w; other symbols: no Phase 1 4h guarantee |
binance_usdm_futures | commodity | Binance USD-M Futures | USD-M futures | true | true | XAUUSDT: 1m, 5m, 15m, 30m, 1h, 4h |
binance_usdm_futures_research | crypto | Binance USD-M Futures | USD-M perpetuals | true | false | BTCUSDT/ETHUSDT: 4h, 1d, 1w |
hyperliquid_perpetual_public | crypto | Hyperliquid public API | perpetual futures | false | false | BTC/ETH/HYPE: 30m, 4h, 1d, 1w |
| 层级 | 技术 | 用途 |
|---|---|---|
| 运行时 | Python 3.11+ | 核心语言 |
| 框架 | FastAPI | REST API |
| 存储 | SQLite (WAL mode) | 本地缓存,无需外部数据库 |
| 验证 | Pydantic v2 | 请求/响应模型 |
kline/
├── src/kline/
│ ├── app.py # FastAPI 应用
│ ├── api.py # 3 个 API 端点
│ ├── models.py # Candle 数据模型 + 枚举
│ ├── ports.py # MarketDataPort + SourceManifest
│ ├── quality.py # stale/gap/out-of-order 检测
│ ├── store.py # SQLite 存储 (upsert + query)
│ ├── registry.py # Adapter 注册与初始化
│ ├── config.py # 环境变量配置
│ └── providers/
│ ├── base.py # Provider Protocol 接口
│ ├── ashare.py # TuShare + Tencent index (A股)
│ ├── sina.py # Sina A-share index fallback
│ ├── treasury.py # Official Treasury levels and 2s10s
│ ├── us.py # Yahoo Finance (美股)
│ ├── crypto.py # Binance 公开 API (加密货币)
│ ├── commodity.py # Yahoo Finance 期货 + 别名
│ └── binance_usdm.py # Binance USD-M Futures live
├── tests/ # unit tests
├── data/ # SQLite 数据库 (自动创建)
├── .env.example
└── pyproject.toml
| 变量 | 说明 | 必填 | 默认值 |
|---|---|---|---|
KLINE_TUSHARE_TOKEN | TuShare Pro token(非 Phase 1 A-share equity 可选) | 否 | — |
KLINE_DB_PATH | SQLite 数据库路径 | 否 | data/kline.db |
KLINE_PORT | 服务端口 | 否 | 8100 |
KLINE_REQUEST_TIMEOUT | 上游请求超时 (秒) | 否 | 30 |
name: klineversion: 0.2.0capability:
summary: "Multi-asset K-line data service with explicit source/cache/quality/fallback policies."in: "ticker + timeframe + optional date range + source + cache_policy + quality + profile"out: "standardized OHLCV candles + provenance (provider, source_mode, policies, quality_flags, is_synthetic=false, served_from, fresh, age_seconds, execution_venue)"guarantees:
- "is_synthetic is always false — real data or an error, never a fabricated placeholder"
- "new brokers integrate through MarketDataPort + SourceManifest + register_adapter(adapter)"
- "source identity and cache/quality/fallback policies are visible in every response"
- "profile=realtime skips cache and applies strict quality checks"
- "profile=execution_live uses Binance USD-M Futures XAUUSDT, skips cache, applies strict quality checks, and requires execution_venue=true"
- "instrument-definition-v1 exposes upstream price/quantity constraints and reports unavailable fee or multiplier fields instead of inventing values"fail:
- "ticker not found → suggestions list"
- "cache_policy=require + no cache → cache_miss"
- "quality=strict + source down/empty/stale/gap/out-of-order → error or blocked envelope"
- "timeframe not supported → supported timeframes list"
- "TuShare-only A-share request without token → setup instructions; Phase 1 index sources do not require TuShare"sources: [tushare_pro, tencent_kline, sina_index, treasury_official_csv, treasury_official_csv_derived, yahoo_finance, yahoo_finance_index, yahoo_finance_etf, yahoo_finance_futures, binance_spot_public, binance_usdm_futures, binance_usdm_futures_research, hyperliquid_perpetual_public, fred_public_csv_macro, fred_public_csv_flow, fred_public_csv_event]api_base_url: http://localhost:8100endpoints:
- path: /api/instruments/{asset_class}/{ticker}method: GETdescription: "Get a versioned upstream execution instrument definition"params:
- name: sourcetype: stringdefault: auto
- name: require_execution_venuetype: booleandefault: false
- path: /api/candles/{asset_class}/{ticker}method: GETdescription: "Get OHLCV candles under explicit source/cache/quality policy"params:
- name: asset_classtype: stringenum: [a_share, us_stock, index, etf, crypto, commodity]required: true
- name: tickertype: stringrequired: true
- name: timeframetype: stringenum: ["1m", "5m", "15m", "30m", "1h", "4h", "1d", "1w"]default: "1d"
- name: limittype: integerdefault: 500
- name: sourcetype: stringenum: [auto, tushare_pro, yahoo_finance, yahoo_finance_index, yahoo_finance_etf, yahoo_finance_futures, binance_spot_public, binance_usdm_futures, binance_usdm_futures_research, hyperliquid_perpetual_public]default: auto
- name: cache_policytype: stringenum: [allow, bypass, require]default: allow
- name: qualitytype: stringenum: [standard, strict]default: standard
- name: profiletype: stringenum: [historical, realtime, execution_live]
- path: /api/ws/candles/{asset_class}/{ticker}method: WSdescription: "Stream realtime candle updates; current stream source is Binance USD-M Futures XAUUSDT"
- path: /api/tickersmethod: GETdescription: "List cached tickers"
- path: /api/healthmethod: GETdescription: "Health check"install_command: "pip install -e ."start_command: "python -m kline"health_check: "GET /api/health"importhttpxasyncdefget_candles(ticker: str, asset_class: str="us_stock", timeframe: str="1d"):
"""获取任意资产的 K 线数据"""base="http://localhost:8100"resp=awaithttpx.AsyncClient().get(
f"{base}/api/candles/{asset_class}/{ticker}",
params={"timeframe": timeframe, "limit": 100},
)
returnresp.json()["candles"]
# 美股candles=awaitget_candles("AAPL")
# A股candles=awaitget_candles("000001", "a_share")
# 加密货币 1 小时线candles=awaitget_candles("BTC", "crypto", "1h")
# 黄金candles=awaitget_candles("GOLD", "commodity")
# 执行场所实时:XAUUSDT Binance USD-M Futures,禁止 cache/fallbackasyncwithhttpx.AsyncClient() asclient:
resp=awaitclient.get(
"http://localhost:8100/api/candles/commodity/XAUUSDT",
params={"timeframe": "1m", "profile": "execution_live", "limit": 200},
)
resp.raise_for_status()
live_candles=resp.json()["candles"]| 项目 | 说明 | 链接 |
|---|---|---|
| quant-data-pipeline | 原始全功能量化数据平台 (kline 从中拆出) | zinan92/quant-data-pipeline |
| trading-copilot | AI 交易分析终端 (44 种方法论) | zinan92/trading-copilot |
MIT