Crypto Quantitative Data Management【SPOT】

Cryptocurrency K-Line data collection and management platform, supporting Binance spot market data.

Features

Data Collection

• Multiple trading pairs (BTCUSDT, ETHUSDT, BNBUSDT, etc.)
• Multiple K-Line intervals (1m, 5m, 15m, 1h, 4h, 1d, etc.)
• Automatic gap detection and backfilling
• Real-time task progress monitoring

Data Management

• View collected data statistics
• Filter by trading pair and K-Line interval
• Display data time range

K-Line Chart

• TradingView lightweight-charts rendering
• 13 Technical indicators:
  • Volume (VOL)
  • MACD
  • RSI
  • KDJ
  • Bollinger Bands (BOLL)
  • CCI
  • Williams %R (WR)
  • ATR
  • ADX
  • SAR
  • OBV
  • MFI
  • Stochastic (Stoch)
• Toggle MA5/MA10/MA20/MA60 moving averages
• Jump to specific time

UI/UX

• Light/Dark theme toggle
• Chinese/English language switch
• Responsive layout

Tech Stack

Frontend

• React 18 + TypeScript
• Ant Design 5
• TradingView lightweight-charts v5
• dayjs for date handling

Backend

• FastAPI (Python 3.10+)
• SQLAlchemy + asyncpg (async database)
• PostgreSQL database
• Redis (optional, for caching)

Quick Start

Prerequisites

• Node.js 18+
• Python 3.10+
• PostgreSQL 14+
• npm or yarn

Installation

1. Clone the project

git clone <repository-url>cd CryptoDataSet

2. Install frontend dependencies

cd frontend
npm install

3. Install backend dependencies

cd backend
pip install -r requirements.txt

4. Configure database

Create PostgreSQL database:

CREATEDATABASEcrypto_kline;

Set environment variable or modify backend/app/core/config.py:

DATABASE_URL="postgresql+asyncpg://postgres:postgres@localhost:5432/crypto_kline"

5. Initialize database tables

cd backend
python -c "from app.core.database import engine, Base; import asyncio; asyncio.run(Base.metadata.create_all(engine))"

6. Start backend server

cd backend
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

7. Start frontend dev server

cd frontend
npm start

Visit http://localhost:3000 to view the application.

Project Structure

CryptoDataSet/
├── frontend/
│ ├── src/
│ │ ├── components/ # React components
│ │ │ ├── DataCollection.tsx # Data collection
│ │ │ ├── DataManagement.tsx # Data management
│ │ │ └── KlineChart.tsx # K-Line chart
│ │ ├── contexts/ # React Context
│ │ │ └── ThemeContext.tsx # Theme context
│ │ ├── i18n/ # Internationalization
│ │ │ ├── locales.ts # Language files
│ │ │ └── LanguageContext.tsx # Language context
│ │ ├── App.tsx # Main app component
│ │ └── App.css # Global styles
│ └── package.json
├── backend/
│ ├── app/
│ │ ├── api/
│ │ │ └── routes.py # API routes
│ │ ├── core/
│ │ │ ├── config.py # Configuration
│ │ │ ├── database.py # Database connection
│ │ │ ├── middleware.py # Middleware (logging, monitoring)
│ │ │ └── rate_limit.py # Rate limiting
│ │ ├── services/
│ │ │ ├── binance_service.py # Binance API
│ │ │ ├── kline_service.py # K-Line service
│ │ │ └── task_manager.py # Task management
│ │ └── main.py # FastAPI entry point
│ └── requirements.txt
└── README.md

API Documentation

After starting the backend server:

• Swagger UI: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc

Main Endpoints

Method	Path	Description
GET	/api/config	Get config (symbols, intervals list)
GET	/api/tasks	Get task list
POST	/api/kline/collect	Trigger data collection
GET	/api/kline/{symbol}/{interval}/info	Get data statistics
GET	/api/kline/{symbol}/{interval}/previous	Get historical K-Lines
GET	/api/kline/{symbol}/{interval}/next	Get future K-Lines
GET	/health	Health check
GET	/metrics	Monitoring metrics

Configuration

Environment Variables

Variable	Description	Example
DATABASE_URL	PostgreSQL database connection string	postgresql+asyncpg://postgres:postgres@localhost:5432/crypto_kline
BINANCE_PROXY	Binance API proxy address (required in some regions like mainland China)	http://127.0.0.1:1080

Configure in backend/.env file:

DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:5432/crypto_kline
BINANCE_PROXY=http://127.0.0.1:1080

Trading Pairs

Modify CONFIG_SYMBOLS in backend/app/api/routes.py:

CONFIG_SYMBOLS= [
"BTCUSDT", "ETHUSDT", "BNBUSDT", "SOLUSDT", "XRPUSDT",
"ADAUSDT", "DOGEUSDT", "AVAXUSDT", "DOTUSDT", "MATICUSDT",
]

K-Line Intervals

Modify CONFIG_INTERVALS in backend/app/api/routes.py:

CONFIG_INTERVALS= ["1m", "3m", "5m", "15m", "30m", "1h", "2h", "4h", "6h", "8h", "12h", "1d", "3d", "1w", "1M"]

Monitoring & Logging

Health Check

curl http://localhost:8000/health

Metrics

curl http://localhost:8000/metrics

View Logs

Backend logs are output to console, including:

• Request logs (method, path, status code, response time)
• Collection progress logs
• Error logs

License

MIT License