Skip to content

PRODUCTION_CHECKLIST.md

Latest commit

History

History
247 lines (188 loc) · 5.33 KB

PRODUCTION_CHECKLIST.md

File metadata and controls

247 lines (188 loc) · 5.33 KB
Raw

Before Production: What to Add

This example is bare minimum for demonstration. Before using in production, add:

Logging (Required)

Current: Simple print statements Add: Proper logging to file

importlogging# Add to main():logger=logging.getLogger(__name__)
logger.setLevel(logging.INFO)
handler=logging.FileHandler('market_data.log')
logger.addHandler(handler)
logger.info(f"Retrieved {len(data)} rows")

Error Handling (Required)

Current: Basic try/except Add: Specific exception handling, retries, and monitoring

try:
data=ld.get_data(...)
exceptConnectionError:
logger.error("Connection failed")
# Retry logic hereexceptExceptionase:
logger.error(f"Error: {e}", exc_info=True)
# Send alert/email

Data Validation (Required)

Current: No validation Add: Check data quality before saving

ifdata.empty:
logger.error("No data returned")
return1iflen(data) <expected_count:
logger.warning(f"Expected {expected_count}, got {len(data)}")

Configuration File (Recommended)

Current: Hardcoded values Add: Config file for instruments and fields

# config.json
{
"instruments": ["IBM.N", "MSFT.O", "APPL.O"],
"fields": ["TR.PriceClose", "TR.Volume"],
"output_dir": "C:\\market_data"
}

Session Management (Recommended)

Current: No timeout handling Add: Session initialization check and timeout

try:
session=ld.open_session()
exceptExceptionase:
logger.error("Failed to open session")
# Ensure session closes even on failure

Monitoring & Alerts (Recommended)

Current: No notifications Add: Email/Slack alerts on failure

ifexit_code!=0:
send_alert(f"Market data job failed: {error_msg}")

Logging (Optional but Useful)

Add structured logging:

importlogging.handlers# Rotate logs by sizehandler=logging.handlers.RotatingFileHandler(
'market_data.log',
maxBytes=10*1024*1024, # 10MBbackupCount=10
)

Database Storage (Optional)

Current: CSV files only Add: Database for historical tracking

# Store in database instead of (or in addition to) CSV# Allows queries like: "Get all closes for IBM in last 30 days"

Schedule Configuration (Recommended)

Current: Hardcoded 7:00 AM Add: Make schedule configurable

# environment variables or config fileSCHEDULE_TIME=os.getenv('SCHEDULE_TIME', '07:00')
SCHEDULE_DAYS=os.getenv('SCHEDULE_DAYS', 'MON,TUE,WED,THU,FRI')

Testing (Required for Production)

Add unit tests:

deftest_csv_creation():
result=main()
assertresult==0assertPath('market_data_*.csv').exists()
deftest_invalid_instruments():
result=main(['INVALID.X'])
assertresult==1

Security (Required for Production)

Current: No credential handling Add: Secure credential management

# Use environment variables or secure vaults# NEVER commit credentials to version controlAPI_KEY=os.getenv('LSEG_API_KEY')

Documentation (Recommended)

Add:

  • Runbook for manual execution
  • Troubleshooting guide for ops team
  • SLA/performance expectations
  • Disaster recovery procedures

Batch File Improvements

Current: Simple run Add:

REM Better error handlingif%ERRORLEVEL%neq0 (
REM Send alertREM Retry
)
REM Start LSEG Workspace if neededREM Wait for it to be readyREM Capture output for auditREM Check for previous failureREM Only run if previous run succeeded

Monitoring Dashboard

Track:

  • Script execution time
  • Number of rows returned
  • Error rates
  • File creation timestamps
  • CSV file sizes

Deployment Checklist

  • Logging configured
  • Error handling for all edge cases
  • Data validation added
  • Configuration file created
  • Credentials in environment variables
  • Unit tests pass
  • Integration tests pass
  • Monitoring/alerts configured
  • Documentation complete
  • Backup procedures in place
  • Tested on target environment
  • Runbook created for ops team

Minimum Production Version Example

importloggingimportosfromdatetimeimportdatetimefrompathlibimportPathfromlsegimportdataasldlogging.basicConfig(
filename='market_data.log',
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s'
)
defmain():
logger=logging.getLogger(__name__)
try:
session=ld.open_session()
data=ld.get_data(['IBM.N', 'MSFT.O'], ['TR.PriceClose'])
ifdata.empty:
logger.error("No data returned")
return1csv_file=f"market_data_{datetime.now().strftime('%Y%m%d')}.csv"data.to_csv(csv_file)
logger.info(f"Saved {len(data)} rows to {csv_file}")
return0exceptExceptionase:
logger.error(f"Error: {e}", exc_info=True)
return1finally:
try:
session.close()
except:
passif__name__=='__main__':
exit(main())

Start with this bare minimum example to demonstrate the concept, then upgrade piece by piece based on your requirements.