Add plan_resume MCP tool for mid-pipeline recovery

[neoneye]

Summary

• Add plan_resume MCP tool that resumes a failed plan without discarding completed intermediary files — plan generation restarts from the first incomplete step, skipping all steps that already produced output files
• Add PIPELINE_VERSION integer constant stamped into plan parameters on create, retry, and resume — on resume, the MCP layer rejects with PIPELINE_VERSION_MISMATCH if the stored version differs from the current version
• Write 001-3-planexe_metadata.json (containing pipeline_version) to the output dir at start, persisting it in the zip snapshot
• Worker-side defense-in-depth: after restoring the zip snapshot, verify pipeline_version from the metadata file before proceeding — sets plan back to failed with a descriptive progress_message on mismatch
• Replace internal terminology ("run", "task", "mid-pipeline") with user-facing language ("plan generation", "step", "before completing all steps", "intermediary files") across tool descriptions, server instructions, and docs
• Add naming migration notes to worker_plan_database/app.py header: new code should use plan_id (not task_id) and plan (not run)
• Rename restore_run_dir_from_zip_snapshot to restore_output_dir_from_zip_snapshot
• Document the plan_resume tool, compatibility caveat, and version check in the MCP interface spec (sections 6.6, 9.3, 9.4)
• Add end-to-end test plan proposal (docs/proposals/112-end-to-end-test-plan.md) with 9 scenarios covering plan_resume, pipeline version mismatch, and full happy path — split into no-LLM and LLM-required groups
• Fix llama_index import compatibility (core.llms.types → core.base.llms.types fallback)
• Fix token metrics session corruption: remove nested app.app_context() from _handle_task_completion — nested contexts tear down the scoped session, corrupting the connection for subsequent db operations like token metrics recording
• Add missing column migrations (thinking_tokens, duration_seconds, success, error_message, raw_usage_data) to ensure_token_metrics_columns so tables created by older model versions get the columns the ORM expects

Test plan

• pytest mcp_cloud/tests/test_plan_resume_tool.py — covers structured content, SSE URL, PLAN_NOT_FOUND, PLAN_NOT_RESUMABLE, PIPELINE_VERSION_MISMATCH, and default model_profile (requires Docker/CI)
• pytest mcp_cloud/tests/test_tool_surface_consistency.py — verifies plan_resume is listed in tool definitions
• pytest mcp_local/ — proxy tests
• pytest worker_plan/worker_plan_api/tests/ — unit tests for shared utilities (passes locally)
• Run a plan end-to-end and verify 001-3-planexe_metadata.json appears in the output zip
• Verify token metrics are recorded without psycopg2 errors during plan generation

🤖 Generated with Claude Code