feat: add LLM-backed DescriptionDiffAlignmentCondition

.watchflow/rules.yaml

@@ -41,3 +41,10 @@ rules:
  event_types: ["pull_request"]
  parameters:
  block_on_unresolved_comments: true
+
+ - description: "PR description must accurately reflect the actual code changes in the diff."
+ enabled: true
+ severity: "medium"
+ event_types: ["pull_request"]
+ parameters:
+ require_description_diff_alignment: true

CHANGELOG.md

@@ -4,6 +4,16 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## [Unreleased]
+
+### Added
+
+- **Description-diff alignment** -- `DescriptionDiffAlignmentCondition` uses
+ the configured AI provider (OpenAI / Bedrock / Vertex AI) to verify that
+ the PR description semantically matches the actual code changes. First
+ LLM-backed condition in Watchflow; adds ~1-3s latency. Gracefully skips
+ (no violation) if the LLM is unavailable.
+
 ## [2026-03-01] -- PR #59
 
 ### Added

README.md

@@ -65,6 +65,7 @@ Rules are **description + event_types + parameters**. The engine matches paramet
 | **Push** | `no_force_push: true` | push | Reject force pushes. |
 | **Files** | `max_file_size_mb: 1` | pull_request | No single file > N MB. |
 | **Files** | `pattern` + `condition_type: "files_match_pattern"` | pull_request | Changed files must (or must not) match glob/regex. |
+| **PR** | `require_description_diff_alignment: true` | pull_request | Description must match code changes (LLM-assisted). |
 | **Time** | `allowed_hours`, `days`, weekend | deployment / workflow | Restrict when actions can run. |
 
 Rules are read from the **default branch** (e.g. `main`). Each webhook delivery is deduplicated by `X-GitHub-Delivery` so handler and processor both run; comments and check runs stay in sync.

docs/concepts/overview.md

@@ -5,7 +5,7 @@ Watchflow is a **rule engine** for GitHub: you define rules in YAML; we evaluate
 ## Design principles
 
 - **Repo-native** — Rules live in `.watchflow/rules.yaml` on the default branch; same mental model as branch protection and CODEOWNERS.
-- **Condition-based enforcement** — Rule evaluation is deterministic: parameters map to conditions (e.g. `require_linked_issue`, `max_lines`, `require_code_owner_reviewers`). No LLM in the hot path for “did this PR violate the rule?”
+- **Condition-based enforcement** — Rule evaluation is deterministic by default: parameters map to conditions (e.g. `require_linked_issue`, `max_lines`, `require_code_owner_reviewers`). Optional LLM-assisted conditions (e.g. `require_description_diff_alignment`) are clearly documented and opt-in.
 - **Webhook-first** — Each delivery is identified by `X-GitHub-Delivery`; handler and processor get distinct task IDs so both run and comments/check runs stay in sync.
 - **Optional intelligence** — Repo analysis and feasibility checks use LLMs to *suggest* rules; enforcement stays rule-driven.
 
@@ -41,7 +41,7 @@ graph TD
 ### Condition registry
 
 - Maps parameter names to condition classes (e.g. `require_linked_issue` → `RequireLinkedIssueCondition`, `max_lines` → `MaxPrLocCondition`, `require_code_owner_reviewers` → `RequireCodeOwnerReviewersCondition`).
-- Supported conditions: linked issue, title pattern, description length, labels, approvals, PR size (lines), CODEOWNERS (path has owner, require owners as reviewers), protected branches, no force push, file size, file pattern, diff pattern scanning, security pattern detection, unresolved comments, test coverage, comment response SLA, signed commits, changelog required, self-approval prevention, cross-team approval, time/deploy rules. See [Configuration](../getting-started/configuration.md).
+- Supported conditions: linked issue, title pattern, description length, labels, approvals, PR size (lines), CODEOWNERS (path has owner, require owners as reviewers), protected branches, no force push, file size, file pattern, diff pattern scanning, security pattern detection, unresolved comments, test coverage, comment response SLA, signed commits, changelog required, self-approval prevention, cross-team approval, description-diff alignment (LLM-assisted), time/deploy rules. See [Configuration](../getting-started/configuration.md).
 
 ### PR enricher
 
@@ -53,12 +53,12 @@ graph TD
 
 ## Where AI is used (and where it isn’t)
 
-- **Rule evaluation** — No. Violations are determined by conditions only.
+- **Rule evaluation** — No LLM by default. Violations are determined by deterministic conditions. Optional LLM-assisted conditions (e.g. `DescriptionDiffAlignmentCondition`) are clearly marked and gracefully degrade on failure.
 - **Acknowledgment parsing** — Optional LLM to interpret reason; can be extended.
 - **Repo analysis** — Yes. `POST /api/v1/rules/recommend` uses an agent to suggest rules from repo structure and PR history; you copy/paste or create a PR.
 - **Feasibility** — Yes. “Can I enforce this rule?” uses an agent to map natural language to supported conditions and suggest YAML.
 
-So: **enforcement is deterministic and condition-based**; **suggestions and feasibility are agent-assisted**. That keeps the hot path simple and auditable.
+So: **enforcement is deterministic and condition-based by default**; **LLM-assisted conditions are opt-in and fail-open**; **suggestions and feasibility are agent-assisted**. That keeps the hot path simple and auditable.
 
 ## Use cases
 

docs/features.md

@@ -57,6 +57,12 @@ Rules are **description + event_types + parameters**. The engine matches **param
 | `require_signed_commits: true` | SignedCommitsCondition | All commits must be cryptographically signed (GPG/SSH/S/MIME). |
 | `require_changelog_update: true` | ChangelogRequiredCondition | Source changes must include a CHANGELOG or `.changeset` update. |
 
+### LLM-assisted
+
+| Parameter | Condition | Description |
+|-----------|-----------|-------------|
+| `require_description_diff_alignment: true` | DescriptionDiffAlignmentCondition | PR description must semantically match the code diff (uses LLM; ~1-3s latency). |
+
 ---
 
 ## Repository analysis → one-click rules PR

docs/getting-started/configuration.md

@@ -238,6 +238,17 @@ parameters:
 
 PRs that modify source code must include a corresponding `CHANGELOG.md` or `.changeset/` update. Docs, tests, and `.github/` paths are excluded.
 
+### LLM-assisted conditions
+
+**Description-diff alignment**
+
+```yaml
+parameters:
+ require_description_diff_alignment: true
+```
+
+Uses the configured AI provider to check whether the PR description semantically reflects the actual code changes. Flags mismatches like "description says fix login but diff only touches billing code." Adds ~1-3s latency per evaluation. If the LLM is unavailable (provider not configured, rate limit), the condition gracefully skips without blocking the PR.
+
 ---
 
 ## Example rules

src/event_processors/pull_request/processor.py

@@ -1,4 +1,6 @@
+import hashlib
 import logging
+import re
 import time
 from typing import Any
 
@@ -208,19 +210,95 @@ async def process(self, task: Task) -> ProcessingResult:
  )
 
  async def _post_violations_to_github(self, task: Task, violations: list[Violation]) -> None:
- """Post violations as comments on the pull request."""
+ """Post violations as comments on the pull request.
+
+ Implements comment-level deduplication by checking existing PR comments
+ and skipping if an identical Watchflow violations comment already exists.
+ """
  try:
  pr_number = task.payload.get("pull_request", {}).get("number")
  if not pr_number or not task.installation_id:
  return
 
- comment_body = github_formatter.format_violations_comment(violations)
+ # Compute content hash for deduplication
+ violations_signature = self._compute_violations_hash(violations)
+
+ # Check if identical comment already exists
+ if await self._has_duplicate_comment(
+ task.repo_full_name, pr_number, violations_signature, task.installation_id
+ ):
+ logger.info(
+ "Skipping duplicate violations comment",
+ extra={
+ "pr_number": pr_number,
+ "repo": task.repo_full_name,
+ "violations_hash": violations_signature,
+ },
+ )
+ return
+
+ # Post new comment with hash marker
+ comment_body = github_formatter.format_violations_comment(violations, content_hash=violations_signature)
  await self.github_client.create_pull_request_comment(
  task.repo_full_name, pr_number, comment_body, task.installation_id
  )
+ logger.info(
+ "Posted violations comment",
+ extra={
+ "pr_number": pr_number,
+ "repo": task.repo_full_name,
+ "violations_count": len(violations),
+ "violations_hash": violations_signature,
+ },
+ )
  except Exception as e:
  logger.error(f"Error posting violations to GitHub: {e}")
 
+ def _compute_violations_hash(self, violations: list[Violation]) -> str:
+ """Compute a stable hash of violations for deduplication.
+
+ Uses rule_description + message + severity to create a fingerprint.
+ This allows detecting identical violation sets regardless of delivery_id.
+ """
+ # Sort violations to ensure consistent ordering
+ sorted_violations = sorted(
+ violations,
+ key=lambda v: (v.rule_description or "", v.message, v.severity.value if v.severity else ""),
+ )
+
+ # Build signature from key fields
+ signature_parts = []
+ for v in sorted_violations:
+ signature_parts.append(f"{v.rule_description}|{v.message}|{v.severity.value if v.severity else ''}")
+
+ signature_string = "::".join(signature_parts)
+ return hashlib.sha256(signature_string.encode()).hexdigest()[:12] # Use first 12 chars for readability
+
+ async def _has_duplicate_comment(
+ self, repo: str, pr_number: int, violations_hash: str, installation_id: int
+ ) -> bool:
+ """Check if a comment with the same violations hash already exists.
+
+ Looks for the hidden HTML marker in existing comments to detect duplicates.
+ """
+ try:
+ existing_comments = await self.github_client.get_issue_comments(repo, pr_number, installation_id)
+
+ # Pattern to extract hash from hidden marker: <!-- watchflow-violations-hash:ABC123 -->
+ hash_pattern = re.compile(r"<!-- watchflow-violations-hash:([a-f0-9]+) -->")
+
+ for comment in existing_comments:
+ body = comment.get("body", "")
+ match = hash_pattern.search(body)
+ if match and match.group(1) == violations_hash:
+ return True
+
+ return False
+ except Exception as e:
+ logger.warning(f"Error checking for duplicate comments: {e}. Proceeding with post.")
+ # Fail open: if we can't check, allow posting to avoid blocking
+ return False
+
  async def prepare_webhook_data(self, task: Task) -> dict[str, Any]:
  """Extract data available in webhook payload."""
  return self.enricher.prepare_webhook_data(task)

src/presentation/github_formatter.py

@@ -190,11 +190,12 @@ def format_rules_not_configured_comment(
  )
 
 
-def format_violations_comment(violations: list[Violation]) -> str:
+def format_violations_comment(violations: list[Violation], content_hash: str | None = None) -> str:
  """Format violations as a GitHub comment.
 
  Args:
  violations: List of rule violations to include in the comment.
+ content_hash: Optional hash to include as a hidden marker for deduplication.
 
  Returns:
  A Markdown formatted string suitable for a Pull Request timeline comment.
@@ -203,7 +204,11 @@ def format_violations_comment(violations: list[Violation]) -> str:
  if not violations:
  return ""
 
- comment = f"### 🛡️ Watchflow Governance Checks\n**Status:** ❌ {len(violations)} Violations Found\n\n"
+ # Add hidden HTML marker for deduplication (not visible in rendered markdown)
+ marker = f"<!-- watchflow-violations-hash:{content_hash} -->\n" if content_hash else ""
+
+ comment = marker
+ comment += f"### 🛡️ Watchflow Governance Checks\n**Status:** ❌ {len(violations)} Violations Found\n\n"
  comment += _build_collapsible_violations_text(violations)
  comment += "---\n"
  comment += (

src/rules/acknowledgment.py

@@ -43,6 +43,7 @@ class RuleID(StrEnum):
  CHANGELOG_REQUIRED = "changelog-required"
  NO_SELF_APPROVAL = "no-self-approval"
  CROSS_TEAM_APPROVAL = "cross-team-approval"
+ DESCRIPTION_DIFF_ALIGNMENT = "description-diff-alignment"
 
 
 # Mapping from violation text patterns to RuleID
@@ -67,6 +68,7 @@ class RuleID(StrEnum):
  "without a corresponding CHANGELOG": RuleID.CHANGELOG_REQUIRED,
  "approved by its own author": RuleID.NO_SELF_APPROVAL,
  "approvals from required teams": RuleID.CROSS_TEAM_APPROVAL,
+ "does not align with code changes": RuleID.DESCRIPTION_DIFF_ALIGNMENT,
 }
 
 # Mapping from RuleID to human-readable descriptions
@@ -91,6 +93,7 @@ class RuleID(StrEnum):
  RuleID.CHANGELOG_REQUIRED: "Source code changes must include a CHANGELOG or .changeset update.",
  RuleID.NO_SELF_APPROVAL: "PR authors cannot approve their own pull requests.",
  RuleID.CROSS_TEAM_APPROVAL: "Pull requests require approvals from specified GitHub teams.",
+ RuleID.DESCRIPTION_DIFF_ALIGNMENT: "PR description must accurately reflect the actual code changes.",
 }
 
 # Comment markers that indicate an acknowledgment comment

src/rules/conditions/__init__.py

@@ -26,6 +26,7 @@
  MaxPrLocCondition,
  TestCoverageCondition,
 )
+from src.rules.conditions.llm_assisted import DescriptionDiffAlignmentCondition
 from src.rules.conditions.pull_request import (
  DiffPatternCondition,
  MinDescriptionLengthCondition,
@@ -71,6 +72,8 @@
  # Compliance
  "SignedCommitsCondition",
  "ChangelogRequiredCondition",
+ # LLM-assisted
+ "DescriptionDiffAlignmentCondition",
  # Temporal
  "AllowedHoursCondition",
  "CommentResponseTimeCondition",