test(agent): invariant oracles for unscripted exploration #400

Closed
#443
Closed
test(agent): invariant oracles for unscripted exploration#400
#443
Assignees
AndresL230
Labels
blockedchapter-2size-Mtest-infra
Milestone
e2e-ch2-agent

Description

@AndresL230
AndresL230
opened on Jul 22, 2026

Context

An agent finds bugs only if something defines "bug." These oracles assert properties that hold regardless of what the model or user did. Most are DB assertions and reuse the #397 psycopg fixtures. Open bug #355 (duplicate subject_root node) is live proof the duplicate-node invariant catches real defects.

Scope

Assert, after any exploration session:

  • Every extracted graph node traces to a chunk that exists in documents.
  • No duplicate nodes for the same concept within a document.
  • Mastery is monotonic under correct answers.
  • Quiz questions reference only nodes the user owns.
  • No 500s, no unhandled console errors, no unencrypted PII in any raw column touched during the run.

Acceptance criteria

  • Oracles run against a completed session and produce a pass/violation report.

Dependencies

Depends on: #399, #397
Blocks: #401

Chapter: 2 (Agent Explorer) | Lane: agent | Size: M | Suggested owner: Jack
Part of epic #403

Metadata

Metadata

Assignees

Labels

blockedchapter-2size-Mtest-infra

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    , 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
     blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

    Skip to content
    test(agent): invariant oracles for unscripted exploration #400
    Closed
    #443
    Closed
    test(agent): invariant oracles for unscripted exploration#400
    #443
    Assignees
    AndresL230
    Labels
    blockedchapter-2size-Mtest-infra
    Milestone
    e2e-ch2-agent
    Description
    @AndresL230
    AndresL230
    opened on Jul 22, 2026
    Context
    An agent finds bugs only if something defines "bug." These oracles assert properties that hold regardless of what the model or user did. Most are DB assertions and reuse the #397 psycopg fixtures. Open bug #355 (duplicate subject_root node) is live proof the duplicate-node invariant catches real defects.
    Scope
    Assert, after any exploration session:
    • Every extracted graph node traces to a chunk that exists in documents.
    • No duplicate nodes for the same concept within a document.
    • Mastery is monotonic under correct answers.
    • Quiz questions reference only nodes the user owns.
    • No 500s, no unhandled console errors, no unencrypted PII in any raw column touched during the run.
    Acceptance criteria
    •  Oracles run against a completed session and produce a pass/violation report.
    Dependencies
    Depends on: #399, #397
    
Blocks: #401
    Chapter: 2 (Agent Explorer) | Lane: agent | Size: M | Suggested owner: Jack
    
Part of epic #403
    Metadata
    Metadata
    Assignees
    Labels
    blockedchapter-2size-Mtest-infra
    Type
    No type
    Projects
    No projects
      Milestone
      Relationships
      None yet
      Development
      No branches or pull requests
      Issue actions
      
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

      Skip to content
      test(agent): invariant oracles for unscripted exploration #400
      Closed
      #443
      Closed
      test(agent): invariant oracles for unscripted exploration#400
      #443
      Assignees
      AndresL230
      Labels
      blockedchapter-2size-Mtest-infra
      Milestone
      e2e-ch2-agent
      Description
      @AndresL230
      AndresL230
      opened on Jul 22, 2026
      Context
      An agent finds bugs only if something defines "bug." These oracles assert properties that hold regardless of what the model or user did. Most are DB assertions and reuse the #397 psycopg fixtures. Open bug #355 (duplicate subject_root node) is live proof the duplicate-node invariant catches real defects.
      Scope
      Assert, after any exploration session:
      • Every extracted graph node traces to a chunk that exists in documents.
      • No duplicate nodes for the same concept within a document.
      • Mastery is monotonic under correct answers.
      • Quiz questions reference only nodes the user owns.
      • No 500s, no unhandled console errors, no unencrypted PII in any raw column touched during the run.
      Acceptance criteria
      •  Oracles run against a completed session and produce a pass/violation report.
      Dependencies
      Depends on: #399, #397
      
Blocks: #401
      Chapter: 2 (Agent Explorer) | Lane: agent | Size: M | Suggested owner: Jack
      
Part of epic #403
      Metadata
      Metadata
      Assignees
      Labels
      blockedchapter-2size-Mtest-infra
      Type
      No type
      Projects
      No projects
        Milestone
        Relationships
        None yet
        Development
        No branches or pull requests
        Issue actions
        
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

        Skip to content
        test(agent): invariant oracles for unscripted exploration #400
        Closed
        #443
        Closed
        test(agent): invariant oracles for unscripted exploration#400
        #443
        Assignees
        AndresL230
        Labels
        blockedchapter-2size-Mtest-infra
        Milestone
        e2e-ch2-agent
        Description
        @AndresL230
        AndresL230
        opened on Jul 22, 2026
        Context
        An agent finds bugs only if something defines "bug." These oracles assert properties that hold regardless of what the model or user did. Most are DB assertions and reuse the #397 psycopg fixtures. Open bug #355 (duplicate subject_root node) is live proof the duplicate-node invariant catches real defects.
        Scope
        Assert, after any exploration session:
        • Every extracted graph node traces to a chunk that exists in documents.
        • No duplicate nodes for the same concept within a document.
        • Mastery is monotonic under correct answers.
        • Quiz questions reference only nodes the user owns.
        • No 500s, no unhandled console errors, no unencrypted PII in any raw column touched during the run.
        Acceptance criteria
        •  Oracles run against a completed session and produce a pass/violation report.
        Dependencies
        Depends on: #399, #397
        
Blocks: #401
        Chapter: 2 (Agent Explorer) | Lane: agent | Size: M | Suggested owner: Jack
        
Part of epic #403
        Metadata
        Metadata
        Assignees
        Labels
        blockedchapter-2size-Mtest-infra
        Type
        No type
        Projects
        No projects
          Milestone
          Relationships
          None yet
          Development
          No branches or pull requests
          Issue actions
          
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

          Skip to content
          test(agent): invariant oracles for unscripted exploration #400
          Closed
          #443
          Closed
          test(agent): invariant oracles for unscripted exploration#400
          #443
          Assignees
          AndresL230
          Labels
          blockedchapter-2size-Mtest-infra
          Milestone
          e2e-ch2-agent
          Description
          @AndresL230
          AndresL230
          opened on Jul 22, 2026
          Context
          An agent finds bugs only if something defines "bug." These oracles assert properties that hold regardless of what the model or user did. Most are DB assertions and reuse the #397 psycopg fixtures. Open bug #355 (duplicate subject_root node) is live proof the duplicate-node invariant catches real defects.
          Scope
          Assert, after any exploration session:
          • Every extracted graph node traces to a chunk that exists in documents.
          • No duplicate nodes for the same concept within a document.
          • Mastery is monotonic under correct answers.
          • Quiz questions reference only nodes the user owns.
          • No 500s, no unhandled console errors, no unencrypted PII in any raw column touched during the run.
          Acceptance criteria
          •  Oracles run against a completed session and produce a pass/violation report.
          Dependencies
          Depends on: #399, #397
          
Blocks: #401
          Chapter: 2 (Agent Explorer) | Lane: agent | Size: M | Suggested owner: Jack
          
Part of epic #403
          Metadata
          Metadata
          Assignees
          Labels
          blockedchapter-2size-Mtest-infra
          Type
          No type
          Projects
          No projects
            Milestone
            Relationships
            None yet
            Development
            No branches or pull requests
            Issue actions
            
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

            Skip to content
            test(agent): invariant oracles for unscripted exploration #400
            Closed
            #443
            Closed
            test(agent): invariant oracles for unscripted exploration#400
            #443
            Assignees
            AndresL230
            Labels
            blockedchapter-2size-Mtest-infra
            Milestone
            e2e-ch2-agent
            Description
            @AndresL230
            AndresL230
            opened on Jul 22, 2026
            Context
            An agent finds bugs only if something defines "bug." These oracles assert properties that hold regardless of what the model or user did. Most are DB assertions and reuse the #397 psycopg fixtures. Open bug #355 (duplicate subject_root node) is live proof the duplicate-node invariant catches real defects.
            Scope
            Assert, after any exploration session:
            • Every extracted graph node traces to a chunk that exists in documents.
            • No duplicate nodes for the same concept within a document.
            • Mastery is monotonic under correct answers.
            • Quiz questions reference only nodes the user owns.
            • No 500s, no unhandled console errors, no unencrypted PII in any raw column touched during the run.
            Acceptance criteria
            •  Oracles run against a completed session and produce a pass/violation report.
            Dependencies
            Depends on: #399, #397
            
Blocks: #401
            Chapter: 2 (Agent Explorer) | Lane: agent | Size: M | Suggested owner: Jack
            
Part of epic #403
            Metadata
            Metadata
            Assignees
            Labels
            blockedchapter-2size-Mtest-infra
            Type
            No type
            Projects
            No projects
              Milestone
              Relationships
              None yet
              Development
              No branches or pull requests
              Issue actions
              
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

              Skip to content
              test(agent): invariant oracles for unscripted exploration #400
              Closed
              #443
              Closed
              test(agent): invariant oracles for unscripted exploration#400
              #443
              Assignees
              AndresL230
              Labels
              blockedchapter-2size-Mtest-infra
              Milestone
              e2e-ch2-agent
              Description
              @AndresL230
              AndresL230
              opened on Jul 22, 2026
              Context
              An agent finds bugs only if something defines "bug." These oracles assert properties that hold regardless of what the model or user did. Most are DB assertions and reuse the #397 psycopg fixtures. Open bug #355 (duplicate subject_root node) is live proof the duplicate-node invariant catches real defects.
              Scope
              Assert, after any exploration session:
              • Every extracted graph node traces to a chunk that exists in documents.
              • No duplicate nodes for the same concept within a document.
              • Mastery is monotonic under correct answers.
              • Quiz questions reference only nodes the user owns.
              • No 500s, no unhandled console errors, no unencrypted PII in any raw column touched during the run.
              Acceptance criteria
              •  Oracles run against a completed session and produce a pass/violation report.
              Dependencies
              Depends on: #399, #397
              
Blocks: #401
              Chapter: 2 (Agent Explorer) | Lane: agent | Size: M | Suggested owner: Jack
              
Part of epic #403
              Metadata
              Metadata
              Assignees
              Labels
              blockedchapter-2size-Mtest-infra
              Type
              No type
              Projects
              No projects
                Milestone
                Relationships
                None yet
                Development
                No branches or pull requests
                Issue actions
                
, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

                Skip to content
                test(agent): invariant oracles for unscripted exploration #400
                Closed
                #443
                Closed
                test(agent): invariant oracles for unscripted exploration#400
                #443
                Assignees
                AndresL230
                Labels
                blockedchapter-2size-Mtest-infra
                Milestone
                e2e-ch2-agent
                Description
                @AndresL230
                AndresL230
                opened on Jul 22, 2026
                Context
                An agent finds bugs only if something defines "bug." These oracles assert properties that hold regardless of what the model or user did. Most are DB assertions and reuse the #397 psycopg fixtures. Open bug #355 (duplicate subject_root node) is live proof the duplicate-node invariant catches real defects.
                Scope
                Assert, after any exploration session:
                • Every extracted graph node traces to a chunk that exists in documents.
                • No duplicate nodes for the same concept within a document.
                • Mastery is monotonic under correct answers.
                • Quiz questions reference only nodes the user owns.
                • No 500s, no unhandled console errors, no unencrypted PII in any raw column touched during the run.
                Acceptance criteria
                •  Oracles run against a completed session and produce a pass/violation report.
                Dependencies
                Depends on: #399, #397
                
Blocks: #401
                Chapter: 2 (Agent Explorer) | Lane: agent | Size: M | Suggested owner: Jack
                
Part of epic #403
                Metadata
                Metadata
                Assignees
                Labels
                blockedchapter-2size-Mtest-infra
                Type
                No type
                Projects
                No projects
                  Milestone
                  Relationships
                  None yet
                  Development
                  No branches or pull requests
                  Issue actions