Skip to content

fix: keep add event stream alive - #213

Draft
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom
wachiravit-thitagran:fix/event-stream-heartbeat
Draft

fix: keep add event stream alive#213
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom
wachiravit-thitagran:fix/event-stream-heartbeat

Conversation

@wachiravit-thitagran

@wachiravit-thitagranwachiravit-thitagran commented Jul 31, 2026

Copy link
Copy Markdown

Warning

This pull request and its code were generated by AI (OpenAI Codex). Please review the implementation and tests carefully before merging.

Problem

POST /api/v1/add can spend longer than Cloudflare's default 125-second Proxy Read Timeout processing an uploaded document without emitting another Server-Sent Events (SSE) body chunk. Cloudflare defines this timeout between successive reads from the origin, so an otherwise healthy long-running ingestion can be terminated if the stream stays silent for too long.

An interrupted stream may appear in Chromium-based clients after the response headers have already returned HTTP 200 as:

net::ERR_HTTP2_PROTOCOL_ERROR 200 (OK)

This is an observed symptom, not proof by itself that Cloudflare is the root cause. Confirming attribution requires comparing the proxied request with a direct-origin request and checking Cloudflare logs, a CF-Ray/HAR, or a browser NetLog.

Why this approach

The HTML Standard recommends sending an SSE comment line (a line beginning with :) roughly every 15 seconds to protect event streams from proxy timeouts.

This fix therefore emits:

: ping

every 15 seconds while a file is being processed. If Cloudflare forwards each SSE chunk normally, the interval stays well below its 125-second read timeout and is expected to prevent this specific timeout.

This condition matters:

  • A normal proxied custom hostname or named Cloudflare Tunnel supports SSE.
  • A Quick Tunnel on *.trycloudflare.com does not support SSE and may buffer text/event-stream; heartbeat comments cannot solve that configuration. Use a named tunnel or another SSE-capable route instead.
  • This change does not address origin crashes, malformed HTTP/2 frames, compression/content-length errors, or request-upload stalls before the streaming response starts.

What changed

  • Run each streamed add operation as an asynchronous task.
  • Emit the valid SSE comment : ping\n\n every 15 seconds while processing remains in progress.
  • Preserve the existing uploaded, file_start, file_done, final, error, and done event contract.
  • Cancel an unfinished task when the client closes the stream.
  • Add a regression test that simulates slow processing, verifies the configured interval remains within 15-30 seconds, observes the keepalive comment, and confirms the final events are still delivered.

SSE clients ignore comment-only frames, so the frontend event payload contract is unchanged. Non-streaming requests are also unchanged.

Validation

Application-level validation:

  • pytest -q tests/test_api.py tests/test_file_size.py — 158 passed
  • mypy openkb/api_helpers.py openkb/api.py — passed
  • ruff check . — passed
  • ruff format --check . — passed
  • git diff --check — passed

The automated test confirms that Starlette emits the keepalive chunks and subsequently completes the SSE stream. It is not an end-to-end Cloudflare integration test.

References

@wachiravit-thitagranwachiravit-thitagran mentioned this pull request Jul 31, 2026
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@wachiravit-thitagran
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
fix: keep add event stream alive by wachiravit-thitagran · Pull Request #213 · VectifyAI/OpenKB · GitHub
Skip to content
fix: keep add event stream alive - #213
Draft
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Draft
fix: keep add event stream alive#213
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Conversation
@wachiravit-thitagran
@wachiravit-thitagranwachiravit-thitagran
commented
Jul 31, 2026

Copy link


Copy Markdown

Warning
This pull request and its code were generated by AI (OpenAI Codex). Please review the implementation and tests carefully before merging.
Problem
POST /api/v1/add can spend longer than Cloudflare's default 125-second Proxy Read Timeout processing an uploaded document without emitting another Server-Sent Events (SSE) body chunk. Cloudflare defines this timeout between successive reads from the origin, so an otherwise healthy long-running ingestion can be terminated if the stream stays silent for too long.
An interrupted stream may appear in Chromium-based clients after the response headers have already returned HTTP 200 as:
net::ERR_HTTP2_PROTOCOL_ERROR 200 (OK)

This is an observed symptom, not proof by itself that Cloudflare is the root cause. Confirming attribution requires comparing the proxied request with a direct-origin request and checking Cloudflare logs, a CF-Ray/HAR, or a browser NetLog.
Why this approach
The HTML Standard recommends sending an SSE comment line (a line beginning with :) roughly every 15 seconds to protect event streams from proxy timeouts.
This fix therefore emits:
: ping

every 15 seconds while a file is being processed. If Cloudflare forwards each SSE chunk normally, the interval stays well below its 125-second read timeout and is expected to prevent this specific timeout.
This condition matters:
  • A normal proxied custom hostname or named Cloudflare Tunnel supports SSE.
  • A Quick Tunnel on *.trycloudflare.com does not support SSE and may buffer text/event-stream; heartbeat comments cannot solve that configuration. Use a named tunnel or another SSE-capable route instead.
  • This change does not address origin crashes, malformed HTTP/2 frames, compression/content-length errors, or request-upload stalls before the streaming response starts.
What changed
  • Run each streamed add operation as an asynchronous task.
  • Emit the valid SSE comment : ping\n\n every 15 seconds while processing remains in progress.
  • Preserve the existing uploaded, file_start, file_done, final, error, and done event contract.
  • Cancel an unfinished task when the client closes the stream.
  • Add a regression test that simulates slow processing, verifies the configured interval remains within 15-30 seconds, observes the keepalive comment, and confirms the final events are still delivered.
SSE clients ignore comment-only frames, so the frontend event payload contract is unchanged. Non-streaming requests are also unchanged.
Validation
Application-level validation:
  • pytest -q tests/test_api.py tests/test_file_size.py — 158 passed
  • mypy openkb/api_helpers.py openkb/api.py — passed
  • ruff check . — passed
  • ruff format --check . — passed
  • git diff --check — passed
The automated test confirms that Starlette emits the keepalive chunks and subsequently completes the SSE stream. It is not an end-to-end Cloudflare integration test.
References
@wachiravit-thitagranwachiravit-thitagran
mentioned this pull request
Jul 31, 2026
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@wachiravit-thitagran

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
fix: keep add event stream alive by wachiravit-thitagran · Pull Request #213 · VectifyAI/OpenKB · GitHub
Skip to content
fix: keep add event stream alive - #213
Draft
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Draft
fix: keep add event stream alive#213
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Conversation
@wachiravit-thitagran
@wachiravit-thitagranwachiravit-thitagran
commented
Jul 31, 2026

Copy link


Copy Markdown

Warning
This pull request and its code were generated by AI (OpenAI Codex). Please review the implementation and tests carefully before merging.
Problem
POST /api/v1/add can spend longer than Cloudflare's default 125-second Proxy Read Timeout processing an uploaded document without emitting another Server-Sent Events (SSE) body chunk. Cloudflare defines this timeout between successive reads from the origin, so an otherwise healthy long-running ingestion can be terminated if the stream stays silent for too long.
An interrupted stream may appear in Chromium-based clients after the response headers have already returned HTTP 200 as:
net::ERR_HTTP2_PROTOCOL_ERROR 200 (OK)

This is an observed symptom, not proof by itself that Cloudflare is the root cause. Confirming attribution requires comparing the proxied request with a direct-origin request and checking Cloudflare logs, a CF-Ray/HAR, or a browser NetLog.
Why this approach
The HTML Standard recommends sending an SSE comment line (a line beginning with :) roughly every 15 seconds to protect event streams from proxy timeouts.
This fix therefore emits:
: ping

every 15 seconds while a file is being processed. If Cloudflare forwards each SSE chunk normally, the interval stays well below its 125-second read timeout and is expected to prevent this specific timeout.
This condition matters:
  • A normal proxied custom hostname or named Cloudflare Tunnel supports SSE.
  • A Quick Tunnel on *.trycloudflare.com does not support SSE and may buffer text/event-stream; heartbeat comments cannot solve that configuration. Use a named tunnel or another SSE-capable route instead.
  • This change does not address origin crashes, malformed HTTP/2 frames, compression/content-length errors, or request-upload stalls before the streaming response starts.
What changed
  • Run each streamed add operation as an asynchronous task.
  • Emit the valid SSE comment : ping\n\n every 15 seconds while processing remains in progress.
  • Preserve the existing uploaded, file_start, file_done, final, error, and done event contract.
  • Cancel an unfinished task when the client closes the stream.
  • Add a regression test that simulates slow processing, verifies the configured interval remains within 15-30 seconds, observes the keepalive comment, and confirms the final events are still delivered.
SSE clients ignore comment-only frames, so the frontend event payload contract is unchanged. Non-streaming requests are also unchanged.
Validation
Application-level validation:
  • pytest -q tests/test_api.py tests/test_file_size.py — 158 passed
  • mypy openkb/api_helpers.py openkb/api.py — passed
  • ruff check . — passed
  • ruff format --check . — passed
  • git diff --check — passed
The automated test confirms that Starlette emits the keepalive chunks and subsequently completes the SSE stream. It is not an end-to-end Cloudflare integration test.
References
@wachiravit-thitagranwachiravit-thitagran
mentioned this pull request
Jul 31, 2026
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@wachiravit-thitagran

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
fix: keep add event stream alive by wachiravit-thitagran · Pull Request #213 · VectifyAI/OpenKB · GitHub
Skip to content
fix: keep add event stream alive - #213
Draft
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Draft
fix: keep add event stream alive#213
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Conversation
@wachiravit-thitagran
@wachiravit-thitagranwachiravit-thitagran
commented
Jul 31, 2026

Copy link


Copy Markdown

Warning
This pull request and its code were generated by AI (OpenAI Codex). Please review the implementation and tests carefully before merging.
Problem
POST /api/v1/add can spend longer than Cloudflare's default 125-second Proxy Read Timeout processing an uploaded document without emitting another Server-Sent Events (SSE) body chunk. Cloudflare defines this timeout between successive reads from the origin, so an otherwise healthy long-running ingestion can be terminated if the stream stays silent for too long.
An interrupted stream may appear in Chromium-based clients after the response headers have already returned HTTP 200 as:
net::ERR_HTTP2_PROTOCOL_ERROR 200 (OK)

This is an observed symptom, not proof by itself that Cloudflare is the root cause. Confirming attribution requires comparing the proxied request with a direct-origin request and checking Cloudflare logs, a CF-Ray/HAR, or a browser NetLog.
Why this approach
The HTML Standard recommends sending an SSE comment line (a line beginning with :) roughly every 15 seconds to protect event streams from proxy timeouts.
This fix therefore emits:
: ping

every 15 seconds while a file is being processed. If Cloudflare forwards each SSE chunk normally, the interval stays well below its 125-second read timeout and is expected to prevent this specific timeout.
This condition matters:
  • A normal proxied custom hostname or named Cloudflare Tunnel supports SSE.
  • A Quick Tunnel on *.trycloudflare.com does not support SSE and may buffer text/event-stream; heartbeat comments cannot solve that configuration. Use a named tunnel or another SSE-capable route instead.
  • This change does not address origin crashes, malformed HTTP/2 frames, compression/content-length errors, or request-upload stalls before the streaming response starts.
What changed
  • Run each streamed add operation as an asynchronous task.
  • Emit the valid SSE comment : ping\n\n every 15 seconds while processing remains in progress.
  • Preserve the existing uploaded, file_start, file_done, final, error, and done event contract.
  • Cancel an unfinished task when the client closes the stream.
  • Add a regression test that simulates slow processing, verifies the configured interval remains within 15-30 seconds, observes the keepalive comment, and confirms the final events are still delivered.
SSE clients ignore comment-only frames, so the frontend event payload contract is unchanged. Non-streaming requests are also unchanged.
Validation
Application-level validation:
  • pytest -q tests/test_api.py tests/test_file_size.py — 158 passed
  • mypy openkb/api_helpers.py openkb/api.py — passed
  • ruff check . — passed
  • ruff format --check . — passed
  • git diff --check — passed
The automated test confirms that Starlette emits the keepalive chunks and subsequently completes the SSE stream. It is not an end-to-end Cloudflare integration test.
References
@wachiravit-thitagranwachiravit-thitagran
mentioned this pull request
Jul 31, 2026
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@wachiravit-thitagran

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
fix: keep add event stream alive by wachiravit-thitagran · Pull Request #213 · VectifyAI/OpenKB · GitHub
Skip to content
fix: keep add event stream alive - #213
Draft
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Draft
fix: keep add event stream alive#213
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Conversation
@wachiravit-thitagran
@wachiravit-thitagranwachiravit-thitagran
commented
Jul 31, 2026

Copy link


Copy Markdown

Warning
This pull request and its code were generated by AI (OpenAI Codex). Please review the implementation and tests carefully before merging.
Problem
POST /api/v1/add can spend longer than Cloudflare's default 125-second Proxy Read Timeout processing an uploaded document without emitting another Server-Sent Events (SSE) body chunk. Cloudflare defines this timeout between successive reads from the origin, so an otherwise healthy long-running ingestion can be terminated if the stream stays silent for too long.
An interrupted stream may appear in Chromium-based clients after the response headers have already returned HTTP 200 as:
net::ERR_HTTP2_PROTOCOL_ERROR 200 (OK)

This is an observed symptom, not proof by itself that Cloudflare is the root cause. Confirming attribution requires comparing the proxied request with a direct-origin request and checking Cloudflare logs, a CF-Ray/HAR, or a browser NetLog.
Why this approach
The HTML Standard recommends sending an SSE comment line (a line beginning with :) roughly every 15 seconds to protect event streams from proxy timeouts.
This fix therefore emits:
: ping

every 15 seconds while a file is being processed. If Cloudflare forwards each SSE chunk normally, the interval stays well below its 125-second read timeout and is expected to prevent this specific timeout.
This condition matters:
  • A normal proxied custom hostname or named Cloudflare Tunnel supports SSE.
  • A Quick Tunnel on *.trycloudflare.com does not support SSE and may buffer text/event-stream; heartbeat comments cannot solve that configuration. Use a named tunnel or another SSE-capable route instead.
  • This change does not address origin crashes, malformed HTTP/2 frames, compression/content-length errors, or request-upload stalls before the streaming response starts.
What changed
  • Run each streamed add operation as an asynchronous task.
  • Emit the valid SSE comment : ping\n\n every 15 seconds while processing remains in progress.
  • Preserve the existing uploaded, file_start, file_done, final, error, and done event contract.
  • Cancel an unfinished task when the client closes the stream.
  • Add a regression test that simulates slow processing, verifies the configured interval remains within 15-30 seconds, observes the keepalive comment, and confirms the final events are still delivered.
SSE clients ignore comment-only frames, so the frontend event payload contract is unchanged. Non-streaming requests are also unchanged.
Validation
Application-level validation:
  • pytest -q tests/test_api.py tests/test_file_size.py — 158 passed
  • mypy openkb/api_helpers.py openkb/api.py — passed
  • ruff check . — passed
  • ruff format --check . — passed
  • git diff --check — passed
The automated test confirms that Starlette emits the keepalive chunks and subsequently completes the SSE stream. It is not an end-to-end Cloudflare integration test.
References
@wachiravit-thitagranwachiravit-thitagran
mentioned this pull request
Jul 31, 2026
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@wachiravit-thitagran

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
fix: keep add event stream alive by wachiravit-thitagran · Pull Request #213 · VectifyAI/OpenKB · GitHub
Skip to content
fix: keep add event stream alive - #213
Draft
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Draft
fix: keep add event stream alive#213
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Conversation
@wachiravit-thitagran
@wachiravit-thitagranwachiravit-thitagran
commented
Jul 31, 2026

Copy link


Copy Markdown

Warning
This pull request and its code were generated by AI (OpenAI Codex). Please review the implementation and tests carefully before merging.
Problem
POST /api/v1/add can spend longer than Cloudflare's default 125-second Proxy Read Timeout processing an uploaded document without emitting another Server-Sent Events (SSE) body chunk. Cloudflare defines this timeout between successive reads from the origin, so an otherwise healthy long-running ingestion can be terminated if the stream stays silent for too long.
An interrupted stream may appear in Chromium-based clients after the response headers have already returned HTTP 200 as:
net::ERR_HTTP2_PROTOCOL_ERROR 200 (OK)

This is an observed symptom, not proof by itself that Cloudflare is the root cause. Confirming attribution requires comparing the proxied request with a direct-origin request and checking Cloudflare logs, a CF-Ray/HAR, or a browser NetLog.
Why this approach
The HTML Standard recommends sending an SSE comment line (a line beginning with :) roughly every 15 seconds to protect event streams from proxy timeouts.
This fix therefore emits:
: ping

every 15 seconds while a file is being processed. If Cloudflare forwards each SSE chunk normally, the interval stays well below its 125-second read timeout and is expected to prevent this specific timeout.
This condition matters:
  • A normal proxied custom hostname or named Cloudflare Tunnel supports SSE.
  • A Quick Tunnel on *.trycloudflare.com does not support SSE and may buffer text/event-stream; heartbeat comments cannot solve that configuration. Use a named tunnel or another SSE-capable route instead.
  • This change does not address origin crashes, malformed HTTP/2 frames, compression/content-length errors, or request-upload stalls before the streaming response starts.
What changed
  • Run each streamed add operation as an asynchronous task.
  • Emit the valid SSE comment : ping\n\n every 15 seconds while processing remains in progress.
  • Preserve the existing uploaded, file_start, file_done, final, error, and done event contract.
  • Cancel an unfinished task when the client closes the stream.
  • Add a regression test that simulates slow processing, verifies the configured interval remains within 15-30 seconds, observes the keepalive comment, and confirms the final events are still delivered.
SSE clients ignore comment-only frames, so the frontend event payload contract is unchanged. Non-streaming requests are also unchanged.
Validation
Application-level validation:
  • pytest -q tests/test_api.py tests/test_file_size.py — 158 passed
  • mypy openkb/api_helpers.py openkb/api.py — passed
  • ruff check . — passed
  • ruff format --check . — passed
  • git diff --check — passed
The automated test confirms that Starlette emits the keepalive chunks and subsequently completes the SSE stream. It is not an end-to-end Cloudflare integration test.
References
@wachiravit-thitagranwachiravit-thitagran
mentioned this pull request
Jul 31, 2026
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@wachiravit-thitagran

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
fix: keep add event stream alive by wachiravit-thitagran · Pull Request #213 · VectifyAI/OpenKB · GitHub
Skip to content
fix: keep add event stream alive - #213
Draft
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Draft
fix: keep add event stream alive#213
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Conversation
@wachiravit-thitagran
@wachiravit-thitagranwachiravit-thitagran
commented
Jul 31, 2026

Copy link


Copy Markdown

Warning
This pull request and its code were generated by AI (OpenAI Codex). Please review the implementation and tests carefully before merging.
Problem
POST /api/v1/add can spend longer than Cloudflare's default 125-second Proxy Read Timeout processing an uploaded document without emitting another Server-Sent Events (SSE) body chunk. Cloudflare defines this timeout between successive reads from the origin, so an otherwise healthy long-running ingestion can be terminated if the stream stays silent for too long.
An interrupted stream may appear in Chromium-based clients after the response headers have already returned HTTP 200 as:
net::ERR_HTTP2_PROTOCOL_ERROR 200 (OK)

This is an observed symptom, not proof by itself that Cloudflare is the root cause. Confirming attribution requires comparing the proxied request with a direct-origin request and checking Cloudflare logs, a CF-Ray/HAR, or a browser NetLog.
Why this approach
The HTML Standard recommends sending an SSE comment line (a line beginning with :) roughly every 15 seconds to protect event streams from proxy timeouts.
This fix therefore emits:
: ping

every 15 seconds while a file is being processed. If Cloudflare forwards each SSE chunk normally, the interval stays well below its 125-second read timeout and is expected to prevent this specific timeout.
This condition matters:
  • A normal proxied custom hostname or named Cloudflare Tunnel supports SSE.
  • A Quick Tunnel on *.trycloudflare.com does not support SSE and may buffer text/event-stream; heartbeat comments cannot solve that configuration. Use a named tunnel or another SSE-capable route instead.
  • This change does not address origin crashes, malformed HTTP/2 frames, compression/content-length errors, or request-upload stalls before the streaming response starts.
What changed
  • Run each streamed add operation as an asynchronous task.
  • Emit the valid SSE comment : ping\n\n every 15 seconds while processing remains in progress.
  • Preserve the existing uploaded, file_start, file_done, final, error, and done event contract.
  • Cancel an unfinished task when the client closes the stream.
  • Add a regression test that simulates slow processing, verifies the configured interval remains within 15-30 seconds, observes the keepalive comment, and confirms the final events are still delivered.
SSE clients ignore comment-only frames, so the frontend event payload contract is unchanged. Non-streaming requests are also unchanged.
Validation
Application-level validation:
  • pytest -q tests/test_api.py tests/test_file_size.py — 158 passed
  • mypy openkb/api_helpers.py openkb/api.py — passed
  • ruff check . — passed
  • ruff format --check . — passed
  • git diff --check — passed
The automated test confirms that Starlette emits the keepalive chunks and subsequently completes the SSE stream. It is not an end-to-end Cloudflare integration test.
References
@wachiravit-thitagranwachiravit-thitagran
mentioned this pull request
Jul 31, 2026
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@wachiravit-thitagran

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
fix: keep add event stream alive by wachiravit-thitagran · Pull Request #213 · VectifyAI/OpenKB · GitHub
Skip to content
fix: keep add event stream alive - #213
Draft
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Draft
fix: keep add event stream alive#213
wachiravit-thitagran wants to merge 1 commit into
VectifyAI:mainfrom 
wachiravit-thitagran:fix/event-stream-heartbeat
Conversation
@wachiravit-thitagran
@wachiravit-thitagranwachiravit-thitagran
commented
Jul 31, 2026

Copy link


Copy Markdown

Warning
This pull request and its code were generated by AI (OpenAI Codex). Please review the implementation and tests carefully before merging.
Problem
POST /api/v1/add can spend longer than Cloudflare's default 125-second Proxy Read Timeout processing an uploaded document without emitting another Server-Sent Events (SSE) body chunk. Cloudflare defines this timeout between successive reads from the origin, so an otherwise healthy long-running ingestion can be terminated if the stream stays silent for too long.
An interrupted stream may appear in Chromium-based clients after the response headers have already returned HTTP 200 as:
net::ERR_HTTP2_PROTOCOL_ERROR 200 (OK)

This is an observed symptom, not proof by itself that Cloudflare is the root cause. Confirming attribution requires comparing the proxied request with a direct-origin request and checking Cloudflare logs, a CF-Ray/HAR, or a browser NetLog.
Why this approach
The HTML Standard recommends sending an SSE comment line (a line beginning with :) roughly every 15 seconds to protect event streams from proxy timeouts.
This fix therefore emits:
: ping

every 15 seconds while a file is being processed. If Cloudflare forwards each SSE chunk normally, the interval stays well below its 125-second read timeout and is expected to prevent this specific timeout.
This condition matters:
  • A normal proxied custom hostname or named Cloudflare Tunnel supports SSE.
  • A Quick Tunnel on *.trycloudflare.com does not support SSE and may buffer text/event-stream; heartbeat comments cannot solve that configuration. Use a named tunnel or another SSE-capable route instead.
  • This change does not address origin crashes, malformed HTTP/2 frames, compression/content-length errors, or request-upload stalls before the streaming response starts.
What changed
  • Run each streamed add operation as an asynchronous task.
  • Emit the valid SSE comment : ping\n\n every 15 seconds while processing remains in progress.
  • Preserve the existing uploaded, file_start, file_done, final, error, and done event contract.
  • Cancel an unfinished task when the client closes the stream.
  • Add a regression test that simulates slow processing, verifies the configured interval remains within 15-30 seconds, observes the keepalive comment, and confirms the final events are still delivered.
SSE clients ignore comment-only frames, so the frontend event payload contract is unchanged. Non-streaming requests are also unchanged.
Validation
Application-level validation:
  • pytest -q tests/test_api.py tests/test_file_size.py — 158 passed
  • mypy openkb/api_helpers.py openkb/api.py — passed
  • ruff check . — passed
  • ruff format --check . — passed
  • git diff --check — passed
The automated test confirms that Starlette emits the keepalive chunks and subsequently completes the SSE stream. It is not an end-to-end Cloudflare integration test.
References
@wachiravit-thitagranwachiravit-thitagran
mentioned this pull request
Jul 31, 2026
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@wachiravit-thitagran