Skip to content

Image uploads corrupted on read due to simple-array comma-splitting base64 data URLs #1075

Description

@plansombl

Description

Every post that contains at least one image is silently corrupted the moment it is read back from the database. The root cause is that TypeORM's simple-array column type stores arrays by naively joining values with a comma (value.join(",")) and reads them back by splitting on comma (value.split(",")) — with no escaping whatsoever.

Base64 data URLs produced by FileReader.readAsDataURL() always contain a literal comma in their MIME prefix (e.g. data:image/png;base64,iVBORw0KGgo...). This comma is indistinguishable from simple-array's delimiter, so the stored string is always parsed incorrectly on read.

Single-image post

Stored fine (nothing to join), but .split(",") still fires unconditionally on read, breaking data:image/png;base64,AAAA... into two entries:

  • data:image/png;base64 — a truncated, invalid data URL with no payload
  • AAAA... — a bare base64 string with no data: prefix, which the browser treats as a relative URL, not an image

Multi-image post

The join-separator comma and each image's own internal comma become completely indistinguishable. Images interleave and split at wrong boundaries, corrupting the entire set.

This is very likely the root cause of the "flying stars / meteor shower" rendering artefacts previously reported — corrupted/partial image decodes rather than someone else's real photos.

Steps to reproduce

  1. Create a post and attach one or more images via the upload flow (post/+page.svelte)
  2. Save the post
  3. Navigate to the post grid / open the post detail
  4. Observe that images render as broken fragments, garbled pieces, or show browser image-load errors

Expected behavior

Uploaded images should be stored and retrieved without corruption. The full, valid base64 data URL should be recovered exactly as it was before storage, and images should render correctly in the post grid and post modal.

Current workaround

A frontend bandage — pairAndJoinChunks — exists in both Post.svelte (lines 32–55) and PostModal.svelte (line 45). It takes the corrupted flat array and re-pairs elements two-at-a-time, rejoining [prefix, payload] back into prefix,payload. This partially recovers single-image posts but remains brittle for multi-image posts where additional internal commas cause further mis-splits.

Proposed fix

Switch the column encoding from simple-array to simple-json (uses JSON.stringify / JSON.parse), which correctly escapes all internal characters including commas.

Both simple-array and simple-json compile down to a plain Postgres text column (confirmed by the original migration SQL — "images" text), so no schema migration is required.

Impact on existing data

Existing rows were stored with the old comma-joined format. Reading them with a JSON.parse decoder will throw a parse error rather than silently returning garbage. Old corrupted rows should be audited and either cleaned up or null-ed out before the fix is deployed, as the original image data cannot be reliably reconstructed from the corrupted fragments.

Steps

  1. Change the images column decorator from simple-array to simple-json in the relevant TypeORM entity
  2. Remove the pairAndJoinChunks workaround from Post.svelte and PostModal.svelte
  3. Handle/migrate (or purge) existing corrupted rows to prevent JSON parse errors at runtime
  4. Verify with a new post containing multiple images that all images are stored and displayed correctly

Supporting Media

N/A

Desktop

  • Device: N/A (server-side / DB layer bug)
  • OS: N/A
  • Browser: N/A
  • Version: N/A

Additional Context

  • Corruption affects every single post with at least one image — no posts are safe under the current encoding
  • If real users are onboarded before this is fixed, their uploaded images will be permanently unrecoverable
  • The fix is low-risk (no SQL schema change needed) but requires careful handling of already-corrupted rows in the DB

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpriority - 1Really important bug. App is broken, major functionality is unusable

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Image uploads corrupted on read due to `simple-array` comma-splitting base64 data URLs · Issue #1075 · MetaState-Prototype-Project/prototype · GitHub
Skip to content

Image uploads corrupted on read due to simple-array comma-splitting base64 data URLs #1075

Description

@plansombl

Description

Every post that contains at least one image is silently corrupted the moment it is read back from the database. The root cause is that TypeORM's simple-array column type stores arrays by naively joining values with a comma (value.join(",")) and reads them back by splitting on comma (value.split(",")) — with no escaping whatsoever.

Base64 data URLs produced by FileReader.readAsDataURL() always contain a literal comma in their MIME prefix (e.g. data:image/png;base64,iVBORw0KGgo...). This comma is indistinguishable from simple-array's delimiter, so the stored string is always parsed incorrectly on read.

Single-image post

Stored fine (nothing to join), but .split(",") still fires unconditionally on read, breaking data:image/png;base64,AAAA... into two entries:

  • data:image/png;base64 — a truncated, invalid data URL with no payload
  • AAAA... — a bare base64 string with no data: prefix, which the browser treats as a relative URL, not an image

Multi-image post

The join-separator comma and each image's own internal comma become completely indistinguishable. Images interleave and split at wrong boundaries, corrupting the entire set.

This is very likely the root cause of the "flying stars / meteor shower" rendering artefacts previously reported — corrupted/partial image decodes rather than someone else's real photos.

Steps to reproduce

  1. Create a post and attach one or more images via the upload flow (post/+page.svelte)
  2. Save the post
  3. Navigate to the post grid / open the post detail
  4. Observe that images render as broken fragments, garbled pieces, or show browser image-load errors

Expected behavior

Uploaded images should be stored and retrieved without corruption. The full, valid base64 data URL should be recovered exactly as it was before storage, and images should render correctly in the post grid and post modal.

Current workaround

A frontend bandage — pairAndJoinChunks — exists in both Post.svelte (lines 32–55) and PostModal.svelte (line 45). It takes the corrupted flat array and re-pairs elements two-at-a-time, rejoining [prefix, payload] back into prefix,payload. This partially recovers single-image posts but remains brittle for multi-image posts where additional internal commas cause further mis-splits.

Proposed fix

Switch the column encoding from simple-array to simple-json (uses JSON.stringify / JSON.parse), which correctly escapes all internal characters including commas.

Both simple-array and simple-json compile down to a plain Postgres text column (confirmed by the original migration SQL — "images" text), so no schema migration is required.

Impact on existing data

Existing rows were stored with the old comma-joined format. Reading them with a JSON.parse decoder will throw a parse error rather than silently returning garbage. Old corrupted rows should be audited and either cleaned up or null-ed out before the fix is deployed, as the original image data cannot be reliably reconstructed from the corrupted fragments.

Steps

  1. Change the images column decorator from simple-array to simple-json in the relevant TypeORM entity
  2. Remove the pairAndJoinChunks workaround from Post.svelte and PostModal.svelte
  3. Handle/migrate (or purge) existing corrupted rows to prevent JSON parse errors at runtime
  4. Verify with a new post containing multiple images that all images are stored and displayed correctly

Supporting Media

N/A

Desktop

  • Device: N/A (server-side / DB layer bug)
  • OS: N/A
  • Browser: N/A
  • Version: N/A

Additional Context

  • Corruption affects every single post with at least one image — no posts are safe under the current encoding
  • If real users are onboarded before this is fixed, their uploaded images will be permanently unrecoverable
  • The fix is low-risk (no SQL schema change needed) but requires careful handling of already-corrupted rows in the DB

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpriority - 1Really important bug. App is broken, major functionality is unusable

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Image uploads corrupted on read due to `simple-array` comma-splitting base64 data URLs · Issue #1075 · MetaState-Prototype-Project/prototype · GitHub
Skip to content

Image uploads corrupted on read due to simple-array comma-splitting base64 data URLs #1075

Description

@plansombl

Description

Every post that contains at least one image is silently corrupted the moment it is read back from the database. The root cause is that TypeORM's simple-array column type stores arrays by naively joining values with a comma (value.join(",")) and reads them back by splitting on comma (value.split(",")) — with no escaping whatsoever.

Base64 data URLs produced by FileReader.readAsDataURL() always contain a literal comma in their MIME prefix (e.g. data:image/png;base64,iVBORw0KGgo...). This comma is indistinguishable from simple-array's delimiter, so the stored string is always parsed incorrectly on read.

Single-image post

Stored fine (nothing to join), but .split(",") still fires unconditionally on read, breaking data:image/png;base64,AAAA... into two entries:

  • data:image/png;base64 — a truncated, invalid data URL with no payload
  • AAAA... — a bare base64 string with no data: prefix, which the browser treats as a relative URL, not an image

Multi-image post

The join-separator comma and each image's own internal comma become completely indistinguishable. Images interleave and split at wrong boundaries, corrupting the entire set.

This is very likely the root cause of the "flying stars / meteor shower" rendering artefacts previously reported — corrupted/partial image decodes rather than someone else's real photos.

Steps to reproduce

  1. Create a post and attach one or more images via the upload flow (post/+page.svelte)
  2. Save the post
  3. Navigate to the post grid / open the post detail
  4. Observe that images render as broken fragments, garbled pieces, or show browser image-load errors

Expected behavior

Uploaded images should be stored and retrieved without corruption. The full, valid base64 data URL should be recovered exactly as it was before storage, and images should render correctly in the post grid and post modal.

Current workaround

A frontend bandage — pairAndJoinChunks — exists in both Post.svelte (lines 32–55) and PostModal.svelte (line 45). It takes the corrupted flat array and re-pairs elements two-at-a-time, rejoining [prefix, payload] back into prefix,payload. This partially recovers single-image posts but remains brittle for multi-image posts where additional internal commas cause further mis-splits.

Proposed fix

Switch the column encoding from simple-array to simple-json (uses JSON.stringify / JSON.parse), which correctly escapes all internal characters including commas.

Both simple-array and simple-json compile down to a plain Postgres text column (confirmed by the original migration SQL — "images" text), so no schema migration is required.

Impact on existing data

Existing rows were stored with the old comma-joined format. Reading them with a JSON.parse decoder will throw a parse error rather than silently returning garbage. Old corrupted rows should be audited and either cleaned up or null-ed out before the fix is deployed, as the original image data cannot be reliably reconstructed from the corrupted fragments.

Steps

  1. Change the images column decorator from simple-array to simple-json in the relevant TypeORM entity
  2. Remove the pairAndJoinChunks workaround from Post.svelte and PostModal.svelte
  3. Handle/migrate (or purge) existing corrupted rows to prevent JSON parse errors at runtime
  4. Verify with a new post containing multiple images that all images are stored and displayed correctly

Supporting Media

N/A

Desktop

  • Device: N/A (server-side / DB layer bug)
  • OS: N/A
  • Browser: N/A
  • Version: N/A

Additional Context

  • Corruption affects every single post with at least one image — no posts are safe under the current encoding
  • If real users are onboarded before this is fixed, their uploaded images will be permanently unrecoverable
  • The fix is low-risk (no SQL schema change needed) but requires careful handling of already-corrupted rows in the DB

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpriority - 1Really important bug. App is broken, major functionality is unusable

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Image uploads corrupted on read due to `simple-array` comma-splitting base64 data URLs · Issue #1075 · MetaState-Prototype-Project/prototype · GitHub
Skip to content

Image uploads corrupted on read due to simple-array comma-splitting base64 data URLs #1075

Description

@plansombl

Description

Every post that contains at least one image is silently corrupted the moment it is read back from the database. The root cause is that TypeORM's simple-array column type stores arrays by naively joining values with a comma (value.join(",")) and reads them back by splitting on comma (value.split(",")) — with no escaping whatsoever.

Base64 data URLs produced by FileReader.readAsDataURL() always contain a literal comma in their MIME prefix (e.g. data:image/png;base64,iVBORw0KGgo...). This comma is indistinguishable from simple-array's delimiter, so the stored string is always parsed incorrectly on read.

Single-image post

Stored fine (nothing to join), but .split(",") still fires unconditionally on read, breaking data:image/png;base64,AAAA... into two entries:

  • data:image/png;base64 — a truncated, invalid data URL with no payload
  • AAAA... — a bare base64 string with no data: prefix, which the browser treats as a relative URL, not an image

Multi-image post

The join-separator comma and each image's own internal comma become completely indistinguishable. Images interleave and split at wrong boundaries, corrupting the entire set.

This is very likely the root cause of the "flying stars / meteor shower" rendering artefacts previously reported — corrupted/partial image decodes rather than someone else's real photos.

Steps to reproduce

  1. Create a post and attach one or more images via the upload flow (post/+page.svelte)
  2. Save the post
  3. Navigate to the post grid / open the post detail
  4. Observe that images render as broken fragments, garbled pieces, or show browser image-load errors

Expected behavior

Uploaded images should be stored and retrieved without corruption. The full, valid base64 data URL should be recovered exactly as it was before storage, and images should render correctly in the post grid and post modal.

Current workaround

A frontend bandage — pairAndJoinChunks — exists in both Post.svelte (lines 32–55) and PostModal.svelte (line 45). It takes the corrupted flat array and re-pairs elements two-at-a-time, rejoining [prefix, payload] back into prefix,payload. This partially recovers single-image posts but remains brittle for multi-image posts where additional internal commas cause further mis-splits.

Proposed fix

Switch the column encoding from simple-array to simple-json (uses JSON.stringify / JSON.parse), which correctly escapes all internal characters including commas.

Both simple-array and simple-json compile down to a plain Postgres text column (confirmed by the original migration SQL — "images" text), so no schema migration is required.

Impact on existing data

Existing rows were stored with the old comma-joined format. Reading them with a JSON.parse decoder will throw a parse error rather than silently returning garbage. Old corrupted rows should be audited and either cleaned up or null-ed out before the fix is deployed, as the original image data cannot be reliably reconstructed from the corrupted fragments.

Steps

  1. Change the images column decorator from simple-array to simple-json in the relevant TypeORM entity
  2. Remove the pairAndJoinChunks workaround from Post.svelte and PostModal.svelte
  3. Handle/migrate (or purge) existing corrupted rows to prevent JSON parse errors at runtime
  4. Verify with a new post containing multiple images that all images are stored and displayed correctly

Supporting Media

N/A

Desktop

  • Device: N/A (server-side / DB layer bug)
  • OS: N/A
  • Browser: N/A
  • Version: N/A

Additional Context

  • Corruption affects every single post with at least one image — no posts are safe under the current encoding
  • If real users are onboarded before this is fixed, their uploaded images will be permanently unrecoverable
  • The fix is low-risk (no SQL schema change needed) but requires careful handling of already-corrupted rows in the DB

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpriority - 1Really important bug. App is broken, major functionality is unusable

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Image uploads corrupted on read due to `simple-array` comma-splitting base64 data URLs · Issue #1075 · MetaState-Prototype-Project/prototype · GitHub
Skip to content

Image uploads corrupted on read due to simple-array comma-splitting base64 data URLs #1075

Description

@plansombl

Description

Every post that contains at least one image is silently corrupted the moment it is read back from the database. The root cause is that TypeORM's simple-array column type stores arrays by naively joining values with a comma (value.join(",")) and reads them back by splitting on comma (value.split(",")) — with no escaping whatsoever.

Base64 data URLs produced by FileReader.readAsDataURL() always contain a literal comma in their MIME prefix (e.g. data:image/png;base64,iVBORw0KGgo...). This comma is indistinguishable from simple-array's delimiter, so the stored string is always parsed incorrectly on read.

Single-image post

Stored fine (nothing to join), but .split(",") still fires unconditionally on read, breaking data:image/png;base64,AAAA... into two entries:

  • data:image/png;base64 — a truncated, invalid data URL with no payload
  • AAAA... — a bare base64 string with no data: prefix, which the browser treats as a relative URL, not an image

Multi-image post

The join-separator comma and each image's own internal comma become completely indistinguishable. Images interleave and split at wrong boundaries, corrupting the entire set.

This is very likely the root cause of the "flying stars / meteor shower" rendering artefacts previously reported — corrupted/partial image decodes rather than someone else's real photos.

Steps to reproduce

  1. Create a post and attach one or more images via the upload flow (post/+page.svelte)
  2. Save the post
  3. Navigate to the post grid / open the post detail
  4. Observe that images render as broken fragments, garbled pieces, or show browser image-load errors

Expected behavior

Uploaded images should be stored and retrieved without corruption. The full, valid base64 data URL should be recovered exactly as it was before storage, and images should render correctly in the post grid and post modal.

Current workaround

A frontend bandage — pairAndJoinChunks — exists in both Post.svelte (lines 32–55) and PostModal.svelte (line 45). It takes the corrupted flat array and re-pairs elements two-at-a-time, rejoining [prefix, payload] back into prefix,payload. This partially recovers single-image posts but remains brittle for multi-image posts where additional internal commas cause further mis-splits.

Proposed fix

Switch the column encoding from simple-array to simple-json (uses JSON.stringify / JSON.parse), which correctly escapes all internal characters including commas.

Both simple-array and simple-json compile down to a plain Postgres text column (confirmed by the original migration SQL — "images" text), so no schema migration is required.

Impact on existing data

Existing rows were stored with the old comma-joined format. Reading them with a JSON.parse decoder will throw a parse error rather than silently returning garbage. Old corrupted rows should be audited and either cleaned up or null-ed out before the fix is deployed, as the original image data cannot be reliably reconstructed from the corrupted fragments.

Steps

  1. Change the images column decorator from simple-array to simple-json in the relevant TypeORM entity
  2. Remove the pairAndJoinChunks workaround from Post.svelte and PostModal.svelte
  3. Handle/migrate (or purge) existing corrupted rows to prevent JSON parse errors at runtime
  4. Verify with a new post containing multiple images that all images are stored and displayed correctly

Supporting Media

N/A

Desktop

  • Device: N/A (server-side / DB layer bug)
  • OS: N/A
  • Browser: N/A
  • Version: N/A

Additional Context

  • Corruption affects every single post with at least one image — no posts are safe under the current encoding
  • If real users are onboarded before this is fixed, their uploaded images will be permanently unrecoverable
  • The fix is low-risk (no SQL schema change needed) but requires careful handling of already-corrupted rows in the DB

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpriority - 1Really important bug. App is broken, major functionality is unusable

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Image uploads corrupted on read due to `simple-array` comma-splitting base64 data URLs · Issue #1075 · MetaState-Prototype-Project/prototype · GitHub
Skip to content

Image uploads corrupted on read due to simple-array comma-splitting base64 data URLs #1075

Description

@plansombl

Description

Every post that contains at least one image is silently corrupted the moment it is read back from the database. The root cause is that TypeORM's simple-array column type stores arrays by naively joining values with a comma (value.join(",")) and reads them back by splitting on comma (value.split(",")) — with no escaping whatsoever.

Base64 data URLs produced by FileReader.readAsDataURL() always contain a literal comma in their MIME prefix (e.g. data:image/png;base64,iVBORw0KGgo...). This comma is indistinguishable from simple-array's delimiter, so the stored string is always parsed incorrectly on read.

Single-image post

Stored fine (nothing to join), but .split(",") still fires unconditionally on read, breaking data:image/png;base64,AAAA... into two entries:

  • data:image/png;base64 — a truncated, invalid data URL with no payload
  • AAAA... — a bare base64 string with no data: prefix, which the browser treats as a relative URL, not an image

Multi-image post

The join-separator comma and each image's own internal comma become completely indistinguishable. Images interleave and split at wrong boundaries, corrupting the entire set.

This is very likely the root cause of the "flying stars / meteor shower" rendering artefacts previously reported — corrupted/partial image decodes rather than someone else's real photos.

Steps to reproduce

  1. Create a post and attach one or more images via the upload flow (post/+page.svelte)
  2. Save the post
  3. Navigate to the post grid / open the post detail
  4. Observe that images render as broken fragments, garbled pieces, or show browser image-load errors

Expected behavior

Uploaded images should be stored and retrieved without corruption. The full, valid base64 data URL should be recovered exactly as it was before storage, and images should render correctly in the post grid and post modal.

Current workaround

A frontend bandage — pairAndJoinChunks — exists in both Post.svelte (lines 32–55) and PostModal.svelte (line 45). It takes the corrupted flat array and re-pairs elements two-at-a-time, rejoining [prefix, payload] back into prefix,payload. This partially recovers single-image posts but remains brittle for multi-image posts where additional internal commas cause further mis-splits.

Proposed fix

Switch the column encoding from simple-array to simple-json (uses JSON.stringify / JSON.parse), which correctly escapes all internal characters including commas.

Both simple-array and simple-json compile down to a plain Postgres text column (confirmed by the original migration SQL — "images" text), so no schema migration is required.

Impact on existing data

Existing rows were stored with the old comma-joined format. Reading them with a JSON.parse decoder will throw a parse error rather than silently returning garbage. Old corrupted rows should be audited and either cleaned up or null-ed out before the fix is deployed, as the original image data cannot be reliably reconstructed from the corrupted fragments.

Steps

  1. Change the images column decorator from simple-array to simple-json in the relevant TypeORM entity
  2. Remove the pairAndJoinChunks workaround from Post.svelte and PostModal.svelte
  3. Handle/migrate (or purge) existing corrupted rows to prevent JSON parse errors at runtime
  4. Verify with a new post containing multiple images that all images are stored and displayed correctly

Supporting Media

N/A

Desktop

  • Device: N/A (server-side / DB layer bug)
  • OS: N/A
  • Browser: N/A
  • Version: N/A

Additional Context

  • Corruption affects every single post with at least one image — no posts are safe under the current encoding
  • If real users are onboarded before this is fixed, their uploaded images will be permanently unrecoverable
  • The fix is low-risk (no SQL schema change needed) but requires careful handling of already-corrupted rows in the DB

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpriority - 1Really important bug. App is broken, major functionality is unusable

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Image uploads corrupted on read due to `simple-array` comma-splitting base64 data URLs · Issue #1075 · MetaState-Prototype-Project/prototype · GitHub
Skip to content

Image uploads corrupted on read due to simple-array comma-splitting base64 data URLs #1075

Description

@plansombl

Description

Every post that contains at least one image is silently corrupted the moment it is read back from the database. The root cause is that TypeORM's simple-array column type stores arrays by naively joining values with a comma (value.join(",")) and reads them back by splitting on comma (value.split(",")) — with no escaping whatsoever.

Base64 data URLs produced by FileReader.readAsDataURL() always contain a literal comma in their MIME prefix (e.g. data:image/png;base64,iVBORw0KGgo...). This comma is indistinguishable from simple-array's delimiter, so the stored string is always parsed incorrectly on read.

Single-image post

Stored fine (nothing to join), but .split(",") still fires unconditionally on read, breaking data:image/png;base64,AAAA... into two entries:

  • data:image/png;base64 — a truncated, invalid data URL with no payload
  • AAAA... — a bare base64 string with no data: prefix, which the browser treats as a relative URL, not an image

Multi-image post

The join-separator comma and each image's own internal comma become completely indistinguishable. Images interleave and split at wrong boundaries, corrupting the entire set.

This is very likely the root cause of the "flying stars / meteor shower" rendering artefacts previously reported — corrupted/partial image decodes rather than someone else's real photos.

Steps to reproduce

  1. Create a post and attach one or more images via the upload flow (post/+page.svelte)
  2. Save the post
  3. Navigate to the post grid / open the post detail
  4. Observe that images render as broken fragments, garbled pieces, or show browser image-load errors

Expected behavior

Uploaded images should be stored and retrieved without corruption. The full, valid base64 data URL should be recovered exactly as it was before storage, and images should render correctly in the post grid and post modal.

Current workaround

A frontend bandage — pairAndJoinChunks — exists in both Post.svelte (lines 32–55) and PostModal.svelte (line 45). It takes the corrupted flat array and re-pairs elements two-at-a-time, rejoining [prefix, payload] back into prefix,payload. This partially recovers single-image posts but remains brittle for multi-image posts where additional internal commas cause further mis-splits.

Proposed fix

Switch the column encoding from simple-array to simple-json (uses JSON.stringify / JSON.parse), which correctly escapes all internal characters including commas.

Both simple-array and simple-json compile down to a plain Postgres text column (confirmed by the original migration SQL — "images" text), so no schema migration is required.

Impact on existing data

Existing rows were stored with the old comma-joined format. Reading them with a JSON.parse decoder will throw a parse error rather than silently returning garbage. Old corrupted rows should be audited and either cleaned up or null-ed out before the fix is deployed, as the original image data cannot be reliably reconstructed from the corrupted fragments.

Steps

  1. Change the images column decorator from simple-array to simple-json in the relevant TypeORM entity
  2. Remove the pairAndJoinChunks workaround from Post.svelte and PostModal.svelte
  3. Handle/migrate (or purge) existing corrupted rows to prevent JSON parse errors at runtime
  4. Verify with a new post containing multiple images that all images are stored and displayed correctly

Supporting Media

N/A

Desktop

  • Device: N/A (server-side / DB layer bug)
  • OS: N/A
  • Browser: N/A
  • Version: N/A

Additional Context

  • Corruption affects every single post with at least one image — no posts are safe under the current encoding
  • If real users are onboarded before this is fixed, their uploaded images will be permanently unrecoverable
  • The fix is low-risk (no SQL schema change needed) but requires careful handling of already-corrupted rows in the DB

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpriority - 1Really important bug. App is broken, major functionality is unusable

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Image uploads corrupted on read due to `simple-array` comma-splitting base64 data URLs · Issue #1075 · MetaState-Prototype-Project/prototype · GitHub
Skip to content

Image uploads corrupted on read due to simple-array comma-splitting base64 data URLs #1075

Description

@plansombl

Description

Every post that contains at least one image is silently corrupted the moment it is read back from the database. The root cause is that TypeORM's simple-array column type stores arrays by naively joining values with a comma (value.join(",")) and reads them back by splitting on comma (value.split(",")) — with no escaping whatsoever.

Base64 data URLs produced by FileReader.readAsDataURL() always contain a literal comma in their MIME prefix (e.g. data:image/png;base64,iVBORw0KGgo...). This comma is indistinguishable from simple-array's delimiter, so the stored string is always parsed incorrectly on read.

Single-image post

Stored fine (nothing to join), but .split(",") still fires unconditionally on read, breaking data:image/png;base64,AAAA... into two entries:

  • data:image/png;base64 — a truncated, invalid data URL with no payload
  • AAAA... — a bare base64 string with no data: prefix, which the browser treats as a relative URL, not an image

Multi-image post

The join-separator comma and each image's own internal comma become completely indistinguishable. Images interleave and split at wrong boundaries, corrupting the entire set.

This is very likely the root cause of the "flying stars / meteor shower" rendering artefacts previously reported — corrupted/partial image decodes rather than someone else's real photos.

Steps to reproduce

  1. Create a post and attach one or more images via the upload flow (post/+page.svelte)
  2. Save the post
  3. Navigate to the post grid / open the post detail
  4. Observe that images render as broken fragments, garbled pieces, or show browser image-load errors

Expected behavior

Uploaded images should be stored and retrieved without corruption. The full, valid base64 data URL should be recovered exactly as it was before storage, and images should render correctly in the post grid and post modal.

Current workaround

A frontend bandage — pairAndJoinChunks — exists in both Post.svelte (lines 32–55) and PostModal.svelte (line 45). It takes the corrupted flat array and re-pairs elements two-at-a-time, rejoining [prefix, payload] back into prefix,payload. This partially recovers single-image posts but remains brittle for multi-image posts where additional internal commas cause further mis-splits.

Proposed fix

Switch the column encoding from simple-array to simple-json (uses JSON.stringify / JSON.parse), which correctly escapes all internal characters including commas.

Both simple-array and simple-json compile down to a plain Postgres text column (confirmed by the original migration SQL — "images" text), so no schema migration is required.

Impact on existing data

Existing rows were stored with the old comma-joined format. Reading them with a JSON.parse decoder will throw a parse error rather than silently returning garbage. Old corrupted rows should be audited and either cleaned up or null-ed out before the fix is deployed, as the original image data cannot be reliably reconstructed from the corrupted fragments.

Steps

  1. Change the images column decorator from simple-array to simple-json in the relevant TypeORM entity
  2. Remove the pairAndJoinChunks workaround from Post.svelte and PostModal.svelte
  3. Handle/migrate (or purge) existing corrupted rows to prevent JSON parse errors at runtime
  4. Verify with a new post containing multiple images that all images are stored and displayed correctly

Supporting Media

N/A

Desktop

  • Device: N/A (server-side / DB layer bug)
  • OS: N/A
  • Browser: N/A
  • Version: N/A

Additional Context

  • Corruption affects every single post with at least one image — no posts are safe under the current encoding
  • If real users are onboarded before this is fixed, their uploaded images will be permanently unrecoverable
  • The fix is low-risk (no SQL schema change needed) but requires careful handling of already-corrupted rows in the DB

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpriority - 1Really important bug. App is broken, major functionality is unusable

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions