[API-321] Migrate tests for blast writes - #342

Merged
schottra merged 6 commits into
mainfrom
comms-blast-tests
Aug 29, 2025
Merged

[API-321] Migrate tests for blast writes#342
schottra merged 6 commits into
mainfrom
comms-blast-tests

Conversation

@schottra

Copy link
Copy Markdown
Contributor

This is a followup to #338 to migrate the quite sizable blasts test file. Source is here: https://github.com/AudiusProject/audius-protocol/blob/ab73d979f98e151ac0aa408c7a82c105c3b6355e/comms/discovery/rpcz/chat_blast_test.go#L760

  • Migrated everything to pgx patterns
  • Split the file into multiple isolated tests instead of one huge test
  • Using fixtures instead of manual inserts for setup data
  • Removed a lot of the 'in the past' time variables in favor of just using time.Now() for operations that are sequential
  • Updated the version of getNewBlasts in the comms directory to match the one we use in the get endpoint. We are being intentionally WET here.
  • Fixed an issue with upgrading blasts to chats where the timestamp that we assign to the new message would be in the wrong timezone if run on a machine not set to UTC. In practice, this won't break on CI or production, but it does break when running on a local machine 😢

As a bonus, also cleaned up the tests from the previous PR to match the patterns here.

Comment threadapi/comms/chat.go
($1, $2, $3, $4, $5)
on conflict do nothing
`, trashid.BlastMessageID(blast.BlastID, params.ChatID), params.ChatID, blast.FromUserID, blast.CreatedAt, blast.BlastID)
`, messageId, params.ChatID, blast.FromUserID, blast.CreatedAt.UTC(), blast.BlastID)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note here the usage of .UTC(). On my machine, when running this, the location() on this time.Time was being set to Local and that was causing us to insert messages in the past. Since chat_blast uses a timestamptz column type and chat_message uses a plain timestamp column type, pgx semantics when inserting will cause it to try and map from the timezone in the incoming variable to the timezone of the machine running postgres.
Kind of gnarly 😬

Comment threadapi/comms/chat_test_queries.go Outdated
Comment threadapi/comms/chat_test_queries.go Outdated
})
}

/* TODO:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think writing them would add a ton of value for the amount of time it would take. Can be persuaded though :-p

@schottra
schottra merged commit df019b2 into mainAug 29, 2025
5 checks passed
@schottra
schottra deleted the comms-blast-tests branch August 29, 2025 18:52
schottra added a commit that referenced this pull request Aug 29, 2025
* origin/main:
[API-321] Migrate tests for blast writes (#342)
Update query for best selling (#347)
Remove membersChange24hrPercent (#346)
[QA-2301] Fix blockees req (#345)
raymondjacobson added a commit that referenced this pull request Jun 9, 2026
#941)
## Summary
Bumps `go-openaudio` to `aee578e` (OpenAudio/go-openaudio#342) and wires
the **core-indexer** to source blocks from the
`CoreService.StreamBlocks` **gRPC stream** instead of polling
`GetBlocks`, behind a new `coreBlockStreamEnabled` env — **default
off**, so prod behavior is unchanged until flipped.
## What #342 gives us
The ETL can now consume blocks over a gRPC server stream: the node
replays history from the resume height, then live-tails, gap-filling any
heights its in-process pubsub drops. The ETL reconnects from its cursor
on error and **auto-falls-back to polling** if the endpoint doesn't
support the stream.
## Changes
- `go.mod`/`go.sum`: `github.com/OpenAudio/go-openaudio` (+ `/pkg/etl`)
→ `v1.3.1-0.20260609211151-aee578ec923b`
- `config/config.go`: new `CoreBlockStreamEnabled`
(`coreBlockStreamEnabled` env, default off)
- `indexer/indexer.go`: when enabled, set `Config.BlockStreamEnabled`
and pass a gRPC (`connect.WithGRPC`) Core client via
`SetBlockStreamClient`. The SDK's unary Core client is retained for
status checks + the polling fallback.
## Rollout
1. Merge with the flag **unset** → no behavior change (still polling).
2. Set `coreBlockStreamEnabled=true` on **one** indexer and validate: it
tails live, resumes gap-free on restart, and (induced) stream errors
fall back to polling cleanly.
3. The one open question is whether the prod LB/ingress in front of the
validator passes long-lived h2 gRPC streams — this single-indexer step
is where that gets confirmed. If it doesn't, leave the flag off; polling
is unaffected.
Validated locally with the harness in OpenAudio/go-openaudio#344
(devnet, `--stream`, catch-up + restart-resume).
## Test plan
- [x] `go mod tidy` (only go.mod/go.sum), `go build ./...`, `go vet
./indexer ./config` clean
- [ ] Staging: flip `coreBlockStreamEnabled=true` on one indexer;
confirm live-tail + gap-free restart + fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@schottra@raymondjacobson
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

[API-321] Migrate tests for blast writes - #342

Merged
schottra merged 6 commits into
mainfrom
comms-blast-tests
Aug 29, 2025
Merged

[API-321] Migrate tests for blast writes#342
schottra merged 6 commits into
mainfrom
comms-blast-tests

Conversation

@schottra

Copy link
Copy Markdown
Contributor

This is a followup to #338 to migrate the quite sizable blasts test file. Source is here: https://github.com/AudiusProject/audius-protocol/blob/ab73d979f98e151ac0aa408c7a82c105c3b6355e/comms/discovery/rpcz/chat_blast_test.go#L760

  • Migrated everything to pgx patterns
  • Split the file into multiple isolated tests instead of one huge test
  • Using fixtures instead of manual inserts for setup data
  • Removed a lot of the 'in the past' time variables in favor of just using time.Now() for operations that are sequential
  • Updated the version of getNewBlasts in the comms directory to match the one we use in the get endpoint. We are being intentionally WET here.
  • Fixed an issue with upgrading blasts to chats where the timestamp that we assign to the new message would be in the wrong timezone if run on a machine not set to UTC. In practice, this won't break on CI or production, but it does break when running on a local machine 😢

As a bonus, also cleaned up the tests from the previous PR to match the patterns here.

Comment threadapi/comms/chat.go
($1, $2, $3, $4, $5)
on conflict do nothing
`, trashid.BlastMessageID(blast.BlastID, params.ChatID), params.ChatID, blast.FromUserID, blast.CreatedAt, blast.BlastID)
`, messageId, params.ChatID, blast.FromUserID, blast.CreatedAt.UTC(), blast.BlastID)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note here the usage of .UTC(). On my machine, when running this, the location() on this time.Time was being set to Local and that was causing us to insert messages in the past. Since chat_blast uses a timestamptz column type and chat_message uses a plain timestamp column type, pgx semantics when inserting will cause it to try and map from the timezone in the incoming variable to the timezone of the machine running postgres.
Kind of gnarly 😬

Comment threadapi/comms/chat_test_queries.go Outdated
Comment threadapi/comms/chat_test_queries.go Outdated
})
}

/* TODO:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think writing them would add a ton of value for the amount of time it would take. Can be persuaded though :-p

@schottra
schottra merged commit df019b2 into mainAug 29, 2025
5 checks passed
@schottra
schottra deleted the comms-blast-tests branch August 29, 2025 18:52
schottra added a commit that referenced this pull request Aug 29, 2025
* origin/main:
[API-321] Migrate tests for blast writes (#342)
Update query for best selling (#347)
Remove membersChange24hrPercent (#346)
[QA-2301] Fix blockees req (#345)
raymondjacobson added a commit that referenced this pull request Jun 9, 2026
#941)
## Summary
Bumps `go-openaudio` to `aee578e` (OpenAudio/go-openaudio#342) and wires
the **core-indexer** to source blocks from the
`CoreService.StreamBlocks` **gRPC stream** instead of polling
`GetBlocks`, behind a new `coreBlockStreamEnabled` env — **default
off**, so prod behavior is unchanged until flipped.
## What #342 gives us
The ETL can now consume blocks over a gRPC server stream: the node
replays history from the resume height, then live-tails, gap-filling any
heights its in-process pubsub drops. The ETL reconnects from its cursor
on error and **auto-falls-back to polling** if the endpoint doesn't
support the stream.
## Changes
- `go.mod`/`go.sum`: `github.com/OpenAudio/go-openaudio` (+ `/pkg/etl`)
→ `v1.3.1-0.20260609211151-aee578ec923b`
- `config/config.go`: new `CoreBlockStreamEnabled`
(`coreBlockStreamEnabled` env, default off)
- `indexer/indexer.go`: when enabled, set `Config.BlockStreamEnabled`
and pass a gRPC (`connect.WithGRPC`) Core client via
`SetBlockStreamClient`. The SDK's unary Core client is retained for
status checks + the polling fallback.
## Rollout
1. Merge with the flag **unset** → no behavior change (still polling).
2. Set `coreBlockStreamEnabled=true` on **one** indexer and validate: it
tails live, resumes gap-free on restart, and (induced) stream errors
fall back to polling cleanly.
3. The one open question is whether the prod LB/ingress in front of the
validator passes long-lived h2 gRPC streams — this single-indexer step
is where that gets confirmed. If it doesn't, leave the flag off; polling
is unaffected.
Validated locally with the harness in OpenAudio/go-openaudio#344
(devnet, `--stream`, catch-up + restart-resume).
## Test plan
- [x] `go mod tidy` (only go.mod/go.sum), `go build ./...`, `go vet
./indexer ./config` clean
- [ ] Staging: flip `coreBlockStreamEnabled=true` on one indexer;
confirm live-tail + gap-free restart + fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@schottra@raymondjacobson
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[API-321] Migrate tests for blast writes - #342

Merged
schottra merged 6 commits into
mainfrom
comms-blast-tests
Aug 29, 2025
Merged

[API-321] Migrate tests for blast writes#342
schottra merged 6 commits into
mainfrom
comms-blast-tests

Conversation

@schottra

Copy link
Copy Markdown
Contributor

This is a followup to #338 to migrate the quite sizable blasts test file. Source is here: https://github.com/AudiusProject/audius-protocol/blob/ab73d979f98e151ac0aa408c7a82c105c3b6355e/comms/discovery/rpcz/chat_blast_test.go#L760

  • Migrated everything to pgx patterns
  • Split the file into multiple isolated tests instead of one huge test
  • Using fixtures instead of manual inserts for setup data
  • Removed a lot of the 'in the past' time variables in favor of just using time.Now() for operations that are sequential
  • Updated the version of getNewBlasts in the comms directory to match the one we use in the get endpoint. We are being intentionally WET here.
  • Fixed an issue with upgrading blasts to chats where the timestamp that we assign to the new message would be in the wrong timezone if run on a machine not set to UTC. In practice, this won't break on CI or production, but it does break when running on a local machine 😢

As a bonus, also cleaned up the tests from the previous PR to match the patterns here.

Comment threadapi/comms/chat.go
($1, $2, $3, $4, $5)
on conflict do nothing
`, trashid.BlastMessageID(blast.BlastID, params.ChatID), params.ChatID, blast.FromUserID, blast.CreatedAt, blast.BlastID)
`, messageId, params.ChatID, blast.FromUserID, blast.CreatedAt.UTC(), blast.BlastID)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note here the usage of .UTC(). On my machine, when running this, the location() on this time.Time was being set to Local and that was causing us to insert messages in the past. Since chat_blast uses a timestamptz column type and chat_message uses a plain timestamp column type, pgx semantics when inserting will cause it to try and map from the timezone in the incoming variable to the timezone of the machine running postgres.
Kind of gnarly 😬

Comment threadapi/comms/chat_test_queries.go Outdated
Comment threadapi/comms/chat_test_queries.go Outdated
})
}

/* TODO:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think writing them would add a ton of value for the amount of time it would take. Can be persuaded though :-p

@schottra
schottra merged commit df019b2 into mainAug 29, 2025
5 checks passed
@schottra
schottra deleted the comms-blast-tests branch August 29, 2025 18:52
schottra added a commit that referenced this pull request Aug 29, 2025
* origin/main:
[API-321] Migrate tests for blast writes (#342)
Update query for best selling (#347)
Remove membersChange24hrPercent (#346)
[QA-2301] Fix blockees req (#345)
raymondjacobson added a commit that referenced this pull request Jun 9, 2026
#941)
## Summary
Bumps `go-openaudio` to `aee578e` (OpenAudio/go-openaudio#342) and wires
the **core-indexer** to source blocks from the
`CoreService.StreamBlocks` **gRPC stream** instead of polling
`GetBlocks`, behind a new `coreBlockStreamEnabled` env — **default
off**, so prod behavior is unchanged until flipped.
## What #342 gives us
The ETL can now consume blocks over a gRPC server stream: the node
replays history from the resume height, then live-tails, gap-filling any
heights its in-process pubsub drops. The ETL reconnects from its cursor
on error and **auto-falls-back to polling** if the endpoint doesn't
support the stream.
## Changes
- `go.mod`/`go.sum`: `github.com/OpenAudio/go-openaudio` (+ `/pkg/etl`)
→ `v1.3.1-0.20260609211151-aee578ec923b`
- `config/config.go`: new `CoreBlockStreamEnabled`
(`coreBlockStreamEnabled` env, default off)
- `indexer/indexer.go`: when enabled, set `Config.BlockStreamEnabled`
and pass a gRPC (`connect.WithGRPC`) Core client via
`SetBlockStreamClient`. The SDK's unary Core client is retained for
status checks + the polling fallback.
## Rollout
1. Merge with the flag **unset** → no behavior change (still polling).
2. Set `coreBlockStreamEnabled=true` on **one** indexer and validate: it
tails live, resumes gap-free on restart, and (induced) stream errors
fall back to polling cleanly.
3. The one open question is whether the prod LB/ingress in front of the
validator passes long-lived h2 gRPC streams — this single-indexer step
is where that gets confirmed. If it doesn't, leave the flag off; polling
is unaffected.
Validated locally with the harness in OpenAudio/go-openaudio#344
(devnet, `--stream`, catch-up + restart-resume).
## Test plan
- [x] `go mod tidy` (only go.mod/go.sum), `go build ./...`, `go vet
./indexer ./config` clean
- [ ] Staging: flip `coreBlockStreamEnabled=true` on one indexer;
confirm live-tail + gap-free restart + fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@schottra@raymondjacobson
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[API-321] Migrate tests for blast writes - #342

Merged
schottra merged 6 commits into
mainfrom
comms-blast-tests
Aug 29, 2025
Merged

[API-321] Migrate tests for blast writes#342
schottra merged 6 commits into
mainfrom
comms-blast-tests

Conversation

@schottra

Copy link
Copy Markdown
Contributor

This is a followup to #338 to migrate the quite sizable blasts test file. Source is here: https://github.com/AudiusProject/audius-protocol/blob/ab73d979f98e151ac0aa408c7a82c105c3b6355e/comms/discovery/rpcz/chat_blast_test.go#L760

  • Migrated everything to pgx patterns
  • Split the file into multiple isolated tests instead of one huge test
  • Using fixtures instead of manual inserts for setup data
  • Removed a lot of the 'in the past' time variables in favor of just using time.Now() for operations that are sequential
  • Updated the version of getNewBlasts in the comms directory to match the one we use in the get endpoint. We are being intentionally WET here.
  • Fixed an issue with upgrading blasts to chats where the timestamp that we assign to the new message would be in the wrong timezone if run on a machine not set to UTC. In practice, this won't break on CI or production, but it does break when running on a local machine 😢

As a bonus, also cleaned up the tests from the previous PR to match the patterns here.

Comment threadapi/comms/chat.go
($1, $2, $3, $4, $5)
on conflict do nothing
`, trashid.BlastMessageID(blast.BlastID, params.ChatID), params.ChatID, blast.FromUserID, blast.CreatedAt, blast.BlastID)
`, messageId, params.ChatID, blast.FromUserID, blast.CreatedAt.UTC(), blast.BlastID)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note here the usage of .UTC(). On my machine, when running this, the location() on this time.Time was being set to Local and that was causing us to insert messages in the past. Since chat_blast uses a timestamptz column type and chat_message uses a plain timestamp column type, pgx semantics when inserting will cause it to try and map from the timezone in the incoming variable to the timezone of the machine running postgres.
Kind of gnarly 😬

Comment threadapi/comms/chat_test_queries.go Outdated
Comment threadapi/comms/chat_test_queries.go Outdated
})
}

/* TODO:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think writing them would add a ton of value for the amount of time it would take. Can be persuaded though :-p

@schottra
schottra merged commit df019b2 into mainAug 29, 2025
5 checks passed
@schottra
schottra deleted the comms-blast-tests branch August 29, 2025 18:52
schottra added a commit that referenced this pull request Aug 29, 2025
* origin/main:
[API-321] Migrate tests for blast writes (#342)
Update query for best selling (#347)
Remove membersChange24hrPercent (#346)
[QA-2301] Fix blockees req (#345)
raymondjacobson added a commit that referenced this pull request Jun 9, 2026
#941)
## Summary
Bumps `go-openaudio` to `aee578e` (OpenAudio/go-openaudio#342) and wires
the **core-indexer** to source blocks from the
`CoreService.StreamBlocks` **gRPC stream** instead of polling
`GetBlocks`, behind a new `coreBlockStreamEnabled` env — **default
off**, so prod behavior is unchanged until flipped.
## What #342 gives us
The ETL can now consume blocks over a gRPC server stream: the node
replays history from the resume height, then live-tails, gap-filling any
heights its in-process pubsub drops. The ETL reconnects from its cursor
on error and **auto-falls-back to polling** if the endpoint doesn't
support the stream.
## Changes
- `go.mod`/`go.sum`: `github.com/OpenAudio/go-openaudio` (+ `/pkg/etl`)
→ `v1.3.1-0.20260609211151-aee578ec923b`
- `config/config.go`: new `CoreBlockStreamEnabled`
(`coreBlockStreamEnabled` env, default off)
- `indexer/indexer.go`: when enabled, set `Config.BlockStreamEnabled`
and pass a gRPC (`connect.WithGRPC`) Core client via
`SetBlockStreamClient`. The SDK's unary Core client is retained for
status checks + the polling fallback.
## Rollout
1. Merge with the flag **unset** → no behavior change (still polling).
2. Set `coreBlockStreamEnabled=true` on **one** indexer and validate: it
tails live, resumes gap-free on restart, and (induced) stream errors
fall back to polling cleanly.
3. The one open question is whether the prod LB/ingress in front of the
validator passes long-lived h2 gRPC streams — this single-indexer step
is where that gets confirmed. If it doesn't, leave the flag off; polling
is unaffected.
Validated locally with the harness in OpenAudio/go-openaudio#344
(devnet, `--stream`, catch-up + restart-resume).
## Test plan
- [x] `go mod tidy` (only go.mod/go.sum), `go build ./...`, `go vet
./indexer ./config` clean
- [ ] Staging: flip `coreBlockStreamEnabled=true` on one indexer;
confirm live-tail + gap-free restart + fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@schottra@raymondjacobson
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

[API-321] Migrate tests for blast writes - #342

Merged
schottra merged 6 commits into
mainfrom
comms-blast-tests
Aug 29, 2025
Merged

[API-321] Migrate tests for blast writes#342
schottra merged 6 commits into
mainfrom
comms-blast-tests

Conversation

@schottra

Copy link
Copy Markdown
Contributor

This is a followup to #338 to migrate the quite sizable blasts test file. Source is here: https://github.com/AudiusProject/audius-protocol/blob/ab73d979f98e151ac0aa408c7a82c105c3b6355e/comms/discovery/rpcz/chat_blast_test.go#L760

  • Migrated everything to pgx patterns
  • Split the file into multiple isolated tests instead of one huge test
  • Using fixtures instead of manual inserts for setup data
  • Removed a lot of the 'in the past' time variables in favor of just using time.Now() for operations that are sequential
  • Updated the version of getNewBlasts in the comms directory to match the one we use in the get endpoint. We are being intentionally WET here.
  • Fixed an issue with upgrading blasts to chats where the timestamp that we assign to the new message would be in the wrong timezone if run on a machine not set to UTC. In practice, this won't break on CI or production, but it does break when running on a local machine 😢

As a bonus, also cleaned up the tests from the previous PR to match the patterns here.

Comment threadapi/comms/chat.go
($1, $2, $3, $4, $5)
on conflict do nothing
`, trashid.BlastMessageID(blast.BlastID, params.ChatID), params.ChatID, blast.FromUserID, blast.CreatedAt, blast.BlastID)
`, messageId, params.ChatID, blast.FromUserID, blast.CreatedAt.UTC(), blast.BlastID)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note here the usage of .UTC(). On my machine, when running this, the location() on this time.Time was being set to Local and that was causing us to insert messages in the past. Since chat_blast uses a timestamptz column type and chat_message uses a plain timestamp column type, pgx semantics when inserting will cause it to try and map from the timezone in the incoming variable to the timezone of the machine running postgres.
Kind of gnarly 😬

Comment threadapi/comms/chat_test_queries.go Outdated
Comment threadapi/comms/chat_test_queries.go Outdated
})
}

/* TODO:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think writing them would add a ton of value for the amount of time it would take. Can be persuaded though :-p

@schottra
schottra merged commit df019b2 into mainAug 29, 2025
5 checks passed
@schottra
schottra deleted the comms-blast-tests branch August 29, 2025 18:52
schottra added a commit that referenced this pull request Aug 29, 2025
* origin/main:
[API-321] Migrate tests for blast writes (#342)
Update query for best selling (#347)
Remove membersChange24hrPercent (#346)
[QA-2301] Fix blockees req (#345)
raymondjacobson added a commit that referenced this pull request Jun 9, 2026
#941)
## Summary
Bumps `go-openaudio` to `aee578e` (OpenAudio/go-openaudio#342) and wires
the **core-indexer** to source blocks from the
`CoreService.StreamBlocks` **gRPC stream** instead of polling
`GetBlocks`, behind a new `coreBlockStreamEnabled` env — **default
off**, so prod behavior is unchanged until flipped.
## What #342 gives us
The ETL can now consume blocks over a gRPC server stream: the node
replays history from the resume height, then live-tails, gap-filling any
heights its in-process pubsub drops. The ETL reconnects from its cursor
on error and **auto-falls-back to polling** if the endpoint doesn't
support the stream.
## Changes
- `go.mod`/`go.sum`: `github.com/OpenAudio/go-openaudio` (+ `/pkg/etl`)
→ `v1.3.1-0.20260609211151-aee578ec923b`
- `config/config.go`: new `CoreBlockStreamEnabled`
(`coreBlockStreamEnabled` env, default off)
- `indexer/indexer.go`: when enabled, set `Config.BlockStreamEnabled`
and pass a gRPC (`connect.WithGRPC`) Core client via
`SetBlockStreamClient`. The SDK's unary Core client is retained for
status checks + the polling fallback.
## Rollout
1. Merge with the flag **unset** → no behavior change (still polling).
2. Set `coreBlockStreamEnabled=true` on **one** indexer and validate: it
tails live, resumes gap-free on restart, and (induced) stream errors
fall back to polling cleanly.
3. The one open question is whether the prod LB/ingress in front of the
validator passes long-lived h2 gRPC streams — this single-indexer step
is where that gets confirmed. If it doesn't, leave the flag off; polling
is unaffected.
Validated locally with the harness in OpenAudio/go-openaudio#344
(devnet, `--stream`, catch-up + restart-resume).
## Test plan
- [x] `go mod tidy` (only go.mod/go.sum), `go build ./...`, `go vet
./indexer ./config` clean
- [ ] Staging: flip `coreBlockStreamEnabled=true` on one indexer;
confirm live-tail + gap-free restart + fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@schottra@raymondjacobson
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[API-321] Migrate tests for blast writes - #342

Merged
schottra merged 6 commits into
mainfrom
comms-blast-tests
Aug 29, 2025
Merged

[API-321] Migrate tests for blast writes#342
schottra merged 6 commits into
mainfrom
comms-blast-tests

Conversation

@schottra

Copy link
Copy Markdown
Contributor

This is a followup to #338 to migrate the quite sizable blasts test file. Source is here: https://github.com/AudiusProject/audius-protocol/blob/ab73d979f98e151ac0aa408c7a82c105c3b6355e/comms/discovery/rpcz/chat_blast_test.go#L760

  • Migrated everything to pgx patterns
  • Split the file into multiple isolated tests instead of one huge test
  • Using fixtures instead of manual inserts for setup data
  • Removed a lot of the 'in the past' time variables in favor of just using time.Now() for operations that are sequential
  • Updated the version of getNewBlasts in the comms directory to match the one we use in the get endpoint. We are being intentionally WET here.
  • Fixed an issue with upgrading blasts to chats where the timestamp that we assign to the new message would be in the wrong timezone if run on a machine not set to UTC. In practice, this won't break on CI or production, but it does break when running on a local machine 😢

As a bonus, also cleaned up the tests from the previous PR to match the patterns here.

Comment threadapi/comms/chat.go
($1, $2, $3, $4, $5)
on conflict do nothing
`, trashid.BlastMessageID(blast.BlastID, params.ChatID), params.ChatID, blast.FromUserID, blast.CreatedAt, blast.BlastID)
`, messageId, params.ChatID, blast.FromUserID, blast.CreatedAt.UTC(), blast.BlastID)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note here the usage of .UTC(). On my machine, when running this, the location() on this time.Time was being set to Local and that was causing us to insert messages in the past. Since chat_blast uses a timestamptz column type and chat_message uses a plain timestamp column type, pgx semantics when inserting will cause it to try and map from the timezone in the incoming variable to the timezone of the machine running postgres.
Kind of gnarly 😬

Comment threadapi/comms/chat_test_queries.go Outdated
Comment threadapi/comms/chat_test_queries.go Outdated
})
}

/* TODO:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think writing them would add a ton of value for the amount of time it would take. Can be persuaded though :-p

@schottra
schottra merged commit df019b2 into mainAug 29, 2025
5 checks passed
@schottra
schottra deleted the comms-blast-tests branch August 29, 2025 18:52
schottra added a commit that referenced this pull request Aug 29, 2025
* origin/main:
[API-321] Migrate tests for blast writes (#342)
Update query for best selling (#347)
Remove membersChange24hrPercent (#346)
[QA-2301] Fix blockees req (#345)
raymondjacobson added a commit that referenced this pull request Jun 9, 2026
#941)
## Summary
Bumps `go-openaudio` to `aee578e` (OpenAudio/go-openaudio#342) and wires
the **core-indexer** to source blocks from the
`CoreService.StreamBlocks` **gRPC stream** instead of polling
`GetBlocks`, behind a new `coreBlockStreamEnabled` env — **default
off**, so prod behavior is unchanged until flipped.
## What #342 gives us
The ETL can now consume blocks over a gRPC server stream: the node
replays history from the resume height, then live-tails, gap-filling any
heights its in-process pubsub drops. The ETL reconnects from its cursor
on error and **auto-falls-back to polling** if the endpoint doesn't
support the stream.
## Changes
- `go.mod`/`go.sum`: `github.com/OpenAudio/go-openaudio` (+ `/pkg/etl`)
→ `v1.3.1-0.20260609211151-aee578ec923b`
- `config/config.go`: new `CoreBlockStreamEnabled`
(`coreBlockStreamEnabled` env, default off)
- `indexer/indexer.go`: when enabled, set `Config.BlockStreamEnabled`
and pass a gRPC (`connect.WithGRPC`) Core client via
`SetBlockStreamClient`. The SDK's unary Core client is retained for
status checks + the polling fallback.
## Rollout
1. Merge with the flag **unset** → no behavior change (still polling).
2. Set `coreBlockStreamEnabled=true` on **one** indexer and validate: it
tails live, resumes gap-free on restart, and (induced) stream errors
fall back to polling cleanly.
3. The one open question is whether the prod LB/ingress in front of the
validator passes long-lived h2 gRPC streams — this single-indexer step
is where that gets confirmed. If it doesn't, leave the flag off; polling
is unaffected.
Validated locally with the harness in OpenAudio/go-openaudio#344
(devnet, `--stream`, catch-up + restart-resume).
## Test plan
- [x] `go mod tidy` (only go.mod/go.sum), `go build ./...`, `go vet
./indexer ./config` clean
- [ ] Staging: flip `coreBlockStreamEnabled=true` on one indexer;
confirm live-tail + gap-free restart + fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@schottra@raymondjacobson
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[API-321] Migrate tests for blast writes - #342

Merged
schottra merged 6 commits into
mainfrom
comms-blast-tests
Aug 29, 2025
Merged

[API-321] Migrate tests for blast writes#342
schottra merged 6 commits into
mainfrom
comms-blast-tests

Conversation

@schottra

Copy link
Copy Markdown
Contributor

This is a followup to #338 to migrate the quite sizable blasts test file. Source is here: https://github.com/AudiusProject/audius-protocol/blob/ab73d979f98e151ac0aa408c7a82c105c3b6355e/comms/discovery/rpcz/chat_blast_test.go#L760

  • Migrated everything to pgx patterns
  • Split the file into multiple isolated tests instead of one huge test
  • Using fixtures instead of manual inserts for setup data
  • Removed a lot of the 'in the past' time variables in favor of just using time.Now() for operations that are sequential
  • Updated the version of getNewBlasts in the comms directory to match the one we use in the get endpoint. We are being intentionally WET here.
  • Fixed an issue with upgrading blasts to chats where the timestamp that we assign to the new message would be in the wrong timezone if run on a machine not set to UTC. In practice, this won't break on CI or production, but it does break when running on a local machine 😢

As a bonus, also cleaned up the tests from the previous PR to match the patterns here.

Comment threadapi/comms/chat.go
($1, $2, $3, $4, $5)
on conflict do nothing
`, trashid.BlastMessageID(blast.BlastID, params.ChatID), params.ChatID, blast.FromUserID, blast.CreatedAt, blast.BlastID)
`, messageId, params.ChatID, blast.FromUserID, blast.CreatedAt.UTC(), blast.BlastID)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note here the usage of .UTC(). On my machine, when running this, the location() on this time.Time was being set to Local and that was causing us to insert messages in the past. Since chat_blast uses a timestamptz column type and chat_message uses a plain timestamp column type, pgx semantics when inserting will cause it to try and map from the timezone in the incoming variable to the timezone of the machine running postgres.
Kind of gnarly 😬

Comment threadapi/comms/chat_test_queries.go Outdated
Comment threadapi/comms/chat_test_queries.go Outdated
})
}

/* TODO:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think writing them would add a ton of value for the amount of time it would take. Can be persuaded though :-p

@schottra
schottra merged commit df019b2 into mainAug 29, 2025
5 checks passed
@schottra
schottra deleted the comms-blast-tests branch August 29, 2025 18:52
schottra added a commit that referenced this pull request Aug 29, 2025
* origin/main:
[API-321] Migrate tests for blast writes (#342)
Update query for best selling (#347)
Remove membersChange24hrPercent (#346)
[QA-2301] Fix blockees req (#345)
raymondjacobson added a commit that referenced this pull request Jun 9, 2026
#941)
## Summary
Bumps `go-openaudio` to `aee578e` (OpenAudio/go-openaudio#342) and wires
the **core-indexer** to source blocks from the
`CoreService.StreamBlocks` **gRPC stream** instead of polling
`GetBlocks`, behind a new `coreBlockStreamEnabled` env — **default
off**, so prod behavior is unchanged until flipped.
## What #342 gives us
The ETL can now consume blocks over a gRPC server stream: the node
replays history from the resume height, then live-tails, gap-filling any
heights its in-process pubsub drops. The ETL reconnects from its cursor
on error and **auto-falls-back to polling** if the endpoint doesn't
support the stream.
## Changes
- `go.mod`/`go.sum`: `github.com/OpenAudio/go-openaudio` (+ `/pkg/etl`)
→ `v1.3.1-0.20260609211151-aee578ec923b`
- `config/config.go`: new `CoreBlockStreamEnabled`
(`coreBlockStreamEnabled` env, default off)
- `indexer/indexer.go`: when enabled, set `Config.BlockStreamEnabled`
and pass a gRPC (`connect.WithGRPC`) Core client via
`SetBlockStreamClient`. The SDK's unary Core client is retained for
status checks + the polling fallback.
## Rollout
1. Merge with the flag **unset** → no behavior change (still polling).
2. Set `coreBlockStreamEnabled=true` on **one** indexer and validate: it
tails live, resumes gap-free on restart, and (induced) stream errors
fall back to polling cleanly.
3. The one open question is whether the prod LB/ingress in front of the
validator passes long-lived h2 gRPC streams — this single-indexer step
is where that gets confirmed. If it doesn't, leave the flag off; polling
is unaffected.
Validated locally with the harness in OpenAudio/go-openaudio#344
(devnet, `--stream`, catch-up + restart-resume).
## Test plan
- [x] `go mod tidy` (only go.mod/go.sum), `go build ./...`, `go vet
./indexer ./config` clean
- [ ] Staging: flip `coreBlockStreamEnabled=true` on one indexer;
confirm live-tail + gap-free restart + fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@schottra@raymondjacobson
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

[API-321] Migrate tests for blast writes - #342

Merged
schottra merged 6 commits into
mainfrom
comms-blast-tests
Aug 29, 2025
Merged

[API-321] Migrate tests for blast writes#342
schottra merged 6 commits into
mainfrom
comms-blast-tests

Conversation

@schottra

Copy link
Copy Markdown
Contributor

This is a followup to #338 to migrate the quite sizable blasts test file. Source is here: https://github.com/AudiusProject/audius-protocol/blob/ab73d979f98e151ac0aa408c7a82c105c3b6355e/comms/discovery/rpcz/chat_blast_test.go#L760

  • Migrated everything to pgx patterns
  • Split the file into multiple isolated tests instead of one huge test
  • Using fixtures instead of manual inserts for setup data
  • Removed a lot of the 'in the past' time variables in favor of just using time.Now() for operations that are sequential
  • Updated the version of getNewBlasts in the comms directory to match the one we use in the get endpoint. We are being intentionally WET here.
  • Fixed an issue with upgrading blasts to chats where the timestamp that we assign to the new message would be in the wrong timezone if run on a machine not set to UTC. In practice, this won't break on CI or production, but it does break when running on a local machine 😢

As a bonus, also cleaned up the tests from the previous PR to match the patterns here.

Comment threadapi/comms/chat.go
($1, $2, $3, $4, $5)
on conflict do nothing
`, trashid.BlastMessageID(blast.BlastID, params.ChatID), params.ChatID, blast.FromUserID, blast.CreatedAt, blast.BlastID)
`, messageId, params.ChatID, blast.FromUserID, blast.CreatedAt.UTC(), blast.BlastID)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note here the usage of .UTC(). On my machine, when running this, the location() on this time.Time was being set to Local and that was causing us to insert messages in the past. Since chat_blast uses a timestamptz column type and chat_message uses a plain timestamp column type, pgx semantics when inserting will cause it to try and map from the timezone in the incoming variable to the timezone of the machine running postgres.
Kind of gnarly 😬

Comment threadapi/comms/chat_test_queries.go Outdated
Comment threadapi/comms/chat_test_queries.go Outdated
})
}

/* TODO:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think writing them would add a ton of value for the amount of time it would take. Can be persuaded though :-p

@schottra
schottra merged commit df019b2 into mainAug 29, 2025
5 checks passed
@schottra
schottra deleted the comms-blast-tests branch August 29, 2025 18:52
schottra added a commit that referenced this pull request Aug 29, 2025
* origin/main:
[API-321] Migrate tests for blast writes (#342)
Update query for best selling (#347)
Remove membersChange24hrPercent (#346)
[QA-2301] Fix blockees req (#345)
raymondjacobson added a commit that referenced this pull request Jun 9, 2026
#941)
## Summary
Bumps `go-openaudio` to `aee578e` (OpenAudio/go-openaudio#342) and wires
the **core-indexer** to source blocks from the
`CoreService.StreamBlocks` **gRPC stream** instead of polling
`GetBlocks`, behind a new `coreBlockStreamEnabled` env — **default
off**, so prod behavior is unchanged until flipped.
## What #342 gives us
The ETL can now consume blocks over a gRPC server stream: the node
replays history from the resume height, then live-tails, gap-filling any
heights its in-process pubsub drops. The ETL reconnects from its cursor
on error and **auto-falls-back to polling** if the endpoint doesn't
support the stream.
## Changes
- `go.mod`/`go.sum`: `github.com/OpenAudio/go-openaudio` (+ `/pkg/etl`)
→ `v1.3.1-0.20260609211151-aee578ec923b`
- `config/config.go`: new `CoreBlockStreamEnabled`
(`coreBlockStreamEnabled` env, default off)
- `indexer/indexer.go`: when enabled, set `Config.BlockStreamEnabled`
and pass a gRPC (`connect.WithGRPC`) Core client via
`SetBlockStreamClient`. The SDK's unary Core client is retained for
status checks + the polling fallback.
## Rollout
1. Merge with the flag **unset** → no behavior change (still polling).
2. Set `coreBlockStreamEnabled=true` on **one** indexer and validate: it
tails live, resumes gap-free on restart, and (induced) stream errors
fall back to polling cleanly.
3. The one open question is whether the prod LB/ingress in front of the
validator passes long-lived h2 gRPC streams — this single-indexer step
is where that gets confirmed. If it doesn't, leave the flag off; polling
is unaffected.
Validated locally with the harness in OpenAudio/go-openaudio#344
(devnet, `--stream`, catch-up + restart-resume).
## Test plan
- [x] `go mod tidy` (only go.mod/go.sum), `go build ./...`, `go vet
./indexer ./config` clean
- [ ] Staging: flip `coreBlockStreamEnabled=true` on one indexer;
confirm live-tail + gap-free restart + fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@schottra@raymondjacobson