Uh oh!
There was an error while loading. Please reload this page.
Add Kinesis Data Streams trigger - #71135
Conversation
aaron-y-chen
commented
Aug 5, 2026
Those failures seem to be caused by the same issue: There is already a PR to fix it: #71118 |
4e1733b to
d7e4df5Compareaaron-y-chen
commented
Aug 5, 2026
Waiting for another PR #71145 to be merged. |
d7e4df5 to
5589794Compareaaron-y-chen
commented
Aug 6, 2026
Okay, all CI failures have been fixed. |
Uh oh!
There was an error while loading. Please reload this page.
aaron-y-chen
commented
Aug 17, 2026
Hi @jason810496, I'd appreciate it if you could take a look when you have time. :) |
vincbeck
commented
Aug 17, 2026
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
aaron-y-chen
commented
Aug 19, 2026
Thanks for the reviews, I believe this version is more concrete than before :) |
jason810496
left a comment
There was a problem hiding this comment.
Thanks for adding the new trigger to compelete #52712.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
amoghrajesh
left a comment
There was a problem hiding this comment.
Blocking due to sync access to asset state store
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
…and update checkpoint methods to be asynchronous
related: #52712
This is the first PR for Kinesis part of #52712, I will integrate the trigger with the common-messaging interface and add a system/e2e test in the next PR.
Summary
Add an asynchronous Kinesis Data Streams trigger for consuming records without occupying worker slots.
The trigger polls all shards, emits JSON-serializable record batches, handles iterator expiry, throttling, pagination, and in-run resharding, and checkpoints per-shard sequence numbers when an asset state store is available. New shards first discovered after a restart honor the configured initial position, so
LATESTmay skip records written during the downtime.This is the provider-side foundation for the common-messaging integration tracked in #52712.
Was generative AI tooling used to co-author this PR?
Generated-by: [GPT 5.6-sol] following the guidelines