Uh oh!
There was an error while loading. Please reload this page.
Conversation
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
…BatchesWhileOffline
…operator-downtime-can-break-eventual-consistency
in order to read the latest assigned err, we need to create a closure
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
uri-99
left a comment
There was a problem hiding this comment.
final detail and i think its ready to approve
Uh oh!
There was an error while loading. Please reload this page.
uri-99
commented
Oct 2, 2024
also add the new .json file to .gitignore to avoid having files with random variables being pushed to the repo |
…operator-downtime-can-break-eventual-consistency
Uh oh!
There was an error while loading. Please reload this page.
Oppen
commented
Oct 3, 2024
OK, seems to work here. |
…operator-downtime-can-break-eventual-consistency
Co-authored-by: Urix <43704209+uri-99@users.noreply.github.com>
Changes
This pr adds a way to recover the operator's tasks from downtime. The way it works now is:
avs_readerfor all the tasks that have not been responded to starting from that block minus100(this is to account for the case that a higher block number task finished first than a previous one, since batches can be processed in parallel). This is to be improved with an algorithm by @Oppen.A new check is also added to not send already responded blocks in the
avs_subscriber'get_latest_task` interval. This problem happened at startup specially, it would always run the latest task doesn't matter how old it was.Test
To make sure everything works, we can run the following test: setup
make anvil_start_with_block,make aggregator_start,make batcher_start_local1:make operator_register_and_start2:make operator_register_and_start CONFIG_FILE=config-files/config-operator-1.yml3:make operator_register_and_start CONFIG_FILE=config-files/config-operator-2.yml4:make operator_register_and_start CONFIG_FILE=config-files/config-operator-3.ymlmake batcher_send_plonk_bls12_381_burst2,3and4make batcher_send_risc0_burst(it is better if you send many batches)2and3again:make operator_start CONFIG_FILE=config-files/config-operator-1.ymlmake operator_start CONFIG_FILE=config-files/config-operator-2.yml4:make operator_start CONFIG_FILE=config-files/config-operator-3.ymlDeploy
Warning
This PR adds the following parameter to the operator configuration
last_processed_batch_filepath: '<>'Advances on #962
Closes#978