feat: aggregator bump fee if transaction was not included

[MarcosNicolau]

Before merging this pr, pr #1304 should be merged first, as it depends on the retry logic.

Changes

Bumps the gasPrice in the aggregator when sending a responded task:

• After a timeout period the aggregator will send a new ReponseToTask with a higher bumped gas price.
• Configurations fields have been added to easily change the behavior:
  • gas_base_bump_percentage: how much to bump the gas price. Configured to 10%
  • gas_bump_incremental_percentage: an extra percentage to bump every retry i*5. Configured to 2%
  • time_to_wait_before_bump: the time to wait for the receipt once we sent the tx. Configured to 36s
• Traces for tasks: a new endpoint in the telemetry to log when the gas price is bumped

• Time series in grafana: shows the number of times the gas was bumped when sending task response across time.

---

Considerations:

For every retry, the gas price is bumped a 10% + 2% * i, where i is the iteration number. i is only incremented when the receipt timeout has passed. That is, if the rpc node fails the fee i won't be incremented. This will run infinitely until the transaction gets accepted unless the checkRespondToTaskFeeLimit fails. In that case, we stop retrying and leave the tx in the mempool. Issue #1389 has been created to cover this case.

Another note is that waitForTransactionReceipt has been refactored. Now it does not have a maxTries anymore (set to 0) and instead receives a new parameter waitTimeout which defines the retries based on a timeout. Also, the maxInterval has been modified to 2 seconds because we can't reliably measure the specific time the tx will be included in a block. Setting a higher value will imply doing fewer retries across the waitTimeout so we might lose the receipt.

Test

Setup a local testnet as usual and introduce the following lines in the avs_writer.go file:

125: ifi<2 {
126: i++127: returnnil, fmt.Errorf("error")
130: }

You should check through the aggregator logs, telemetry traces, and grafana charts, that the gas price was indeed bumped.

[JulianVentura]

Looks good and works on my machine

[uri-99]

There is something odd with the bumper, i added the lines of code in the description to make it fail, it bumped correctly but then skyrocketed on because it had no delay?

[MarcosNicolau]

There is something odd with the bumper, i added the lines of code in the description to make it fail, it bumped correctly but then skyrocketed on because it had no delay?

I tried to reproduce the behavior but I couldn't. Would you try again with the latest changes pls.

[avilagaston9]

Maybe for a separate PR, but we need to add some numbers to graph the retry logic. See:

aligned_layer/batcher/aligned-batcher/src/lib.rs

Line 1422 in b03a625

pubasyncfncancel_create_new_task_tx(&self,old_tx_gas_price:U256){

[avilagaston9]

Suggested change

	// Incremental percentage increase for each retry attempt (i*5%)
	// Incremental percentage increase for each retry attempt (i*2%)

[avilagaston9]

LGTM! Left some nit comments

[avilagaston9]

We should add comments explaining how we are using this function. We are using different bump percentages depending on which price that we are bumping.