drain and fill channel plugin

[m-schmoock]

This plugin offers three new commands: drain, fill and setbalance.
It works similar to the rebalance plugin but with some key differences:

• Does not need a second scid parameter but figures out the other one by routing.
• Can have chunked payments that increases routing probability but can result in partial completion. If cunks is not set, number of required chunks is autodetected.
• Does not consume or calculate msat amounts, but uses percentages. Default 100 for drain and fill and 50 for setbalance.

Note: If draining 100%, the plugin guesses the correct HTLC fee by try and error. For new servers that report HTLC fee via exception (ElementsProject/lightning#2691) it takes the exact amount.

Issues:

• Can run into "route not found" errors afters several "drain"s on multiple channels. Reason yet unknown.

[m-schmoock]

I removed the "work in progress" TAG, as the plugin is very usable aready and I dont have good clues about the remaining TODOs and optimizations.

I could need a review now... volunteers?

Note: If you get "Could not find a route" errors, when draining a channel, try using lightning-cli drain <scid> 100 4 it tells the pluging to drain by 100% using 4 chunks. Im not sure, but it feels we have routing issues since the dijkstra update. Maybe reviewer finds another reason... The "fill" command doesn't have routing issues... Everytime I go through the generated 'exclude' list for the failed drain routing, it seems correctly build...

[darosior]

Here is what caught my eyes (it's mostly stylish) :

• A docstring by function would be useful
• Type hints : you use them for some function, and not for some other, or even for some arguments of a function and not some other.
• Some nits (see below)

I also got an error when trying drain :

Error while processing drain: TypeError('__int__ returned non-int (type float)')

[darosior]

Btw if you think it's mergeable you should remove the WIP tag

[m-schmoock]

@darosior

I removed the WIP flag, still Im going to fix this one way or another, now that we talked about this issues it is at least known. If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

F.y.i. I will have more time on from November, since I reduced my regular job to just 25hrs a week :). This was going to be my first task anyway.

Great ! To work on Lightning ?

Yes !

[darosior]

If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

I don't have one so we'll have to wait for ackbot ;-)

[darosior]

Mostly nits and questions, which most of them will need user testing to be resolved. I think it's mergeable (modulo cleanup and pylightning version ;-))

[m-schmoock]

I updated the documentation a bit to make clear how it is currently supposed to work.

[darosior]

Ok so I have a channel with 60% theirs, 40% ours.

 ├────────┼─────────────┤ 245x2x0

I think the wording is misleading as if I want to drain, say 30 percent of the channel (so that the balance results in 90% theirs, 10% ours)

 ├──┼──────────────────────┤ 245x2x0

I dont

cli drain <scid> 30

But

cli drain <scid> 90

So effectively passing to the drain command the expected percent on their side. That's why I proposed to change the wording to :

cli set_theirs <scid> 90

And, if we keep the drain command, to pass it the expected percent to drain from my side (30%).

The same goes for fill :
Now that they effectively have 90% of the channel capacity,

 ├──┼──────────────────────┤ 245x2x0

and I want my side to have 90% capacity (so passing from ours: 10%; theirs: 90% to ours: 90%; theirs: 10%)

 ├────────────────────┼─┤ 245x2x0

I would expect to drain from them 80% of the capacity (hence fill 80% on my side) but in order to do that with the current wording I have to :

cli fill <scid> 90

That's why I propose to rename it to

cli set_ours <scid> 90

[cdecker]

Sorry for the delay, I seem to have missed the non-WIP window 😉

I'm quite happy to merge the plugin even if it has a edge-case that may fail some times. fwiw I have a c-lightning PR pending that should allow us to start testing plugins, so pinning down the issue and reproducing it will become easier: ElementsProject/lightning#3218

[darosior]

🎉 ! 😁

[m-schmoock]

Hi,

After my first two days of my new 'Altersteilzeit' I managed to get this ready.

@darosior I added the setbalance command and rewrote the drainfill to behave more intuitive. thanks for the feedback. The new command works like a charm.
@cdecker I finally removed the WIP tag. Can we make final reviews and merge?

cheers,
Michael

[m-schmoock]

You guys also might want to test this, but for my setup and testnet and mainnet it works smoothly.

[darosior]

testnet and mainnet it works smoothly.

So you don't have the spontaneous errors you were talking about ?

[m-schmoock]

Still have them from time to time, which is why I think it's not plug-in but daemon related.
It would also be good to know if I am the only one experiencing them...

[darosior]

ACK 2f825de

Tested this on my node, and it behaves exactly as I would expect it to behave. Playing with the chunk feature allows to rebalance channels it was not possible to rebalance previously: this is a really cool plugin ! 🎉

[m-schmoock]

This plugin offers three new commands: drain, fill and setbalance.
It works similar to the rebalance plugin but with some key differences:

• Does not need a second scid parameter but figures out the other one by routing.
• Can have chunked payments that increases routing probability but can result in partial completion. If cunks is not set, number of required chunks is autodetected.
• Does not consume or calculate msat amounts, but uses percentages. Default 100 for drain and fill and 50 for setbalance.

Note: If draining 100%, the plugin guesses the correct HTLC fee by try and error. For new servers that report HTLC fee via exception (ElementsProject/lightning#2691) it takes the exact amount.

Issues:

• Can run into "route not found" errors afters several "drain"s on multiple channels. Reason yet unknown.

[m-schmoock]

I removed the "work in progress" TAG, as the plugin is very usable aready and I dont have good clues about the remaining TODOs and optimizations.

I could need a review now... volunteers?

Note: If you get "Could not find a route" errors, when draining a channel, try using lightning-cli drain <scid> 100 4 it tells the pluging to drain by 100% using 4 chunks. Im not sure, but it feels we have routing issues since the dijkstra update. Maybe reviewer finds another reason... The "fill" command doesn't have routing issues... Everytime I go through the generated 'exclude' list for the failed drain routing, it seems correctly build...

[darosior]

Here is what caught my eyes (it's mostly stylish) :

• A docstring by function would be useful
• Type hints : you use them for some function, and not for some other, or even for some arguments of a function and not some other.
• Some nits (see below)

I also got an error when trying drain :

Error while processing drain: TypeError('__int__ returned non-int (type float)')

[darosior]

Btw if you think it's mergeable you should remove the WIP tag

[m-schmoock]

@darosior

I removed the WIP flag, still Im going to fix this one way or another, now that we talked about this issues it is at least known. If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

F.y.i. I will have more time on from November, since I reduced my regular job to just 25hrs a week :). This was going to be my first task anyway.

Great ! To work on Lightning ?

Yes !

[darosior]

If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

I don't have one so we'll have to wait for ackbot ;-)

[darosior]

Mostly nits and questions, which most of them will need user testing to be resolved. I think it's mergeable (modulo cleanup and pylightning version ;-))

[m-schmoock]

I updated the documentation a bit to make clear how it is currently supposed to work.

[darosior]

Ok so I have a channel with 60% theirs, 40% ours.

 ├────────┼─────────────┤ 245x2x0

I think the wording is misleading as if I want to drain, say 30 percent of the channel (so that the balance results in 90% theirs, 10% ours)

 ├──┼──────────────────────┤ 245x2x0

I dont

cli drain <scid> 30

But

cli drain <scid> 90

So effectively passing to the drain command the expected percent on their side. That's why I proposed to change the wording to :

cli set_theirs <scid> 90

And, if we keep the drain command, to pass it the expected percent to drain from my side (30%).

The same goes for fill :
Now that they effectively have 90% of the channel capacity,

 ├──┼──────────────────────┤ 245x2x0

and I want my side to have 90% capacity (so passing from ours: 10%; theirs: 90% to ours: 90%; theirs: 10%)

 ├────────────────────┼─┤ 245x2x0

I would expect to drain from them 80% of the capacity (hence fill 80% on my side) but in order to do that with the current wording I have to :

cli fill <scid> 90

That's why I propose to rename it to

cli set_ours <scid> 90

[cdecker]

Sorry for the delay, I seem to have missed the non-WIP window 😉

I'm quite happy to merge the plugin even if it has a edge-case that may fail some times. fwiw I have a c-lightning PR pending that should allow us to start testing plugins, so pinning down the issue and reproducing it will become easier: ElementsProject/lightning#3218

[darosior]

🎉 ! 😁

[m-schmoock]

Hi,

After my first two days of my new 'Altersteilzeit' I managed to get this ready.

@darosior I added the setbalance command and rewrote the drainfill to behave more intuitive. thanks for the feedback. The new command works like a charm.
@cdecker I finally removed the WIP tag. Can we make final reviews and merge?

cheers,
Michael

[m-schmoock]

You guys also might want to test this, but for my setup and testnet and mainnet it works smoothly.

[darosior]

testnet and mainnet it works smoothly.

So you don't have the spontaneous errors you were talking about ?

[m-schmoock]

Still have them from time to time, which is why I think it's not plug-in but daemon related.
It would also be good to know if I am the only one experiencing them...

[darosior]

ACK 2f825de

Tested this on my node, and it behaves exactly as I would expect it to behave. Playing with the chunk feature allows to rebalance channels it was not possible to rebalance previously: this is a really cool plugin ! 🎉

[m-schmoock]

This plugin offers three new commands: drain, fill and setbalance.
It works similar to the rebalance plugin but with some key differences:

• Does not need a second scid parameter but figures out the other one by routing.
• Can have chunked payments that increases routing probability but can result in partial completion. If cunks is not set, number of required chunks is autodetected.
• Does not consume or calculate msat amounts, but uses percentages. Default 100 for drain and fill and 50 for setbalance.

Note: If draining 100%, the plugin guesses the correct HTLC fee by try and error. For new servers that report HTLC fee via exception (ElementsProject/lightning#2691) it takes the exact amount.

Issues:

• Can run into "route not found" errors afters several "drain"s on multiple channels. Reason yet unknown.

[m-schmoock]

I removed the "work in progress" TAG, as the plugin is very usable aready and I dont have good clues about the remaining TODOs and optimizations.

I could need a review now... volunteers?

Note: If you get "Could not find a route" errors, when draining a channel, try using lightning-cli drain <scid> 100 4 it tells the pluging to drain by 100% using 4 chunks. Im not sure, but it feels we have routing issues since the dijkstra update. Maybe reviewer finds another reason... The "fill" command doesn't have routing issues... Everytime I go through the generated 'exclude' list for the failed drain routing, it seems correctly build...

[darosior]

Here is what caught my eyes (it's mostly stylish) :

• A docstring by function would be useful
• Type hints : you use them for some function, and not for some other, or even for some arguments of a function and not some other.
• Some nits (see below)

I also got an error when trying drain :

Error while processing drain: TypeError('__int__ returned non-int (type float)')

[darosior]

Btw if you think it's mergeable you should remove the WIP tag

[m-schmoock]

@darosior

I removed the WIP flag, still Im going to fix this one way or another, now that we talked about this issues it is at least known. If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

F.y.i. I will have more time on from November, since I reduced my regular job to just 25hrs a week :). This was going to be my first task anyway.

Great ! To work on Lightning ?

Yes !

[darosior]

If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

I don't have one so we'll have to wait for ackbot ;-)

[darosior]

Mostly nits and questions, which most of them will need user testing to be resolved. I think it's mergeable (modulo cleanup and pylightning version ;-))

[m-schmoock]

I updated the documentation a bit to make clear how it is currently supposed to work.

[darosior]

Ok so I have a channel with 60% theirs, 40% ours.

 ├────────┼─────────────┤ 245x2x0

I think the wording is misleading as if I want to drain, say 30 percent of the channel (so that the balance results in 90% theirs, 10% ours)

 ├──┼──────────────────────┤ 245x2x0

I dont

cli drain <scid> 30

But

cli drain <scid> 90

So effectively passing to the drain command the expected percent on their side. That's why I proposed to change the wording to :

cli set_theirs <scid> 90

And, if we keep the drain command, to pass it the expected percent to drain from my side (30%).

The same goes for fill :
Now that they effectively have 90% of the channel capacity,

 ├──┼──────────────────────┤ 245x2x0

and I want my side to have 90% capacity (so passing from ours: 10%; theirs: 90% to ours: 90%; theirs: 10%)

 ├────────────────────┼─┤ 245x2x0

I would expect to drain from them 80% of the capacity (hence fill 80% on my side) but in order to do that with the current wording I have to :

cli fill <scid> 90

That's why I propose to rename it to

cli set_ours <scid> 90

[cdecker]

Sorry for the delay, I seem to have missed the non-WIP window 😉

I'm quite happy to merge the plugin even if it has a edge-case that may fail some times. fwiw I have a c-lightning PR pending that should allow us to start testing plugins, so pinning down the issue and reproducing it will become easier: ElementsProject/lightning#3218

[darosior]

🎉 ! 😁

[m-schmoock]

Hi,

After my first two days of my new 'Altersteilzeit' I managed to get this ready.

@darosior I added the setbalance command and rewrote the drainfill to behave more intuitive. thanks for the feedback. The new command works like a charm.
@cdecker I finally removed the WIP tag. Can we make final reviews and merge?

cheers,
Michael

[m-schmoock]

You guys also might want to test this, but for my setup and testnet and mainnet it works smoothly.

[darosior]

testnet and mainnet it works smoothly.

So you don't have the spontaneous errors you were talking about ?

[m-schmoock]

Still have them from time to time, which is why I think it's not plug-in but daemon related.
It would also be good to know if I am the only one experiencing them...

[darosior]

ACK 2f825de

Tested this on my node, and it behaves exactly as I would expect it to behave. Playing with the chunk feature allows to rebalance channels it was not possible to rebalance previously: this is a really cool plugin ! 🎉

[m-schmoock]

This plugin offers three new commands: drain, fill and setbalance.
It works similar to the rebalance plugin but with some key differences:

• Does not need a second scid parameter but figures out the other one by routing.
• Can have chunked payments that increases routing probability but can result in partial completion. If cunks is not set, number of required chunks is autodetected.
• Does not consume or calculate msat amounts, but uses percentages. Default 100 for drain and fill and 50 for setbalance.

Note: If draining 100%, the plugin guesses the correct HTLC fee by try and error. For new servers that report HTLC fee via exception (ElementsProject/lightning#2691) it takes the exact amount.

Issues:

• Can run into "route not found" errors afters several "drain"s on multiple channels. Reason yet unknown.

[m-schmoock]

I removed the "work in progress" TAG, as the plugin is very usable aready and I dont have good clues about the remaining TODOs and optimizations.

I could need a review now... volunteers?

Note: If you get "Could not find a route" errors, when draining a channel, try using lightning-cli drain <scid> 100 4 it tells the pluging to drain by 100% using 4 chunks. Im not sure, but it feels we have routing issues since the dijkstra update. Maybe reviewer finds another reason... The "fill" command doesn't have routing issues... Everytime I go through the generated 'exclude' list for the failed drain routing, it seems correctly build...

[darosior]

Here is what caught my eyes (it's mostly stylish) :

• A docstring by function would be useful
• Type hints : you use them for some function, and not for some other, or even for some arguments of a function and not some other.
• Some nits (see below)

I also got an error when trying drain :

Error while processing drain: TypeError('__int__ returned non-int (type float)')

[darosior]

Btw if you think it's mergeable you should remove the WIP tag

[m-schmoock]

@darosior

I removed the WIP flag, still Im going to fix this one way or another, now that we talked about this issues it is at least known. If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

F.y.i. I will have more time on from November, since I reduced my regular job to just 25hrs a week :). This was going to be my first task anyway.

Great ! To work on Lightning ?

Yes !

[darosior]

If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

I don't have one so we'll have to wait for ackbot ;-)

[darosior]

Mostly nits and questions, which most of them will need user testing to be resolved. I think it's mergeable (modulo cleanup and pylightning version ;-))

[m-schmoock]

I updated the documentation a bit to make clear how it is currently supposed to work.

[darosior]

Ok so I have a channel with 60% theirs, 40% ours.

 ├────────┼─────────────┤ 245x2x0

I think the wording is misleading as if I want to drain, say 30 percent of the channel (so that the balance results in 90% theirs, 10% ours)

 ├──┼──────────────────────┤ 245x2x0

I dont

cli drain <scid> 30

But

cli drain <scid> 90

So effectively passing to the drain command the expected percent on their side. That's why I proposed to change the wording to :

cli set_theirs <scid> 90

And, if we keep the drain command, to pass it the expected percent to drain from my side (30%).

The same goes for fill :
Now that they effectively have 90% of the channel capacity,

 ├──┼──────────────────────┤ 245x2x0

and I want my side to have 90% capacity (so passing from ours: 10%; theirs: 90% to ours: 90%; theirs: 10%)

 ├────────────────────┼─┤ 245x2x0

I would expect to drain from them 80% of the capacity (hence fill 80% on my side) but in order to do that with the current wording I have to :

cli fill <scid> 90

That's why I propose to rename it to

cli set_ours <scid> 90

[cdecker]

Sorry for the delay, I seem to have missed the non-WIP window 😉

I'm quite happy to merge the plugin even if it has a edge-case that may fail some times. fwiw I have a c-lightning PR pending that should allow us to start testing plugins, so pinning down the issue and reproducing it will become easier: ElementsProject/lightning#3218

[darosior]

🎉 ! 😁

[m-schmoock]

Hi,

After my first two days of my new 'Altersteilzeit' I managed to get this ready.

@darosior I added the setbalance command and rewrote the drainfill to behave more intuitive. thanks for the feedback. The new command works like a charm.
@cdecker I finally removed the WIP tag. Can we make final reviews and merge?

cheers,
Michael

[m-schmoock]

You guys also might want to test this, but for my setup and testnet and mainnet it works smoothly.

[darosior]

testnet and mainnet it works smoothly.

So you don't have the spontaneous errors you were talking about ?

[m-schmoock]

Still have them from time to time, which is why I think it's not plug-in but daemon related.
It would also be good to know if I am the only one experiencing them...

[darosior]

ACK 2f825de

Tested this on my node, and it behaves exactly as I would expect it to behave. Playing with the chunk feature allows to rebalance channels it was not possible to rebalance previously: this is a really cool plugin ! 🎉

[m-schmoock]

This plugin offers three new commands: drain, fill and setbalance.
It works similar to the rebalance plugin but with some key differences:

• Does not need a second scid parameter but figures out the other one by routing.
• Can have chunked payments that increases routing probability but can result in partial completion. If cunks is not set, number of required chunks is autodetected.
• Does not consume or calculate msat amounts, but uses percentages. Default 100 for drain and fill and 50 for setbalance.

Note: If draining 100%, the plugin guesses the correct HTLC fee by try and error. For new servers that report HTLC fee via exception (ElementsProject/lightning#2691) it takes the exact amount.

Issues:

• Can run into "route not found" errors afters several "drain"s on multiple channels. Reason yet unknown.

[m-schmoock]

I removed the "work in progress" TAG, as the plugin is very usable aready and I dont have good clues about the remaining TODOs and optimizations.

I could need a review now... volunteers?

Note: If you get "Could not find a route" errors, when draining a channel, try using lightning-cli drain <scid> 100 4 it tells the pluging to drain by 100% using 4 chunks. Im not sure, but it feels we have routing issues since the dijkstra update. Maybe reviewer finds another reason... The "fill" command doesn't have routing issues... Everytime I go through the generated 'exclude' list for the failed drain routing, it seems correctly build...

[darosior]

Here is what caught my eyes (it's mostly stylish) :

• A docstring by function would be useful
• Type hints : you use them for some function, and not for some other, or even for some arguments of a function and not some other.
• Some nits (see below)

I also got an error when trying drain :

Error while processing drain: TypeError('__int__ returned non-int (type float)')

[darosior]

Btw if you think it's mergeable you should remove the WIP tag

[m-schmoock]

@darosior

I removed the WIP flag, still Im going to fix this one way or another, now that we talked about this issues it is at least known. If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

F.y.i. I will have more time on from November, since I reduced my regular job to just 25hrs a week :). This was going to be my first task anyway.

Great ! To work on Lightning ?

Yes !

[darosior]

If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

I don't have one so we'll have to wait for ackbot ;-)

[darosior]

Mostly nits and questions, which most of them will need user testing to be resolved. I think it's mergeable (modulo cleanup and pylightning version ;-))

[m-schmoock]

I updated the documentation a bit to make clear how it is currently supposed to work.

[darosior]

Ok so I have a channel with 60% theirs, 40% ours.

 ├────────┼─────────────┤ 245x2x0

I think the wording is misleading as if I want to drain, say 30 percent of the channel (so that the balance results in 90% theirs, 10% ours)

 ├──┼──────────────────────┤ 245x2x0

I dont

cli drain <scid> 30

But

cli drain <scid> 90

So effectively passing to the drain command the expected percent on their side. That's why I proposed to change the wording to :

cli set_theirs <scid> 90

And, if we keep the drain command, to pass it the expected percent to drain from my side (30%).

The same goes for fill :
Now that they effectively have 90% of the channel capacity,

 ├──┼──────────────────────┤ 245x2x0

and I want my side to have 90% capacity (so passing from ours: 10%; theirs: 90% to ours: 90%; theirs: 10%)

 ├────────────────────┼─┤ 245x2x0

I would expect to drain from them 80% of the capacity (hence fill 80% on my side) but in order to do that with the current wording I have to :

cli fill <scid> 90

That's why I propose to rename it to

cli set_ours <scid> 90

[cdecker]

Sorry for the delay, I seem to have missed the non-WIP window 😉

I'm quite happy to merge the plugin even if it has a edge-case that may fail some times. fwiw I have a c-lightning PR pending that should allow us to start testing plugins, so pinning down the issue and reproducing it will become easier: ElementsProject/lightning#3218

[darosior]

🎉 ! 😁

[m-schmoock]

Hi,

After my first two days of my new 'Altersteilzeit' I managed to get this ready.

@darosior I added the setbalance command and rewrote the drainfill to behave more intuitive. thanks for the feedback. The new command works like a charm.
@cdecker I finally removed the WIP tag. Can we make final reviews and merge?

cheers,
Michael

[m-schmoock]

You guys also might want to test this, but for my setup and testnet and mainnet it works smoothly.

[darosior]

testnet and mainnet it works smoothly.

So you don't have the spontaneous errors you were talking about ?

[m-schmoock]

Still have them from time to time, which is why I think it's not plug-in but daemon related.
It would also be good to know if I am the only one experiencing them...

[darosior]

ACK 2f825de

Tested this on my node, and it behaves exactly as I would expect it to behave. Playing with the chunk feature allows to rebalance channels it was not possible to rebalance previously: this is a really cool plugin ! 🎉

[m-schmoock]

This plugin offers three new commands: drain, fill and setbalance.
It works similar to the rebalance plugin but with some key differences:

• Does not need a second scid parameter but figures out the other one by routing.
• Can have chunked payments that increases routing probability but can result in partial completion. If cunks is not set, number of required chunks is autodetected.
• Does not consume or calculate msat amounts, but uses percentages. Default 100 for drain and fill and 50 for setbalance.

Note: If draining 100%, the plugin guesses the correct HTLC fee by try and error. For new servers that report HTLC fee via exception (ElementsProject/lightning#2691) it takes the exact amount.

Issues:

• Can run into "route not found" errors afters several "drain"s on multiple channels. Reason yet unknown.

[m-schmoock]

I removed the "work in progress" TAG, as the plugin is very usable aready and I dont have good clues about the remaining TODOs and optimizations.

I could need a review now... volunteers?

Note: If you get "Could not find a route" errors, when draining a channel, try using lightning-cli drain <scid> 100 4 it tells the pluging to drain by 100% using 4 chunks. Im not sure, but it feels we have routing issues since the dijkstra update. Maybe reviewer finds another reason... The "fill" command doesn't have routing issues... Everytime I go through the generated 'exclude' list for the failed drain routing, it seems correctly build...

[darosior]

Here is what caught my eyes (it's mostly stylish) :

• A docstring by function would be useful
• Type hints : you use them for some function, and not for some other, or even for some arguments of a function and not some other.
• Some nits (see below)

I also got an error when trying drain :

Error while processing drain: TypeError('__int__ returned non-int (type float)')

[darosior]

Btw if you think it's mergeable you should remove the WIP tag

[m-schmoock]

@darosior

I removed the WIP flag, still Im going to fix this one way or another, now that we talked about this issues it is at least known. If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

F.y.i. I will have more time on from November, since I reduced my regular job to just 25hrs a week :). This was going to be my first task anyway.

Great ! To work on Lightning ?

Yes !

[darosior]

If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

I don't have one so we'll have to wait for ackbot ;-)

[darosior]

Mostly nits and questions, which most of them will need user testing to be resolved. I think it's mergeable (modulo cleanup and pylightning version ;-))

[m-schmoock]

I updated the documentation a bit to make clear how it is currently supposed to work.

[darosior]

Ok so I have a channel with 60% theirs, 40% ours.

 ├────────┼─────────────┤ 245x2x0

I think the wording is misleading as if I want to drain, say 30 percent of the channel (so that the balance results in 90% theirs, 10% ours)

 ├──┼──────────────────────┤ 245x2x0

I dont

cli drain <scid> 30

But

cli drain <scid> 90

So effectively passing to the drain command the expected percent on their side. That's why I proposed to change the wording to :

cli set_theirs <scid> 90

And, if we keep the drain command, to pass it the expected percent to drain from my side (30%).

The same goes for fill :
Now that they effectively have 90% of the channel capacity,

 ├──┼──────────────────────┤ 245x2x0

and I want my side to have 90% capacity (so passing from ours: 10%; theirs: 90% to ours: 90%; theirs: 10%)

 ├────────────────────┼─┤ 245x2x0

I would expect to drain from them 80% of the capacity (hence fill 80% on my side) but in order to do that with the current wording I have to :

cli fill <scid> 90

That's why I propose to rename it to

cli set_ours <scid> 90

[cdecker]

Sorry for the delay, I seem to have missed the non-WIP window 😉

I'm quite happy to merge the plugin even if it has a edge-case that may fail some times. fwiw I have a c-lightning PR pending that should allow us to start testing plugins, so pinning down the issue and reproducing it will become easier: ElementsProject/lightning#3218

[darosior]

🎉 ! 😁

[m-schmoock]

Hi,

After my first two days of my new 'Altersteilzeit' I managed to get this ready.

@darosior I added the setbalance command and rewrote the drainfill to behave more intuitive. thanks for the feedback. The new command works like a charm.
@cdecker I finally removed the WIP tag. Can we make final reviews and merge?

cheers,
Michael

[m-schmoock]

You guys also might want to test this, but for my setup and testnet and mainnet it works smoothly.

[darosior]

testnet and mainnet it works smoothly.

So you don't have the spontaneous errors you were talking about ?

[m-schmoock]

Still have them from time to time, which is why I think it's not plug-in but daemon related.
It would also be good to know if I am the only one experiencing them...

[darosior]

ACK 2f825de

Tested this on my node, and it behaves exactly as I would expect it to behave. Playing with the chunk feature allows to rebalance channels it was not possible to rebalance previously: this is a really cool plugin ! 🎉

[m-schmoock]

This plugin offers three new commands: drain, fill and setbalance.
It works similar to the rebalance plugin but with some key differences:

• Does not need a second scid parameter but figures out the other one by routing.
• Can have chunked payments that increases routing probability but can result in partial completion. If cunks is not set, number of required chunks is autodetected.
• Does not consume or calculate msat amounts, but uses percentages. Default 100 for drain and fill and 50 for setbalance.

Note: If draining 100%, the plugin guesses the correct HTLC fee by try and error. For new servers that report HTLC fee via exception (ElementsProject/lightning#2691) it takes the exact amount.

Issues:

• Can run into "route not found" errors afters several "drain"s on multiple channels. Reason yet unknown.

[m-schmoock]

I removed the "work in progress" TAG, as the plugin is very usable aready and I dont have good clues about the remaining TODOs and optimizations.

I could need a review now... volunteers?

Note: If you get "Could not find a route" errors, when draining a channel, try using lightning-cli drain <scid> 100 4 it tells the pluging to drain by 100% using 4 chunks. Im not sure, but it feels we have routing issues since the dijkstra update. Maybe reviewer finds another reason... The "fill" command doesn't have routing issues... Everytime I go through the generated 'exclude' list for the failed drain routing, it seems correctly build...

[darosior]

Here is what caught my eyes (it's mostly stylish) :

• A docstring by function would be useful
• Type hints : you use them for some function, and not for some other, or even for some arguments of a function and not some other.
• Some nits (see below)

I also got an error when trying drain :

Error while processing drain: TypeError('__int__ returned non-int (type float)')

[darosior]

Btw if you think it's mergeable you should remove the WIP tag

[m-schmoock]

@darosior

I removed the WIP flag, still Im going to fix this one way or another, now that we talked about this issues it is at least known. If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

F.y.i. I will have more time on from November, since I reduced my regular job to just 25hrs a week :). This was going to be my first task anyway.

Great ! To work on Lightning ?

Yes !

[darosior]

If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

I don't have one so we'll have to wait for ackbot ;-)

[darosior]

Mostly nits and questions, which most of them will need user testing to be resolved. I think it's mergeable (modulo cleanup and pylightning version ;-))

[m-schmoock]

I updated the documentation a bit to make clear how it is currently supposed to work.

[darosior]

Ok so I have a channel with 60% theirs, 40% ours.

 ├────────┼─────────────┤ 245x2x0

I think the wording is misleading as if I want to drain, say 30 percent of the channel (so that the balance results in 90% theirs, 10% ours)

 ├──┼──────────────────────┤ 245x2x0

I dont

cli drain <scid> 30

But

cli drain <scid> 90

So effectively passing to the drain command the expected percent on their side. That's why I proposed to change the wording to :

cli set_theirs <scid> 90

And, if we keep the drain command, to pass it the expected percent to drain from my side (30%).

The same goes for fill :
Now that they effectively have 90% of the channel capacity,

 ├──┼──────────────────────┤ 245x2x0

and I want my side to have 90% capacity (so passing from ours: 10%; theirs: 90% to ours: 90%; theirs: 10%)

 ├────────────────────┼─┤ 245x2x0

I would expect to drain from them 80% of the capacity (hence fill 80% on my side) but in order to do that with the current wording I have to :

cli fill <scid> 90

That's why I propose to rename it to

cli set_ours <scid> 90

[cdecker]

Sorry for the delay, I seem to have missed the non-WIP window 😉

I'm quite happy to merge the plugin even if it has a edge-case that may fail some times. fwiw I have a c-lightning PR pending that should allow us to start testing plugins, so pinning down the issue and reproducing it will become easier: ElementsProject/lightning#3218

[darosior]

🎉 ! 😁

[m-schmoock]

Hi,

After my first two days of my new 'Altersteilzeit' I managed to get this ready.

@darosior I added the setbalance command and rewrote the drainfill to behave more intuitive. thanks for the feedback. The new command works like a charm.
@cdecker I finally removed the WIP tag. Can we make final reviews and merge?

cheers,
Michael

[m-schmoock]

You guys also might want to test this, but for my setup and testnet and mainnet it works smoothly.

[darosior]

testnet and mainnet it works smoothly.

So you don't have the spontaneous errors you were talking about ?

[m-schmoock]

Still have them from time to time, which is why I think it's not plug-in but daemon related.
It would also be good to know if I am the only one experiencing them...

[darosior]

ACK 2f825de

Tested this on my node, and it behaves exactly as I would expect it to behave. Playing with the chunk feature allows to rebalance channels it was not possible to rebalance previously: this is a really cool plugin ! 🎉

[m-schmoock]

This plugin offers three new commands: drain, fill and setbalance.
It works similar to the rebalance plugin but with some key differences:

• Does not need a second scid parameter but figures out the other one by routing.
• Can have chunked payments that increases routing probability but can result in partial completion. If cunks is not set, number of required chunks is autodetected.
• Does not consume or calculate msat amounts, but uses percentages. Default 100 for drain and fill and 50 for setbalance.

Note: If draining 100%, the plugin guesses the correct HTLC fee by try and error. For new servers that report HTLC fee via exception (ElementsProject/lightning#2691) it takes the exact amount.

Issues:

• Can run into "route not found" errors afters several "drain"s on multiple channels. Reason yet unknown.

[m-schmoock]

I removed the "work in progress" TAG, as the plugin is very usable aready and I dont have good clues about the remaining TODOs and optimizations.

I could need a review now... volunteers?

Note: If you get "Could not find a route" errors, when draining a channel, try using lightning-cli drain <scid> 100 4 it tells the pluging to drain by 100% using 4 chunks. Im not sure, but it feels we have routing issues since the dijkstra update. Maybe reviewer finds another reason... The "fill" command doesn't have routing issues... Everytime I go through the generated 'exclude' list for the failed drain routing, it seems correctly build...

[darosior]

Here is what caught my eyes (it's mostly stylish) :

• A docstring by function would be useful
• Type hints : you use them for some function, and not for some other, or even for some arguments of a function and not some other.
• Some nits (see below)

I also got an error when trying drain :

Error while processing drain: TypeError('__int__ returned non-int (type float)')

[darosior]

Btw if you think it's mergeable you should remove the WIP tag

[m-schmoock]

@darosior

I removed the WIP flag, still Im going to fix this one way or another, now that we talked about this issues it is at least known. If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

F.y.i. I will have more time on from November, since I reduced my regular job to just 25hrs a week :). This was going to be my first task anyway.

Great ! To work on Lightning ?

Yes !

[darosior]

If you are fine we can merge, I lost my commit access because I have been inactive for a while, so we need someone else to merge it.

I don't have one so we'll have to wait for ackbot ;-)

[darosior]

Mostly nits and questions, which most of them will need user testing to be resolved. I think it's mergeable (modulo cleanup and pylightning version ;-))

[m-schmoock]

I updated the documentation a bit to make clear how it is currently supposed to work.

[darosior]

Ok so I have a channel with 60% theirs, 40% ours.

 ├────────┼─────────────┤ 245x2x0

I think the wording is misleading as if I want to drain, say 30 percent of the channel (so that the balance results in 90% theirs, 10% ours)

 ├──┼──────────────────────┤ 245x2x0

I dont

cli drain <scid> 30

But

cli drain <scid> 90

So effectively passing to the drain command the expected percent on their side. That's why I proposed to change the wording to :

cli set_theirs <scid> 90

And, if we keep the drain command, to pass it the expected percent to drain from my side (30%).

The same goes for fill :
Now that they effectively have 90% of the channel capacity,

 ├──┼──────────────────────┤ 245x2x0

and I want my side to have 90% capacity (so passing from ours: 10%; theirs: 90% to ours: 90%; theirs: 10%)

 ├────────────────────┼─┤ 245x2x0

I would expect to drain from them 80% of the capacity (hence fill 80% on my side) but in order to do that with the current wording I have to :

cli fill <scid> 90

That's why I propose to rename it to

cli set_ours <scid> 90

[cdecker]

Sorry for the delay, I seem to have missed the non-WIP window 😉

I'm quite happy to merge the plugin even if it has a edge-case that may fail some times. fwiw I have a c-lightning PR pending that should allow us to start testing plugins, so pinning down the issue and reproducing it will become easier: ElementsProject/lightning#3218

[darosior]

🎉 ! 😁

[m-schmoock]

Hi,

After my first two days of my new 'Altersteilzeit' I managed to get this ready.

@darosior I added the setbalance command and rewrote the drainfill to behave more intuitive. thanks for the feedback. The new command works like a charm.
@cdecker I finally removed the WIP tag. Can we make final reviews and merge?

cheers,
Michael

[m-schmoock]

You guys also might want to test this, but for my setup and testnet and mainnet it works smoothly.

[darosior]

testnet and mainnet it works smoothly.

So you don't have the spontaneous errors you were talking about ?

[m-schmoock]

Still have them from time to time, which is why I think it's not plug-in but daemon related.
It would also be good to know if I am the only one experiencing them...

[darosior]

ACK 2f825de

Tested this on my node, and it behaves exactly as I would expect it to behave. Playing with the chunk feature allows to rebalance channels it was not possible to rebalance previously: this is a really cool plugin ! 🎉