Auto deploy regtest nodes in background

[rajarshimaitra]

Description

Fixes#55. I decided to go for one feature flag for each backends as they made dependency management easier. Because different backend will have different dependency, and we don't wanna have redundant ones. Also it makes the commands a little shorter.

Notes to the reviewers

Right now its not that useful because the backend APIs are not exposed. So I can't really do much other than sync/send in the command line. (unlike testnet we cannot send coins to bdk from outside of local regtest node). So to make it useful in repl mode as described in the issue, we also have to expose the blockchain apis in bdk-cli command in some way (at least the basic stuffs, address, send, generate). Looking for suggestions on this.

usage looks like this:

cargo run --features regtest-rpc -- wallet --descriptor "wpkh(tpubEBr4i6yk5nf5DAaJpsi9N2pPYBeJ7fZ5Z9rmN4977iYLCGco1VyjB9tvvuvYtfZzjD5A8igzgw3HeWeeKFmanHYqksqZXYXGsw5zjnj7KM9/*)" sync
{}

Although I am having trouble connecting to the esplora end of electrs. Not sure whats the issue, will have to look more.

Checklists

All Submissions:

• I've signed all my commits
• I followed the contribution guidelines
• I ran cargo fmt and cargo clippy before committing

New Features:

• I've added tests for the new feature
• I've added docs for the new feature
• I've updated CHANGELOG.md

[rajarshimaitra]

Fixed merge conflict with some enhancements.

[rajarshimaitra]

Updated the commit history for better structuring.

[notmandatory]

@rajarshimaitra Looks like this one needs a rebase or if you're tied up I can do the rebase but would have to rewrite your commits. Once that's done I'll do some testing and see if I can figure out how to expose the core rpc in repl mode.

[rajarshimaitra]

Rebased and resolved conflicts..

[notmandatory]

I spent some time today testing and reviewing and this is a great start! There are still the two (related) issues you found:

• 1. no way to control the bitcoind node (ie. create new blocks, get regtest coins)
• 2. sync doesn't work for esplora

I believe issue 2 is because electrsd won't sync until at least one block is created by bitcoind; so if we fix 1 we should also fix 2. My idea for fixing 1 is to change the bitcoind lib to allow us to set a reusable (not randomly generated tmp) directory for the bitcoind data dir; and maybe the same for electrsd. Then we can add some simple RPC commands to bdk-cli to update the bitcoind (and electrsd) database and that data will persist for future invocations of bdk-cli.

But for now as long as we add the comment that the regtest-* features are experimental this should be OK to merge. Then we have more time to to play with and get merged any PRs to bitcoind (and electrsd) that we need to have it all working end to end.

[notmandatory]

Add some little experimental warnings and this is good to go.

[rajarshimaitra]

Thanks @notmandatory .. Updated as per suggested and rebased..

[rajarshimaitra]

Rebased on latest master.

[notmandatory]

Also need to do a cargo fmt.

[rajarshimaitra]

• Reverted back default network to testnet
• Fixed test failure

[notmandatory]

ACK 930b0c4

[rajarshimaitra]

Description

Fixes#55. I decided to go for one feature flag for each backends as they made dependency management easier. Because different backend will have different dependency, and we don't wanna have redundant ones. Also it makes the commands a little shorter.

Notes to the reviewers

Right now its not that useful because the backend APIs are not exposed. So I can't really do much other than sync/send in the command line. (unlike testnet we cannot send coins to bdk from outside of local regtest node). So to make it useful in repl mode as described in the issue, we also have to expose the blockchain apis in bdk-cli command in some way (at least the basic stuffs, address, send, generate). Looking for suggestions on this.

usage looks like this:

cargo run --features regtest-rpc -- wallet --descriptor "wpkh(tpubEBr4i6yk5nf5DAaJpsi9N2pPYBeJ7fZ5Z9rmN4977iYLCGco1VyjB9tvvuvYtfZzjD5A8igzgw3HeWeeKFmanHYqksqZXYXGsw5zjnj7KM9/*)" sync
{}

Although I am having trouble connecting to the esplora end of electrs. Not sure whats the issue, will have to look more.

Checklists

All Submissions:

• I've signed all my commits
• I followed the contribution guidelines
• I ran cargo fmt and cargo clippy before committing

New Features:

• I've added tests for the new feature
• I've added docs for the new feature
• I've updated CHANGELOG.md

[rajarshimaitra]

Fixed merge conflict with some enhancements.

[rajarshimaitra]

Updated the commit history for better structuring.

[notmandatory]

@rajarshimaitra Looks like this one needs a rebase or if you're tied up I can do the rebase but would have to rewrite your commits. Once that's done I'll do some testing and see if I can figure out how to expose the core rpc in repl mode.

[rajarshimaitra]

Rebased and resolved conflicts..

[notmandatory]

I spent some time today testing and reviewing and this is a great start! There are still the two (related) issues you found:

• 1. no way to control the bitcoind node (ie. create new blocks, get regtest coins)
• 2. sync doesn't work for esplora

I believe issue 2 is because electrsd won't sync until at least one block is created by bitcoind; so if we fix 1 we should also fix 2. My idea for fixing 1 is to change the bitcoind lib to allow us to set a reusable (not randomly generated tmp) directory for the bitcoind data dir; and maybe the same for electrsd. Then we can add some simple RPC commands to bdk-cli to update the bitcoind (and electrsd) database and that data will persist for future invocations of bdk-cli.

But for now as long as we add the comment that the regtest-* features are experimental this should be OK to merge. Then we have more time to to play with and get merged any PRs to bitcoind (and electrsd) that we need to have it all working end to end.

[notmandatory]

Add some little experimental warnings and this is good to go.

[rajarshimaitra]

Thanks @notmandatory .. Updated as per suggested and rebased..

[rajarshimaitra]

Rebased on latest master.

[notmandatory]

Also need to do a cargo fmt.

[rajarshimaitra]

• Reverted back default network to testnet
• Fixed test failure

[notmandatory]

ACK 930b0c4

[rajarshimaitra]

Description

Fixes#55. I decided to go for one feature flag for each backends as they made dependency management easier. Because different backend will have different dependency, and we don't wanna have redundant ones. Also it makes the commands a little shorter.

Notes to the reviewers

Right now its not that useful because the backend APIs are not exposed. So I can't really do much other than sync/send in the command line. (unlike testnet we cannot send coins to bdk from outside of local regtest node). So to make it useful in repl mode as described in the issue, we also have to expose the blockchain apis in bdk-cli command in some way (at least the basic stuffs, address, send, generate). Looking for suggestions on this.

usage looks like this:

cargo run --features regtest-rpc -- wallet --descriptor "wpkh(tpubEBr4i6yk5nf5DAaJpsi9N2pPYBeJ7fZ5Z9rmN4977iYLCGco1VyjB9tvvuvYtfZzjD5A8igzgw3HeWeeKFmanHYqksqZXYXGsw5zjnj7KM9/*)" sync
{}

Although I am having trouble connecting to the esplora end of electrs. Not sure whats the issue, will have to look more.

Checklists

All Submissions:

• I've signed all my commits
• I followed the contribution guidelines
• I ran cargo fmt and cargo clippy before committing

New Features:

• I've added tests for the new feature
• I've added docs for the new feature
• I've updated CHANGELOG.md

[rajarshimaitra]

Fixed merge conflict with some enhancements.

[rajarshimaitra]

Updated the commit history for better structuring.

[notmandatory]

@rajarshimaitra Looks like this one needs a rebase or if you're tied up I can do the rebase but would have to rewrite your commits. Once that's done I'll do some testing and see if I can figure out how to expose the core rpc in repl mode.

[rajarshimaitra]

Rebased and resolved conflicts..

[notmandatory]

I spent some time today testing and reviewing and this is a great start! There are still the two (related) issues you found:

• 1. no way to control the bitcoind node (ie. create new blocks, get regtest coins)
• 2. sync doesn't work for esplora

I believe issue 2 is because electrsd won't sync until at least one block is created by bitcoind; so if we fix 1 we should also fix 2. My idea for fixing 1 is to change the bitcoind lib to allow us to set a reusable (not randomly generated tmp) directory for the bitcoind data dir; and maybe the same for electrsd. Then we can add some simple RPC commands to bdk-cli to update the bitcoind (and electrsd) database and that data will persist for future invocations of bdk-cli.

But for now as long as we add the comment that the regtest-* features are experimental this should be OK to merge. Then we have more time to to play with and get merged any PRs to bitcoind (and electrsd) that we need to have it all working end to end.

[notmandatory]

Add some little experimental warnings and this is good to go.

[rajarshimaitra]

Thanks @notmandatory .. Updated as per suggested and rebased..

[rajarshimaitra]

Rebased on latest master.

[notmandatory]

Also need to do a cargo fmt.

[rajarshimaitra]

• Reverted back default network to testnet
• Fixed test failure

[notmandatory]

ACK 930b0c4

[rajarshimaitra]

Description

Fixes#55. I decided to go for one feature flag for each backends as they made dependency management easier. Because different backend will have different dependency, and we don't wanna have redundant ones. Also it makes the commands a little shorter.

Notes to the reviewers

Right now its not that useful because the backend APIs are not exposed. So I can't really do much other than sync/send in the command line. (unlike testnet we cannot send coins to bdk from outside of local regtest node). So to make it useful in repl mode as described in the issue, we also have to expose the blockchain apis in bdk-cli command in some way (at least the basic stuffs, address, send, generate). Looking for suggestions on this.

usage looks like this:

cargo run --features regtest-rpc -- wallet --descriptor "wpkh(tpubEBr4i6yk5nf5DAaJpsi9N2pPYBeJ7fZ5Z9rmN4977iYLCGco1VyjB9tvvuvYtfZzjD5A8igzgw3HeWeeKFmanHYqksqZXYXGsw5zjnj7KM9/*)" sync
{}

Although I am having trouble connecting to the esplora end of electrs. Not sure whats the issue, will have to look more.

Checklists

All Submissions:

• I've signed all my commits
• I followed the contribution guidelines
• I ran cargo fmt and cargo clippy before committing

New Features:

• I've added tests for the new feature
• I've added docs for the new feature
• I've updated CHANGELOG.md

[rajarshimaitra]

Fixed merge conflict with some enhancements.

[rajarshimaitra]

Updated the commit history for better structuring.

[notmandatory]

@rajarshimaitra Looks like this one needs a rebase or if you're tied up I can do the rebase but would have to rewrite your commits. Once that's done I'll do some testing and see if I can figure out how to expose the core rpc in repl mode.

[rajarshimaitra]

Rebased and resolved conflicts..

[notmandatory]

I spent some time today testing and reviewing and this is a great start! There are still the two (related) issues you found:

• 1. no way to control the bitcoind node (ie. create new blocks, get regtest coins)
• 2. sync doesn't work for esplora

I believe issue 2 is because electrsd won't sync until at least one block is created by bitcoind; so if we fix 1 we should also fix 2. My idea for fixing 1 is to change the bitcoind lib to allow us to set a reusable (not randomly generated tmp) directory for the bitcoind data dir; and maybe the same for electrsd. Then we can add some simple RPC commands to bdk-cli to update the bitcoind (and electrsd) database and that data will persist for future invocations of bdk-cli.

But for now as long as we add the comment that the regtest-* features are experimental this should be OK to merge. Then we have more time to to play with and get merged any PRs to bitcoind (and electrsd) that we need to have it all working end to end.

[notmandatory]

Add some little experimental warnings and this is good to go.

[rajarshimaitra]

Thanks @notmandatory .. Updated as per suggested and rebased..

[rajarshimaitra]

Rebased on latest master.

[notmandatory]

Also need to do a cargo fmt.

[rajarshimaitra]

• Reverted back default network to testnet
• Fixed test failure

[notmandatory]

ACK 930b0c4

[rajarshimaitra]

Description

Fixes#55. I decided to go for one feature flag for each backends as they made dependency management easier. Because different backend will have different dependency, and we don't wanna have redundant ones. Also it makes the commands a little shorter.

Notes to the reviewers

Right now its not that useful because the backend APIs are not exposed. So I can't really do much other than sync/send in the command line. (unlike testnet we cannot send coins to bdk from outside of local regtest node). So to make it useful in repl mode as described in the issue, we also have to expose the blockchain apis in bdk-cli command in some way (at least the basic stuffs, address, send, generate). Looking for suggestions on this.

usage looks like this:

cargo run --features regtest-rpc -- wallet --descriptor "wpkh(tpubEBr4i6yk5nf5DAaJpsi9N2pPYBeJ7fZ5Z9rmN4977iYLCGco1VyjB9tvvuvYtfZzjD5A8igzgw3HeWeeKFmanHYqksqZXYXGsw5zjnj7KM9/*)" sync
{}

Although I am having trouble connecting to the esplora end of electrs. Not sure whats the issue, will have to look more.

Checklists

All Submissions:

• I've signed all my commits
• I followed the contribution guidelines
• I ran cargo fmt and cargo clippy before committing

New Features:

• I've added tests for the new feature
• I've added docs for the new feature
• I've updated CHANGELOG.md

[rajarshimaitra]

Fixed merge conflict with some enhancements.

[rajarshimaitra]

Updated the commit history for better structuring.

[notmandatory]

@rajarshimaitra Looks like this one needs a rebase or if you're tied up I can do the rebase but would have to rewrite your commits. Once that's done I'll do some testing and see if I can figure out how to expose the core rpc in repl mode.

[rajarshimaitra]

Rebased and resolved conflicts..

[notmandatory]

I spent some time today testing and reviewing and this is a great start! There are still the two (related) issues you found:

• 1. no way to control the bitcoind node (ie. create new blocks, get regtest coins)
• 2. sync doesn't work for esplora

I believe issue 2 is because electrsd won't sync until at least one block is created by bitcoind; so if we fix 1 we should also fix 2. My idea for fixing 1 is to change the bitcoind lib to allow us to set a reusable (not randomly generated tmp) directory for the bitcoind data dir; and maybe the same for electrsd. Then we can add some simple RPC commands to bdk-cli to update the bitcoind (and electrsd) database and that data will persist for future invocations of bdk-cli.

But for now as long as we add the comment that the regtest-* features are experimental this should be OK to merge. Then we have more time to to play with and get merged any PRs to bitcoind (and electrsd) that we need to have it all working end to end.

[notmandatory]

Add some little experimental warnings and this is good to go.

[rajarshimaitra]

Thanks @notmandatory .. Updated as per suggested and rebased..

[rajarshimaitra]

Rebased on latest master.

[notmandatory]

Also need to do a cargo fmt.

[rajarshimaitra]

• Reverted back default network to testnet
• Fixed test failure

[notmandatory]

ACK 930b0c4

[rajarshimaitra]

Description

Fixes#55. I decided to go for one feature flag for each backends as they made dependency management easier. Because different backend will have different dependency, and we don't wanna have redundant ones. Also it makes the commands a little shorter.

Notes to the reviewers

Right now its not that useful because the backend APIs are not exposed. So I can't really do much other than sync/send in the command line. (unlike testnet we cannot send coins to bdk from outside of local regtest node). So to make it useful in repl mode as described in the issue, we also have to expose the blockchain apis in bdk-cli command in some way (at least the basic stuffs, address, send, generate). Looking for suggestions on this.

usage looks like this:

cargo run --features regtest-rpc -- wallet --descriptor "wpkh(tpubEBr4i6yk5nf5DAaJpsi9N2pPYBeJ7fZ5Z9rmN4977iYLCGco1VyjB9tvvuvYtfZzjD5A8igzgw3HeWeeKFmanHYqksqZXYXGsw5zjnj7KM9/*)" sync
{}

Although I am having trouble connecting to the esplora end of electrs. Not sure whats the issue, will have to look more.

Checklists

All Submissions:

• I've signed all my commits
• I followed the contribution guidelines
• I ran cargo fmt and cargo clippy before committing

New Features:

• I've added tests for the new feature
• I've added docs for the new feature
• I've updated CHANGELOG.md

[rajarshimaitra]

Fixed merge conflict with some enhancements.

[rajarshimaitra]

Updated the commit history for better structuring.

[notmandatory]

@rajarshimaitra Looks like this one needs a rebase or if you're tied up I can do the rebase but would have to rewrite your commits. Once that's done I'll do some testing and see if I can figure out how to expose the core rpc in repl mode.

[rajarshimaitra]

Rebased and resolved conflicts..

[notmandatory]

I spent some time today testing and reviewing and this is a great start! There are still the two (related) issues you found:

• 1. no way to control the bitcoind node (ie. create new blocks, get regtest coins)
• 2. sync doesn't work for esplora

I believe issue 2 is because electrsd won't sync until at least one block is created by bitcoind; so if we fix 1 we should also fix 2. My idea for fixing 1 is to change the bitcoind lib to allow us to set a reusable (not randomly generated tmp) directory for the bitcoind data dir; and maybe the same for electrsd. Then we can add some simple RPC commands to bdk-cli to update the bitcoind (and electrsd) database and that data will persist for future invocations of bdk-cli.

But for now as long as we add the comment that the regtest-* features are experimental this should be OK to merge. Then we have more time to to play with and get merged any PRs to bitcoind (and electrsd) that we need to have it all working end to end.

[notmandatory]

Add some little experimental warnings and this is good to go.

[rajarshimaitra]

Thanks @notmandatory .. Updated as per suggested and rebased..

[rajarshimaitra]

Rebased on latest master.

[notmandatory]

Also need to do a cargo fmt.

[rajarshimaitra]

• Reverted back default network to testnet
• Fixed test failure

[notmandatory]

ACK 930b0c4

[rajarshimaitra]

Description

Fixes#55. I decided to go for one feature flag for each backends as they made dependency management easier. Because different backend will have different dependency, and we don't wanna have redundant ones. Also it makes the commands a little shorter.

Notes to the reviewers

Right now its not that useful because the backend APIs are not exposed. So I can't really do much other than sync/send in the command line. (unlike testnet we cannot send coins to bdk from outside of local regtest node). So to make it useful in repl mode as described in the issue, we also have to expose the blockchain apis in bdk-cli command in some way (at least the basic stuffs, address, send, generate). Looking for suggestions on this.

usage looks like this:

cargo run --features regtest-rpc -- wallet --descriptor "wpkh(tpubEBr4i6yk5nf5DAaJpsi9N2pPYBeJ7fZ5Z9rmN4977iYLCGco1VyjB9tvvuvYtfZzjD5A8igzgw3HeWeeKFmanHYqksqZXYXGsw5zjnj7KM9/*)" sync
{}

Although I am having trouble connecting to the esplora end of electrs. Not sure whats the issue, will have to look more.

Checklists

All Submissions:

• I've signed all my commits
• I followed the contribution guidelines
• I ran cargo fmt and cargo clippy before committing

New Features:

• I've added tests for the new feature
• I've added docs for the new feature
• I've updated CHANGELOG.md

[rajarshimaitra]

Fixed merge conflict with some enhancements.

[rajarshimaitra]

Updated the commit history for better structuring.

[notmandatory]

@rajarshimaitra Looks like this one needs a rebase or if you're tied up I can do the rebase but would have to rewrite your commits. Once that's done I'll do some testing and see if I can figure out how to expose the core rpc in repl mode.

[rajarshimaitra]

Rebased and resolved conflicts..

[notmandatory]

I spent some time today testing and reviewing and this is a great start! There are still the two (related) issues you found:

• 1. no way to control the bitcoind node (ie. create new blocks, get regtest coins)
• 2. sync doesn't work for esplora

I believe issue 2 is because electrsd won't sync until at least one block is created by bitcoind; so if we fix 1 we should also fix 2. My idea for fixing 1 is to change the bitcoind lib to allow us to set a reusable (not randomly generated tmp) directory for the bitcoind data dir; and maybe the same for electrsd. Then we can add some simple RPC commands to bdk-cli to update the bitcoind (and electrsd) database and that data will persist for future invocations of bdk-cli.

But for now as long as we add the comment that the regtest-* features are experimental this should be OK to merge. Then we have more time to to play with and get merged any PRs to bitcoind (and electrsd) that we need to have it all working end to end.

[notmandatory]

Add some little experimental warnings and this is good to go.

[rajarshimaitra]

Thanks @notmandatory .. Updated as per suggested and rebased..

[rajarshimaitra]

Rebased on latest master.

[notmandatory]

Also need to do a cargo fmt.

[rajarshimaitra]

• Reverted back default network to testnet
• Fixed test failure

[notmandatory]

ACK 930b0c4

[rajarshimaitra]

Description

Fixes#55. I decided to go for one feature flag for each backends as they made dependency management easier. Because different backend will have different dependency, and we don't wanna have redundant ones. Also it makes the commands a little shorter.

Notes to the reviewers

Right now its not that useful because the backend APIs are not exposed. So I can't really do much other than sync/send in the command line. (unlike testnet we cannot send coins to bdk from outside of local regtest node). So to make it useful in repl mode as described in the issue, we also have to expose the blockchain apis in bdk-cli command in some way (at least the basic stuffs, address, send, generate). Looking for suggestions on this.

usage looks like this:

cargo run --features regtest-rpc -- wallet --descriptor "wpkh(tpubEBr4i6yk5nf5DAaJpsi9N2pPYBeJ7fZ5Z9rmN4977iYLCGco1VyjB9tvvuvYtfZzjD5A8igzgw3HeWeeKFmanHYqksqZXYXGsw5zjnj7KM9/*)" sync
{}

Although I am having trouble connecting to the esplora end of electrs. Not sure whats the issue, will have to look more.

Checklists

All Submissions:

• I've signed all my commits
• I followed the contribution guidelines
• I ran cargo fmt and cargo clippy before committing

New Features:

• I've added tests for the new feature
• I've added docs for the new feature
• I've updated CHANGELOG.md

[rajarshimaitra]

Fixed merge conflict with some enhancements.

[rajarshimaitra]

Updated the commit history for better structuring.

[notmandatory]

@rajarshimaitra Looks like this one needs a rebase or if you're tied up I can do the rebase but would have to rewrite your commits. Once that's done I'll do some testing and see if I can figure out how to expose the core rpc in repl mode.

[rajarshimaitra]

Rebased and resolved conflicts..

[notmandatory]

I spent some time today testing and reviewing and this is a great start! There are still the two (related) issues you found:

• 1. no way to control the bitcoind node (ie. create new blocks, get regtest coins)
• 2. sync doesn't work for esplora

I believe issue 2 is because electrsd won't sync until at least one block is created by bitcoind; so if we fix 1 we should also fix 2. My idea for fixing 1 is to change the bitcoind lib to allow us to set a reusable (not randomly generated tmp) directory for the bitcoind data dir; and maybe the same for electrsd. Then we can add some simple RPC commands to bdk-cli to update the bitcoind (and electrsd) database and that data will persist for future invocations of bdk-cli.

But for now as long as we add the comment that the regtest-* features are experimental this should be OK to merge. Then we have more time to to play with and get merged any PRs to bitcoind (and electrsd) that we need to have it all working end to end.

[notmandatory]

Add some little experimental warnings and this is good to go.

[rajarshimaitra]

Thanks @notmandatory .. Updated as per suggested and rebased..

[rajarshimaitra]

Rebased on latest master.

[notmandatory]

Also need to do a cargo fmt.

[rajarshimaitra]

• Reverted back default network to testnet
• Fixed test failure

[notmandatory]

ACK 930b0c4