Implement backup, restore and reset of app data via API

[tlindi]

Application Data Management

Many applications data would be nice to have backup initiation possibility. And also restore and "Remove" (aka Factory reset) in case of some miss hap.

Instead of creating such routines separately for each App, should one API based functions be likely better way.

Checklist

• mentioned related open issue, if any Feature Request - Add Alby Hub to Marketplace #878
• To be initially used with one App - suggested Alby Hub as it's PR comment suggested this need.
• • Add albyhub (v1.15.0) #902 (comment)
• [ - ] Backup data
• [ - ] Restore data
• Reset data - Concept implemented on [API] Implement App storage folder reset #945 (and fixed at c0f0a66)
• • [API] Implement App storage folder reset #945 Tested with Upgrade Alby Hub to v1.17.2 #946
• Update docs at https://mynodebtc.github.io/developer/add-app.html

List of test device(s)

• [ + ] tested successfully on local MyNode, if yes, list the device(s) below
• VM - MyNodeBTC 0.3.37 + Fixes

@tehelsper - mind to comment if functions are planned to proper files, and if you like idea / way to do this?

[tehelsper]

I like the concept.

[tlindi]

I like the concept.

Looking forward to first implement and test Data Removal in few coming days.

[tehelsper]

Can this be closed now that PR #945 was merged?

[tlindi]

Can this be closed now that PR #945 was merged?

Sir, unfor not - I think.

I have not yet fould time to fix issue mentioned here:
#940 (comment)

And backup & restore are still to be implement.

[tlindi]

I had an idea to make backups version dependant, like here:
https://github.com/tlindi/mynode-phoenixd/blob/main/scripts/uninstall_phoenixd.sh

(And restore on install_phoenixd)

But as If your #945 does both I'm just happy with it.

(And willing to test as soon as I get my rig online ✌🏻🙏🏻👍🏻 )

[tehelsper]

This is getting complex - I'm leaning towards not merging, especially with versioned backups. That means there's tracking, displaying in the UI, potential for old backups not working with new app versions, etc... An option to download a tar.gz file for each app data folder may be better.

[tehelsper]

I'm fine with the simple reset part that is mostly done as of now.

[tlindi]

This is getting complex - I'm leaning towards not merging, especially with versioned backups. That means there's tracking, displaying in the UI, potential for old backups not working with new app versions, etc... An option to download a tar.gz file for each app data folder may be better.

Possibility to download tgz of data would be clean and simple.

[tlindi]

I'm fine with the simple reset part that is mostly done as of now.

I'll fix app->short_name and remove references to backup and restore.

[tlindi]

Reset works now after this commit on master.

And as this PR has much of hazzle - I'm okay for this to be Closed without merge.

If download storage_folder as tar.gz button appears onto Apps info pages someday, I'll be grateful.

[tlindi]

Small issue to re-test.
Alby Hub custom configuration with phoenixd and reset worked flawlessly.
But configuration (default) with LDK and reset caused App Deactivation in log and restart tries to install docker version "latest" which fails.

Log attached here albyhub.log

[tlindi]

Small issue to re-test.

Re-tested - Uninstalled and then reinstalled. Afterwards reset works. Hickup maybe caused concurrently ongoing BTCPay Server re install and not rebooted.

I'd say Work's for me :D

[tehelsper]

OK, I'll try the reset on AlbyHub once I get to working on the automated setup. That error makes it seem like it's trying to re-install the docker container after clearing data for some reason.

Actually, now that I think about it, several apps probably can't be reset entirely. Some installations place specific files in the data folder and may not properly set those files up in the pre-startup scripts. For example, the mempool clear only deletes a subset of the data folders within its data folder. Just a thought that the "reset" would need to be tested on each app it gets enabled on.

[tlindi]

Just a thought that the "reset" would need to be tested on each app it gets enabled on.

Yes, sure testing is need.
Test must be done by devs before enabling reset button onto "info page" by adding add data_manageable=true to Dynamic App .json file.

[tlindi]

OK, I'll try the reset on AlbyHub once I get to working on the automated setup. That error makes it seem like it's trying to re-install the docker container after clearing data for some reason.

Could you please also make switch to disable automated config to LND, for advanced users to be able to manually config custom setup? (like me, for manually setup Alby Hub to connect phoenixd Community App by me)?

[tlindi]

That error makes it seem like it's trying to re-install the docker container after clearing data for some reason.

I re-tested issue, and I could not replicate. Non-starting Alby Hub after reset from Default setup could have been caused by the fact that I was reinstalling BTCPay same time as i did reset. But more tests never do harm...