Add Non-conditional Put/Delete functionality by G8XSU · Pull Request #17 · lightningdevkit/vss-server

G8XSU · 2023-08-18T06:11:37Z

Similar to popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

DynamoDB: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html
If-match in cosmos: https://learn.microsoft.com/en-us/rest/api/cosmos-db/replace-a-document

jkczyz

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

jkczyz · 2023-08-18T17:33:35Z

+ // After any non-conditional upsert, `version` against the `key` is reset to '1'. Hence the next
+ // `PutObjectRequest` for the `key` can either be a non-conditional upsert or a conditional write


Or DeleteObjectRequest, too?

Since delete will delete the object, there is nothing left-over afterwards.

Oh, you mean delete it after put, yes we could of-course do get/list/delete as well.
Only wanted to clarify for next write operation.

G8XSU · 2023-08-18T19:03:29Z

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

Yes that risk exists, this risk is similar to using single-device storage and having a multi-device app.
It can be easily circumvented by client storing config in storage. For e.g. for iphones that could be iCloud or vss.

It will be directly exposed in thin-client and will not be exposed in phase-2's PrimarySecondaryStorage.

jkczyz

Just a few minor comments. Otherwise, looks good.

Just like popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

G8XSU · 2023-08-18T06:11:37Z

Similar to popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

DynamoDB: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html
If-match in cosmos: https://learn.microsoft.com/en-us/rest/api/cosmos-db/replace-a-document

jkczyz

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

jkczyz · 2023-08-18T17:33:35Z

+ // After any non-conditional upsert, `version` against the `key` is reset to '1'. Hence the next
+ // `PutObjectRequest` for the `key` can either be a non-conditional upsert or a conditional write


Or DeleteObjectRequest, too?

Since delete will delete the object, there is nothing left-over afterwards.

Oh, you mean delete it after put, yes we could of-course do get/list/delete as well.
Only wanted to clarify for next write operation.

G8XSU · 2023-08-18T19:03:29Z

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

Yes that risk exists, this risk is similar to using single-device storage and having a multi-device app.
It can be easily circumvented by client storing config in storage. For e.g. for iphones that could be iCloud or vss.

It will be directly exposed in thin-client and will not be exposed in phase-2's PrimarySecondaryStorage.

jkczyz

Just a few minor comments. Otherwise, looks good.

Just like popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

G8XSU · 2023-08-18T06:11:37Z

Similar to popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

DynamoDB: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html
If-match in cosmos: https://learn.microsoft.com/en-us/rest/api/cosmos-db/replace-a-document

jkczyz

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

jkczyz · 2023-08-18T17:33:35Z

+ // After any non-conditional upsert, `version` against the `key` is reset to '1'. Hence the next
+ // `PutObjectRequest` for the `key` can either be a non-conditional upsert or a conditional write


Or DeleteObjectRequest, too?

Since delete will delete the object, there is nothing left-over afterwards.

Oh, you mean delete it after put, yes we could of-course do get/list/delete as well.
Only wanted to clarify for next write operation.

G8XSU · 2023-08-18T19:03:29Z

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

Yes that risk exists, this risk is similar to using single-device storage and having a multi-device app.
It can be easily circumvented by client storing config in storage. For e.g. for iphones that could be iCloud or vss.

It will be directly exposed in thin-client and will not be exposed in phase-2's PrimarySecondaryStorage.

jkczyz

Just a few minor comments. Otherwise, looks good.

Just like popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

G8XSU · 2023-08-18T06:11:37Z

Similar to popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

DynamoDB: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html
If-match in cosmos: https://learn.microsoft.com/en-us/rest/api/cosmos-db/replace-a-document

jkczyz

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

jkczyz · 2023-08-18T17:33:35Z

+ // After any non-conditional upsert, `version` against the `key` is reset to '1'. Hence the next
+ // `PutObjectRequest` for the `key` can either be a non-conditional upsert or a conditional write


Or DeleteObjectRequest, too?

Since delete will delete the object, there is nothing left-over afterwards.

Oh, you mean delete it after put, yes we could of-course do get/list/delete as well.
Only wanted to clarify for next write operation.

G8XSU · 2023-08-18T19:03:29Z

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

Yes that risk exists, this risk is similar to using single-device storage and having a multi-device app.
It can be easily circumvented by client storing config in storage. For e.g. for iphones that could be iCloud or vss.

It will be directly exposed in thin-client and will not be exposed in phase-2's PrimarySecondaryStorage.

jkczyz

Just a few minor comments. Otherwise, looks good.

Just like popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

G8XSU · 2023-08-18T06:11:37Z

Similar to popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

DynamoDB: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html
If-match in cosmos: https://learn.microsoft.com/en-us/rest/api/cosmos-db/replace-a-document

jkczyz

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

jkczyz · 2023-08-18T17:33:35Z

+ // After any non-conditional upsert, `version` against the `key` is reset to '1'. Hence the next
+ // `PutObjectRequest` for the `key` can either be a non-conditional upsert or a conditional write


Or DeleteObjectRequest, too?

Since delete will delete the object, there is nothing left-over afterwards.

Oh, you mean delete it after put, yes we could of-course do get/list/delete as well.
Only wanted to clarify for next write operation.

G8XSU · 2023-08-18T19:03:29Z

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

Yes that risk exists, this risk is similar to using single-device storage and having a multi-device app.
It can be easily circumvented by client storing config in storage. For e.g. for iphones that could be iCloud or vss.

It will be directly exposed in thin-client and will not be exposed in phase-2's PrimarySecondaryStorage.

jkczyz

Just a few minor comments. Otherwise, looks good.

Just like popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

G8XSU · 2023-08-18T06:11:37Z

Similar to popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

DynamoDB: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html
If-match in cosmos: https://learn.microsoft.com/en-us/rest/api/cosmos-db/replace-a-document

jkczyz

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

jkczyz · 2023-08-18T17:33:35Z

+ // After any non-conditional upsert, `version` against the `key` is reset to '1'. Hence the next
+ // `PutObjectRequest` for the `key` can either be a non-conditional upsert or a conditional write


Or DeleteObjectRequest, too?

Since delete will delete the object, there is nothing left-over afterwards.

Oh, you mean delete it after put, yes we could of-course do get/list/delete as well.
Only wanted to clarify for next write operation.

G8XSU · 2023-08-18T19:03:29Z

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

Yes that risk exists, this risk is similar to using single-device storage and having a multi-device app.
It can be easily circumvented by client storing config in storage. For e.g. for iphones that could be iCloud or vss.

It will be directly exposed in thin-client and will not be exposed in phase-2's PrimarySecondaryStorage.

jkczyz

Just a few minor comments. Otherwise, looks good.

Just like popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

G8XSU · 2023-08-18T06:11:37Z

Similar to popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

DynamoDB: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html
If-match in cosmos: https://learn.microsoft.com/en-us/rest/api/cosmos-db/replace-a-document

jkczyz

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

jkczyz · 2023-08-18T17:33:35Z

+ // After any non-conditional upsert, `version` against the `key` is reset to '1'. Hence the next
+ // `PutObjectRequest` for the `key` can either be a non-conditional upsert or a conditional write


Or DeleteObjectRequest, too?

Since delete will delete the object, there is nothing left-over afterwards.

Oh, you mean delete it after put, yes we could of-course do get/list/delete as well.
Only wanted to clarify for next write operation.

G8XSU · 2023-08-18T19:03:29Z

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

Yes that risk exists, this risk is similar to using single-device storage and having a multi-device app.
It can be easily circumvented by client storing config in storage. For e.g. for iphones that could be iCloud or vss.

It will be directly exposed in thin-client and will not be exposed in phase-2's PrimarySecondaryStorage.

jkczyz

Just a few minor comments. Otherwise, looks good.

Just like popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

G8XSU · 2023-08-18T06:11:37Z

Similar to popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

DynamoDB: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html
If-match in cosmos: https://learn.microsoft.com/en-us/rest/api/cosmos-db/replace-a-document

jkczyz

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

jkczyz · 2023-08-18T17:33:35Z

+ // After any non-conditional upsert, `version` against the `key` is reset to '1'. Hence the next
+ // `PutObjectRequest` for the `key` can either be a non-conditional upsert or a conditional write


Or DeleteObjectRequest, too?

Since delete will delete the object, there is nothing left-over afterwards.

Oh, you mean delete it after put, yes we could of-course do get/list/delete as well.
Only wanted to clarify for next write operation.

G8XSU · 2023-08-18T19:03:29Z

How will this be exposed in the clients? Is there risk of accidentally corrupting data if a device has misconfigured its client to use non-conditional writes while another device is properly configured to do conditional writes? Is there any way to guard agains this?

Yes that risk exists, this risk is similar to using single-device storage and having a multi-device app.
It can be easily circumvented by client storing config in storage. For e.g. for iphones that could be iCloud or vss.

It will be directly exposed in thin-client and will not be exposed in phase-2's PrimarySecondaryStorage.

jkczyz

Just a few minor comments. Otherwise, looks good.

Just like popular kvstores, we should also support non-conditional put/delete operations. This opens up new patterns for usage of VSS, allows clients to easily onboard to VSS (without versioning, just for remote-backups) or to just use global_version for versioning and avoid key-level version checks.

G8XSU requested a review from jkczyz August 18, 2023 06:11

G8XSU mentioned this pull request Aug 18, 2023
VSS Tracking Issue #9
Closed
31 tasks

jkczyz reviewed Aug 18, 2023
View reviewed changes

G8XSU force-pushed the non-conditional-put-delete branch from b8a745c to 4984189Compare August 18, 2023 19:04

G8XSU requested a review from jkczyz August 18, 2023 19:05

jkczyz approved these changes Aug 18, 2023
View reviewed changes

Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated

G8XSU force-pushed the non-conditional-put-delete branch from 4984189 to 14744aeCompare August 18, 2023 19:45

G8XSU merged commit e1a88af into lightningdevkit:mainAug 18, 2023

G8XSU requested a review from jkczyz August 18, 2023 06:11

G8XSU mentioned this pull request Aug 18, 2023
VSS Tracking Issue #9
Closed
31 tasks

jkczyz reviewed Aug 18, 2023
View reviewed changes

G8XSU force-pushed the non-conditional-put-delete branch from b8a745c to 4984189Compare August 18, 2023 19:04

G8XSU requested a review from jkczyz August 18, 2023 19:05

jkczyz approved these changes Aug 18, 2023
View reviewed changes

Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated

G8XSU force-pushed the non-conditional-put-delete branch from 4984189 to 14744aeCompare August 18, 2023 19:45

G8XSU merged commit e1a88af into lightningdevkit:mainAug 18, 2023

G8XSU requested a review from jkczyz August 18, 2023 06:11

G8XSU mentioned this pull request Aug 18, 2023
VSS Tracking Issue #9
Closed
31 tasks

jkczyz reviewed Aug 18, 2023
View reviewed changes

G8XSU force-pushed the non-conditional-put-delete branch from b8a745c to 4984189Compare August 18, 2023 19:04

G8XSU requested a review from jkczyz August 18, 2023 19:05

jkczyz approved these changes Aug 18, 2023
View reviewed changes

Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated

G8XSU force-pushed the non-conditional-put-delete branch from 4984189 to 14744aeCompare August 18, 2023 19:45

G8XSU merged commit e1a88af into lightningdevkit:mainAug 18, 2023

G8XSU requested a review from jkczyz August 18, 2023 06:11

G8XSU mentioned this pull request Aug 18, 2023
VSS Tracking Issue #9
Closed
31 tasks

jkczyz reviewed Aug 18, 2023
View reviewed changes

G8XSU force-pushed the non-conditional-put-delete branch from b8a745c to 4984189Compare August 18, 2023 19:04

G8XSU requested a review from jkczyz August 18, 2023 19:05

jkczyz approved these changes Aug 18, 2023
View reviewed changes

Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated

G8XSU force-pushed the non-conditional-put-delete branch from 4984189 to 14744aeCompare August 18, 2023 19:45

G8XSU merged commit e1a88af into lightningdevkit:mainAug 18, 2023

G8XSU requested a review from jkczyz August 18, 2023 06:11

G8XSU mentioned this pull request Aug 18, 2023
VSS Tracking Issue #9
Closed
31 tasks

jkczyz reviewed Aug 18, 2023
View reviewed changes

G8XSU force-pushed the non-conditional-put-delete branch from b8a745c to 4984189Compare August 18, 2023 19:04

G8XSU requested a review from jkczyz August 18, 2023 19:05

jkczyz approved these changes Aug 18, 2023
View reviewed changes

Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated

G8XSU force-pushed the non-conditional-put-delete branch from 4984189 to 14744aeCompare August 18, 2023 19:45

G8XSU merged commit e1a88af into lightningdevkit:mainAug 18, 2023

G8XSU requested a review from jkczyz August 18, 2023 06:11

G8XSU mentioned this pull request Aug 18, 2023
VSS Tracking Issue #9
Closed
31 tasks

jkczyz reviewed Aug 18, 2023
View reviewed changes

G8XSU force-pushed the non-conditional-put-delete branch from b8a745c to 4984189Compare August 18, 2023 19:04

G8XSU requested a review from jkczyz August 18, 2023 19:05

jkczyz approved these changes Aug 18, 2023
View reviewed changes

Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated

G8XSU force-pushed the non-conditional-put-delete branch from 4984189 to 14744aeCompare August 18, 2023 19:45

G8XSU merged commit e1a88af into lightningdevkit:mainAug 18, 2023

G8XSU requested a review from jkczyz August 18, 2023 06:11

G8XSU mentioned this pull request Aug 18, 2023
VSS Tracking Issue #9
Closed
31 tasks

jkczyz reviewed Aug 18, 2023
View reviewed changes

G8XSU force-pushed the non-conditional-put-delete branch from b8a745c to 4984189Compare August 18, 2023 19:04

G8XSU requested a review from jkczyz August 18, 2023 19:05

jkczyz approved these changes Aug 18, 2023
View reviewed changes

Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated

G8XSU force-pushed the non-conditional-put-delete branch from 4984189 to 14744aeCompare August 18, 2023 19:45

G8XSU merged commit e1a88af into lightningdevkit:mainAug 18, 2023

G8XSU requested a review from jkczyz August 18, 2023 06:11

G8XSU mentioned this pull request Aug 18, 2023
VSS Tracking Issue #9
Closed
31 tasks

jkczyz reviewed Aug 18, 2023
View reviewed changes

G8XSU force-pushed the non-conditional-put-delete branch from b8a745c to 4984189Compare August 18, 2023 19:04

G8XSU requested a review from jkczyz August 18, 2023 19:05

jkczyz approved these changes Aug 18, 2023
View reviewed changes

Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated
Comment threadapp/src/main/proto/vss.proto Outdated

G8XSU force-pushed the non-conditional-put-delete branch from 4984189 to 14744aeCompare August 18, 2023 19:45

G8XSU merged commit e1a88af into lightningdevkit:mainAug 18, 2023

		// After any non-conditional upsert, `version` against the `key` is reset to '1'. Hence the next
		// `PutObjectRequest` for the `key` can either be a non-conditional upsert or a conditional write

Uh oh!

Conversation

G8XSU commented Aug 18, 2023

Uh oh!

jkczyz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

G8XSU commented Aug 18, 2023

Uh oh!

jkczyz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

G8XSU commented Aug 18, 2023

Uh oh!

jkczyz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

G8XSU commented Aug 18, 2023

Uh oh!

jkczyz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

G8XSU commented Aug 18, 2023

Uh oh!

jkczyz left a comment

Choose a reason for hiding this comment