Handle unmodified sensitived fields when updating connections

[pierrejeambrun]

closes: #52301
closes: #53753

How it works:
For connection, 'password' and 'extras' are merged with their original value when doing an update. The function works similarly to the redact function, it will recursively handle all sort of data types and detect sensitive values that were not modified in the 'new_value' and then restore the value from the unredacted previous value.

See the warning note:

After adding a key 'new_key' to the extra and saving this is what we get in the UI:

And from the CLI we can see that both password, and extra redacted field were preserved:

The only downside is that we cannot 'insert' a real '***' for redacted value because this is how we detect that the value didn't change. I think it's a fair limitation, '***' shouldn't never be a valid value for a sensitive field anyway. @ashb is working on a follow up PR to instead use unicode characters that looks like '***' but are not, to make it even less likely that it will be blocking for users. (They would have to chose a very weird value for their secret).

Another example, it also handle well arrays:

password1 and password3 value in the array are retained.

[pierrejeambrun]

Really nice test coverage @pierrejeambrun.

The changes are looking fine to me, one thing however I want to show is the experience when we try to update with an invalid json.

You just found a bug. I was able to reproduce on main, seems to happen when 'extra_json' is a str.

I created the issue: #53963, more info there.

[pierrejeambrun]

Initial value: {"tokens":["1234","abcd","as12"]}
If I update the "abcd" to "xyz"
so updated value becomes {"tokens":",xyz,"}. It becomes a single value.

@shubhamraj-git I was able to reproduce on main, I just opened an issue there:
#53961

[amoghrajesh]

Thanks for addressing my concerns and for the follow up issues, lgtm

[github-actions]

Backport failed to create: v3-0-test. View the failure log Run details

Status	Branch	Result
❌	v3-0-test

You can attempt to backport this manually by running:

cherry_picker 0abcfdf v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[pierrejeambrun]

Manual backport #53973

[pierrejeambrun]

closes: #52301
closes: #53753

How it works:
For connection, 'password' and 'extras' are merged with their original value when doing an update. The function works similarly to the redact function, it will recursively handle all sort of data types and detect sensitive values that were not modified in the 'new_value' and then restore the value from the unredacted previous value.

See the warning note:

After adding a key 'new_key' to the extra and saving this is what we get in the UI:

And from the CLI we can see that both password, and extra redacted field were preserved:

The only downside is that we cannot 'insert' a real '***' for redacted value because this is how we detect that the value didn't change. I think it's a fair limitation, '***' shouldn't never be a valid value for a sensitive field anyway. @ashb is working on a follow up PR to instead use unicode characters that looks like '***' but are not, to make it even less likely that it will be blocking for users. (They would have to chose a very weird value for their secret).

Another example, it also handle well arrays:

password1 and password3 value in the array are retained.

[pierrejeambrun]

Really nice test coverage @pierrejeambrun.

The changes are looking fine to me, one thing however I want to show is the experience when we try to update with an invalid json.

You just found a bug. I was able to reproduce on main, seems to happen when 'extra_json' is a str.

I created the issue: #53963, more info there.

[pierrejeambrun]

Initial value: {"tokens":["1234","abcd","as12"]}
If I update the "abcd" to "xyz"
so updated value becomes {"tokens":",xyz,"}. It becomes a single value.

@shubhamraj-git I was able to reproduce on main, I just opened an issue there:
#53961

[amoghrajesh]

Thanks for addressing my concerns and for the follow up issues, lgtm

[github-actions]

Backport failed to create: v3-0-test. View the failure log Run details

Status	Branch	Result
❌	v3-0-test

You can attempt to backport this manually by running:

cherry_picker 0abcfdf v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[pierrejeambrun]

Manual backport #53973

[pierrejeambrun]

closes: #52301
closes: #53753

How it works:
For connection, 'password' and 'extras' are merged with their original value when doing an update. The function works similarly to the redact function, it will recursively handle all sort of data types and detect sensitive values that were not modified in the 'new_value' and then restore the value from the unredacted previous value.

See the warning note:

After adding a key 'new_key' to the extra and saving this is what we get in the UI:

And from the CLI we can see that both password, and extra redacted field were preserved:

The only downside is that we cannot 'insert' a real '***' for redacted value because this is how we detect that the value didn't change. I think it's a fair limitation, '***' shouldn't never be a valid value for a sensitive field anyway. @ashb is working on a follow up PR to instead use unicode characters that looks like '***' but are not, to make it even less likely that it will be blocking for users. (They would have to chose a very weird value for their secret).

Another example, it also handle well arrays:

password1 and password3 value in the array are retained.

[pierrejeambrun]

Really nice test coverage @pierrejeambrun.

The changes are looking fine to me, one thing however I want to show is the experience when we try to update with an invalid json.

You just found a bug. I was able to reproduce on main, seems to happen when 'extra_json' is a str.

I created the issue: #53963, more info there.

[pierrejeambrun]

Initial value: {"tokens":["1234","abcd","as12"]}
If I update the "abcd" to "xyz"
so updated value becomes {"tokens":",xyz,"}. It becomes a single value.

@shubhamraj-git I was able to reproduce on main, I just opened an issue there:
#53961

[amoghrajesh]

Thanks for addressing my concerns and for the follow up issues, lgtm

[github-actions]

Backport failed to create: v3-0-test. View the failure log Run details

Status	Branch	Result
❌	v3-0-test

You can attempt to backport this manually by running:

cherry_picker 0abcfdf v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[pierrejeambrun]

Manual backport #53973

[pierrejeambrun]

closes: #52301
closes: #53753

How it works:
For connection, 'password' and 'extras' are merged with their original value when doing an update. The function works similarly to the redact function, it will recursively handle all sort of data types and detect sensitive values that were not modified in the 'new_value' and then restore the value from the unredacted previous value.

See the warning note:

After adding a key 'new_key' to the extra and saving this is what we get in the UI:

And from the CLI we can see that both password, and extra redacted field were preserved:

The only downside is that we cannot 'insert' a real '***' for redacted value because this is how we detect that the value didn't change. I think it's a fair limitation, '***' shouldn't never be a valid value for a sensitive field anyway. @ashb is working on a follow up PR to instead use unicode characters that looks like '***' but are not, to make it even less likely that it will be blocking for users. (They would have to chose a very weird value for their secret).

Another example, it also handle well arrays:

password1 and password3 value in the array are retained.

[pierrejeambrun]

Really nice test coverage @pierrejeambrun.

The changes are looking fine to me, one thing however I want to show is the experience when we try to update with an invalid json.

You just found a bug. I was able to reproduce on main, seems to happen when 'extra_json' is a str.

I created the issue: #53963, more info there.

[pierrejeambrun]

Initial value: {"tokens":["1234","abcd","as12"]}
If I update the "abcd" to "xyz"
so updated value becomes {"tokens":",xyz,"}. It becomes a single value.

@shubhamraj-git I was able to reproduce on main, I just opened an issue there:
#53961

[amoghrajesh]

Thanks for addressing my concerns and for the follow up issues, lgtm

[github-actions]

Backport failed to create: v3-0-test. View the failure log Run details

Status	Branch	Result
❌	v3-0-test

You can attempt to backport this manually by running:

cherry_picker 0abcfdf v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[pierrejeambrun]

Manual backport #53973

[pierrejeambrun]

closes: #52301
closes: #53753

How it works:
For connection, 'password' and 'extras' are merged with their original value when doing an update. The function works similarly to the redact function, it will recursively handle all sort of data types and detect sensitive values that were not modified in the 'new_value' and then restore the value from the unredacted previous value.

See the warning note:

After adding a key 'new_key' to the extra and saving this is what we get in the UI:

And from the CLI we can see that both password, and extra redacted field were preserved:

The only downside is that we cannot 'insert' a real '***' for redacted value because this is how we detect that the value didn't change. I think it's a fair limitation, '***' shouldn't never be a valid value for a sensitive field anyway. @ashb is working on a follow up PR to instead use unicode characters that looks like '***' but are not, to make it even less likely that it will be blocking for users. (They would have to chose a very weird value for their secret).

Another example, it also handle well arrays:

password1 and password3 value in the array are retained.

[pierrejeambrun]

Really nice test coverage @pierrejeambrun.

The changes are looking fine to me, one thing however I want to show is the experience when we try to update with an invalid json.

You just found a bug. I was able to reproduce on main, seems to happen when 'extra_json' is a str.

I created the issue: #53963, more info there.

[pierrejeambrun]

Initial value: {"tokens":["1234","abcd","as12"]}
If I update the "abcd" to "xyz"
so updated value becomes {"tokens":",xyz,"}. It becomes a single value.

@shubhamraj-git I was able to reproduce on main, I just opened an issue there:
#53961

[amoghrajesh]

Thanks for addressing my concerns and for the follow up issues, lgtm

[github-actions]

Backport failed to create: v3-0-test. View the failure log Run details

Status	Branch	Result
❌	v3-0-test

You can attempt to backport this manually by running:

cherry_picker 0abcfdf v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[pierrejeambrun]

Manual backport #53973

[pierrejeambrun]

closes: #52301
closes: #53753

How it works:
For connection, 'password' and 'extras' are merged with their original value when doing an update. The function works similarly to the redact function, it will recursively handle all sort of data types and detect sensitive values that were not modified in the 'new_value' and then restore the value from the unredacted previous value.

See the warning note:

After adding a key 'new_key' to the extra and saving this is what we get in the UI:

And from the CLI we can see that both password, and extra redacted field were preserved:

The only downside is that we cannot 'insert' a real '***' for redacted value because this is how we detect that the value didn't change. I think it's a fair limitation, '***' shouldn't never be a valid value for a sensitive field anyway. @ashb is working on a follow up PR to instead use unicode characters that looks like '***' but are not, to make it even less likely that it will be blocking for users. (They would have to chose a very weird value for their secret).

Another example, it also handle well arrays:

password1 and password3 value in the array are retained.

[pierrejeambrun]

Really nice test coverage @pierrejeambrun.

The changes are looking fine to me, one thing however I want to show is the experience when we try to update with an invalid json.

You just found a bug. I was able to reproduce on main, seems to happen when 'extra_json' is a str.

I created the issue: #53963, more info there.

[pierrejeambrun]

Initial value: {"tokens":["1234","abcd","as12"]}
If I update the "abcd" to "xyz"
so updated value becomes {"tokens":",xyz,"}. It becomes a single value.

@shubhamraj-git I was able to reproduce on main, I just opened an issue there:
#53961

[amoghrajesh]

Thanks for addressing my concerns and for the follow up issues, lgtm

[github-actions]

Backport failed to create: v3-0-test. View the failure log Run details

Status	Branch	Result
❌	v3-0-test

You can attempt to backport this manually by running:

cherry_picker 0abcfdf v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[pierrejeambrun]

Manual backport #53973

[pierrejeambrun]

closes: #52301
closes: #53753

How it works:
For connection, 'password' and 'extras' are merged with their original value when doing an update. The function works similarly to the redact function, it will recursively handle all sort of data types and detect sensitive values that were not modified in the 'new_value' and then restore the value from the unredacted previous value.

See the warning note:

After adding a key 'new_key' to the extra and saving this is what we get in the UI:

And from the CLI we can see that both password, and extra redacted field were preserved:

The only downside is that we cannot 'insert' a real '***' for redacted value because this is how we detect that the value didn't change. I think it's a fair limitation, '***' shouldn't never be a valid value for a sensitive field anyway. @ashb is working on a follow up PR to instead use unicode characters that looks like '***' but are not, to make it even less likely that it will be blocking for users. (They would have to chose a very weird value for their secret).

Another example, it also handle well arrays:

password1 and password3 value in the array are retained.

[pierrejeambrun]

Really nice test coverage @pierrejeambrun.

The changes are looking fine to me, one thing however I want to show is the experience when we try to update with an invalid json.

You just found a bug. I was able to reproduce on main, seems to happen when 'extra_json' is a str.

I created the issue: #53963, more info there.

[pierrejeambrun]

Initial value: {"tokens":["1234","abcd","as12"]}
If I update the "abcd" to "xyz"
so updated value becomes {"tokens":",xyz,"}. It becomes a single value.

@shubhamraj-git I was able to reproduce on main, I just opened an issue there:
#53961

[amoghrajesh]

Thanks for addressing my concerns and for the follow up issues, lgtm

[github-actions]

Backport failed to create: v3-0-test. View the failure log Run details

Status	Branch	Result
❌	v3-0-test

You can attempt to backport this manually by running:

cherry_picker 0abcfdf v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[pierrejeambrun]

Manual backport #53973

[pierrejeambrun]

closes: #52301
closes: #53753

How it works:
For connection, 'password' and 'extras' are merged with their original value when doing an update. The function works similarly to the redact function, it will recursively handle all sort of data types and detect sensitive values that were not modified in the 'new_value' and then restore the value from the unredacted previous value.

See the warning note:

After adding a key 'new_key' to the extra and saving this is what we get in the UI:

And from the CLI we can see that both password, and extra redacted field were preserved:

The only downside is that we cannot 'insert' a real '***' for redacted value because this is how we detect that the value didn't change. I think it's a fair limitation, '***' shouldn't never be a valid value for a sensitive field anyway. @ashb is working on a follow up PR to instead use unicode characters that looks like '***' but are not, to make it even less likely that it will be blocking for users. (They would have to chose a very weird value for their secret).

Another example, it also handle well arrays:

password1 and password3 value in the array are retained.

[pierrejeambrun]

Really nice test coverage @pierrejeambrun.

The changes are looking fine to me, one thing however I want to show is the experience when we try to update with an invalid json.

You just found a bug. I was able to reproduce on main, seems to happen when 'extra_json' is a str.

I created the issue: #53963, more info there.

[pierrejeambrun]

Initial value: {"tokens":["1234","abcd","as12"]}
If I update the "abcd" to "xyz"
so updated value becomes {"tokens":",xyz,"}. It becomes a single value.

@shubhamraj-git I was able to reproduce on main, I just opened an issue there:
#53961

[amoghrajesh]

Thanks for addressing my concerns and for the follow up issues, lgtm

[github-actions]

Backport failed to create: v3-0-test. View the failure log Run details

Status	Branch	Result
❌	v3-0-test

You can attempt to backport this manually by running:

cherry_picker 0abcfdf v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[pierrejeambrun]

Manual backport #53973