Uh oh!
There was an error while loading. Please reload this page.
Added support for lifecycle.started for clusters - #5150
Conversation
d14c9b4 to
75e051fCompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| // ClusterRemote extends compute.ClusterDetails with a synthetic Lifecycle field so that | ||
| // RemoteType satisfies TestRemoteSuperset (every field in ClusterState exists in ClusterRemote). | ||
| // Lifecycle.Started is populated by DoRead from the cluster's running state. | ||
| type ClusterRemote struct { |
There was a problem hiding this comment.
can we use the same struct for ClusterRemote and ClusterState?
| // lifecycle.started=true: fire Start; WaitAfterUpdate polls for RUNNING. | ||
| _, err := r.client.Clusters.Start(ctx, compute.StartCluster{ClusterId: id}) | ||
| return nil, err | ||
| } else if !desiredStarted && alreadyRunning { |
There was a problem hiding this comment.
Should we also call delete on other states? Like PENDING | RESTARTING | RESIZING | UNKNOWN | ERROR? And poll waiting for the state transition if the state is TERMINATING?
There was a problem hiding this comment.
Do we guarentee TERMINATED if started = false?
There was a problem hiding this comment.
It's a good question. I believe we should only explicitly manpulate it if it's in a known good state remotely, if it's not it's better not to
There was a problem hiding this comment.
It's a good question. I believe we should only explicitly manpulate it if it's in a known good state remotely, if it's not it's better not to
If I read it correctly we will call Delete if started=false and remote state is anything but RUNNING.
According to docs, we can skip it for TERMINATING/TERMINATED, although it is safe to call.
If the cluster is already in a TERMINATING or TERMINATED state, nothing will happen.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
## Changes Pass id to WaitAfterXXX methods ## Why Follow up from here #5150 (comment) ## Tests Tests pas <!-- If your PR needs to be included in the release notes for next release, add a separate entry in NEXT_CHANGELOG.md as part of your PR. -->
## Changes Pass id to WaitAfterXXX methods ## Why Follow up from here databricks#5150 (comment) ## Tests Tests pas <!-- If your PR needs to be included in the release notes for next release, add a separate entry in NEXT_CHANGELOG.md as part of your PR. -->
| @@ -0,0 +1,3 @@ | |||
| Local = true | |||
| Cloud = true | |||
There was a problem hiding this comment.
seems unnecessary to run this on cloud, it's pure validation.
| if errors.As(err, &apiErr) && apiErr.ErrorCode == "INVALID_STATE" { | ||
| return nil, retries.Continues(fmt.Sprintf("cluster %s cannot be modified in its current state: %s", id, apiErr.Message)) | ||
| func (r *ResourceCluster) DoUpdate(ctx context.Context, id string, config *ClusterState, entry *PlanEntry) (*ClusterRemote, error) { | ||
| if hasClusterChanges(entry) { |
| // lifecycle.started=true: fire Start; WaitAfterUpdate polls for RUNNING. | ||
| _, err := r.client.Clusters.Start(ctx, compute.StartCluster{ClusterId: id}) | ||
| return nil, err | ||
| } else if !desiredStarted && alreadyRunning { |
There was a problem hiding this comment.
It's a good question. I believe we should only explicitly manpulate it if it's in a known good state remotely, if it's not it's better not to
If I read it correctly we will call Delete if started=false and remote state is anything but RUNNING.
According to docs, we can skip it for TERMINATING/TERMINATED, although it is safe to call.
If the cluster is already in a TERMINATING or TERMINATED state, nothing will happen.
eng-dev-ecosystem-bot
commented
May 27, 2026
Commit: aca8452 |
Uh oh!
There was an error while loading. Please reload this page.
eng-dev-ecosystem-bot
commented
May 27, 2026
Commit: a802f4a |
## Changes Adds lifecycle.started support for clusters in the direct deployment engine, mirroring the same feature for apps (#4672). ## Why Without this field, clusters defined in a bundle are always left in whatever state the API puts them in after creation. Users have no way to declare "ensure this cluster is running after every deploy." `lifecycle.started: true` guarantees the cluster is RUNNING after bundle deploy. `lifecycle.started: false` creates the cluster but immediately terminates it, and subsequent deploys that detect drift (e.g., someone started the cluster manually) will stop it again. Note: `WaitAfterCreate` always waits for RUNNING first — real clusters start in PENDING state and must be polled. For `started=false`, we wait for RUNNING then terminate; this avoids races with the API that would reject a terminate on a still-pending cluster. ## Tests Added acceptance tests <!-- If your PR needs to be included in the release notes for next release, add a separate entry in NEXT_CHANGELOG.md as part of your PR. -->
Changes
Adds lifecycle.started support for clusters in the direct deployment engine, mirroring the same feature for apps (#4672).
Why
Without this field, clusters defined in a bundle are always left in whatever state the API puts them in after creation.
Users have no way to declare "ensure this cluster is running after every deploy."
lifecycle.started: trueguarantees the cluster is RUNNING after bundle deploy.lifecycle.started: falsecreates the cluster but immediately terminates it, and subsequent deploys that detect drift (e.g., someone started the cluster manually) will stop it again.Note:
WaitAfterCreatealways waits for RUNNING first — real clusters start in PENDING state and must be polled. Forstarted=false, we wait for RUNNING then terminate; this avoids races with the API that would reject a terminate on a still-pending cluster.Tests
Added acceptance tests