Skip to content

DevWorkspace Operator crashes with OOMKilled #611

Description

@sleshchenko

That's a strange bug that happened on DevSanbox Prod.
To reproduce it we may try to redo load testing Tina did.
Screenshot_20210923_122309

> kc get pod devworkspace-controller-manager-*** -o=yaml
- containerID: cri-o://4e196253a9478ddc52a19bedadf162561fcc2b2aa4a6d3d2b62d143601f87266
image: registry.redhat.io/devworkspace/devworkspace-rhel8-operator@sha256:e68ec2fe7ac27e59641bdfc7794ae99fdfaa60e5b6d0cc0e3f20ab3f7a31bc11
imageID: registry.redhat.io/devworkspace/devworkspace-rhel8-operator@sha256:e68ec2fe7ac27e59641bdfc7794ae99fdfaa60e5b6d0cc0e3f20ab3f7a31bc11
lastState:
terminated:
containerID: cri-o://4e196253a9478ddc52a19bedadf162561fcc2b2aa4a6d3d2b62d143601f87266
exitCode: 137
finishedAt: "2021-09-23T09:22:13Z"
reason: OOMKilled
startedAt: "2021-09-23T09:21:33Z"
name: devworkspace-controller
ready: false
restartCount: 7
started: false
state:
waiting:
message: back-off 5m0s restarting failed container=devworkspace-controller
pod=devworkspace-controller-manager-6c99cdf947-ch5bg_openshift-operators(d29753b5-6155-4ff9-a920-c27457242e5b)
reason: CrashLoopBackOff

DevWorkspace hanged on Waiting for SA, anything reconcile loop related appeared in DWO logs.

Metadata

Metadata

Assignees

Labels

sprint/currentIs assigned to issues which are planned to work on in the current team sprint

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions