Add manager for simple Kubernetes job creation by stv0g · Pull Request #47 · VILLASframework/controller

stv0g · 2022-12-02T11:12:31Z

In GitLab by @iripiri on Sep 23, 2021, 13:35

relates to #26

Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

…eter and config file Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

…added to the controller Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

rather set their state to stopped Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

…t and test stages Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

stv0g · 2025-04-02T14:30:27Z

Hi @iripiri,

Could we somehow break this PR up? Its containing over 80 commits which is almost impossible to review..

Or, I see a lot of small bug fixing commits, could you squash those into the respective commits to which they logically belong? (Not everthing into a single commit, as this would be equally hard to review :/).

iripiri · 2025-04-03T07:05:47Z

Hi @stv0g,
I will do so! Though I'm not completely done here, I am testing & fixing right now, it's been a long time..
Could you review #61 first? That one is done

Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de> pipeline fixes Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de> updatet setuptools version to support license expression Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de> update license definition Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de> read version dynamically Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de> fixes Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de> move version to correct init file Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

…t to find controller executable Signed-off-by: SystemsPurge <naktiyoussef@proton.me>

stv0g · 2025-05-21T08:16:58Z

Could you review #61 first? That one is done

#61 is merged.

I think we are almost ready to merge this one, after the conflicts have been resolved.

stv0g

First little round of review is done.

Ideally, we should split this PR into two:

Migration to pyproject.toml
New kubernetes-simple component

stv0g · 2025-05-21T08:17:55Z

@@ -0,0 +1,17 @@
+---
+broker:
+ url: amqp://admin:vieQuoo2sieDahHee8ohM5aThaibiPei@villas-broker:5672/


Is this a production credential? Looks like your RabbitMQ deployment in K8s?

stv0g · 2025-05-21T08:19:44Z

- 'psutil',
- 'pyusb'
- ],
 data_files=[


Cant we add this to pyproject.toml as well?
Glob expressions are supported there as well:
https://setuptools.pypa.io/en/latest/userguide/datafiles.html#package-data

stv0g · 2025-05-21T08:22:13Z

+ # job isn't immediately deleted
+ # let the user see that something is happening


Suggested change
# job isn't immediately deleted
# let the user see that something is happening
# Job isn't immediately deleted.
# Let the user see that something is happening.

stv0g · 2025-05-21T08:23:16Z

 self.properties['pod_names'] = []
+ # job isn't immediately deleted
+ # let the user see that something is happening
+ time.sleep(7)


Sorry, but this is a bad.. Why should we artifically block execution here?
This also stops the controller from processing other actions and stuff..

stv0g · 2025-05-21T08:24:47Z

+from villas.controller.components.managers.kubernetes import KubernetesManager
+from villas.controller.components.simulators.kubernetes import KubernetesJob
+
+parameters_simple = {


Please turn this into a function which constructs the dictionary and returns it.
The patching of the struct below is not nice, since it modifies the global variable.

stv0g · 2025-05-21T08:25:19Z

+ name = params.get('name')
+ privileged = params.get('privileged', False)
+ uuid = params.get('uuid')
+ self.logger.info('uuid:')


Please improve the logging message here. This does not need to be spread across two invocations.

stv0g · 2025-05-21T08:25:52Z

+ super().create(payload)
+ return


Suggested change
super().create(payload)
return
returnsuper().create(payload)

stv0g · 2025-05-21T08:26:22Z

+ parameters = parameters_simple
+ parameters['name'] = sim_name
+ job = parameters['properties']['job']
+ job['metadata']['name'] = jobname
+ job['spec']['activeDeadlineSeconds'] = adls
+ job_container = job['spec']['template']['spec']['containers'][0]
+ job_container['image'] = image
+ job_container['securityContext']['privileged'] = privileged
+
+ parameters['job'] = job
+
+ if name:
+ parameters['name'] = name
+
+ if uuid:
+ parameters['uuid'] = uuid


As noted above, please construct the parameters within a dedicated function.

iripiri force-pushed the simple-kubernetes-manager branch from 7a3a130 to 0c166d5Compare August 14, 2023 14:38

iripiriand others added 29 commits August 14, 2023 16:42

add manager for simple kubernetes job creation
de44dea
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

suppress state changes from event repetitions
8959757
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

try create of superclass if parameters are missing
d69a26f
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

delete configmap when job is deleted
298df49
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

fix api_url for villas-node manager
f13ea00
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

simplify config handling
df28149
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

move global status to Controller mixin class
e0d569c
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

make work dir configurable
a0759cf
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

pass payload instead of message to action handlers
a9bde8d
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

fix mixed up state vs status properties in some components
1f30e93
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

fix mixed up state vs status properties in some components
4e23053
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

fix mixed up state vs status properties in some components
426cbf2
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

added first version of an HTTP REST API
22ba436
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

creating AMQP queue with durable, exclusive and auto_delete flags
4360d29
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

add first draft of OpenAPI spec
124f55f
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

fix simple manager
5800d0b
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

use 'villas' namespace as default
1f695f5
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

use villas-controller namespace as default
17080eb
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

move schemas to YAML files and load them from there for all components
d705699
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

config: show a useful error message when started without broker param…
208f8e8
…eter and config file Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

fix typo
71a4872
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

config: fix bug when started without config file
075fbfc
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

validate action parameters against schema
aa6432c
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

fix linting errors
b79c286
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

fix some errors in the API spec and schema
bac3246
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

validate schemas and openapi doc against meta jsonschemas
38c698c
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

raise a SimulationException if an IC with an existing UUID should be …
a27be13
…added to the controller Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

provide version number within status field of status update
f87608c
Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

relay: do not remove vanished sessions by setting state to gone
e0f7829
rather set their state to stopped Signed-off-by: Iris Koester <iris.m.koester@gmail.com>

stv0g changed the title ~~add manager for simple kubernetes job creation~~Add manager for simple kubernetes job creationMar 25, 2025

iripiri added 8 commits March 26, 2025 08:23

changed python docker image due to pip3 command not found in precommi…
67a8761
…t and test stages Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

changed back image version and changed CI tag
7f5242d
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

update schema checking
e06754c
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

commented schema checking to run CI, needs to get revised
d0c70d1
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

debug test fail
2ccadc5
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

add current workind directory to PATH
e4dd2b3
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

added init file to villas folder
555fe20
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

remove changes made during pipeline debug/fix
5ac9fa1
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

stv0g changed the title ~~Add manager for simple kubernetes job creation~~Add manager for simple Kubernetes job creationMar 31, 2025

iripiri added 9 commits March 31, 2025 10:34

get version (fix ModuleNotFound error), remove tags
52e7338
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

add long description which got lost in last commit
fd0d5c8
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

updated json schema checking
5fdb890
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

added blank lines (flake8)
44e8da4
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

add init.py for modules to be found in tests
05fdaa4
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

merged with pipeline fixes
dd13225
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

watch the resources inside the controller namespace
64f2143
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

start watcher thread AFTER reading environment variables
79a56d6
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

corrected setting the namespace
3df9f3d
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

iripiriand others added 5 commits April 22, 2025 16:04

remove requirements files
d7dd022
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

merged build updates
487a387
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

fix errors, update config
0336da3
Signed-off-by: iripiri <ikoester@eonerc.rwth-aachen.de>

fix: temporary fix for entrypoint of Dockerfile, causing container no…
82bc2ad
…t to find controller executable Signed-off-by: SystemsPurge <naktiyoussef@proton.me>

stv0g commented May 21, 2025
View reviewed changes

stv0g mentioned this pull request Jun 12, 2025
Attempt at making dpsim work consistently #63
Draft

iripiri mentioned this pull request Jul 1, 2025
Cleaned simple kubernetes manager #66
Open

Navigation Menu

Uh oh!

Add manager for simple Kubernetes job creation - #47

Add manager for simple Kubernetes job creation#47
stv0g wants to merge 92 commits into
masterfrom
simple-kubernetes-manager

stv0g commented Dec 2, 2022

Uh oh!

stv0g commented Apr 2, 2025

Uh oh!

iripiri commented Apr 3, 2025

Uh oh!

stv0g commented May 21, 2025

Uh oh!

stv0g left a comment

Uh oh!

stv0gMay 21, 2025

Uh oh!

stv0gMay 21, 2025

Uh oh!

stv0gMay 21, 2025

Uh oh!

stv0gMay 21, 2025

Uh oh!

stv0gMay 21, 2025

Uh oh!

stv0gMay 21, 2025

Uh oh!

stv0gMay 21, 2025

Uh oh!

stv0gMay 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		# job isn't immediately deleted
		# let the user see that something is happening

Uh oh!

Conversation

stv0g commented Dec 2, 2022

Uh oh!

stv0g commented Apr 2, 2025

Uh oh!

iripiri commented Apr 3, 2025

Uh oh!

stv0g commented May 21, 2025

Uh oh!

stv0g left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants