Adding the application Flower

[adityapgupta]

Hello, I am Aditya Gupta from IISc, working with Prof. Yogesh Simmhan. We wish to add support for a new application called Flower (a framework to perform federated learning) and subsequently introduce some faults for it.

I tried writing a basic implementation for the same and ran into some issues. I had a docker compose file to initialise the containers and then run federated training on them via flower. I tried to convert this to k8s using the kind image provided by you guys. The relevant docker compose and k8s files are in the flower folder (this was placed inside the submodule aiops-applications, but I could not figure out how to git push there), and the images have been uploaded to my dockerhub. I have also added a flower.py file inside aiopslab/services/apps to initialise the k8s pods.

The main issue is that my pods get initialised but they are not able to communicate with each other. This is my first time using k8s, so I am unable to debug this on my end (it looks like a port misconfig issue, but I havent been able to solve it). The application works with a docker compose up for reference. Kindly go through flower/kubernetes to check my k8s yaml files. Once this is fixed, we will proceed to add faults.

PS: I am running on WSL2 with Ubuntu 20.04. Testing was done by manually doing kubectl apply -Rf flower/kubernetes but the flower.py file also works. The flower application is run with flwr run aiopslab/orchestrator/problems/flower local-deployment --stream

The relevant code is in PR #31

[HacksonClark]

Hi Aditya Gupta (@adityapgupta), excited to have you working on the project! I currently maintain the aiopslab-applications repository so if you need something added there I can do it, though I think you should be able to make a PR there. Let me know if that isn't the case.

As for your networking issue I'm not going to be able to help fix that as you're adding a completely new application, that is your responsibility. From your issue description, it seems like a service discovery issue, so maybe look at your service definitions and network policies to start with. You might also just need to update your calls in your pod code to use the Kubernetes DNS.

[gaganso]

Hi Aditya Gupta (@adityapgupta), thanks for this! Have you taken a look at this - https://github.com/hpn-bristol/kubeFlower?

[adityapgupta]

Hello Gagan Somashekar (@gaganso), I did come across this repo yesterday but our initial attempt to make it work failed. Let me give this one more shot if adding docker compose is going to be tough.