M-CMP ADMIN CLI (mcc)

This repository provides a Multi-Cloud ADMIN CLI.
The name of this tool is mcc(Multi-Cloud admin CLI).
A sub-system of M-CMP platform to deploy and manage Multi-Cloud Infrastructures.

[NOTE]
mcc is currently under development.
So, we do not recommend using the current release in production.
Please note that the functionalities of mcc are not stable and secure yet.
If you have any difficulties in using mcc, please let us know.
(Open an issue or Join the M-CMP Slack)

mcc Overview

• Management tool that supports the installation, execution, status information provision, termination, and API calls of the M-CMP system.
• Currently, infra subcommand is only support docker compose base infra install and management.
  • infra subcommand
• If you want to checkout how to run the whole subsystem on the single instance on CSP Instance, see this document.

Development & Test Environment

• Go 1.25.0 (minimum required version)
• Docker version 27.3.1
• Docker Compose version v2.29

Install Docker & Docker Compose V2

• Install Docker Engine on Ubuntu

checkout the commands down below.

sudo apt-get install -y apt-transport-https ca-certificates curl gnupg-agent software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli docker-compose-plugin

Optionally: Grant Docker Permission to Current User.

• After installation, you may need to allow your current user to access the Docker daemon socket to avoid permission denied errors.

# Start Docker service
sudo systemctl start docker
sudo systemctl enable docker
# Add current user to the 'docker' group
sudo usermod -aG docker $USER# Apply changes immediately (no need to log out)
newgrp docker
# Verify Docker access
docker ps

Quick Start (TL;DR)

For the full step-by-step guide, see the Quick Guide section below.

1. Configure environment

cd mc-admin-cli/conf/docker/conf/mc-iam-manager
cp .env.setup .env
# Edit .env — set required values such as platform admin ID and password

2. Run installAll.sh

cd mc-admin-cli/bin
./installAll.sh

When prompted for a domain:

• Dev / Local PC — press Enter (uses default mciam.local)
• Dev / Remote VM — enter the VM's IP address or domain name
• Production — enter your public FQDN (required; DNS A-record must point to this server)

3. Verify all containers are healthy

./mcc infra info

Wait until all containers show healthy. mc-iam-manager-post-initial showing Exited (0) is normal.

4. Open the web console

Navigate to https://<server>:3001 in your browser.

⚠️ Use https, not http. When the browser shows a security warning, click Continue (or Advanced → Proceed).

Default credentials: mcmp / mcmp_password

---

Quick Guide

This section describes the minimal process for those who want to set up quickly.
For more detailed installation guide, please refer to the Running on Single Instance Guide document.

Step 1. Clone the Repository

For stable infrastructure deployment, it is recommended to explicitly clone a specific released version using the -b flag instead of using the latest development version.

git clone https://github.com/m-cmp/mc-admin-cli.git -b v0.5.0
cd mc-admin-cli/bin

For Contributors, clone the latest development branch instead.

git clone https://github.com/m-cmp/mc-admin-cli.git
cd mc-admin-cli/bin

Step 1-1. Verify mcc Binary (Optional)

Before proceeding, confirm the pre-built binary runs on your system:

./mcc --version

If you see an error like:

./mcc: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found

the pre-built binary requires a newer GLIBC than your OS provides (e.g. Ubuntu 20.04 ships GLIBC 2.31). Rebuild from source as a static binary — see Build a Static Binary below — then return here.

Step 2. Choose Deployment Mode

Decide which mode fits your environment. installAll.sh will prompt you interactively — no manual .env editing required.

	Mode A — Local / Dev	Mode B — Production
Domain	Any local name (default: mciam.local)	Public FQDN (e.g. iam.example.com)
TLS Certificate	Self-signed (auto-generated)	Let's Encrypt via certbot
DNS	Auto-added to /etc/hosts	Public DNS A-record → server IP
installAll.sh flag	--mode dev	--mode prod
Browser warning	Certificate warning (acceptable for dev)	No warning (trusted CA)

Mode B prerequisite — DNS A-record for your domain must point to this server's public IP before running installAll.sh.

Step 3. Run installAll.sh

installAll.sh automatically:

1. Creates .env files from .env.setup templates if they do not exist
2. Prompts for Mode (A/B) and domain — or accepts them as CLI flags
3. Injects the domain into the environment files
4. Generates TLS certificates (Mode A: self-signed, Mode B: Let's Encrypt) and nginx config

# Interactive mode — prompts for mode, domain, and run mode
./installAll.sh
# Non-interactive: Mode A — default domain (mciam.local), start in background
./installAll.sh --mode dev --run background
# Non-interactive: Mode A — custom local domain
./installAll.sh --mode dev --domain myhost.local --run background
# Non-interactive: Mode B — real domain, start in background
./installAll.sh --mode prod --domain iam.example.com --run background
# Generate config files only, skip container start
./installAll.sh --mode dev --run skip

Step 4. Start the Platform

If you used --run skip in Step 3, start all containers now:

./mcc infra run

Step 5. Verify Startup

Wait a few minutes for all containers to become healthy, then run the checks below.

(a) Container health — all containers healthy, mc-web-console-api last:

./mcc infra info

Look for no unhealthy entries. mc-iam-manager-post-initial will show as Exited (0) — that is expected.

(b) mc-infra-manager readyz:

./mcc rest get -u default -p default http://localhost:1323/tumblebug/readyz

Expected: {"message":"CB-Tumblebug is ready","ready":true}

(c) Keycloak OIDC discovery (replace <DOMAIN> with your MC_IAM_MANAGER_PUBLIC_DOMAIN):

curl -k https://<DOMAIN>/auth/realms/mciam/.well-known/openid-configuration | grep issuer

Expected: "issuer": "https://<DOMAIN>/auth/realms/mciam" — must start with https:// and include /auth/.

(d) mc-iam-manager-post-initial 8-step setup:

docker logs mc-iam-manager-post-initial | tail -5

Expected last lines:

=== Automated setup completed successfully ===
[Success] MC-IAM-Manager initialization completed successfully!

If the container exited with a non-zero code or the success message is missing, re-run the init script:

./iam_manager_init.sh

Or run individual steps manually using conf/docker/conf/mc-iam-manager/1_setup_manual.sh.

(e) iframe HTTPS proxy endpoints (used by the web console for embedded views):

curl -kI https://<DOMAIN>:33002 # Grafana dashboard proxy
curl -kI https://<DOMAIN>:7781 # Cost Optimizer FE proxy
curl -k https://<DOMAIN>:9090/api/costopti/be/readyz # Cost Optimizer BE proxy
curl -k https://<DOMAIN>:9000/actuator/health 2>/dev/null ||true# Cost Optimizer alarm proxy

Expected: :33002 and :7781 return HTTP/2 200; :9090/readyz returns Application is ready.

Step 6. Initialize CB-Tumblebug & Access the Web Console

Once mc-web-console-api is healthy, initialize CB-Tumblebug using the following instructions:

• Quick Start Guide – CB-Tumblebug
• In the guide, running init.sh is the required step.

Access the web console with the default credentials:

• Mode A: https://mciam.local:3001 (accept the self-signed certificate warning)
• Mode B: https://<DOMAIN>:3001
• Username: mcmp
• Password: mcmp_password

Step 7. Clean Up

If you want to completely reset the environment (removes all Docker containers, images, and volumes):
[WARNING] All Docker environments and existing work history on the system will be deleted.

cd mc-admin-cli/bin
./cleanAll.sh

When to run cleanAll.sh: Always run a full clean before switching deployment modes (dev ↔ prod) or changing the domain. Re-running installAll.sh over an existing setup without cleaning first can leave stale certificates, nginx config, or DB state that conflicts with the new configuration.

If you only need to reset containers/volumes but want to avoid re-pulling every image on the next installAll.sh run (e.g. to stay under a Docker Hub pull quota), use --keep-current-images. This still deletes containers/volumes/networks as usual, keeps the image versions currently pinned in conf/docker/docker-compose.yaml, and only removes older versions of those same images:

./cleanAll.sh --keep-current-images

---

Known Issues

Cost Optimizer iframe — BE API (in-place workaround required)

The Cost Optimizer frontend JavaScript bundle contains hardcoded URL logic that selects the backend API host at runtime:

Access method	BE/alarm URL selected	Result
localhost	http://localhost:9090	OK (same-origin, no TLS)
IP address	https://{ip}:9090	OK — IAM nginx HTTPS proxy on :9090
Domain	https://{domain}:9090	OK — IAM nginx HTTPS proxy on :9090

After every container recreation, apply the following in-place patch to the FE bundle:

# Find the actual bundle filename first
JS=$(docker exec mc-cost-optimizer-fe ls /usr/share/nginx/html/assets/index-*.js 2>/dev/null | head -1)
docker exec mc-cost-optimizer-fe sh -c " # IP branch: http:// → https:// sed -i 's|t=\`http://\${r}:9090\`,i=\`http://\${r}:9000\`|t=\`https://\${r}:9090\`,i=\`https://\${r}:9000\`|g' $JS # Domain branch: no-port https:// → explicit :9090/:9000 sed -i 's|t=\`https://\${r}\`,i=\`https://\${r}\`|t=\`https://\${r}:9090\`,i=\`https://\${r}:9000\`|g' $JS"

Root fix: The mc-cost-optimizer-fe source code needs to be updated so it always uses https://{host}:9090 for IP and domain cases. See todo_mc-cost-optimizer.md for the development team handoff.

HSTS Cache — switching between dev and prod on the same domain

When you switch from Mode A (self-signed cert) to Mode B (Let's Encrypt) using the same domain, the browser's HSTS cache may block the connection during the transition.

Workaround:

1. Use an incognito/private window for the first access after switching, or
2. Clear the HSTS cache manually:
   • Chrome/Edge: navigate to chrome://net-internals/#hsts → "Delete domain security policies" → enter your domain → Delete
   • Firefox: use a new browser profile or delete SiteSecurityServiceState.txt from your profile folder

TLS Certificate Auto-Renewal (Mode B)

When running in Mode B (Let's Encrypt), certbot renews the certificate automatically via systemd certbot.timer (twice daily). The certificate is renewed 30 days before expiry.

Webroot setup (required once after installation)

Mode B uses the webroot authenticator so nginx keeps running during renewal. If your installation used the standalone authenticator (older setup), switch it once:

sudo certbot certonly \
--webroot \
-w <mc-admin-cli-path>/conf/docker/container-volume/certbot/www \
-d <your-domain> \
--force-renewal

Verify the renewal config was updated:

sudo grep "authenticator" /etc/letsencrypt/renewal/<your-domain>.conf
# Expected: authenticator = webroot

Deploy hook — nginx reload after renewal

After renewal, the new certificate must be loaded into the running nginx container. Install the deploy hook once:

sudo cp conf/docker/scripts/certbot-deploy-hook.sh \
/etc/letsencrypt/renewal-hooks/deploy/reload-nginx-docker.sh
sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/reload-nginx-docker.sh

Verify auto-renewal

sudo certbot renew --dry-run
# Expected: "all simulated renewals succeeded"

---

Firewall Port Information

The following ports should be registered in the firewall if needed:

MC-INFRA-CONNECTOR

Service	Port	Protocol	Description
mc-infra-connector	1024	TCP	CB-Spider API

MC-INFRA-MANAGER

Service	Port	Protocol	Description
mc-infra-manager	1323	TCP	CB-Tumblebug API
mc-infra-manager-etcd	2379, 2380	TCP	etcd cluster
mc-infra-manager-postgres	6432	TCP	PostgreSQL DB
mc-infra-manager-openbao	8200	TCP	OpenBao (Vault fork, secret storage)

MC-IAM-MANAGER

Service	Port	Protocol	Description
mc-iam-manager	5000	TCP	IAM Manager API
mc-iam-manager-db	5432	TCP	PostgreSQL DB
mc-iam-manager-kc	8080	TCP	Keycloak
mc-iam-manager-nginx	80, 443	TCP	Nginx (HTTP redirect + HTTPS)
mc-iam-manager-nginx	3001	TCP	Web Console Frontend (HTTPS proxy)
mc-iam-manager-nginx	33002	TCP	Grafana (iframe-safe HTTPS reverse proxy)
mc-iam-manager-nginx	7781	TCP	Cost Optimizer FE (iframe-safe HTTPS reverse proxy)

MC-COST-OPTIMIZER

Service	Port	Protocol	Description
mc-cost-optimizer-fe	7780	TCP	Cost Optimizer Frontend (internal, accessed via :7781 HTTPS proxy)
mc-cost-optimizer-be	9090	TCP	Cost Optimizer Backend (internal, accessed via IAM nginx :9090 HTTPS proxy)
mc-cost-optimizer-cost-collector	8881	TCP	Cost Collector
mc-cost-optimizer-cost-processor	18082	TCP	Cost Processor
mc-cost-optimizer-cost-selector	8083	TCP	Cost Selector
mc-cost-optimizer-alarm-service	9000	TCP	Alarm Service (internal, accessed via IAM nginx :9000 HTTPS proxy)
mc-cost-optimizer-asset-collector	8091	TCP	Asset Collector
mc-cost-optimizer-db	3307	TCP	MariaDB

MC-APPLICATION-MANAGER

Service	Port	Protocol	Description
mc-application-manager-jenkins	9800	TCP	Jenkins
mc-application-manager-sonatype-nexus	8081, 5500	TCP	Nexus Repository
mc-application-manager	18084	TCP	Application Manager API

MC-WORKFLOW-MANAGER

Service	Port	Protocol	Description
mc-workflow-manager-jenkins	9880	TCP	Jenkins
mc-workflow-manager	18083	TCP	Workflow Manager API

MC-DATA-MANAGER

Service	Port	Protocol	Description
mc-data-manager	3300	TCP	Data Manager API

MC-WEB-CONSOLE

Service	Port	Protocol	Description
mc-web-console-db	15432	TCP	PostgreSQL DB
mc-web-console-api	3000	TCP	Web Console API
mc-web-console-front	3001	TCP	Web Console Frontend

MC-OBSERVABILITY

Service	Port	Protocol	Description
mc-observability-manager	18080	TCP	Observability Manager
mc-observability-infra	33000	TCP	Observability Infrastructure
mc-observability-rabbitmq	5672, 1883, 15672	TCP	RabbitMQ (AMQP, MQTT, Management)
mc-observability-maria	3306	TCP	MariaDB
mc-observability-influx	8086	TCP	InfluxDB
mc-observability-influx-2	8087	TCP	InfluxDB 2
mc-observability-loki	3100	TCP	Loki Log Aggregation
mc-observability-tempo	3200, 4317, 4318	TCP	Tempo Distributed Tracing
mc-observability-grafana	33001	TCP	Grafana
mc-observability-insight	9001	TCP	Observability Insight
mc-observability-insight-scheduler	9002	TCP	Insight Scheduler
mc-observability-mcp-grafana	8000	TCP	MCP Grafana Server (LLM-based analysis)
mc-observability-mcp-mariadb	8001	TCP	MCP MariaDB Server (LLM-based analysis)
mc-observability-mcp-influx	8002	TCP	MCP InfluxDB Server (LLM-based analysis)

Total 49 ports are configured for external access.

The following ports must be registered in the firewall:

Required Firewall Services

Service	Port	Protocol	Description
mc-iam-manager-nginx	80, 443	TCP	Nginx entry point (HTTP redirect + HTTPS web console)
mc-iam-manager-nginx	3001	TCP	Web Console Frontend (HTTPS)
mc-iam-manager-nginx	33002	TCP	Grafana iframe proxy (HTTPS)
mc-iam-manager-nginx	7781	TCP	Cost Optimizer FE iframe proxy (HTTPS)
mc-iam-manager-nginx	9090	TCP	Cost Optimizer BE HTTPS proxy (required for iframe API calls)
mc-iam-manager-nginx	9000	TCP	Cost Optimizer alarm HTTPS proxy (required for iframe API calls)
mc-web-console-api	3000	TCP	Web Console API

---

Troubleshooting

mc-iam-manager Stays Unhealthy After Install

If ./mcc infra info shows mc-iam-manager as unhealthy and docker logs mc-iam-manager-post-initial ends with ERROR: 1_setup_auto.sh Script execution failed, the post-init container started before mc-iam-manager finished its first boot.

Recovery steps:

# 1. Confirm all prerequisites are healthycd bin && ./mcc infra info
# 2. Remove the exited post-init container, then re-run it (idempotent — safe to repeat)
docker rm mc-iam-manager-post-initial 2>/dev/null
./mcc infra run -s mc-iam-manager-post-initial
docker logs -f mc-iam-manager-post-initial
# Each of the 8 setup steps should finish with ✓# 3. Verify
curl -s http://localhost:5000/readyz | jq .# Expected: "status": "healthy"

---

Build from Source

Build a Static Binary

Use this when the pre-built bin/mcc fails with a GLIBC version error (e.g. on Ubuntu 20.04 or other systems with GLIBC < 2.34):

cd mc-admin-cli/src
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o ../bin/mcc

CGO_ENABLED=0 produces a fully statically linked binary with no GLIBC dependency. The resulting binary runs on any Linux distribution regardless of the installed GLIBC version.

Minimum Go version: 1.21 or later. Install Go from https://go.dev/dl/ if not already available.

Platform-Specific Builds (Makefile)

cd mc-admin-cli/src
# Install / update dependencies
go get -u
# Build for current platform (default: linux amd64)
make
# Cross-compile for other targets
make win # Windows amd64
make mac # macOS amd64
make linux-arm # Linux arm64
make win86 # Windows 386
make mac-arm # macOS arm64

How to use the mcc

mc-admin-cli/bin$ ./mcc -h
The mcc is a tool to operate Cloud-Barista system. Usage:
mcc [command]
Available Commands:
api Call the M-CMP system's Open APIs as services and actions
infra A tool to operate M-CMP system
help Help about any command
rest rest api call
Flags:
-h, --help help for mcc
Use "mcc [command] --help" for more information about a command.

For more detailed explanations, see the articles below.

• infra sub-command guide
• rest sub-command guide

docker-compose.yaml

The necessary service information for the M-CMP System configuration is defined in the mc-admin-cli/docker-compose-mode-files/docker-compose.yaml file.(By default, it is set to build the desired configuration and data volume in the docker-compose-mode-files folder.)
If you want to change the information for each container you want to deploy, modify the mc-admin-cli/docker-compose-mode-files/docker-compose.yaml file or use the -f option.

infra subcommand

For more information, check out the infra subcommand document.

For now, it supports infra's run/stop/info/pull/remove commands.

Use the -h option at the end of the sub-command requiring assistance, or executing 'mcc' without any options will display the help manual.

Usage:
mcc infra [flags]
mcc infra [command]
Available Commands:
info Get information of M-CMP System
pull Pull images of M-CMP System containers
remove Stop and Remove M-CMP System
run Setup and Run M-CMP System
stop Stop M-CMP System
Flags:
-h, --help help for infra
Use "mcc infra [command] --help" for more information about a command.

infra subcommand examples

Simple usage examples for infra subcommand

- ./mcc infra pull [-f ../conf/docker/docker-compose.yaml]
- ./mcc infra run [-f ../conf/docker/docker-compose.yaml] -d
- ./mcc infra info
- ./mcc infra stop [-f ../conf/docker/docker-compose.yaml]
- ./mcc infra remove [-f ../conf/docker/docker-compose.yaml] -v -i

k8s subcommand

K8S is not currently supported and will be supported in the near future.

rest subcommand

The rest subcommands are developed around the basic features of REST to make it easy to use the open APIs of M-CMP-related frameworks from the CLI. For now, it supports get/post/delete/put/patch commands.

For more information, check out the rest subcommand document.

rest api call
Usage:
mcc rest [flags]
mcc rest [command]
Available Commands:
delete REST API calls with DELETE methods
get REST API calls with GET methods
patch REST API calls with PATCH methods
post REST API calls with POST methods
put REST API calls with PUT methods
Flags:
--authScheme string sets the auth scheme type in the HTTP request.(Exam. OAuth)(The default auth scheme is Bearer)
--authToken string sets the auth token of the 'Authorization' header for all HTTP requests.(The default auth scheme is 'Bearer')
-d, --data string Data to send to the server
-f, --file string Data to send to the server from file
-I, --head Show response headers only
-H, --header strings Pass custom header(s) to server
-h, --help help for rest
-p, --password string Password for basic authentication
-u, --user string Username for basic authentication
-v, --verbose Show more detail information
Use "mcc rest [command] --help" for more information about a command.

rest command examples

Simple usage examples for rest commands

./mcc rest get -u default -p default http://localhost:1323/tumblebug/health
./mcc rest post https://reqres.in/api/users -d '{
"name": "morpheus",
"job": "leader"
}'

api subcommand

For more information, check out the infra subcommand document. The api subcommands are developed to make it easy to use the open APIs of M-CMP-related frameworks from the CLI.

Call the action of the service defined in api.yaml. Usage:
mcc api [flags]
mcc api [command]
Available Commands:
tool Swagger JSON parsing tool to assist in writing api.yaml files
Flags:
-a, --action string Action to perform
-c, --config string config file (default "../conf/api.yaml")
-d, --data string Data to send to the server
-f, --file string Data to send to the server from file
-h, --help help for api
-l, --list Show Service or Action list
-m, --method string HTTP Method
-p, --pathParam string Variable path info set "key1:value1 key2:value2" for URIs
-q, --queryString string Use if you have a query string to add to URIs
-s, --service string Service to perform
-v, --verbose Show more detail information
Use "mcc api [command] --help" for more information about a command.

api subcommand examples

Simple usage examples for api subcommand.

./mcc api --help
./mcc api --list
./mcc api --service spider --list
./mcc api --service spider --action ListCloudOS
./mcc api --service spider --action GetCloudDriver --pathParam driver_name:AWS
./mcc api --service spider --action GetRegionZone --pathParam region_name:ap-northeast-3 --queryString ConnectionName:aws-config01