Support Build & Setup Environment Variables

[metaskills]

In my case I have a postCreateCommand that clones another organization repo. This works great under Codespaces but is hanging when using this project. I think what I see is that 🏃 start container will stop here.

Cloning into '/workspaces/mysharedrepo'...
Username for 'https://github.com/':

And never move from that spot. The lifecycle script is doing something like this.

git clone "https://${GITHUB_TOKEN}@github.com/myorg/mysharedrepo.git" /workspaces/mysharedrepo
cd /workspaces/mysharedrepo

Any recommendations to get that working?

[metaskills]

Would it make sense to have the env: variables be accessible to both the build and start stages? That might help here.

[metaskills]

FWIW, I would be happy if there was ANY environment variable during the build / start process so I could make some runtime decisions?

[Oblynx]

Hi! I wanted to request exactly the same feature.

I would like to run a postCreateCommand that pulls a private repo. I specify the secrets as environment variables, but they aren't picked up at the build stage. When I move the step that pulls the private repo to the runCmd, it works. Therefore, the only issue is that the environment variables specified in env aren't passed to the build phase.

[joshspicer]

Hey! This proposal makes sense to me. Stuart Leeks (@stuartleeks) I suspect you'll agree this is OK, but curious if there's any thoughts you have counter?

[stuartleeks]

I was wondering whether there would be any scenarios where we would want to set the build environment variables independently of the runtime ones.

E.g. would we want to have buildEnv and env, or just have env that is passed to both?

[Oblynx]

You can start with passing env to both. Then if an issue is reported with this behavior, we can split it up.

[metaskills]

You can start with passing env to both. Then if an issue is reported with this behavior, we can split it up.

Agreed here too. Thank you.

[Oblynx]

Hello! Just bumping this issue, is it being tracked/planned?

[stuartleeks]

I've was looking at adding support for this to the action, and was a little confused by the results of my testing for a while so thought I'd drop a comment here.

There's a PR to improve support for this (#211) but depending on how your devcontainer.json is set up, you may be able to make this work already.

To do this, you need to have the remoteEnv in your devcontainer.json referencing a value from localEnv:

	 "remoteEnv": {
		"YOUR_ENV" : "${localEnv:YOUR_ENV}"
	}

This instructs the devcontainer CLI to set the remote env in the container based on the localEnv context.

The localEnv context can be set using the env property at the step level (i.e. not nested under the with block):

 - name: Build and run Dev Container task
 uses: devcontainers/ci@v0.2
 env:
 YOUR_ENV_VAR: some_value
 with:
 runCmd: |
 make the-thing

#211 will add update the action so that env vars set in the env under the with block are passed to the container. This will provide a way to set container env vars without needing to have it set in devcontainer.json.

I'll raise an issue to add examples of this to the docs.

[metaskills]

I'm trying this out now and still can not figure out how to use an environment variable when the devcontainer is being built. I've read the docs in the PR work Stuart Leeks (@stuartleeks) did but still can't figure it out. Is the env only available to runCmd?

[stuartleeks]

Ken Collins (@metaskills) there is an example in the PR tests here

I added some initial extra content to the docs as well: https://github.com/devcontainers/ci/blob/main/docs/github-action.md#environment-variables

[moleary-gsa]

Hey Folks - I have been trying to get me head around this for the last 2 days with no luck.

I use a .env file for things like AWS credentials and database credentials.

When I run the following devcontainer.json locally it works fine:

// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/alpine
{
	"name": "Python",
	// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
	"image": "mcr.microsoft.com/devcontainers/python:0-3.10",
	// Set *default* container specific settings.json values on container create.
	"customizations": {
		"vscode": {
			"settings": {
				"python.languageServer": "Pylance",
				"python.linting.enabled": true,
				"python.formatting.provider": "black",
				"editor.formatOnSave": true
			},
			"extensions": [
				"ms-toolsai.jupyter",
				"ms-python.python",
				"ms-python.vscode-pylance",
				"njpwerner.autodocstring",
				"oderwat.indent-rainbow",
				"bungcip.better-toml",
				"ms-python.black-formatter",
				"donjayamanne.python-extension-pack",
				"donjayamanne.python-environment-manager"
			]
		}
	},
	// Features to add to the dev container. More info: https://containers.dev/features.
	// "features": {},
	// Use 'forwardPorts' to make a list of ports inside the container available locally.
	// "forwardPorts": [],
	// Use 'postCreateCommand' to run commands after the container is created.
	"postCreateCommand": "make setup",
	// Configure tool-specific properties.
	// "customizations": {},
	// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
	"remoteUser": "root",
	"runArgs": [
		"--env-file",
		"${localWorkspaceFolder}/.env"
	],
}

The critical part here is the postCreateCommand which is a make command that does a few things:

1. Download the AWS CLI
2. Install poetry package manager
3. Use AWS CLI to authenticate poetry to interact with our private Codeartifact repository
4. Install python dependencies (including one on our internal Codeartifact repository)

I know this should be prebuilt but I am taking things in stages as we incorporate dev containers into our standard way of working.

So when I rebuild this container locally, the runArgs entry pulls in a local .env file into the environment which works once it's there. The first time this repo gets cloned there is an error because the .env file is not available. I edit the repo in a recovery dev container, add the .env file with all the variables I need then rebuild. this works perfectly locally and only needs to happen once so it's not a huge deal.

When I try to run this in github actions there is literally no way I have been able to find to make this approach work and have spent almost 3 days fighting it now. Until I found this issue! I added the following section to the bottom of my devcontainer.json:

"remoteEnv": {
	"ENV": "${localEnv:ENV}",
	"AWS_DEFAULT_REGION": "${localEnv:AWS_DEFAULT_REGION}",
	"AWS_ACCESS_KEY_ID": "${localEnv:AWS_ACCESS_KEY_ID}",
	"AWS_SECRET_ACCESS_KEY": "${localEnv:AWS_SECRET_ACCESS_KEY}",
	"AWS_ACCOUNT_ID": "${localEnv:AWS_ACCOUNT_ID}",
	"CODEARTIFACT_DOMAIN": "${localEnv:CODEARTIFACT_DOMAIN}",
	"CODEARTIFACT_REPOSITORY": "${localEnv:CODEARTIFACT_REPOSITORY}",
	"CODEARTIFACT_USER": "${localEnv:CODEARTIFACT_USER}",
	"FC_USER": "${localEnv:FC_USER}",
	"FC_PASSWORD": "${localEnv:FC_PASSWORD}",
	"ENV_FILE": "${localWorkspaceFolder}/.env"
}

I add the following step to my job:

- name: Setup environment
 run: |
 echo "${{ secrets.ENV_FILE }}" >> $GITHUB_ENV
 echo "FC_PASSWORD=${{ secrets.FC_PASSWORD }}" >> $GITHUB_ENV
 echo "AWS_ACCESS_KEY_ID=${{ secrets.AWS_ACCESS_KEY_ID }}" >> $GITHUB_ENV
 echo "AWS_SECRET_ACCESS_KEY=${{ secrets.AWS_SECRET_ACCESS_KEY }}" >> $GITHUB_ENV

and this works fine in Github actions.

The issue is now when I build this locally, the variables specified in the remoteEnv object are not set so they don't get set in the container and the runArgs --env-file is completely ignored.

So I now can run my code locally or on GitHub actions, but not both!

Am I missing something ?