Experiment with shelling out to parse files

[bkeepers]

This was part of an experiment I did a while ago in response to #430 to use shell to parse .env files. The basics are working, but there is still a lot of work and some difficult problems.

Ideally, dotenv woud just be able to exec("source .env"), but Ruby doesn't offer a way to execute a command in the current process and then return control back to ruby (and there are some behavioral differences that I don't think we want, which I'll get to in a minute). exec will replace the current process and then exit once the command is run. spawn and backticks return a subshell, so any exported variables don't get returned.

So this PR adds dotenv-source, which is a shell command that just sources a file and then prints the env after. ShellParser#source calls that shell script, and then parse the printed env into a hash and returns just changed values from the current ENV (and eventually will update ENV).

Biggest problem: conditional assignment

dotenv (by default) does conditional assignment and you can override defaults at runtime. Given this file:

VAR_1=default
VAR_2="$VAR_1 and some stuff"

Here is the current behavior:

$ dotenv ruby -e "puts ENV['VAR_2']"default and some extra
$ VAR_1="changed" dotenv ruby -e "puts ENV['VAR_2']"changed and some stuff

But shell/bash uses : ${VAR:="default"} for conditional assignment, so sourcing this .env with the ShellParser has different behavior:

$ VAR_1=ignored ruby -Ilib -r dotenv/shell_parser -e 'puts Dotenv::ShellParser.new.source(".env")["VAR_2"]'default and some stuff

So we need to find a way to support the current behavior without forcing people to change syntax.

---

TODO:

• Solve conditional assignment problem
• Get remaining tests passing (and either deprecate or document syntax that is not valid in shell, like VAR=spaces without quotes)
• Add way to opt into this new parser (maybe DOTENV_PARSER=shell at top of file?)

cc @graywolf-at-work
cc #422

[bkeepers]

This was part of an experiment I did a while ago in response to #430 to use shell to parse .env files. The basics are working, but there is still a lot of work and some difficult problems.

Ideally, dotenv woud just be able to exec("source .env"), but Ruby doesn't offer a way to execute a command in the current process and then return control back to ruby (and there are some behavioral differences that I don't think we want, which I'll get to in a minute). exec will replace the current process and then exit once the command is run. spawn and backticks return a subshell, so any exported variables don't get returned.

So this PR adds dotenv-source, which is a shell command that just sources a file and then prints the env after. ShellParser#source calls that shell script, and then parse the printed env into a hash and returns just changed values from the current ENV (and eventually will update ENV).

Biggest problem: conditional assignment

dotenv (by default) does conditional assignment and you can override defaults at runtime. Given this file:

VAR_1=default
VAR_2="$VAR_1 and some stuff"

Here is the current behavior:

$ dotenv ruby -e "puts ENV['VAR_2']"default and some extra
$ VAR_1="changed" dotenv ruby -e "puts ENV['VAR_2']"changed and some stuff

But shell/bash uses : ${VAR:="default"} for conditional assignment, so sourcing this .env with the ShellParser has different behavior:

$ VAR_1=ignored ruby -Ilib -r dotenv/shell_parser -e 'puts Dotenv::ShellParser.new.source(".env")["VAR_2"]'default and some stuff

So we need to find a way to support the current behavior without forcing people to change syntax.

---

TODO:

• Solve conditional assignment problem
• Get remaining tests passing (and either deprecate or document syntax that is not valid in shell, like VAR=spaces without quotes)
• Add way to opt into this new parser (maybe DOTENV_PARSER=shell at top of file?)

cc @graywolf-at-work
cc #422

[bkeepers]

This was part of an experiment I did a while ago in response to #430 to use shell to parse .env files. The basics are working, but there is still a lot of work and some difficult problems.

Ideally, dotenv woud just be able to exec("source .env"), but Ruby doesn't offer a way to execute a command in the current process and then return control back to ruby (and there are some behavioral differences that I don't think we want, which I'll get to in a minute). exec will replace the current process and then exit once the command is run. spawn and backticks return a subshell, so any exported variables don't get returned.

So this PR adds dotenv-source, which is a shell command that just sources a file and then prints the env after. ShellParser#source calls that shell script, and then parse the printed env into a hash and returns just changed values from the current ENV (and eventually will update ENV).

Biggest problem: conditional assignment

dotenv (by default) does conditional assignment and you can override defaults at runtime. Given this file:

VAR_1=default
VAR_2="$VAR_1 and some stuff"

Here is the current behavior:

$ dotenv ruby -e "puts ENV['VAR_2']"default and some extra
$ VAR_1="changed" dotenv ruby -e "puts ENV['VAR_2']"changed and some stuff

But shell/bash uses : ${VAR:="default"} for conditional assignment, so sourcing this .env with the ShellParser has different behavior:

$ VAR_1=ignored ruby -Ilib -r dotenv/shell_parser -e 'puts Dotenv::ShellParser.new.source(".env")["VAR_2"]'default and some stuff

So we need to find a way to support the current behavior without forcing people to change syntax.

---

TODO:

• Solve conditional assignment problem
• Get remaining tests passing (and either deprecate or document syntax that is not valid in shell, like VAR=spaces without quotes)
• Add way to opt into this new parser (maybe DOTENV_PARSER=shell at top of file?)

cc @graywolf-at-work
cc #422

[bkeepers]

This was part of an experiment I did a while ago in response to #430 to use shell to parse .env files. The basics are working, but there is still a lot of work and some difficult problems.

Ideally, dotenv woud just be able to exec("source .env"), but Ruby doesn't offer a way to execute a command in the current process and then return control back to ruby (and there are some behavioral differences that I don't think we want, which I'll get to in a minute). exec will replace the current process and then exit once the command is run. spawn and backticks return a subshell, so any exported variables don't get returned.

So this PR adds dotenv-source, which is a shell command that just sources a file and then prints the env after. ShellParser#source calls that shell script, and then parse the printed env into a hash and returns just changed values from the current ENV (and eventually will update ENV).

Biggest problem: conditional assignment

dotenv (by default) does conditional assignment and you can override defaults at runtime. Given this file:

VAR_1=default
VAR_2="$VAR_1 and some stuff"

Here is the current behavior:

$ dotenv ruby -e "puts ENV['VAR_2']"default and some extra
$ VAR_1="changed" dotenv ruby -e "puts ENV['VAR_2']"changed and some stuff

But shell/bash uses : ${VAR:="default"} for conditional assignment, so sourcing this .env with the ShellParser has different behavior:

$ VAR_1=ignored ruby -Ilib -r dotenv/shell_parser -e 'puts Dotenv::ShellParser.new.source(".env")["VAR_2"]'default and some stuff

So we need to find a way to support the current behavior without forcing people to change syntax.

---

TODO:

• Solve conditional assignment problem
• Get remaining tests passing (and either deprecate or document syntax that is not valid in shell, like VAR=spaces without quotes)
• Add way to opt into this new parser (maybe DOTENV_PARSER=shell at top of file?)

cc @graywolf-at-work
cc #422

[bkeepers]

This was part of an experiment I did a while ago in response to #430 to use shell to parse .env files. The basics are working, but there is still a lot of work and some difficult problems.

Ideally, dotenv woud just be able to exec("source .env"), but Ruby doesn't offer a way to execute a command in the current process and then return control back to ruby (and there are some behavioral differences that I don't think we want, which I'll get to in a minute). exec will replace the current process and then exit once the command is run. spawn and backticks return a subshell, so any exported variables don't get returned.

So this PR adds dotenv-source, which is a shell command that just sources a file and then prints the env after. ShellParser#source calls that shell script, and then parse the printed env into a hash and returns just changed values from the current ENV (and eventually will update ENV).

Biggest problem: conditional assignment

dotenv (by default) does conditional assignment and you can override defaults at runtime. Given this file:

VAR_1=default
VAR_2="$VAR_1 and some stuff"

Here is the current behavior:

$ dotenv ruby -e "puts ENV['VAR_2']"default and some extra
$ VAR_1="changed" dotenv ruby -e "puts ENV['VAR_2']"changed and some stuff

But shell/bash uses : ${VAR:="default"} for conditional assignment, so sourcing this .env with the ShellParser has different behavior:

$ VAR_1=ignored ruby -Ilib -r dotenv/shell_parser -e 'puts Dotenv::ShellParser.new.source(".env")["VAR_2"]'default and some stuff

So we need to find a way to support the current behavior without forcing people to change syntax.

---

TODO:

• Solve conditional assignment problem
• Get remaining tests passing (and either deprecate or document syntax that is not valid in shell, like VAR=spaces without quotes)
• Add way to opt into this new parser (maybe DOTENV_PARSER=shell at top of file?)

cc @graywolf-at-work
cc #422

[bkeepers]

This was part of an experiment I did a while ago in response to #430 to use shell to parse .env files. The basics are working, but there is still a lot of work and some difficult problems.

Ideally, dotenv woud just be able to exec("source .env"), but Ruby doesn't offer a way to execute a command in the current process and then return control back to ruby (and there are some behavioral differences that I don't think we want, which I'll get to in a minute). exec will replace the current process and then exit once the command is run. spawn and backticks return a subshell, so any exported variables don't get returned.

So this PR adds dotenv-source, which is a shell command that just sources a file and then prints the env after. ShellParser#source calls that shell script, and then parse the printed env into a hash and returns just changed values from the current ENV (and eventually will update ENV).

Biggest problem: conditional assignment

dotenv (by default) does conditional assignment and you can override defaults at runtime. Given this file:

VAR_1=default
VAR_2="$VAR_1 and some stuff"

Here is the current behavior:

$ dotenv ruby -e "puts ENV['VAR_2']"default and some extra
$ VAR_1="changed" dotenv ruby -e "puts ENV['VAR_2']"changed and some stuff

But shell/bash uses : ${VAR:="default"} for conditional assignment, so sourcing this .env with the ShellParser has different behavior:

$ VAR_1=ignored ruby -Ilib -r dotenv/shell_parser -e 'puts Dotenv::ShellParser.new.source(".env")["VAR_2"]'default and some stuff

So we need to find a way to support the current behavior without forcing people to change syntax.

---

TODO:

• Solve conditional assignment problem
• Get remaining tests passing (and either deprecate or document syntax that is not valid in shell, like VAR=spaces without quotes)
• Add way to opt into this new parser (maybe DOTENV_PARSER=shell at top of file?)

cc @graywolf-at-work
cc #422

[bkeepers]

This was part of an experiment I did a while ago in response to #430 to use shell to parse .env files. The basics are working, but there is still a lot of work and some difficult problems.

Ideally, dotenv woud just be able to exec("source .env"), but Ruby doesn't offer a way to execute a command in the current process and then return control back to ruby (and there are some behavioral differences that I don't think we want, which I'll get to in a minute). exec will replace the current process and then exit once the command is run. spawn and backticks return a subshell, so any exported variables don't get returned.

So this PR adds dotenv-source, which is a shell command that just sources a file and then prints the env after. ShellParser#source calls that shell script, and then parse the printed env into a hash and returns just changed values from the current ENV (and eventually will update ENV).

Biggest problem: conditional assignment

dotenv (by default) does conditional assignment and you can override defaults at runtime. Given this file:

VAR_1=default
VAR_2="$VAR_1 and some stuff"

Here is the current behavior:

$ dotenv ruby -e "puts ENV['VAR_2']"default and some extra
$ VAR_1="changed" dotenv ruby -e "puts ENV['VAR_2']"changed and some stuff

But shell/bash uses : ${VAR:="default"} for conditional assignment, so sourcing this .env with the ShellParser has different behavior:

$ VAR_1=ignored ruby -Ilib -r dotenv/shell_parser -e 'puts Dotenv::ShellParser.new.source(".env")["VAR_2"]'default and some stuff

So we need to find a way to support the current behavior without forcing people to change syntax.

---

TODO:

• Solve conditional assignment problem
• Get remaining tests passing (and either deprecate or document syntax that is not valid in shell, like VAR=spaces without quotes)
• Add way to opt into this new parser (maybe DOTENV_PARSER=shell at top of file?)

cc @graywolf-at-work
cc #422

[bkeepers]

This was part of an experiment I did a while ago in response to #430 to use shell to parse .env files. The basics are working, but there is still a lot of work and some difficult problems.

Ideally, dotenv woud just be able to exec("source .env"), but Ruby doesn't offer a way to execute a command in the current process and then return control back to ruby (and there are some behavioral differences that I don't think we want, which I'll get to in a minute). exec will replace the current process and then exit once the command is run. spawn and backticks return a subshell, so any exported variables don't get returned.

So this PR adds dotenv-source, which is a shell command that just sources a file and then prints the env after. ShellParser#source calls that shell script, and then parse the printed env into a hash and returns just changed values from the current ENV (and eventually will update ENV).

Biggest problem: conditional assignment

dotenv (by default) does conditional assignment and you can override defaults at runtime. Given this file:

VAR_1=default
VAR_2="$VAR_1 and some stuff"

Here is the current behavior:

$ dotenv ruby -e "puts ENV['VAR_2']"default and some extra
$ VAR_1="changed" dotenv ruby -e "puts ENV['VAR_2']"changed and some stuff

But shell/bash uses : ${VAR:="default"} for conditional assignment, so sourcing this .env with the ShellParser has different behavior:

$ VAR_1=ignored ruby -Ilib -r dotenv/shell_parser -e 'puts Dotenv::ShellParser.new.source(".env")["VAR_2"]'default and some stuff

So we need to find a way to support the current behavior without forcing people to change syntax.

---

TODO:

• Solve conditional assignment problem
• Get remaining tests passing (and either deprecate or document syntax that is not valid in shell, like VAR=spaces without quotes)
• Add way to opt into this new parser (maybe DOTENV_PARSER=shell at top of file?)

cc @graywolf-at-work
cc #422