fix(demo): make the docker demo datasources reachable

[nadeem4]

Two bugs, both of which had to be fixed for the Docker demo to work at all

Bug 1 — ${env:...} in any non-password field broke adapter validation

DatasourceRegistry.resolved_connection() wrapped every resolved ${...}
placeholder in SecretStr, not just the passwords:

ifisinstance(value, str) andvalue.startswith("${") andvalue.endswith("}"):
secret_val=self.find_and_resolve_secret(value)
resolved_connection[key] =SecretStr(secret_val) # unconditional

The wrap could not be made selective at that point, because ConnectionConfig
carries no type information to work from — it declares a single field, type: str,
with extra: "allow". Everything else (host, port, user, password, driver)
arrives as an untyped extra. The real types live in each adapter's own model, e.g.
PostgresConnectionConfig: host: str, user: str, password: SecretStr. The
registry has no way to tell a password apart from a hostname.

So user: "${env:DEMO_REF_USER}" reached the adapter as a SecretStr and validation
blew up:

pydantic_core._pydantic_core.ValidationError: 3 validation errors for PostgresConnectionConfig
host
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
user
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
port
Input should be a valid integer [type=int_type, input_value=SecretStr('**********'), input_type=SecretStr]

Every DEMO_DOCKER_DATASOURCES entry uses ${env:...} for user, so the Docker
demo has never been able to register a datasource.

The fix is to stop wrapping. Pydantic already coerces a plain string into
SecretStr for fields declared that way, so returning the resolved plain string
gives user a str and password a SecretStr — both correct, with masking
preserved exactly where it is declared.

Password masking — one consequence, handled

resolved_connection has a single caller, which immediately .model_dump()s the
result into connection_args. Tracing where that goes turned up two places where
the plaintext password was previously masked by SecretStr's repr and would now
be exposed:

1. DatasourceAPI.get_datasource_details() returns adapter.connection_args
   verbatim to callers. Now masked by name via mask_connection_args.
2. A pydantic ValidationError renders the whole input dict when a required
   field is missing, and nl2sql doctor prints that message straight to the
   console. Adapter construction is now wrapped so the message is redacted, and
   re-raised from None so the traceback does not carry the cause either.

The adapter itself still receives the real value — only the rendered views are
masked. The connection URI has always contained the plaintext password; that is
unchanged by this PR.

Bug 2 — localhost is the wrong host inside the app container

DEMO_DOCKER_DATASOURCES hardcoded host: "localhost" with the published ports
(5433, 5434, 3307, 1434). That is right from the developer's machine and wrong from
inside the app container, which has to reach the databases by Compose service name
on their internal ports.

Both callers have to keep working, so host and port are now env-driven with
host-appropriate defaults:

Caller	Host	Port	Source
nl2sql --env demo index / run	localhost	published (5433/5434/3307/1434)	.env.demo
the app container	Compose service name	internal (5432/5432/3306/1433)	environment: on app, which overrides env_file

This uses the same ${env:...} convention already in place for user and password —
which is exactly why Bug 1 had to be fixed first, or the new host and port
references would have hit the same SecretStr failure.

Tests

New tests in the existing files:

• resolved_connection returns str for non-secret fields, and a real
  PostgresConnectionConfig still receives a SecretStr password.
• A DEMO_DOCKER_DATASOURCES-shaped config with ${env:...} in user, host
  and port registers and produces the expected URI. This fails against main
  with the three-error ValidationError quoted above.
• Registration errors do not echo the resolved password; get_datasource_details
  masks it.
• The generated Compose app service sets the service-name/internal-port
  overrides, the generated .env defaults to localhost + published ports, and
  the databases keep their published ports for host access.

Verification

• Unit: 255 passed, 1 skipped, 47 deselected (was 247/1/47; +8 new), run twice.
• Key-free integration: 28 passed, run twice.
• nl2sql setup --demo --docker, then docker compose config --services:
  • default → manufacturing_ref, manufacturing_supply, manufacturing_ops, app
  • --profile mssql → the above plus manufacturing_history
  • Compose's own resolved output confirms the app service ends up with
    DEMO_REF_HOST: manufacturing_ref / DEMO_REF_PORT: "5432", overriding the
    localhost:5433 in .env.demo.
• End-to-end without Docker: with the DEMO_* vars set, all four docker-shaped
  datasources resolve to plain str and build sane URIs, e.g.
  postgresql://ref_admin:********@manufacturing_ref:5432/manufacturing_ref and
  mssql+pyodbc://history_admin:********@manufacturing_history:1433/manufacturing_history?driver=ODBC+Driver+17+for+SQL+Server.

No images were pulled and no containers were started. Docs updated in
docs/getting_started/demo.md.

[nadeem4]

Two bugs, both of which had to be fixed for the Docker demo to work at all

Bug 1 — ${env:...} in any non-password field broke adapter validation

DatasourceRegistry.resolved_connection() wrapped every resolved ${...}
placeholder in SecretStr, not just the passwords:

ifisinstance(value, str) andvalue.startswith("${") andvalue.endswith("}"):
secret_val=self.find_and_resolve_secret(value)
resolved_connection[key] =SecretStr(secret_val) # unconditional

The wrap could not be made selective at that point, because ConnectionConfig
carries no type information to work from — it declares a single field, type: str,
with extra: "allow". Everything else (host, port, user, password, driver)
arrives as an untyped extra. The real types live in each adapter's own model, e.g.
PostgresConnectionConfig: host: str, user: str, password: SecretStr. The
registry has no way to tell a password apart from a hostname.

So user: "${env:DEMO_REF_USER}" reached the adapter as a SecretStr and validation
blew up:

pydantic_core._pydantic_core.ValidationError: 3 validation errors for PostgresConnectionConfig
host
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
user
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
port
Input should be a valid integer [type=int_type, input_value=SecretStr('**********'), input_type=SecretStr]

Every DEMO_DOCKER_DATASOURCES entry uses ${env:...} for user, so the Docker
demo has never been able to register a datasource.

The fix is to stop wrapping. Pydantic already coerces a plain string into
SecretStr for fields declared that way, so returning the resolved plain string
gives user a str and password a SecretStr — both correct, with masking
preserved exactly where it is declared.

Password masking — one consequence, handled

resolved_connection has a single caller, which immediately .model_dump()s the
result into connection_args. Tracing where that goes turned up two places where
the plaintext password was previously masked by SecretStr's repr and would now
be exposed:

1. DatasourceAPI.get_datasource_details() returns adapter.connection_args
   verbatim to callers. Now masked by name via mask_connection_args.
2. A pydantic ValidationError renders the whole input dict when a required
   field is missing, and nl2sql doctor prints that message straight to the
   console. Adapter construction is now wrapped so the message is redacted, and
   re-raised from None so the traceback does not carry the cause either.

The adapter itself still receives the real value — only the rendered views are
masked. The connection URI has always contained the plaintext password; that is
unchanged by this PR.

Bug 2 — localhost is the wrong host inside the app container

DEMO_DOCKER_DATASOURCES hardcoded host: "localhost" with the published ports
(5433, 5434, 3307, 1434). That is right from the developer's machine and wrong from
inside the app container, which has to reach the databases by Compose service name
on their internal ports.

Both callers have to keep working, so host and port are now env-driven with
host-appropriate defaults:

Caller	Host	Port	Source
nl2sql --env demo index / run	localhost	published (5433/5434/3307/1434)	.env.demo
the app container	Compose service name	internal (5432/5432/3306/1433)	environment: on app, which overrides env_file

This uses the same ${env:...} convention already in place for user and password —
which is exactly why Bug 1 had to be fixed first, or the new host and port
references would have hit the same SecretStr failure.

Tests

New tests in the existing files:

• resolved_connection returns str for non-secret fields, and a real
  PostgresConnectionConfig still receives a SecretStr password.
• A DEMO_DOCKER_DATASOURCES-shaped config with ${env:...} in user, host
  and port registers and produces the expected URI. This fails against main
  with the three-error ValidationError quoted above.
• Registration errors do not echo the resolved password; get_datasource_details
  masks it.
• The generated Compose app service sets the service-name/internal-port
  overrides, the generated .env defaults to localhost + published ports, and
  the databases keep their published ports for host access.

Verification

• Unit: 255 passed, 1 skipped, 47 deselected (was 247/1/47; +8 new), run twice.
• Key-free integration: 28 passed, run twice.
• nl2sql setup --demo --docker, then docker compose config --services:
  • default → manufacturing_ref, manufacturing_supply, manufacturing_ops, app
  • --profile mssql → the above plus manufacturing_history
  • Compose's own resolved output confirms the app service ends up with
    DEMO_REF_HOST: manufacturing_ref / DEMO_REF_PORT: "5432", overriding the
    localhost:5433 in .env.demo.
• End-to-end without Docker: with the DEMO_* vars set, all four docker-shaped
  datasources resolve to plain str and build sane URIs, e.g.
  postgresql://ref_admin:********@manufacturing_ref:5432/manufacturing_ref and
  mssql+pyodbc://history_admin:********@manufacturing_history:1433/manufacturing_history?driver=ODBC+Driver+17+for+SQL+Server.

No images were pulled and no containers were started. Docs updated in
docs/getting_started/demo.md.

[nadeem4]

Two bugs, both of which had to be fixed for the Docker demo to work at all

Bug 1 — ${env:...} in any non-password field broke adapter validation

DatasourceRegistry.resolved_connection() wrapped every resolved ${...}
placeholder in SecretStr, not just the passwords:

ifisinstance(value, str) andvalue.startswith("${") andvalue.endswith("}"):
secret_val=self.find_and_resolve_secret(value)
resolved_connection[key] =SecretStr(secret_val) # unconditional

The wrap could not be made selective at that point, because ConnectionConfig
carries no type information to work from — it declares a single field, type: str,
with extra: "allow". Everything else (host, port, user, password, driver)
arrives as an untyped extra. The real types live in each adapter's own model, e.g.
PostgresConnectionConfig: host: str, user: str, password: SecretStr. The
registry has no way to tell a password apart from a hostname.

So user: "${env:DEMO_REF_USER}" reached the adapter as a SecretStr and validation
blew up:

pydantic_core._pydantic_core.ValidationError: 3 validation errors for PostgresConnectionConfig
host
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
user
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
port
Input should be a valid integer [type=int_type, input_value=SecretStr('**********'), input_type=SecretStr]

Every DEMO_DOCKER_DATASOURCES entry uses ${env:...} for user, so the Docker
demo has never been able to register a datasource.

The fix is to stop wrapping. Pydantic already coerces a plain string into
SecretStr for fields declared that way, so returning the resolved plain string
gives user a str and password a SecretStr — both correct, with masking
preserved exactly where it is declared.

Password masking — one consequence, handled

resolved_connection has a single caller, which immediately .model_dump()s the
result into connection_args. Tracing where that goes turned up two places where
the plaintext password was previously masked by SecretStr's repr and would now
be exposed:

1. DatasourceAPI.get_datasource_details() returns adapter.connection_args
   verbatim to callers. Now masked by name via mask_connection_args.
2. A pydantic ValidationError renders the whole input dict when a required
   field is missing, and nl2sql doctor prints that message straight to the
   console. Adapter construction is now wrapped so the message is redacted, and
   re-raised from None so the traceback does not carry the cause either.

The adapter itself still receives the real value — only the rendered views are
masked. The connection URI has always contained the plaintext password; that is
unchanged by this PR.

Bug 2 — localhost is the wrong host inside the app container

DEMO_DOCKER_DATASOURCES hardcoded host: "localhost" with the published ports
(5433, 5434, 3307, 1434). That is right from the developer's machine and wrong from
inside the app container, which has to reach the databases by Compose service name
on their internal ports.

Both callers have to keep working, so host and port are now env-driven with
host-appropriate defaults:

Caller	Host	Port	Source
nl2sql --env demo index / run	localhost	published (5433/5434/3307/1434)	.env.demo
the app container	Compose service name	internal (5432/5432/3306/1433)	environment: on app, which overrides env_file

This uses the same ${env:...} convention already in place for user and password —
which is exactly why Bug 1 had to be fixed first, or the new host and port
references would have hit the same SecretStr failure.

Tests

New tests in the existing files:

• resolved_connection returns str for non-secret fields, and a real
  PostgresConnectionConfig still receives a SecretStr password.
• A DEMO_DOCKER_DATASOURCES-shaped config with ${env:...} in user, host
  and port registers and produces the expected URI. This fails against main
  with the three-error ValidationError quoted above.
• Registration errors do not echo the resolved password; get_datasource_details
  masks it.
• The generated Compose app service sets the service-name/internal-port
  overrides, the generated .env defaults to localhost + published ports, and
  the databases keep their published ports for host access.

Verification

• Unit: 255 passed, 1 skipped, 47 deselected (was 247/1/47; +8 new), run twice.
• Key-free integration: 28 passed, run twice.
• nl2sql setup --demo --docker, then docker compose config --services:
  • default → manufacturing_ref, manufacturing_supply, manufacturing_ops, app
  • --profile mssql → the above plus manufacturing_history
  • Compose's own resolved output confirms the app service ends up with
    DEMO_REF_HOST: manufacturing_ref / DEMO_REF_PORT: "5432", overriding the
    localhost:5433 in .env.demo.
• End-to-end without Docker: with the DEMO_* vars set, all four docker-shaped
  datasources resolve to plain str and build sane URIs, e.g.
  postgresql://ref_admin:********@manufacturing_ref:5432/manufacturing_ref and
  mssql+pyodbc://history_admin:********@manufacturing_history:1433/manufacturing_history?driver=ODBC+Driver+17+for+SQL+Server.

No images were pulled and no containers were started. Docs updated in
docs/getting_started/demo.md.

[nadeem4]

Two bugs, both of which had to be fixed for the Docker demo to work at all

Bug 1 — ${env:...} in any non-password field broke adapter validation

DatasourceRegistry.resolved_connection() wrapped every resolved ${...}
placeholder in SecretStr, not just the passwords:

ifisinstance(value, str) andvalue.startswith("${") andvalue.endswith("}"):
secret_val=self.find_and_resolve_secret(value)
resolved_connection[key] =SecretStr(secret_val) # unconditional

The wrap could not be made selective at that point, because ConnectionConfig
carries no type information to work from — it declares a single field, type: str,
with extra: "allow". Everything else (host, port, user, password, driver)
arrives as an untyped extra. The real types live in each adapter's own model, e.g.
PostgresConnectionConfig: host: str, user: str, password: SecretStr. The
registry has no way to tell a password apart from a hostname.

So user: "${env:DEMO_REF_USER}" reached the adapter as a SecretStr and validation
blew up:

pydantic_core._pydantic_core.ValidationError: 3 validation errors for PostgresConnectionConfig
host
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
user
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
port
Input should be a valid integer [type=int_type, input_value=SecretStr('**********'), input_type=SecretStr]

Every DEMO_DOCKER_DATASOURCES entry uses ${env:...} for user, so the Docker
demo has never been able to register a datasource.

The fix is to stop wrapping. Pydantic already coerces a plain string into
SecretStr for fields declared that way, so returning the resolved plain string
gives user a str and password a SecretStr — both correct, with masking
preserved exactly where it is declared.

Password masking — one consequence, handled

resolved_connection has a single caller, which immediately .model_dump()s the
result into connection_args. Tracing where that goes turned up two places where
the plaintext password was previously masked by SecretStr's repr and would now
be exposed:

1. DatasourceAPI.get_datasource_details() returns adapter.connection_args
   verbatim to callers. Now masked by name via mask_connection_args.
2. A pydantic ValidationError renders the whole input dict when a required
   field is missing, and nl2sql doctor prints that message straight to the
   console. Adapter construction is now wrapped so the message is redacted, and
   re-raised from None so the traceback does not carry the cause either.

The adapter itself still receives the real value — only the rendered views are
masked. The connection URI has always contained the plaintext password; that is
unchanged by this PR.

Bug 2 — localhost is the wrong host inside the app container

DEMO_DOCKER_DATASOURCES hardcoded host: "localhost" with the published ports
(5433, 5434, 3307, 1434). That is right from the developer's machine and wrong from
inside the app container, which has to reach the databases by Compose service name
on their internal ports.

Both callers have to keep working, so host and port are now env-driven with
host-appropriate defaults:

Caller	Host	Port	Source
nl2sql --env demo index / run	localhost	published (5433/5434/3307/1434)	.env.demo
the app container	Compose service name	internal (5432/5432/3306/1433)	environment: on app, which overrides env_file

This uses the same ${env:...} convention already in place for user and password —
which is exactly why Bug 1 had to be fixed first, or the new host and port
references would have hit the same SecretStr failure.

Tests

New tests in the existing files:

• resolved_connection returns str for non-secret fields, and a real
  PostgresConnectionConfig still receives a SecretStr password.
• A DEMO_DOCKER_DATASOURCES-shaped config with ${env:...} in user, host
  and port registers and produces the expected URI. This fails against main
  with the three-error ValidationError quoted above.
• Registration errors do not echo the resolved password; get_datasource_details
  masks it.
• The generated Compose app service sets the service-name/internal-port
  overrides, the generated .env defaults to localhost + published ports, and
  the databases keep their published ports for host access.

Verification

• Unit: 255 passed, 1 skipped, 47 deselected (was 247/1/47; +8 new), run twice.
• Key-free integration: 28 passed, run twice.
• nl2sql setup --demo --docker, then docker compose config --services:
  • default → manufacturing_ref, manufacturing_supply, manufacturing_ops, app
  • --profile mssql → the above plus manufacturing_history
  • Compose's own resolved output confirms the app service ends up with
    DEMO_REF_HOST: manufacturing_ref / DEMO_REF_PORT: "5432", overriding the
    localhost:5433 in .env.demo.
• End-to-end without Docker: with the DEMO_* vars set, all four docker-shaped
  datasources resolve to plain str and build sane URIs, e.g.
  postgresql://ref_admin:********@manufacturing_ref:5432/manufacturing_ref and
  mssql+pyodbc://history_admin:********@manufacturing_history:1433/manufacturing_history?driver=ODBC+Driver+17+for+SQL+Server.

No images were pulled and no containers were started. Docs updated in
docs/getting_started/demo.md.

[nadeem4]

Two bugs, both of which had to be fixed for the Docker demo to work at all

Bug 1 — ${env:...} in any non-password field broke adapter validation

DatasourceRegistry.resolved_connection() wrapped every resolved ${...}
placeholder in SecretStr, not just the passwords:

ifisinstance(value, str) andvalue.startswith("${") andvalue.endswith("}"):
secret_val=self.find_and_resolve_secret(value)
resolved_connection[key] =SecretStr(secret_val) # unconditional

The wrap could not be made selective at that point, because ConnectionConfig
carries no type information to work from — it declares a single field, type: str,
with extra: "allow". Everything else (host, port, user, password, driver)
arrives as an untyped extra. The real types live in each adapter's own model, e.g.
PostgresConnectionConfig: host: str, user: str, password: SecretStr. The
registry has no way to tell a password apart from a hostname.

So user: "${env:DEMO_REF_USER}" reached the adapter as a SecretStr and validation
blew up:

pydantic_core._pydantic_core.ValidationError: 3 validation errors for PostgresConnectionConfig
host
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
user
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
port
Input should be a valid integer [type=int_type, input_value=SecretStr('**********'), input_type=SecretStr]

Every DEMO_DOCKER_DATASOURCES entry uses ${env:...} for user, so the Docker
demo has never been able to register a datasource.

The fix is to stop wrapping. Pydantic already coerces a plain string into
SecretStr for fields declared that way, so returning the resolved plain string
gives user a str and password a SecretStr — both correct, with masking
preserved exactly where it is declared.

Password masking — one consequence, handled

resolved_connection has a single caller, which immediately .model_dump()s the
result into connection_args. Tracing where that goes turned up two places where
the plaintext password was previously masked by SecretStr's repr and would now
be exposed:

1. DatasourceAPI.get_datasource_details() returns adapter.connection_args
   verbatim to callers. Now masked by name via mask_connection_args.
2. A pydantic ValidationError renders the whole input dict when a required
   field is missing, and nl2sql doctor prints that message straight to the
   console. Adapter construction is now wrapped so the message is redacted, and
   re-raised from None so the traceback does not carry the cause either.

The adapter itself still receives the real value — only the rendered views are
masked. The connection URI has always contained the plaintext password; that is
unchanged by this PR.

Bug 2 — localhost is the wrong host inside the app container

DEMO_DOCKER_DATASOURCES hardcoded host: "localhost" with the published ports
(5433, 5434, 3307, 1434). That is right from the developer's machine and wrong from
inside the app container, which has to reach the databases by Compose service name
on their internal ports.

Both callers have to keep working, so host and port are now env-driven with
host-appropriate defaults:

Caller	Host	Port	Source
nl2sql --env demo index / run	localhost	published (5433/5434/3307/1434)	.env.demo
the app container	Compose service name	internal (5432/5432/3306/1433)	environment: on app, which overrides env_file

This uses the same ${env:...} convention already in place for user and password —
which is exactly why Bug 1 had to be fixed first, or the new host and port
references would have hit the same SecretStr failure.

Tests

New tests in the existing files:

• resolved_connection returns str for non-secret fields, and a real
  PostgresConnectionConfig still receives a SecretStr password.
• A DEMO_DOCKER_DATASOURCES-shaped config with ${env:...} in user, host
  and port registers and produces the expected URI. This fails against main
  with the three-error ValidationError quoted above.
• Registration errors do not echo the resolved password; get_datasource_details
  masks it.
• The generated Compose app service sets the service-name/internal-port
  overrides, the generated .env defaults to localhost + published ports, and
  the databases keep their published ports for host access.

Verification

• Unit: 255 passed, 1 skipped, 47 deselected (was 247/1/47; +8 new), run twice.
• Key-free integration: 28 passed, run twice.
• nl2sql setup --demo --docker, then docker compose config --services:
  • default → manufacturing_ref, manufacturing_supply, manufacturing_ops, app
  • --profile mssql → the above plus manufacturing_history
  • Compose's own resolved output confirms the app service ends up with
    DEMO_REF_HOST: manufacturing_ref / DEMO_REF_PORT: "5432", overriding the
    localhost:5433 in .env.demo.
• End-to-end without Docker: with the DEMO_* vars set, all four docker-shaped
  datasources resolve to plain str and build sane URIs, e.g.
  postgresql://ref_admin:********@manufacturing_ref:5432/manufacturing_ref and
  mssql+pyodbc://history_admin:********@manufacturing_history:1433/manufacturing_history?driver=ODBC+Driver+17+for+SQL+Server.

No images were pulled and no containers were started. Docs updated in
docs/getting_started/demo.md.

[nadeem4]

Two bugs, both of which had to be fixed for the Docker demo to work at all

Bug 1 — ${env:...} in any non-password field broke adapter validation

DatasourceRegistry.resolved_connection() wrapped every resolved ${...}
placeholder in SecretStr, not just the passwords:

ifisinstance(value, str) andvalue.startswith("${") andvalue.endswith("}"):
secret_val=self.find_and_resolve_secret(value)
resolved_connection[key] =SecretStr(secret_val) # unconditional

The wrap could not be made selective at that point, because ConnectionConfig
carries no type information to work from — it declares a single field, type: str,
with extra: "allow". Everything else (host, port, user, password, driver)
arrives as an untyped extra. The real types live in each adapter's own model, e.g.
PostgresConnectionConfig: host: str, user: str, password: SecretStr. The
registry has no way to tell a password apart from a hostname.

So user: "${env:DEMO_REF_USER}" reached the adapter as a SecretStr and validation
blew up:

pydantic_core._pydantic_core.ValidationError: 3 validation errors for PostgresConnectionConfig
host
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
user
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
port
Input should be a valid integer [type=int_type, input_value=SecretStr('**********'), input_type=SecretStr]

Every DEMO_DOCKER_DATASOURCES entry uses ${env:...} for user, so the Docker
demo has never been able to register a datasource.

The fix is to stop wrapping. Pydantic already coerces a plain string into
SecretStr for fields declared that way, so returning the resolved plain string
gives user a str and password a SecretStr — both correct, with masking
preserved exactly where it is declared.

Password masking — one consequence, handled

resolved_connection has a single caller, which immediately .model_dump()s the
result into connection_args. Tracing where that goes turned up two places where
the plaintext password was previously masked by SecretStr's repr and would now
be exposed:

1. DatasourceAPI.get_datasource_details() returns adapter.connection_args
   verbatim to callers. Now masked by name via mask_connection_args.
2. A pydantic ValidationError renders the whole input dict when a required
   field is missing, and nl2sql doctor prints that message straight to the
   console. Adapter construction is now wrapped so the message is redacted, and
   re-raised from None so the traceback does not carry the cause either.

The adapter itself still receives the real value — only the rendered views are
masked. The connection URI has always contained the plaintext password; that is
unchanged by this PR.

Bug 2 — localhost is the wrong host inside the app container

DEMO_DOCKER_DATASOURCES hardcoded host: "localhost" with the published ports
(5433, 5434, 3307, 1434). That is right from the developer's machine and wrong from
inside the app container, which has to reach the databases by Compose service name
on their internal ports.

Both callers have to keep working, so host and port are now env-driven with
host-appropriate defaults:

Caller	Host	Port	Source
nl2sql --env demo index / run	localhost	published (5433/5434/3307/1434)	.env.demo
the app container	Compose service name	internal (5432/5432/3306/1433)	environment: on app, which overrides env_file

This uses the same ${env:...} convention already in place for user and password —
which is exactly why Bug 1 had to be fixed first, or the new host and port
references would have hit the same SecretStr failure.

Tests

New tests in the existing files:

• resolved_connection returns str for non-secret fields, and a real
  PostgresConnectionConfig still receives a SecretStr password.
• A DEMO_DOCKER_DATASOURCES-shaped config with ${env:...} in user, host
  and port registers and produces the expected URI. This fails against main
  with the three-error ValidationError quoted above.
• Registration errors do not echo the resolved password; get_datasource_details
  masks it.
• The generated Compose app service sets the service-name/internal-port
  overrides, the generated .env defaults to localhost + published ports, and
  the databases keep their published ports for host access.

Verification

• Unit: 255 passed, 1 skipped, 47 deselected (was 247/1/47; +8 new), run twice.
• Key-free integration: 28 passed, run twice.
• nl2sql setup --demo --docker, then docker compose config --services:
  • default → manufacturing_ref, manufacturing_supply, manufacturing_ops, app
  • --profile mssql → the above plus manufacturing_history
  • Compose's own resolved output confirms the app service ends up with
    DEMO_REF_HOST: manufacturing_ref / DEMO_REF_PORT: "5432", overriding the
    localhost:5433 in .env.demo.
• End-to-end without Docker: with the DEMO_* vars set, all four docker-shaped
  datasources resolve to plain str and build sane URIs, e.g.
  postgresql://ref_admin:********@manufacturing_ref:5432/manufacturing_ref and
  mssql+pyodbc://history_admin:********@manufacturing_history:1433/manufacturing_history?driver=ODBC+Driver+17+for+SQL+Server.

No images were pulled and no containers were started. Docs updated in
docs/getting_started/demo.md.

[nadeem4]

Two bugs, both of which had to be fixed for the Docker demo to work at all

Bug 1 — ${env:...} in any non-password field broke adapter validation

DatasourceRegistry.resolved_connection() wrapped every resolved ${...}
placeholder in SecretStr, not just the passwords:

ifisinstance(value, str) andvalue.startswith("${") andvalue.endswith("}"):
secret_val=self.find_and_resolve_secret(value)
resolved_connection[key] =SecretStr(secret_val) # unconditional

The wrap could not be made selective at that point, because ConnectionConfig
carries no type information to work from — it declares a single field, type: str,
with extra: "allow". Everything else (host, port, user, password, driver)
arrives as an untyped extra. The real types live in each adapter's own model, e.g.
PostgresConnectionConfig: host: str, user: str, password: SecretStr. The
registry has no way to tell a password apart from a hostname.

So user: "${env:DEMO_REF_USER}" reached the adapter as a SecretStr and validation
blew up:

pydantic_core._pydantic_core.ValidationError: 3 validation errors for PostgresConnectionConfig
host
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
user
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
port
Input should be a valid integer [type=int_type, input_value=SecretStr('**********'), input_type=SecretStr]

Every DEMO_DOCKER_DATASOURCES entry uses ${env:...} for user, so the Docker
demo has never been able to register a datasource.

The fix is to stop wrapping. Pydantic already coerces a plain string into
SecretStr for fields declared that way, so returning the resolved plain string
gives user a str and password a SecretStr — both correct, with masking
preserved exactly where it is declared.

Password masking — one consequence, handled

resolved_connection has a single caller, which immediately .model_dump()s the
result into connection_args. Tracing where that goes turned up two places where
the plaintext password was previously masked by SecretStr's repr and would now
be exposed:

1. DatasourceAPI.get_datasource_details() returns adapter.connection_args
   verbatim to callers. Now masked by name via mask_connection_args.
2. A pydantic ValidationError renders the whole input dict when a required
   field is missing, and nl2sql doctor prints that message straight to the
   console. Adapter construction is now wrapped so the message is redacted, and
   re-raised from None so the traceback does not carry the cause either.

The adapter itself still receives the real value — only the rendered views are
masked. The connection URI has always contained the plaintext password; that is
unchanged by this PR.

Bug 2 — localhost is the wrong host inside the app container

DEMO_DOCKER_DATASOURCES hardcoded host: "localhost" with the published ports
(5433, 5434, 3307, 1434). That is right from the developer's machine and wrong from
inside the app container, which has to reach the databases by Compose service name
on their internal ports.

Both callers have to keep working, so host and port are now env-driven with
host-appropriate defaults:

Caller	Host	Port	Source
nl2sql --env demo index / run	localhost	published (5433/5434/3307/1434)	.env.demo
the app container	Compose service name	internal (5432/5432/3306/1433)	environment: on app, which overrides env_file

This uses the same ${env:...} convention already in place for user and password —
which is exactly why Bug 1 had to be fixed first, or the new host and port
references would have hit the same SecretStr failure.

Tests

New tests in the existing files:

• resolved_connection returns str for non-secret fields, and a real
  PostgresConnectionConfig still receives a SecretStr password.
• A DEMO_DOCKER_DATASOURCES-shaped config with ${env:...} in user, host
  and port registers and produces the expected URI. This fails against main
  with the three-error ValidationError quoted above.
• Registration errors do not echo the resolved password; get_datasource_details
  masks it.
• The generated Compose app service sets the service-name/internal-port
  overrides, the generated .env defaults to localhost + published ports, and
  the databases keep their published ports for host access.

Verification

• Unit: 255 passed, 1 skipped, 47 deselected (was 247/1/47; +8 new), run twice.
• Key-free integration: 28 passed, run twice.
• nl2sql setup --demo --docker, then docker compose config --services:
  • default → manufacturing_ref, manufacturing_supply, manufacturing_ops, app
  • --profile mssql → the above plus manufacturing_history
  • Compose's own resolved output confirms the app service ends up with
    DEMO_REF_HOST: manufacturing_ref / DEMO_REF_PORT: "5432", overriding the
    localhost:5433 in .env.demo.
• End-to-end without Docker: with the DEMO_* vars set, all four docker-shaped
  datasources resolve to plain str and build sane URIs, e.g.
  postgresql://ref_admin:********@manufacturing_ref:5432/manufacturing_ref and
  mssql+pyodbc://history_admin:********@manufacturing_history:1433/manufacturing_history?driver=ODBC+Driver+17+for+SQL+Server.

No images were pulled and no containers were started. Docs updated in
docs/getting_started/demo.md.

[nadeem4]

Two bugs, both of which had to be fixed for the Docker demo to work at all

Bug 1 — ${env:...} in any non-password field broke adapter validation

DatasourceRegistry.resolved_connection() wrapped every resolved ${...}
placeholder in SecretStr, not just the passwords:

ifisinstance(value, str) andvalue.startswith("${") andvalue.endswith("}"):
secret_val=self.find_and_resolve_secret(value)
resolved_connection[key] =SecretStr(secret_val) # unconditional

The wrap could not be made selective at that point, because ConnectionConfig
carries no type information to work from — it declares a single field, type: str,
with extra: "allow". Everything else (host, port, user, password, driver)
arrives as an untyped extra. The real types live in each adapter's own model, e.g.
PostgresConnectionConfig: host: str, user: str, password: SecretStr. The
registry has no way to tell a password apart from a hostname.

So user: "${env:DEMO_REF_USER}" reached the adapter as a SecretStr and validation
blew up:

pydantic_core._pydantic_core.ValidationError: 3 validation errors for PostgresConnectionConfig
host
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
user
Input should be a valid string [type=string_type, input_value=SecretStr('**********'), input_type=SecretStr]
port
Input should be a valid integer [type=int_type, input_value=SecretStr('**********'), input_type=SecretStr]

Every DEMO_DOCKER_DATASOURCES entry uses ${env:...} for user, so the Docker
demo has never been able to register a datasource.

The fix is to stop wrapping. Pydantic already coerces a plain string into
SecretStr for fields declared that way, so returning the resolved plain string
gives user a str and password a SecretStr — both correct, with masking
preserved exactly where it is declared.

Password masking — one consequence, handled

resolved_connection has a single caller, which immediately .model_dump()s the
result into connection_args. Tracing where that goes turned up two places where
the plaintext password was previously masked by SecretStr's repr and would now
be exposed:

1. DatasourceAPI.get_datasource_details() returns adapter.connection_args
   verbatim to callers. Now masked by name via mask_connection_args.
2. A pydantic ValidationError renders the whole input dict when a required
   field is missing, and nl2sql doctor prints that message straight to the
   console. Adapter construction is now wrapped so the message is redacted, and
   re-raised from None so the traceback does not carry the cause either.

The adapter itself still receives the real value — only the rendered views are
masked. The connection URI has always contained the plaintext password; that is
unchanged by this PR.

Bug 2 — localhost is the wrong host inside the app container

DEMO_DOCKER_DATASOURCES hardcoded host: "localhost" with the published ports
(5433, 5434, 3307, 1434). That is right from the developer's machine and wrong from
inside the app container, which has to reach the databases by Compose service name
on their internal ports.

Both callers have to keep working, so host and port are now env-driven with
host-appropriate defaults:

Caller	Host	Port	Source
nl2sql --env demo index / run	localhost	published (5433/5434/3307/1434)	.env.demo
the app container	Compose service name	internal (5432/5432/3306/1433)	environment: on app, which overrides env_file

This uses the same ${env:...} convention already in place for user and password —
which is exactly why Bug 1 had to be fixed first, or the new host and port
references would have hit the same SecretStr failure.

Tests

New tests in the existing files:

• resolved_connection returns str for non-secret fields, and a real
  PostgresConnectionConfig still receives a SecretStr password.
• A DEMO_DOCKER_DATASOURCES-shaped config with ${env:...} in user, host
  and port registers and produces the expected URI. This fails against main
  with the three-error ValidationError quoted above.
• Registration errors do not echo the resolved password; get_datasource_details
  masks it.
• The generated Compose app service sets the service-name/internal-port
  overrides, the generated .env defaults to localhost + published ports, and
  the databases keep their published ports for host access.

Verification

• Unit: 255 passed, 1 skipped, 47 deselected (was 247/1/47; +8 new), run twice.
• Key-free integration: 28 passed, run twice.
• nl2sql setup --demo --docker, then docker compose config --services:
  • default → manufacturing_ref, manufacturing_supply, manufacturing_ops, app
  • --profile mssql → the above plus manufacturing_history
  • Compose's own resolved output confirms the app service ends up with
    DEMO_REF_HOST: manufacturing_ref / DEMO_REF_PORT: "5432", overriding the
    localhost:5433 in .env.demo.
• End-to-end without Docker: with the DEMO_* vars set, all four docker-shaped
  datasources resolve to plain str and build sane URIs, e.g.
  postgresql://ref_admin:********@manufacturing_ref:5432/manufacturing_ref and
  mssql+pyodbc://history_admin:********@manufacturing_history:1433/manufacturing_history?driver=ODBC+Driver+17+for+SQL+Server.

No images were pulled and no containers were started. Docs updated in
docs/getting_started/demo.md.