Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/security/cvm-boundaries.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -126,7 +126,7 @@ dstack uses encrypted environment variables to allow app developers to securely
This file is not measured to RTMRs. But it is highly recommended to add application-specific integrity checks on encrypted environment variables at the application layer. See [security-best-practices.md](./security-best-practices.md) for more details.

### .user-config
This is an optional application-specific configuration file that applications inside the CVM can access. dstack OS simply stores it at /dstack/.host-shared/.user-config without any measurement or additional processing.
This is an optional application-specific configuration file that applications inside the CVM can access. dstack OS simply stores it at /dstack/.host-shared/.user-config without any measurement or additional processing, unless `requirements.launch_token_hash` is set in app-compose.json — in that case the guest reads the launch token from JSON path `dstack.launch_token` in this file and fails closed at boot, before key provisioning, unless its SHA-256 matches the pinned hash.

Application developers should perform integrity checks on user_config at the application layer if necessary.

Expand Down
12 changes: 12 additions & 0 deletions docs/security/security-best-practices.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -40,6 +40,18 @@ If your App is intended for end users who need to verify what code your App is r

dstack provides encrypted environment variable functionality. Although the CVM physical machine controller cannot view encrypted environment variables, they may forge encrypted environment variables because the CVM encryption public key is known to everyone. Therefore, Apps need to perform auth checks on encrypted environment variables at the application layer. LAUNCH_TOKEN pattern is one method to prevent unauthorized envs replacement. For details, refer to the deployment script of [dstack-gateway](https://github.com/Dstack-TEE/dstack/blob/1b8a4516826b02f9d7f747eddac244dcd68fc325/gateway/dstack-app/deploy-to-vmm.sh#L150-L165).

Newer dstack OS images support the LAUNCH_TOKEN pattern natively via `requirements.launch_token_hash` in app-compose.json. When this field is set, the guest reads the launch token from `user_config` at JSON path `dstack.launch_token` and refuses to boot — before any keys are provisioned — unless its digest matches the hash pinned in the (compose-hash-measured) app-compose.json. When the field is absent, `user_config` is not parsed and stays fully application-defined. Set manifest_version to `"3"` (string) when using `requirements` so older guests fail closed instead of silently ignoring it.

The digest is domain-separated so it stays distinct from the legacy plain-`sha256(token)` convention and from generic precomputed tables:

```bash
LAUNCH_TOKEN_HASH=$(printf 'dstack-launch-token/v1:%s' "$TOKEN" | sha256sum | cut -d' ' -f1)
```

Because `launch_token_hash` is public, a guessable token can be recovered offline by brute force. Guests reject tokens shorter than 32 bytes, but length alone does not guarantee entropy — always generate the token randomly, e.g. `tr -dc 'a-zA-Z0-9' < /dev/urandom | head -c 32`.

Also understand the protection boundary of this mechanism: the guest verifies the token before any keys are provisioned, which means the token must reach the guest through `user_config` — a channel the host can read. The requirement therefore stops parties who only know the public app-compose.json from launching the app, but once a host has hosted a deployment it learns the token and can later relaunch instances of that compose with substituted encrypted envs. Mitigations: generate a fresh token per deployment and remove stale compose hashes from the on-chain whitelist; if the token must stay secret from the host, use the app-layer `APP_LAUNCH_TOKEN` encrypted-env pattern above instead (its check necessarily runs after key provisioning).

If you use dstack-vmm's built-in UI, the prelaunch script has already been automatically filled in for you:

![Prelaunch Script](../assets/prelaunch-script.png)
Expand Down
64 changes: 63 additions & 1 deletion dstack-types/src/lib.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -131,16 +131,46 @@ pub struct Requirements {
/// with ACPI tables measured, while `false` requires lite mode.
#[serde(skip_serializing_if = "Option::is_none")]
pub tdx_measure_acpi_tables: Option<bool>,
/// Hex digest of the launch token carried in `user_config` at JSON path
/// `dstack.launch_token`, computed as
/// `sha256("dstack-launch-token/v1:" || token)` (see
/// [`launch_token_hash`]). When set, guests fail closed before key
/// provisioning unless the token hashes to this value; when absent,
/// `user_config` is not parsed at all.
///
/// This hash is public, so the token must not be guessable: guests reject
/// tokens shorter than 32 bytes, and deployers should use a random token
/// (e.g. 32 random alphanumeric characters).
#[serde(skip_serializing_if = "Option::is_none")]
pub launch_token_hash: Option<String>,
}

impl Requirements {
pub fn is_empty(&self) -> bool {
self.os_version.is_none()
&& self.platforms.is_none()
&& self.tdx_measure_acpi_tables.is_none()
&& self.launch_token_hash.is_none()
}
}

/// Domain-separation prefix for [`launch_token_hash`]. It keeps the digest
/// distinct from a plain `sha256(token)` (as used by the legacy app-layer
/// top-level `launch_token_hash` convention) and from generic precomputed
/// tables.
pub const LAUNCH_TOKEN_HASH_DOMAIN: &str = "dstack-launch-token/v1:";

/// Canonical `requirements.launch_token_hash` digest of a launch token:
/// `sha256("dstack-launch-token/v1:" || token)`.
///
/// Shell equivalent: `printf 'dstack-launch-token/v1:%s' "$TOKEN" | sha256sum`.
pub fn launch_token_hash(token: &str) -> [u8; 32] {
let mut data = Vec::with_capacity(LAUNCH_TOKEN_HASH_DOMAIN.len() + token.len());
data.extend_from_slice(LAUNCH_TOKEN_HASH_DOMAIN.as_bytes());
data.extend_from_slice(token.as_bytes());
sha256(&data)
}

fn deserialize_manifest_version<'de, D>(deserializer: D) -> Result<String, D::Error>
where
D: serde::Deserializer<'de>,
Expand DownExpand Up@@ -389,7 +419,8 @@ mod app_compose_tests {
"requirements": {
"os_version": ">=0.6.1",
"platforms": ["dstack-gcp-tdx", "dstack-tdx"],
"tdx_measure_acpi_tables": true
"tdx_measure_acpi_tables": true,
"launch_token_hash": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
}
}))
.unwrap();
Expand All@@ -400,6 +431,10 @@ mod app_compose_tests {
Some(vec!["dstack-gcp-tdx".to_string(), "dstack-tdx".to_string()])
);
assert_eq!(requirements.tdx_measure_acpi_tables, Some(true));
assert_eq!(
requirements.launch_token_hash.as_deref(),
Some("9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08")
);

let err = serde_json::from_value::<AppCompose>(serde_json::json!({
"manifest_version": "3",
Expand DownExpand Up@@ -451,6 +486,33 @@ mod app_compose_tests {
let requirements = acpi_tables.requirements.as_ref().unwrap();
assert_eq!(requirements.tdx_measure_acpi_tables, Some(false));
assert!(!requirements.is_empty());

let launch_token: AppCompose = serde_json::from_value(serde_json::json!({
"manifest_version": "3",
"name": "test",
"runner": "docker-compose",
"requirements": {
"launch_token_hash": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
}
}))
.unwrap();
let requirements = launch_token.requirements.as_ref().unwrap();
assert!(requirements.launch_token_hash.is_some());
assert!(!requirements.is_empty());
}

#[test]
fn launch_token_hash_is_domain_separated() {
assert_eq!(
hex::encode(launch_token_hash("unit-test-launch-token-0000000001")),
"28faa1319055d733ad9651f5ab7689c15b04609846bcd27b3c5bc8df6246f5a3"
);
// Not a plain sha256 of the token (the legacy app-layer convention).
use sha2::{Digest, Sha256};
assert_ne!(
launch_token_hash("unit-test-launch-token-0000000001").to_vec(),
Sha256::digest("unit-test-launch-token-0000000001".as_bytes()).to_vec()
);
}
}

Expand Down
135 changes: 132 additions & 3 deletions dstack-util/src/system_setup.rs
Original file line numberDiff line numberDiff line change
Expand Up@@ -229,6 +229,10 @@ impl HostShareDir {
fn instance_info_file(&self) -> PathBuf {
self.base_dir.join(INSTANCE_INFO)
}

fn user_config_file(&self) -> PathBuf {
self.base_dir.join(USER_CONFIG)
}
}

struct HostShared {
Expand DownExpand Up@@ -774,7 +778,9 @@ fn verify_manifest_version(app_compose: &AppCompose) -> Result<u32> {
Ok(manifest_version)
}

fn verify_app_compose_policy(app_compose: &AppCompose, sys_config: &SysConfig) -> Result<()> {
fn verify_app_compose_policy(shared: &HostShared) -> Result<()> {
let app_compose = &shared.app_compose;
let sys_config = &shared.sys_config;
verify_manifest_feature_requirements(app_compose)?;
let Some(requirements) = app_compose.requirements.as_ref() else {
return Ok(());
Expand All@@ -801,6 +807,14 @@ fn verify_app_compose_policy(app_compose: &AppCompose, sys_config: &SysConfig) -
current_platform,
)?;
}
if let Some(launch_token_hash) = requirements.launch_token_hash.as_deref() {
// Only touch user_config when the requirement is present; otherwise it
// is opaque application data and must not be parsed here.
let user_config = fs::read_to_string(shared.dir.user_config_file())
.context("failed to read user_config for requirements.launch_token_hash")?;
let token = launch_token_from_user_config(&user_config)?;
verify_launch_token_requirement(launch_token_hash, &token)?;
}
Ok(())
}

Expand DownExpand Up@@ -911,6 +925,55 @@ fn verify_tdx_measure_acpi_tables_requirement(
Ok(())
}

/// Minimum launch token length in bytes. `launch_token_hash` is public (it is
/// part of app-compose.json), so short tokens can be recovered offline via
/// brute force or precomputed tables. Length cannot prove entropy, but it
/// rejects the trivially guessable tokens; deployers should still generate
/// random tokens (e.g. 32 random alphanumeric characters).
const LAUNCH_TOKEN_MIN_LEN: usize = 32;

/// Enforce the launch-token pattern: the compose-hash-measured
/// `requirements.launch_token_hash` must match the domain-separated digest of
/// the launch token (see [`dstack_types::launch_token_hash`]). This binds a
/// deployment to a token known only to the deployer, so a host cannot launch
/// the app with substituted inputs.
fn verify_launch_token_requirement(launch_token_hash: &str, token: &str) -> Result<()> {
let expected = hex::decode(launch_token_hash)
.context("invalid requirements.launch_token_hash: not a hex string")?;
if expected.len() != 32 {
bail!(
"invalid requirements.launch_token_hash: expected 32-byte sha256 hex, got {} bytes",
expected.len()
);
}
if token.len() < LAUNCH_TOKEN_MIN_LEN {
bail!(
"launch token too short: got {} bytes, minimum is {LAUNCH_TOKEN_MIN_LEN}; use a random token since launch_token_hash is public",
token.len()
);
}
if dstack_types::launch_token_hash(token)[..] != expected[..] {
bail!("launch token mismatch: sha256(\"{}\" || launch token) does not match requirements.launch_token_hash", dstack_types::LAUNCH_TOKEN_HASH_DOMAIN);
}
info!("launch token requirement satisfied");
Ok(())
}

/// Extract the launch token from `user_config` at JSON path
/// `dstack.launch_token`. Callers must only invoke this when
/// `requirements.launch_token_hash` is set; otherwise `user_config` is opaque
/// application data and must not be parsed.
fn launch_token_from_user_config(user_config: &str) -> Result<String> {
let user_config: Value = serde_json::from_str(user_config)
.context("failed to parse user_config as JSON for requirements.launch_token_hash")?;
let token = user_config
.pointer("/dstack/launch_token")
.context("user_config is missing dstack.launch_token")?
.as_str()
.context("user_config dstack.launch_token is not a string")?;
Ok(token.to_string())
}

fn read_current_os_version() -> Result<String> {
const OS_RELEASE_PATHS: &[&str] = &["/etc/os-release", "/usr/lib/os-release"];
for path in OS_RELEASE_PATHS {
Expand DownExpand Up@@ -974,8 +1037,7 @@ pub async fn cmd_sys_setup(args: SetupArgs) -> Result<()> {
}

async fn do_sys_setup(stage0: Stage0<'_>) -> Result<()> {
verify_app_compose_policy(&stage0.shared.app_compose, &stage0.shared.sys_config)
.context("Failed to verify app-compose policy")?;
verify_app_compose_policy(&stage0.shared).context("Failed to verify app-compose policy")?;
if stage0.shared.app_compose.secure_time {
info!("Waiting for the system time to be synchronized");
cmd! {
Expand DownExpand Up@@ -2306,6 +2368,73 @@ fn test_tdx_measure_acpi_tables_requirement_ignored_on_non_tdx() {
.unwrap();
}

#[cfg(test)]
const TEST_LAUNCH_TOKEN: &str = "unit-test-launch-token-0000000001";
#[cfg(test)]
// sha256("dstack-launch-token/v1:" || TEST_LAUNCH_TOKEN)
const TEST_LAUNCH_TOKEN_HASH: &str =
"28faa1319055d733ad9651f5ab7689c15b04609846bcd27b3c5bc8df6246f5a3";

#[test]
fn test_launch_token_requirement_accepts_matching_token() {
verify_launch_token_requirement(TEST_LAUNCH_TOKEN_HASH, TEST_LAUNCH_TOKEN).unwrap();
}

#[test]
fn test_launch_token_requirement_rejects_wrong_token() {
let err = verify_launch_token_requirement(
TEST_LAUNCH_TOKEN_HASH,
"wrong-launch-token-00000000000001",
)
.unwrap_err();
assert!(err.to_string().contains("launch token mismatch"));
}

#[test]
fn test_launch_token_requirement_rejects_short_token() {
// sha256("dstack-launch-token/v1:test"): a matching but brute-forceable
// token must be rejected.
let err = verify_launch_token_requirement(
"e128cf5f3c3633d3a1f450d3d4bece260b20f9afb667de4bbff6dd985f1e5d1a",
"test",
)
.unwrap_err();
assert!(err.to_string().contains("launch token too short"));
let err = verify_launch_token_requirement(TEST_LAUNCH_TOKEN_HASH, "").unwrap_err();
assert!(err.to_string().contains("launch token too short"));
// 31 bytes is one short of the minimum.
let err = verify_launch_token_requirement(TEST_LAUNCH_TOKEN_HASH, &"a".repeat(31)).unwrap_err();
assert!(err.to_string().contains("launch token too short"));
}

#[test]
fn test_launch_token_requirement_rejects_invalid_hash() {
let err = verify_launch_token_requirement("zz", TEST_LAUNCH_TOKEN).unwrap_err();
assert!(err.to_string().contains("not a hex string"));
let err = verify_launch_token_requirement("9f86d0", TEST_LAUNCH_TOKEN).unwrap_err();
assert!(err.to_string().contains("expected 32-byte sha256 hex"));
}

#[test]
fn test_launch_token_from_user_config_extracts_token() {
let user_config = r#"{"dstack":{"launch_token":"test"},"app":{"foo":"bar"}}"#;
assert_eq!(launch_token_from_user_config(user_config).unwrap(), "test");
}

#[test]
fn test_launch_token_from_user_config_rejects_missing_or_invalid_token() {
let err = launch_token_from_user_config(r#"{}"#).unwrap_err();
assert!(err.to_string().contains("missing dstack.launch_token"));
let err = launch_token_from_user_config(r#"{"dstack":{}}"#).unwrap_err();
assert!(err.to_string().contains("missing dstack.launch_token"));
let err = launch_token_from_user_config(r#"{"dstack":{"launch_token":42}}"#).unwrap_err();
assert!(err.to_string().contains("not a string"));
let err = launch_token_from_user_config("not json").unwrap_err();
assert!(err
.to_string()
.contains("failed to parse user_config as JSON"));
}

#[test]
fn test_os_release_value_parses_quoted_version_id() {
let content = r#"
Expand Down
1 change: 1 addition & 0 deletions sdk/go/dstack/compose_hash.go
Original file line numberDiff line numberDiff line change
Expand Up@@ -42,6 +42,7 @@ type Requirements struct {
OsVersion string `json:"os_version,omitempty"`
Platforms *[]RequirementPlatform `json:"platforms,omitempty"`
TdxMeasureAcpiTables *bool `json:"tdx_measure_acpi_tables,omitempty"`
LaunchTokenHash string `json:"launch_token_hash,omitempty"`
}

// AppCompose represents the application composition structure
Expand Down
1 change: 1 addition & 0 deletions sdk/js/src/get-compose-hash.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -52,6 +52,7 @@ export interface Requirements extends SortableObject {
os_version?: string;
platforms?: RequirementPlatform[];
tdx_measure_acpi_tables?: boolean;
launch_token_hash?: string;
}

export interface AppCompose extends SortableObject {
Expand Down
4 changes: 4 additions & 0 deletions sdk/python/src/dstack_sdk/get_compose_hash.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -54,11 +54,13 @@ def __init__(
os_version: Optional[str] = None,
platforms: Optional[List[str]] = None,
tdx_measure_acpi_tables: Optional[bool] = None,
launch_token_hash: Optional[str] = None,
) -> None:
"""Initialize a new ``Requirements`` instance."""
self.os_version = os_version
self.platforms = platforms
self.tdx_measure_acpi_tables = tdx_measure_acpi_tables
self.launch_token_hash = launch_token_hash

def to_dict(self) -> Dict[str, Any]:
"""Return a dictionary representation excluding ``None`` fields."""
Expand All@@ -69,6 +71,8 @@ def to_dict(self) -> Dict[str, Any]:
result["platforms"] = self.platforms
if self.tdx_measure_acpi_tables is not None:
result["tdx_measure_acpi_tables"] = self.tdx_measure_acpi_tables
if self.launch_token_hash is not None:
result["launch_token_hash"] = self.launch_token_hash
return result


Expand Down
1 change: 1 addition & 0 deletions vmm/ui/src/composables/useVmManager.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -44,6 +44,7 @@ type Requirements = {
os_version?: string;
platforms?: RequirementPlatform[];
tdx_measure_acpi_tables?: boolean;
launch_token_hash?: string;
Comment thread
kvinwang marked this conversation as resolved.
};

const x25519 = require('../lib/x25519.js');
Expand Down
Loading