feat(go-sdk): add wasi hostcalls used by the Go SDK by leonm1 · Pull Request #427 · proxy-wasm/proxy-wasm-cpp-host

leonm1 · 2024-12-13T18:55:45Z

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present.

PiotrSikora

Thanks for working on this!

Two high-level comments:

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls? For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?
Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

leonm1 · 2024-12-18T22:13:45Z

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls?

Answered elsewhere as well, but I was mistaken about some of the filesystem syscalls. This PR now represents the full minimal set of added hostcalls for a helloworld go plugin to initialize.

For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?

Yield should be fine due to the linked behavior. For polling, I do wonder about that — technically we could support some of the subscription types, but I'm not sure how much value they add in a single-threaded environment anyway, since we only support FDs 1 and 2, which never get "closed" and wouldn't "block" from the wasm modules perspective.

Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

I tested the CPP host with all of service extensions' cpp and rust plugins. Since these are added hostcalls, they should not affect currently-working wasm modules. I attempted to add explicit calls to each of sched_yield, poll, and fdstat_set_flags (which corresponds to posix's fcntl), and confirmed each translated the libc call to the corresponding wasi hostcall and none complained about the ENOSYS (or ESUCCESS, in sched_yield's case).

PiotrSikora · 2024-12-19T17:04:23Z

Since these are added hostcalls, they should not affect currently-working wasm modules.

Right, this won't break any of the existing plugins. My point was mostly that with those changes (well, the earlier version of this PR) plugins that attempt to open files would load successfully, but could break at runtime.

PiotrSikora

Thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

martijneken

LGTM, thanks!

mpwarres

LGTM, thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: zty98751 <zty98751@alibaba-inc.com>

… behavior (proxy-wasm#434) (#11) * fix: CI branch name master -> main (proxy-wasm#398) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Bump Abseil to fix Linux build issues (proxy-wasm#400) Bump Abseil to fix Linux build issues Pick up this fix: abseil/abseil-cpp#1187 Bump past Envoy to pick up another fix found in fuzz tests: proxy-wasm#399 (comment) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Update cargo-raze -> crate_universe (proxy-wasm#399) - Updated platforms for crate_universe compatibility - Supports upgrade to wasmsign2 - Includes workaround for Windows path length issue Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Move from unavailable macos-11 to macos-13 (proxy-wasm#401) See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources This does not fixproxy-wasm#384, but does resurface those errors. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump Bazel from 5.2.0 to 6.5.0 (proxy-wasm#402) Bump Bazel from 5.2.0 to 6.5.0 This breaks the s390x build which relied on an external Docker image. I made some strides in fixing s390x, but it's not yet working. Deferred to proxy-wasm#405. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump rules_python and rules_fuzzing (proxy-wasm#404) Upgrade rules_python (0.34.0) and rules_fuzzing (0.5.2) This requires extracting WORKSPACE phases into more phases: - dependencies -- py_repositories() and toolchains - dependencies_python() -- pip_parse module loading - dependencies_import() -- python/fuzzing/other deps The new structure roughly matches Envoy WORKSPACE: - envoy_dependencies() - envoy_dependencies_extra() -- not needed here - envoy_python_dependencies() - envoy_dependency_imports() Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update CI to use Ubuntu 22.04 / clang 14 (proxy-wasm#408) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update rules_rust to v0.42.1 (with Rust v1.77.2). (proxy-wasm#410) * Update rules_rust * Update rust and vendor * rust_oom -> rg_oom * Change rust version --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Update wasmtime (v24.0.0) (proxy-wasm#406) Removes Wasmtime + Windows CI because rules_rust has recently dropped Windows: https://github.com/bazelbuild/rules_rust/blob/main/docs/index.md#supported-platforms Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix: Upgrade deprecated artifact upload/download handlers (proxy-wasm#415) Seen on proxy-wasm#380 CI: Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v2`. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/ Signed-off-by: Martijn Stevenson <mstevenson@google.com> * bump wamr to 2.1.1 and able to consume precompiled content (proxy-wasm#380) - skip leading paddings in .aot section Signed-off-by: liang.he@intel.com <liang.he@intel.com> * compdb add the compdb support to the proxy_wasm_cpp_host (proxy-wasm#419) * compdb add the compdb support to the proxy_wasm_cpp_host Signed-off-by: wangbaiping <wbphub@gmail.com> * Fix references to prefix_wasm_api (proxy-wasm#420) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * feat(go-sdk): add wasi hostcalls used by the Go SDK (proxy-wasm#427) The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> * chore: workflow runner fixes (proxy-wasm#436) Assorted changes to get workflows working again: - Update format workflows to use ubuntu-22.04 - Update windows-2019 to windows-2022 and add a missing <string> include needed to build with that - Enable manual triggering of workflows Fixesproxy-wasm#435 --------- Signed-off-by: Michael Warres <mpw@google.com> * feat: add knob to customise on{Request,Response}Headers StopIteration behavior (proxy-wasm#434) Add protected ContextBase::allow_on_headers_stop_iteration_ field that can be used by host implementations to control whether or not ContextBase propagates FilterHeaderStatus::StopIteration returned by onRequestHeaders() or onResponseHeaders() without modification. Follow-on envoyproxy/envoy#40213 adds an option in Envoy WasmFilter PluginConfig that sets the value of this field. For details, see [Envoy Wasm / Proxy-Wasm support for FilterHeadersStatus::StopIteration](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?usp=sharing). This PR is one part of implementing [Option B: WasmFilter config knob](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?tab=t.0#bookmark=id.5wxldlapsp54). Note that default behavior of proxy-wasm-cpp-host and ContextBase is unchanged. --------- Signed-off-by: Michael Warres <mpw@google.com> --------- Signed-off-by: Martijn Stevenson <mstevenson@google.com> Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: liang.he@intel.com <liang.he@intel.com> Signed-off-by: wangbaiping <wbphub@gmail.com> Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: Michael Warres <mpw@google.com> Co-authored-by: martijneken <mstevenson@google.com> Co-authored-by: Keith Mattix II <keithmattix2@gmail.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: liang.he <liang.he@intel.com> Co-authored-by: code <wbphub@gmail.com> Co-authored-by: Matt Leon <ml@mattleon.com> Co-authored-by: Michael Warres <mpw@google.com>

In #427, a portion of the wasi hostcalls was added, but not all. Now, all the wasi hostcalls have been included, and their stability has been verified in our multiple go 1.24 compiled wasm plugins --------- Signed-off-by: zty98751 <zty98751@alibaba-inc.com> Signed-off-by: Matt Leon <mattleon@google.com> Co-authored-by: zty98751 <zty98751@alibaba-inc.com>

leonm1 · 2024-12-13T18:55:45Z

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present.

PiotrSikora

Thanks for working on this!

Two high-level comments:

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls? For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?
Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

leonm1 · 2024-12-18T22:13:45Z

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls?

Answered elsewhere as well, but I was mistaken about some of the filesystem syscalls. This PR now represents the full minimal set of added hostcalls for a helloworld go plugin to initialize.

For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?

Yield should be fine due to the linked behavior. For polling, I do wonder about that — technically we could support some of the subscription types, but I'm not sure how much value they add in a single-threaded environment anyway, since we only support FDs 1 and 2, which never get "closed" and wouldn't "block" from the wasm modules perspective.

Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

I tested the CPP host with all of service extensions' cpp and rust plugins. Since these are added hostcalls, they should not affect currently-working wasm modules. I attempted to add explicit calls to each of sched_yield, poll, and fdstat_set_flags (which corresponds to posix's fcntl), and confirmed each translated the libc call to the corresponding wasi hostcall and none complained about the ENOSYS (or ESUCCESS, in sched_yield's case).

PiotrSikora · 2024-12-19T17:04:23Z

Since these are added hostcalls, they should not affect currently-working wasm modules.

Right, this won't break any of the existing plugins. My point was mostly that with those changes (well, the earlier version of this PR) plugins that attempt to open files would load successfully, but could break at runtime.

PiotrSikora

Thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

martijneken

LGTM, thanks!

mpwarres

LGTM, thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: zty98751 <zty98751@alibaba-inc.com>

… behavior (proxy-wasm#434) (#11) * fix: CI branch name master -> main (proxy-wasm#398) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Bump Abseil to fix Linux build issues (proxy-wasm#400) Bump Abseil to fix Linux build issues Pick up this fix: abseil/abseil-cpp#1187 Bump past Envoy to pick up another fix found in fuzz tests: proxy-wasm#399 (comment) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Update cargo-raze -> crate_universe (proxy-wasm#399) - Updated platforms for crate_universe compatibility - Supports upgrade to wasmsign2 - Includes workaround for Windows path length issue Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Move from unavailable macos-11 to macos-13 (proxy-wasm#401) See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources This does not fixproxy-wasm#384, but does resurface those errors. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump Bazel from 5.2.0 to 6.5.0 (proxy-wasm#402) Bump Bazel from 5.2.0 to 6.5.0 This breaks the s390x build which relied on an external Docker image. I made some strides in fixing s390x, but it's not yet working. Deferred to proxy-wasm#405. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump rules_python and rules_fuzzing (proxy-wasm#404) Upgrade rules_python (0.34.0) and rules_fuzzing (0.5.2) This requires extracting WORKSPACE phases into more phases: - dependencies -- py_repositories() and toolchains - dependencies_python() -- pip_parse module loading - dependencies_import() -- python/fuzzing/other deps The new structure roughly matches Envoy WORKSPACE: - envoy_dependencies() - envoy_dependencies_extra() -- not needed here - envoy_python_dependencies() - envoy_dependency_imports() Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update CI to use Ubuntu 22.04 / clang 14 (proxy-wasm#408) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update rules_rust to v0.42.1 (with Rust v1.77.2). (proxy-wasm#410) * Update rules_rust * Update rust and vendor * rust_oom -> rg_oom * Change rust version --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Update wasmtime (v24.0.0) (proxy-wasm#406) Removes Wasmtime + Windows CI because rules_rust has recently dropped Windows: https://github.com/bazelbuild/rules_rust/blob/main/docs/index.md#supported-platforms Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix: Upgrade deprecated artifact upload/download handlers (proxy-wasm#415) Seen on proxy-wasm#380 CI: Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v2`. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/ Signed-off-by: Martijn Stevenson <mstevenson@google.com> * bump wamr to 2.1.1 and able to consume precompiled content (proxy-wasm#380) - skip leading paddings in .aot section Signed-off-by: liang.he@intel.com <liang.he@intel.com> * compdb add the compdb support to the proxy_wasm_cpp_host (proxy-wasm#419) * compdb add the compdb support to the proxy_wasm_cpp_host Signed-off-by: wangbaiping <wbphub@gmail.com> * Fix references to prefix_wasm_api (proxy-wasm#420) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * feat(go-sdk): add wasi hostcalls used by the Go SDK (proxy-wasm#427) The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> * chore: workflow runner fixes (proxy-wasm#436) Assorted changes to get workflows working again: - Update format workflows to use ubuntu-22.04 - Update windows-2019 to windows-2022 and add a missing <string> include needed to build with that - Enable manual triggering of workflows Fixesproxy-wasm#435 --------- Signed-off-by: Michael Warres <mpw@google.com> * feat: add knob to customise on{Request,Response}Headers StopIteration behavior (proxy-wasm#434) Add protected ContextBase::allow_on_headers_stop_iteration_ field that can be used by host implementations to control whether or not ContextBase propagates FilterHeaderStatus::StopIteration returned by onRequestHeaders() or onResponseHeaders() without modification. Follow-on envoyproxy/envoy#40213 adds an option in Envoy WasmFilter PluginConfig that sets the value of this field. For details, see [Envoy Wasm / Proxy-Wasm support for FilterHeadersStatus::StopIteration](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?usp=sharing). This PR is one part of implementing [Option B: WasmFilter config knob](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?tab=t.0#bookmark=id.5wxldlapsp54). Note that default behavior of proxy-wasm-cpp-host and ContextBase is unchanged. --------- Signed-off-by: Michael Warres <mpw@google.com> --------- Signed-off-by: Martijn Stevenson <mstevenson@google.com> Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: liang.he@intel.com <liang.he@intel.com> Signed-off-by: wangbaiping <wbphub@gmail.com> Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: Michael Warres <mpw@google.com> Co-authored-by: martijneken <mstevenson@google.com> Co-authored-by: Keith Mattix II <keithmattix2@gmail.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: liang.he <liang.he@intel.com> Co-authored-by: code <wbphub@gmail.com> Co-authored-by: Matt Leon <ml@mattleon.com> Co-authored-by: Michael Warres <mpw@google.com>

In #427, a portion of the wasi hostcalls was added, but not all. Now, all the wasi hostcalls have been included, and their stability has been verified in our multiple go 1.24 compiled wasm plugins --------- Signed-off-by: zty98751 <zty98751@alibaba-inc.com> Signed-off-by: Matt Leon <mattleon@google.com> Co-authored-by: zty98751 <zty98751@alibaba-inc.com>

leonm1 · 2024-12-13T18:55:45Z

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present.

PiotrSikora

Thanks for working on this!

Two high-level comments:

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls? For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?
Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

leonm1 · 2024-12-18T22:13:45Z

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls?

Answered elsewhere as well, but I was mistaken about some of the filesystem syscalls. This PR now represents the full minimal set of added hostcalls for a helloworld go plugin to initialize.

For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?

Yield should be fine due to the linked behavior. For polling, I do wonder about that — technically we could support some of the subscription types, but I'm not sure how much value they add in a single-threaded environment anyway, since we only support FDs 1 and 2, which never get "closed" and wouldn't "block" from the wasm modules perspective.

Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

I tested the CPP host with all of service extensions' cpp and rust plugins. Since these are added hostcalls, they should not affect currently-working wasm modules. I attempted to add explicit calls to each of sched_yield, poll, and fdstat_set_flags (which corresponds to posix's fcntl), and confirmed each translated the libc call to the corresponding wasi hostcall and none complained about the ENOSYS (or ESUCCESS, in sched_yield's case).

PiotrSikora · 2024-12-19T17:04:23Z

Since these are added hostcalls, they should not affect currently-working wasm modules.

Right, this won't break any of the existing plugins. My point was mostly that with those changes (well, the earlier version of this PR) plugins that attempt to open files would load successfully, but could break at runtime.

PiotrSikora

Thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

martijneken

LGTM, thanks!

mpwarres

LGTM, thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: zty98751 <zty98751@alibaba-inc.com>

… behavior (proxy-wasm#434) (#11) * fix: CI branch name master -> main (proxy-wasm#398) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Bump Abseil to fix Linux build issues (proxy-wasm#400) Bump Abseil to fix Linux build issues Pick up this fix: abseil/abseil-cpp#1187 Bump past Envoy to pick up another fix found in fuzz tests: proxy-wasm#399 (comment) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Update cargo-raze -> crate_universe (proxy-wasm#399) - Updated platforms for crate_universe compatibility - Supports upgrade to wasmsign2 - Includes workaround for Windows path length issue Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Move from unavailable macos-11 to macos-13 (proxy-wasm#401) See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources This does not fixproxy-wasm#384, but does resurface those errors. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump Bazel from 5.2.0 to 6.5.0 (proxy-wasm#402) Bump Bazel from 5.2.0 to 6.5.0 This breaks the s390x build which relied on an external Docker image. I made some strides in fixing s390x, but it's not yet working. Deferred to proxy-wasm#405. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump rules_python and rules_fuzzing (proxy-wasm#404) Upgrade rules_python (0.34.0) and rules_fuzzing (0.5.2) This requires extracting WORKSPACE phases into more phases: - dependencies -- py_repositories() and toolchains - dependencies_python() -- pip_parse module loading - dependencies_import() -- python/fuzzing/other deps The new structure roughly matches Envoy WORKSPACE: - envoy_dependencies() - envoy_dependencies_extra() -- not needed here - envoy_python_dependencies() - envoy_dependency_imports() Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update CI to use Ubuntu 22.04 / clang 14 (proxy-wasm#408) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update rules_rust to v0.42.1 (with Rust v1.77.2). (proxy-wasm#410) * Update rules_rust * Update rust and vendor * rust_oom -> rg_oom * Change rust version --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Update wasmtime (v24.0.0) (proxy-wasm#406) Removes Wasmtime + Windows CI because rules_rust has recently dropped Windows: https://github.com/bazelbuild/rules_rust/blob/main/docs/index.md#supported-platforms Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix: Upgrade deprecated artifact upload/download handlers (proxy-wasm#415) Seen on proxy-wasm#380 CI: Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v2`. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/ Signed-off-by: Martijn Stevenson <mstevenson@google.com> * bump wamr to 2.1.1 and able to consume precompiled content (proxy-wasm#380) - skip leading paddings in .aot section Signed-off-by: liang.he@intel.com <liang.he@intel.com> * compdb add the compdb support to the proxy_wasm_cpp_host (proxy-wasm#419) * compdb add the compdb support to the proxy_wasm_cpp_host Signed-off-by: wangbaiping <wbphub@gmail.com> * Fix references to prefix_wasm_api (proxy-wasm#420) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * feat(go-sdk): add wasi hostcalls used by the Go SDK (proxy-wasm#427) The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> * chore: workflow runner fixes (proxy-wasm#436) Assorted changes to get workflows working again: - Update format workflows to use ubuntu-22.04 - Update windows-2019 to windows-2022 and add a missing <string> include needed to build with that - Enable manual triggering of workflows Fixesproxy-wasm#435 --------- Signed-off-by: Michael Warres <mpw@google.com> * feat: add knob to customise on{Request,Response}Headers StopIteration behavior (proxy-wasm#434) Add protected ContextBase::allow_on_headers_stop_iteration_ field that can be used by host implementations to control whether or not ContextBase propagates FilterHeaderStatus::StopIteration returned by onRequestHeaders() or onResponseHeaders() without modification. Follow-on envoyproxy/envoy#40213 adds an option in Envoy WasmFilter PluginConfig that sets the value of this field. For details, see [Envoy Wasm / Proxy-Wasm support for FilterHeadersStatus::StopIteration](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?usp=sharing). This PR is one part of implementing [Option B: WasmFilter config knob](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?tab=t.0#bookmark=id.5wxldlapsp54). Note that default behavior of proxy-wasm-cpp-host and ContextBase is unchanged. --------- Signed-off-by: Michael Warres <mpw@google.com> --------- Signed-off-by: Martijn Stevenson <mstevenson@google.com> Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: liang.he@intel.com <liang.he@intel.com> Signed-off-by: wangbaiping <wbphub@gmail.com> Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: Michael Warres <mpw@google.com> Co-authored-by: martijneken <mstevenson@google.com> Co-authored-by: Keith Mattix II <keithmattix2@gmail.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: liang.he <liang.he@intel.com> Co-authored-by: code <wbphub@gmail.com> Co-authored-by: Matt Leon <ml@mattleon.com> Co-authored-by: Michael Warres <mpw@google.com>

In #427, a portion of the wasi hostcalls was added, but not all. Now, all the wasi hostcalls have been included, and their stability has been verified in our multiple go 1.24 compiled wasm plugins --------- Signed-off-by: zty98751 <zty98751@alibaba-inc.com> Signed-off-by: Matt Leon <mattleon@google.com> Co-authored-by: zty98751 <zty98751@alibaba-inc.com>

leonm1 · 2024-12-13T18:55:45Z

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present.

PiotrSikora

Thanks for working on this!

Two high-level comments:

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls? For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?
Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

leonm1 · 2024-12-18T22:13:45Z

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls?

Answered elsewhere as well, but I was mistaken about some of the filesystem syscalls. This PR now represents the full minimal set of added hostcalls for a helloworld go plugin to initialize.

For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?

Yield should be fine due to the linked behavior. For polling, I do wonder about that — technically we could support some of the subscription types, but I'm not sure how much value they add in a single-threaded environment anyway, since we only support FDs 1 and 2, which never get "closed" and wouldn't "block" from the wasm modules perspective.

Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

I tested the CPP host with all of service extensions' cpp and rust plugins. Since these are added hostcalls, they should not affect currently-working wasm modules. I attempted to add explicit calls to each of sched_yield, poll, and fdstat_set_flags (which corresponds to posix's fcntl), and confirmed each translated the libc call to the corresponding wasi hostcall and none complained about the ENOSYS (or ESUCCESS, in sched_yield's case).

PiotrSikora · 2024-12-19T17:04:23Z

Since these are added hostcalls, they should not affect currently-working wasm modules.

Right, this won't break any of the existing plugins. My point was mostly that with those changes (well, the earlier version of this PR) plugins that attempt to open files would load successfully, but could break at runtime.

PiotrSikora

Thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

martijneken

LGTM, thanks!

mpwarres

LGTM, thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: zty98751 <zty98751@alibaba-inc.com>

… behavior (proxy-wasm#434) (#11) * fix: CI branch name master -> main (proxy-wasm#398) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Bump Abseil to fix Linux build issues (proxy-wasm#400) Bump Abseil to fix Linux build issues Pick up this fix: abseil/abseil-cpp#1187 Bump past Envoy to pick up another fix found in fuzz tests: proxy-wasm#399 (comment) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Update cargo-raze -> crate_universe (proxy-wasm#399) - Updated platforms for crate_universe compatibility - Supports upgrade to wasmsign2 - Includes workaround for Windows path length issue Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Move from unavailable macos-11 to macos-13 (proxy-wasm#401) See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources This does not fixproxy-wasm#384, but does resurface those errors. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump Bazel from 5.2.0 to 6.5.0 (proxy-wasm#402) Bump Bazel from 5.2.0 to 6.5.0 This breaks the s390x build which relied on an external Docker image. I made some strides in fixing s390x, but it's not yet working. Deferred to proxy-wasm#405. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump rules_python and rules_fuzzing (proxy-wasm#404) Upgrade rules_python (0.34.0) and rules_fuzzing (0.5.2) This requires extracting WORKSPACE phases into more phases: - dependencies -- py_repositories() and toolchains - dependencies_python() -- pip_parse module loading - dependencies_import() -- python/fuzzing/other deps The new structure roughly matches Envoy WORKSPACE: - envoy_dependencies() - envoy_dependencies_extra() -- not needed here - envoy_python_dependencies() - envoy_dependency_imports() Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update CI to use Ubuntu 22.04 / clang 14 (proxy-wasm#408) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update rules_rust to v0.42.1 (with Rust v1.77.2). (proxy-wasm#410) * Update rules_rust * Update rust and vendor * rust_oom -> rg_oom * Change rust version --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Update wasmtime (v24.0.0) (proxy-wasm#406) Removes Wasmtime + Windows CI because rules_rust has recently dropped Windows: https://github.com/bazelbuild/rules_rust/blob/main/docs/index.md#supported-platforms Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix: Upgrade deprecated artifact upload/download handlers (proxy-wasm#415) Seen on proxy-wasm#380 CI: Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v2`. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/ Signed-off-by: Martijn Stevenson <mstevenson@google.com> * bump wamr to 2.1.1 and able to consume precompiled content (proxy-wasm#380) - skip leading paddings in .aot section Signed-off-by: liang.he@intel.com <liang.he@intel.com> * compdb add the compdb support to the proxy_wasm_cpp_host (proxy-wasm#419) * compdb add the compdb support to the proxy_wasm_cpp_host Signed-off-by: wangbaiping <wbphub@gmail.com> * Fix references to prefix_wasm_api (proxy-wasm#420) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * feat(go-sdk): add wasi hostcalls used by the Go SDK (proxy-wasm#427) The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> * chore: workflow runner fixes (proxy-wasm#436) Assorted changes to get workflows working again: - Update format workflows to use ubuntu-22.04 - Update windows-2019 to windows-2022 and add a missing <string> include needed to build with that - Enable manual triggering of workflows Fixesproxy-wasm#435 --------- Signed-off-by: Michael Warres <mpw@google.com> * feat: add knob to customise on{Request,Response}Headers StopIteration behavior (proxy-wasm#434) Add protected ContextBase::allow_on_headers_stop_iteration_ field that can be used by host implementations to control whether or not ContextBase propagates FilterHeaderStatus::StopIteration returned by onRequestHeaders() or onResponseHeaders() without modification. Follow-on envoyproxy/envoy#40213 adds an option in Envoy WasmFilter PluginConfig that sets the value of this field. For details, see [Envoy Wasm / Proxy-Wasm support for FilterHeadersStatus::StopIteration](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?usp=sharing). This PR is one part of implementing [Option B: WasmFilter config knob](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?tab=t.0#bookmark=id.5wxldlapsp54). Note that default behavior of proxy-wasm-cpp-host and ContextBase is unchanged. --------- Signed-off-by: Michael Warres <mpw@google.com> --------- Signed-off-by: Martijn Stevenson <mstevenson@google.com> Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: liang.he@intel.com <liang.he@intel.com> Signed-off-by: wangbaiping <wbphub@gmail.com> Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: Michael Warres <mpw@google.com> Co-authored-by: martijneken <mstevenson@google.com> Co-authored-by: Keith Mattix II <keithmattix2@gmail.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: liang.he <liang.he@intel.com> Co-authored-by: code <wbphub@gmail.com> Co-authored-by: Matt Leon <ml@mattleon.com> Co-authored-by: Michael Warres <mpw@google.com>

In #427, a portion of the wasi hostcalls was added, but not all. Now, all the wasi hostcalls have been included, and their stability has been verified in our multiple go 1.24 compiled wasm plugins --------- Signed-off-by: zty98751 <zty98751@alibaba-inc.com> Signed-off-by: Matt Leon <mattleon@google.com> Co-authored-by: zty98751 <zty98751@alibaba-inc.com>

leonm1 · 2024-12-13T18:55:45Z

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present.

PiotrSikora

Thanks for working on this!

Two high-level comments:

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls? For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?
Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

leonm1 · 2024-12-18T22:13:45Z

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls?

Answered elsewhere as well, but I was mistaken about some of the filesystem syscalls. This PR now represents the full minimal set of added hostcalls for a helloworld go plugin to initialize.

For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?

Yield should be fine due to the linked behavior. For polling, I do wonder about that — technically we could support some of the subscription types, but I'm not sure how much value they add in a single-threaded environment anyway, since we only support FDs 1 and 2, which never get "closed" and wouldn't "block" from the wasm modules perspective.

Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

I tested the CPP host with all of service extensions' cpp and rust plugins. Since these are added hostcalls, they should not affect currently-working wasm modules. I attempted to add explicit calls to each of sched_yield, poll, and fdstat_set_flags (which corresponds to posix's fcntl), and confirmed each translated the libc call to the corresponding wasi hostcall and none complained about the ENOSYS (or ESUCCESS, in sched_yield's case).

PiotrSikora · 2024-12-19T17:04:23Z

Since these are added hostcalls, they should not affect currently-working wasm modules.

Right, this won't break any of the existing plugins. My point was mostly that with those changes (well, the earlier version of this PR) plugins that attempt to open files would load successfully, but could break at runtime.

PiotrSikora

Thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

martijneken

LGTM, thanks!

mpwarres

LGTM, thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: zty98751 <zty98751@alibaba-inc.com>

… behavior (proxy-wasm#434) (#11) * fix: CI branch name master -> main (proxy-wasm#398) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Bump Abseil to fix Linux build issues (proxy-wasm#400) Bump Abseil to fix Linux build issues Pick up this fix: abseil/abseil-cpp#1187 Bump past Envoy to pick up another fix found in fuzz tests: proxy-wasm#399 (comment) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Update cargo-raze -> crate_universe (proxy-wasm#399) - Updated platforms for crate_universe compatibility - Supports upgrade to wasmsign2 - Includes workaround for Windows path length issue Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Move from unavailable macos-11 to macos-13 (proxy-wasm#401) See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources This does not fixproxy-wasm#384, but does resurface those errors. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump Bazel from 5.2.0 to 6.5.0 (proxy-wasm#402) Bump Bazel from 5.2.0 to 6.5.0 This breaks the s390x build which relied on an external Docker image. I made some strides in fixing s390x, but it's not yet working. Deferred to proxy-wasm#405. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump rules_python and rules_fuzzing (proxy-wasm#404) Upgrade rules_python (0.34.0) and rules_fuzzing (0.5.2) This requires extracting WORKSPACE phases into more phases: - dependencies -- py_repositories() and toolchains - dependencies_python() -- pip_parse module loading - dependencies_import() -- python/fuzzing/other deps The new structure roughly matches Envoy WORKSPACE: - envoy_dependencies() - envoy_dependencies_extra() -- not needed here - envoy_python_dependencies() - envoy_dependency_imports() Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update CI to use Ubuntu 22.04 / clang 14 (proxy-wasm#408) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update rules_rust to v0.42.1 (with Rust v1.77.2). (proxy-wasm#410) * Update rules_rust * Update rust and vendor * rust_oom -> rg_oom * Change rust version --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Update wasmtime (v24.0.0) (proxy-wasm#406) Removes Wasmtime + Windows CI because rules_rust has recently dropped Windows: https://github.com/bazelbuild/rules_rust/blob/main/docs/index.md#supported-platforms Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix: Upgrade deprecated artifact upload/download handlers (proxy-wasm#415) Seen on proxy-wasm#380 CI: Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v2`. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/ Signed-off-by: Martijn Stevenson <mstevenson@google.com> * bump wamr to 2.1.1 and able to consume precompiled content (proxy-wasm#380) - skip leading paddings in .aot section Signed-off-by: liang.he@intel.com <liang.he@intel.com> * compdb add the compdb support to the proxy_wasm_cpp_host (proxy-wasm#419) * compdb add the compdb support to the proxy_wasm_cpp_host Signed-off-by: wangbaiping <wbphub@gmail.com> * Fix references to prefix_wasm_api (proxy-wasm#420) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * feat(go-sdk): add wasi hostcalls used by the Go SDK (proxy-wasm#427) The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> * chore: workflow runner fixes (proxy-wasm#436) Assorted changes to get workflows working again: - Update format workflows to use ubuntu-22.04 - Update windows-2019 to windows-2022 and add a missing <string> include needed to build with that - Enable manual triggering of workflows Fixesproxy-wasm#435 --------- Signed-off-by: Michael Warres <mpw@google.com> * feat: add knob to customise on{Request,Response}Headers StopIteration behavior (proxy-wasm#434) Add protected ContextBase::allow_on_headers_stop_iteration_ field that can be used by host implementations to control whether or not ContextBase propagates FilterHeaderStatus::StopIteration returned by onRequestHeaders() or onResponseHeaders() without modification. Follow-on envoyproxy/envoy#40213 adds an option in Envoy WasmFilter PluginConfig that sets the value of this field. For details, see [Envoy Wasm / Proxy-Wasm support for FilterHeadersStatus::StopIteration](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?usp=sharing). This PR is one part of implementing [Option B: WasmFilter config knob](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?tab=t.0#bookmark=id.5wxldlapsp54). Note that default behavior of proxy-wasm-cpp-host and ContextBase is unchanged. --------- Signed-off-by: Michael Warres <mpw@google.com> --------- Signed-off-by: Martijn Stevenson <mstevenson@google.com> Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: liang.he@intel.com <liang.he@intel.com> Signed-off-by: wangbaiping <wbphub@gmail.com> Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: Michael Warres <mpw@google.com> Co-authored-by: martijneken <mstevenson@google.com> Co-authored-by: Keith Mattix II <keithmattix2@gmail.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: liang.he <liang.he@intel.com> Co-authored-by: code <wbphub@gmail.com> Co-authored-by: Matt Leon <ml@mattleon.com> Co-authored-by: Michael Warres <mpw@google.com>

In #427, a portion of the wasi hostcalls was added, but not all. Now, all the wasi hostcalls have been included, and their stability has been verified in our multiple go 1.24 compiled wasm plugins --------- Signed-off-by: zty98751 <zty98751@alibaba-inc.com> Signed-off-by: Matt Leon <mattleon@google.com> Co-authored-by: zty98751 <zty98751@alibaba-inc.com>

leonm1 · 2024-12-13T18:55:45Z

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present.

PiotrSikora

Thanks for working on this!

Two high-level comments:

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls? For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?
Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

leonm1 · 2024-12-18T22:13:45Z

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls?

Answered elsewhere as well, but I was mistaken about some of the filesystem syscalls. This PR now represents the full minimal set of added hostcalls for a helloworld go plugin to initialize.

For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?

Yield should be fine due to the linked behavior. For polling, I do wonder about that — technically we could support some of the subscription types, but I'm not sure how much value they add in a single-threaded environment anyway, since we only support FDs 1 and 2, which never get "closed" and wouldn't "block" from the wasm modules perspective.

Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

I tested the CPP host with all of service extensions' cpp and rust plugins. Since these are added hostcalls, they should not affect currently-working wasm modules. I attempted to add explicit calls to each of sched_yield, poll, and fdstat_set_flags (which corresponds to posix's fcntl), and confirmed each translated the libc call to the corresponding wasi hostcall and none complained about the ENOSYS (or ESUCCESS, in sched_yield's case).

PiotrSikora · 2024-12-19T17:04:23Z

Since these are added hostcalls, they should not affect currently-working wasm modules.

Right, this won't break any of the existing plugins. My point was mostly that with those changes (well, the earlier version of this PR) plugins that attempt to open files would load successfully, but could break at runtime.

PiotrSikora

Thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

martijneken

LGTM, thanks!

mpwarres

LGTM, thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: zty98751 <zty98751@alibaba-inc.com>

… behavior (proxy-wasm#434) (#11) * fix: CI branch name master -> main (proxy-wasm#398) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Bump Abseil to fix Linux build issues (proxy-wasm#400) Bump Abseil to fix Linux build issues Pick up this fix: abseil/abseil-cpp#1187 Bump past Envoy to pick up another fix found in fuzz tests: proxy-wasm#399 (comment) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Update cargo-raze -> crate_universe (proxy-wasm#399) - Updated platforms for crate_universe compatibility - Supports upgrade to wasmsign2 - Includes workaround for Windows path length issue Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Move from unavailable macos-11 to macos-13 (proxy-wasm#401) See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources This does not fixproxy-wasm#384, but does resurface those errors. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump Bazel from 5.2.0 to 6.5.0 (proxy-wasm#402) Bump Bazel from 5.2.0 to 6.5.0 This breaks the s390x build which relied on an external Docker image. I made some strides in fixing s390x, but it's not yet working. Deferred to proxy-wasm#405. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump rules_python and rules_fuzzing (proxy-wasm#404) Upgrade rules_python (0.34.0) and rules_fuzzing (0.5.2) This requires extracting WORKSPACE phases into more phases: - dependencies -- py_repositories() and toolchains - dependencies_python() -- pip_parse module loading - dependencies_import() -- python/fuzzing/other deps The new structure roughly matches Envoy WORKSPACE: - envoy_dependencies() - envoy_dependencies_extra() -- not needed here - envoy_python_dependencies() - envoy_dependency_imports() Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update CI to use Ubuntu 22.04 / clang 14 (proxy-wasm#408) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update rules_rust to v0.42.1 (with Rust v1.77.2). (proxy-wasm#410) * Update rules_rust * Update rust and vendor * rust_oom -> rg_oom * Change rust version --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Update wasmtime (v24.0.0) (proxy-wasm#406) Removes Wasmtime + Windows CI because rules_rust has recently dropped Windows: https://github.com/bazelbuild/rules_rust/blob/main/docs/index.md#supported-platforms Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix: Upgrade deprecated artifact upload/download handlers (proxy-wasm#415) Seen on proxy-wasm#380 CI: Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v2`. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/ Signed-off-by: Martijn Stevenson <mstevenson@google.com> * bump wamr to 2.1.1 and able to consume precompiled content (proxy-wasm#380) - skip leading paddings in .aot section Signed-off-by: liang.he@intel.com <liang.he@intel.com> * compdb add the compdb support to the proxy_wasm_cpp_host (proxy-wasm#419) * compdb add the compdb support to the proxy_wasm_cpp_host Signed-off-by: wangbaiping <wbphub@gmail.com> * Fix references to prefix_wasm_api (proxy-wasm#420) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * feat(go-sdk): add wasi hostcalls used by the Go SDK (proxy-wasm#427) The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> * chore: workflow runner fixes (proxy-wasm#436) Assorted changes to get workflows working again: - Update format workflows to use ubuntu-22.04 - Update windows-2019 to windows-2022 and add a missing <string> include needed to build with that - Enable manual triggering of workflows Fixesproxy-wasm#435 --------- Signed-off-by: Michael Warres <mpw@google.com> * feat: add knob to customise on{Request,Response}Headers StopIteration behavior (proxy-wasm#434) Add protected ContextBase::allow_on_headers_stop_iteration_ field that can be used by host implementations to control whether or not ContextBase propagates FilterHeaderStatus::StopIteration returned by onRequestHeaders() or onResponseHeaders() without modification. Follow-on envoyproxy/envoy#40213 adds an option in Envoy WasmFilter PluginConfig that sets the value of this field. For details, see [Envoy Wasm / Proxy-Wasm support for FilterHeadersStatus::StopIteration](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?usp=sharing). This PR is one part of implementing [Option B: WasmFilter config knob](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?tab=t.0#bookmark=id.5wxldlapsp54). Note that default behavior of proxy-wasm-cpp-host and ContextBase is unchanged. --------- Signed-off-by: Michael Warres <mpw@google.com> --------- Signed-off-by: Martijn Stevenson <mstevenson@google.com> Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: liang.he@intel.com <liang.he@intel.com> Signed-off-by: wangbaiping <wbphub@gmail.com> Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: Michael Warres <mpw@google.com> Co-authored-by: martijneken <mstevenson@google.com> Co-authored-by: Keith Mattix II <keithmattix2@gmail.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: liang.he <liang.he@intel.com> Co-authored-by: code <wbphub@gmail.com> Co-authored-by: Matt Leon <ml@mattleon.com> Co-authored-by: Michael Warres <mpw@google.com>

In #427, a portion of the wasi hostcalls was added, but not all. Now, all the wasi hostcalls have been included, and their stability has been verified in our multiple go 1.24 compiled wasm plugins --------- Signed-off-by: zty98751 <zty98751@alibaba-inc.com> Signed-off-by: Matt Leon <mattleon@google.com> Co-authored-by: zty98751 <zty98751@alibaba-inc.com>

leonm1 · 2024-12-13T18:55:45Z

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present.

PiotrSikora

Thanks for working on this!

Two high-level comments:

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls? For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?
Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

leonm1 · 2024-12-18T22:13:45Z

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls?

Answered elsewhere as well, but I was mistaken about some of the filesystem syscalls. This PR now represents the full minimal set of added hostcalls for a helloworld go plugin to initialize.

For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?

Yield should be fine due to the linked behavior. For polling, I do wonder about that — technically we could support some of the subscription types, but I'm not sure how much value they add in a single-threaded environment anyway, since we only support FDs 1 and 2, which never get "closed" and wouldn't "block" from the wasm modules perspective.

Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

I tested the CPP host with all of service extensions' cpp and rust plugins. Since these are added hostcalls, they should not affect currently-working wasm modules. I attempted to add explicit calls to each of sched_yield, poll, and fdstat_set_flags (which corresponds to posix's fcntl), and confirmed each translated the libc call to the corresponding wasi hostcall and none complained about the ENOSYS (or ESUCCESS, in sched_yield's case).

PiotrSikora · 2024-12-19T17:04:23Z

Since these are added hostcalls, they should not affect currently-working wasm modules.

Right, this won't break any of the existing plugins. My point was mostly that with those changes (well, the earlier version of this PR) plugins that attempt to open files would load successfully, but could break at runtime.

PiotrSikora

Thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

martijneken

LGTM, thanks!

mpwarres

LGTM, thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: zty98751 <zty98751@alibaba-inc.com>

… behavior (proxy-wasm#434) (#11) * fix: CI branch name master -> main (proxy-wasm#398) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Bump Abseil to fix Linux build issues (proxy-wasm#400) Bump Abseil to fix Linux build issues Pick up this fix: abseil/abseil-cpp#1187 Bump past Envoy to pick up another fix found in fuzz tests: proxy-wasm#399 (comment) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Update cargo-raze -> crate_universe (proxy-wasm#399) - Updated platforms for crate_universe compatibility - Supports upgrade to wasmsign2 - Includes workaround for Windows path length issue Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Move from unavailable macos-11 to macos-13 (proxy-wasm#401) See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources This does not fixproxy-wasm#384, but does resurface those errors. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump Bazel from 5.2.0 to 6.5.0 (proxy-wasm#402) Bump Bazel from 5.2.0 to 6.5.0 This breaks the s390x build which relied on an external Docker image. I made some strides in fixing s390x, but it's not yet working. Deferred to proxy-wasm#405. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump rules_python and rules_fuzzing (proxy-wasm#404) Upgrade rules_python (0.34.0) and rules_fuzzing (0.5.2) This requires extracting WORKSPACE phases into more phases: - dependencies -- py_repositories() and toolchains - dependencies_python() -- pip_parse module loading - dependencies_import() -- python/fuzzing/other deps The new structure roughly matches Envoy WORKSPACE: - envoy_dependencies() - envoy_dependencies_extra() -- not needed here - envoy_python_dependencies() - envoy_dependency_imports() Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update CI to use Ubuntu 22.04 / clang 14 (proxy-wasm#408) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update rules_rust to v0.42.1 (with Rust v1.77.2). (proxy-wasm#410) * Update rules_rust * Update rust and vendor * rust_oom -> rg_oom * Change rust version --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Update wasmtime (v24.0.0) (proxy-wasm#406) Removes Wasmtime + Windows CI because rules_rust has recently dropped Windows: https://github.com/bazelbuild/rules_rust/blob/main/docs/index.md#supported-platforms Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix: Upgrade deprecated artifact upload/download handlers (proxy-wasm#415) Seen on proxy-wasm#380 CI: Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v2`. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/ Signed-off-by: Martijn Stevenson <mstevenson@google.com> * bump wamr to 2.1.1 and able to consume precompiled content (proxy-wasm#380) - skip leading paddings in .aot section Signed-off-by: liang.he@intel.com <liang.he@intel.com> * compdb add the compdb support to the proxy_wasm_cpp_host (proxy-wasm#419) * compdb add the compdb support to the proxy_wasm_cpp_host Signed-off-by: wangbaiping <wbphub@gmail.com> * Fix references to prefix_wasm_api (proxy-wasm#420) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * feat(go-sdk): add wasi hostcalls used by the Go SDK (proxy-wasm#427) The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> * chore: workflow runner fixes (proxy-wasm#436) Assorted changes to get workflows working again: - Update format workflows to use ubuntu-22.04 - Update windows-2019 to windows-2022 and add a missing <string> include needed to build with that - Enable manual triggering of workflows Fixesproxy-wasm#435 --------- Signed-off-by: Michael Warres <mpw@google.com> * feat: add knob to customise on{Request,Response}Headers StopIteration behavior (proxy-wasm#434) Add protected ContextBase::allow_on_headers_stop_iteration_ field that can be used by host implementations to control whether or not ContextBase propagates FilterHeaderStatus::StopIteration returned by onRequestHeaders() or onResponseHeaders() without modification. Follow-on envoyproxy/envoy#40213 adds an option in Envoy WasmFilter PluginConfig that sets the value of this field. For details, see [Envoy Wasm / Proxy-Wasm support for FilterHeadersStatus::StopIteration](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?usp=sharing). This PR is one part of implementing [Option B: WasmFilter config knob](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?tab=t.0#bookmark=id.5wxldlapsp54). Note that default behavior of proxy-wasm-cpp-host and ContextBase is unchanged. --------- Signed-off-by: Michael Warres <mpw@google.com> --------- Signed-off-by: Martijn Stevenson <mstevenson@google.com> Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: liang.he@intel.com <liang.he@intel.com> Signed-off-by: wangbaiping <wbphub@gmail.com> Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: Michael Warres <mpw@google.com> Co-authored-by: martijneken <mstevenson@google.com> Co-authored-by: Keith Mattix II <keithmattix2@gmail.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: liang.he <liang.he@intel.com> Co-authored-by: code <wbphub@gmail.com> Co-authored-by: Matt Leon <ml@mattleon.com> Co-authored-by: Michael Warres <mpw@google.com>

In #427, a portion of the wasi hostcalls was added, but not all. Now, all the wasi hostcalls have been included, and their stability has been verified in our multiple go 1.24 compiled wasm plugins --------- Signed-off-by: zty98751 <zty98751@alibaba-inc.com> Signed-off-by: Matt Leon <mattleon@google.com> Co-authored-by: zty98751 <zty98751@alibaba-inc.com>

leonm1 · 2024-12-13T18:55:45Z

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present.

PiotrSikora

Thanks for working on this!

Two high-level comments:

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls? For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?
Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

leonm1 · 2024-12-18T22:13:45Z

It would be good to understand why Go SDK wants to import (and use?) those hostcalls. For filesystem calls - does it try to access some files on startup, and why it doesn't use preopen hostcalls?

Answered elsewhere as well, but I was mistaken about some of the filesystem syscalls. This PR now represents the full minimal set of added hostcalls for a helloworld go plugin to initialize.

For yield and polling - does it actually work without making progress or are non-trivial plugins going to deadlock?

Yield should be fine due to the linked behavior. For polling, I do wonder about that — technically we could support some of the subscription types, but I'm not sure how much value they add in a single-threaded environment anyway, since we only support FDs 1 and 2, which never get "closed" and wouldn't "block" from the wasm modules perspective.

Since we're exposing those hostcalls globally, how does it affect other the SDKs? If you try to open files in Rust or C++ files, will they gracefully return user errors (i.e. file not found or similar) or crash due to a missing code path for ENOSYS?

I tested the CPP host with all of service extensions' cpp and rust plugins. Since these are added hostcalls, they should not affect currently-working wasm modules. I attempted to add explicit calls to each of sched_yield, poll, and fdstat_set_flags (which corresponds to posix's fcntl), and confirmed each translated the libc call to the corresponding wasi hostcall and none complained about the ENOSYS (or ESUCCESS, in sched_yield's case).

PiotrSikora · 2024-12-19T17:04:23Z

Since these are added hostcalls, they should not affect currently-working wasm modules.

Right, this won't break any of the existing plugins. My point was mostly that with those changes (well, the earlier version of this PR) plugins that attempt to open files would load successfully, but could break at runtime.

PiotrSikora

Thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

martijneken

LGTM, thanks!

mpwarres

LGTM, thanks!

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com>

The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: zty98751 <zty98751@alibaba-inc.com>

… behavior (proxy-wasm#434) (#11) * fix: CI branch name master -> main (proxy-wasm#398) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Bump Abseil to fix Linux build issues (proxy-wasm#400) Bump Abseil to fix Linux build issues Pick up this fix: abseil/abseil-cpp#1187 Bump past Envoy to pick up another fix found in fuzz tests: proxy-wasm#399 (comment) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Update cargo-raze -> crate_universe (proxy-wasm#399) - Updated platforms for crate_universe compatibility - Supports upgrade to wasmsign2 - Includes workaround for Windows path length issue Signed-off-by: Martijn Stevenson <mstevenson@google.com> * fix: Move from unavailable macos-11 to macos-13 (proxy-wasm#401) See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources This does not fixproxy-wasm#384, but does resurface those errors. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump Bazel from 5.2.0 to 6.5.0 (proxy-wasm#402) Bump Bazel from 5.2.0 to 6.5.0 This breaks the s390x build which relied on an external Docker image. I made some strides in fixing s390x, but it's not yet working. Deferred to proxy-wasm#405. Signed-off-by: Martijn Stevenson <mstevenson@google.com> * chore: bump rules_python and rules_fuzzing (proxy-wasm#404) Upgrade rules_python (0.34.0) and rules_fuzzing (0.5.2) This requires extracting WORKSPACE phases into more phases: - dependencies -- py_repositories() and toolchains - dependencies_python() -- pip_parse module loading - dependencies_import() -- python/fuzzing/other deps The new structure roughly matches Envoy WORKSPACE: - envoy_dependencies() - envoy_dependencies_extra() -- not needed here - envoy_python_dependencies() - envoy_dependency_imports() Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update CI to use Ubuntu 22.04 / clang 14 (proxy-wasm#408) Signed-off-by: Martijn Stevenson <mstevenson@google.com> * Update rules_rust to v0.42.1 (with Rust v1.77.2). (proxy-wasm#410) * Update rules_rust * Update rust and vendor * rust_oom -> rg_oom * Change rust version --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Update wasmtime (v24.0.0) (proxy-wasm#406) Removes Wasmtime + Windows CI because rules_rust has recently dropped Windows: https://github.com/bazelbuild/rules_rust/blob/main/docs/index.md#supported-platforms Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * fix: Upgrade deprecated artifact upload/download handlers (proxy-wasm#415) Seen on proxy-wasm#380 CI: Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v2`. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/ Signed-off-by: Martijn Stevenson <mstevenson@google.com> * bump wamr to 2.1.1 and able to consume precompiled content (proxy-wasm#380) - skip leading paddings in .aot section Signed-off-by: liang.he@intel.com <liang.he@intel.com> * compdb add the compdb support to the proxy_wasm_cpp_host (proxy-wasm#419) * compdb add the compdb support to the proxy_wasm_cpp_host Signed-off-by: wangbaiping <wbphub@gmail.com> * Fix references to prefix_wasm_api (proxy-wasm#420) Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * feat(go-sdk): add wasi hostcalls used by the Go SDK (proxy-wasm#427) The full Go sdk imports hostcalls not currently exported to the wasm module, making the wasm module fail on instantiation. Per discussion with the Go core maintainers, these functions do not need to be implemented, but they must be present. Signed-off-by: Matt Leon <mattleon@google.com> * chore: workflow runner fixes (proxy-wasm#436) Assorted changes to get workflows working again: - Update format workflows to use ubuntu-22.04 - Update windows-2019 to windows-2022 and add a missing <string> include needed to build with that - Enable manual triggering of workflows Fixesproxy-wasm#435 --------- Signed-off-by: Michael Warres <mpw@google.com> * feat: add knob to customise on{Request,Response}Headers StopIteration behavior (proxy-wasm#434) Add protected ContextBase::allow_on_headers_stop_iteration_ field that can be used by host implementations to control whether or not ContextBase propagates FilterHeaderStatus::StopIteration returned by onRequestHeaders() or onResponseHeaders() without modification. Follow-on envoyproxy/envoy#40213 adds an option in Envoy WasmFilter PluginConfig that sets the value of this field. For details, see [Envoy Wasm / Proxy-Wasm support for FilterHeadersStatus::StopIteration](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?usp=sharing). This PR is one part of implementing [Option B: WasmFilter config knob](https://docs.google.com/document/d/1Whd1C0k-H2NHrPOmlAqqauFz6ObSTP017juJIYyciB0/edit?tab=t.0#bookmark=id.5wxldlapsp54). Note that default behavior of proxy-wasm-cpp-host and ContextBase is unchanged. --------- Signed-off-by: Michael Warres <mpw@google.com> --------- Signed-off-by: Martijn Stevenson <mstevenson@google.com> Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> Signed-off-by: liang.he@intel.com <liang.he@intel.com> Signed-off-by: wangbaiping <wbphub@gmail.com> Signed-off-by: Matt Leon <mattleon@google.com> Signed-off-by: Michael Warres <mpw@google.com> Co-authored-by: martijneken <mstevenson@google.com> Co-authored-by: Keith Mattix II <keithmattix2@gmail.com> Co-authored-by: Keith Mattix II <keithmattix@microsoft.com> Co-authored-by: liang.he <liang.he@intel.com> Co-authored-by: code <wbphub@gmail.com> Co-authored-by: Matt Leon <ml@mattleon.com> Co-authored-by: Michael Warres <mpw@google.com>

In #427, a portion of the wasi hostcalls was added, but not all. Now, all the wasi hostcalls have been included, and their stability has been verified in our multiple go 1.24 compiled wasm plugins --------- Signed-off-by: zty98751 <zty98751@alibaba-inc.com> Signed-off-by: Matt Leon <mattleon@google.com> Co-authored-by: zty98751 <zty98751@alibaba-inc.com>

leonm1 requested review from PiotrSikora, martijneken and mpwarres as code owners December 13, 2024 18:55

leonm1 force-pushed the feat/go-sdk branch 4 times, most recently from b9b4a5e to 1af87ceCompare December 13, 2024 19:48

mpwarres reviewed Dec 13, 2024
View reviewed changes

Comment threadsrc/exports.cc Outdated
Comment threadinclude/proxy-wasm/exports.h Outdated
Comment threadsrc/exports.cc Outdated

martijneken reviewed Dec 13, 2024
View reviewed changes

Comment threadsrc/exports.cc Outdated

PiotrSikora requested changes Dec 16, 2024
View reviewed changes

Comment threadsrc/exports.cc Outdated
Comment threadsrc/exports.cc Outdated
Comment threadsrc/exports.cc Outdated

leonm1 mentioned this pull request Dec 17, 2024
Retarget SDK to upstream Go toolchain 1.24 proxy-wasm/proxy-wasm-go-sdk#1
Merged

leonm1 force-pushed the feat/go-sdk branch from 1af87ce to 814b428Compare December 18, 2024 22:06

leonm1 requested review from PiotrSikora, martijneken and mpwarres December 18, 2024 22:14

PiotrSikora reviewed Dec 19, 2024
View reviewed changes

Comment threadsrc/wasm.cc Outdated

PiotrSikora reviewed Dec 19, 2024
View reviewed changes

Comment threadsrc/wasm.cc Outdated

PiotrSikora approved these changes Dec 19, 2024
View reviewed changes

leonm1 force-pushed the feat/go-sdk branch from 814b428 to c9193c6Compare December 19, 2024 18:12

martijneken approved these changes Dec 19, 2024
View reviewed changes

mpwarres approved these changes Dec 19, 2024
View reviewed changes

martijneken merged commit c4d7bb0 into proxy-wasm:mainDec 19, 2024

martijneken mentioned this pull request Feb 17, 2025
Proxy Wasm Go SDK is deprecated sablierapp/sablier#426
Closed

acouvreur mentioned this pull request Feb 21, 2025
unhandled WASI function Kong/ngx_wasm_module#682
Closed

johnlanni mentioned this pull request May 9, 2025
add all wasi hostcalls used by Go SDK #433
Closed

PiotrSikora mentioned this pull request Feb 26, 2026
wasi_snapshot_preview.poll_oneoff must return success #497
Closed

leonm1 mentioned this pull request Mar 27, 2026
add all wasi hostcalls used by Go SDK #533
Merged