From cee593e01fbf343184604bb5f7b9d9ac1fa99260 Mon Sep 17 00:00:00 2001 From: Wu Sheng Date: Sat, 15 Aug 2026 22:56:16 +0800 Subject: [PATCH] docs: move user documentation into docs/ for website hosting Split the README into a docs/ tree that the SkyWalking website can import, following the layout used by the Python and Go agents (docs/README.md as the overview page, docs/menu.yml as the sidebar, pages under docs/en/). - README.md becomes a short entry point: requirements, install, quick start, and links into docs/. - CONTRIBUTING.md points at the build, plugin development, and release guides. - docs/en/ adds setup (quick start, configuration, start and stop), features (tracing, runtime metrics), plugins (supported libraries, AWS SDK v2), advanced use (serverless, webpack, troubleshooting), and contribution (build and test, plugin development, release) pages. - docs/How-to-release.md stays as a stub, keeping the #vote-check anchor that earlier [VOTE] emails link to. - scripts/release.sh: the [VOTE] email now points at the in-tarball doc paths, since the moved guides ship inside the source package. Configuration values, meter names, plugin behavior, and the build and test commands were verified against src/ and .github/workflows/test.yaml. Two long-standing README errors are corrected: SW_AGENT_INSTANCE defaults to the host name (not a random value) and SW_AGENT_LOGGING_LEVEL defaults to error (not info). The website side needs a follow-up PR in apache/skywalking-website adding repoUrl and a Next entry for the Node.js Agent in data/docs.yml. Co-Authored-By: Claude Opus 5 (1M context) --- CONTRIBUTING.md | 35 +-- README.md | 204 +++++------------- docs/How-to-release.md | 227 +------------------ docs/README.md | 40 ++++ docs/en/advanced/serverless.md | 95 ++++++++ docs/en/advanced/troubleshooting.md | 96 +++++++++ docs/en/advanced/webpack.md | 74 +++++++ docs/en/contribution/build-and-test.md | 94 ++++++++ docs/en/contribution/plugin-development.md | 89 ++++++++ docs/en/contribution/release.md | 239 +++++++++++++++++++++ docs/en/features/runtime-metrics.md | 61 ++++++ docs/en/features/tracing.md | 88 ++++++++ docs/en/plugins/aws-sdk-v2.md | 96 +++++++++ docs/en/plugins/supported-libraries.md | 62 ++++++ docs/en/setup/configuration.md | 109 ++++++++++ docs/en/setup/quick-start.md | 69 ++++++ docs/en/setup/startup-and-shutdown.md | 61 ++++++ docs/menu.yml | 56 +++++ scripts/release.sh | 4 +- 19 files changed, 1393 insertions(+), 406 deletions(-) create mode 100644 docs/README.md create mode 100644 docs/en/advanced/serverless.md create mode 100644 docs/en/advanced/troubleshooting.md create mode 100644 docs/en/advanced/webpack.md create mode 100644 docs/en/contribution/build-and-test.md create mode 100644 docs/en/contribution/plugin-development.md create mode 100644 docs/en/contribution/release.md create mode 100644 docs/en/features/runtime-metrics.md create mode 100644 docs/en/features/tracing.md create mode 100644 docs/en/plugins/aws-sdk-v2.md create mode 100644 docs/en/plugins/supported-libraries.md create mode 100644 docs/en/setup/configuration.md create mode 100644 docs/en/setup/quick-start.md create mode 100644 docs/en/setup/startup-and-shutdown.md create mode 100644 docs/menu.yml diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e651767..68514dd 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,31 +1,12 @@ -# Compiling and Building +# Contributing to the Apache SkyWalking Node.js Agent -We build skywalking-nodejs with NodeJS 20 (the current LTS baseline; CI tests Node 20, 22, and 24). -If you don't have a suitable NodeJS installed, use a version manager such as -[nvm](https://github.com/nvm-sh/nvm) to manage multiple node versions, or build inside a Docker -container: +Thank you for helping the project. -```shell -# Suppose you have the source codes in folder skywalking-nodejs -docker run -it --rm -v $(pwd)/skywalking-nodejs:/workspace -w /workspace node:20 bash -``` +Use the documentation in this repository: -The gRPC / protobuf definitions live in the `protocol/` git submodule, so initialize it first (or -clone the repository with `--recurse-submodules`). Then install dependencies — the `prepare` hook -generates the protobuf stubs into `src/proto/` — and compile: +- [Build and test](docs/en/contribution/build-and-test.md) +- [Plugin development](docs/en/contribution/plugin-development.md) +- [Release guide](docs/en/contribution/release.md) for maintainers -```shell -git submodule update --init --recursive -npm install -npm run build -``` - -`npm run build` compiles the TypeScript sources into `lib/`. Other useful commands: - -```shell -npm run lint # ESLint: code style + Apache license headers -npm run test # plugin tests (require Docker) -``` - -Warnings can be ignored, but if an error prevents you from continuing, try `rm -rf node_modules/` -and rerun the commands above. +Please discuss large changes in an issue or on the `dev@skywalking.apache.org` mailing list before +doing a large amount of work. diff --git a/README.md b/README.md index bc31f3c..70e8ba3 100644 --- a/README.md +++ b/README.md @@ -1,189 +1,81 @@ -# SkyWalking NodeJS Agent +# Apache SkyWalking Node.js Agent -Sky Walking logo +Apache SkyWalking logo -**SkyWalking-NodeJS**: The NodeJS Agent for Apache SkyWalking, which provides the native tracing abilities for NodeJS backend project. +The Apache SkyWalking Node.js Agent reports distributed traces and Node.js runtime metrics to an +Apache SkyWalking OAP server. It instruments supported Node.js libraries without changes to their +source code. -**SkyWalking**: an APM(application performance monitor) system, especially designed for -microservices, cloud native and container-based (Docker, Kubernetes, Mesos) architectures. - -[![GitHub stars](https://img.shields.io/github/stars/apache/skywalking-nodejs.svg?style=for-the-badge&label=Stars&logo=github)](https://github.com/apache/skywalking-nodejs) -[![Twitter Follow](https://img.shields.io/twitter/follow/asfskywalking.svg?style=for-the-badge&label=Follow&logo=twitter)](https://twitter.com/AsfSkyWalking) +[![Build](https://github.com/apache/skywalking-nodejs/workflows/Build/badge.svg?branch=master)](https://github.com/apache/skywalking-nodejs/actions?query=branch%3Amaster+event%3Apush+workflow%3A%22Build%22) +[![npm version](https://badge.fury.io/js/skywalking-backend-js.svg)](https://www.npmjs.com/package/skywalking-backend-js) +[![GitHub stars](https://img.shields.io/github/stars/apache/skywalking-nodejs.svg?label=Stars&logo=github)](https://github.com/apache/skywalking-nodejs) +## Requirements -[![Build](https://github.com/apache/skywalking-nodejs/workflows/Build/badge.svg?branch=master)](https://github.com/apache/skywalking-nodejs/actions?query=branch%3Amaster+event%3Apush+workflow%3A%22Build%22) -[![npm version](https://badge.fury.io/js/skywalking-backend-js.svg)](https://badge.fury.io/js/skywalking-backend-js) +- Node.js 20 or later. +- A compatible Apache SkyWalking OAP server. See the + [agent and OAP compatibility guide](https://skywalking.apache.org/docs/main/next/en/setup/service-agent/agent-compatibility/). -## Install SkyWalking NodeJS package from npmjs +## Install ```bash -$ npm install --save skywalking-backend-js +npm install skywalking-backend-js ``` -## Set up NodeJS Agent +## Quick start -SkyWalking NodeJS SDK requires SkyWalking backend (OAP) 8.0+ and NodeJS >= 20, -other versions are not tested and SkyWalking NodeJS SDK may or may not work, -please make sure to use the supported versions before reporting any issue. +Start the agent before loading the modules that it must instrument: ```typescript import agent from 'skywalking-backend-js'; -agent.start(); -``` - -This will use default configurations to start the SkyWalking agent above, if you want to specify your own configurations, here are two methods. - -- Pass those values to `agent.start` method, such as: - -```typescript agent.start({ - serviceName: 'my-service-name', - serviceInstance: 'my-service-instance-name', - collectorAddress: 'my.collector.address:port', + serviceName: 'checkout-service', + collectorAddress: '127.0.0.1:11800', }); ``` -Note that all options given (including empty/null values) will override the corresponding default values, e.g. `agent.start({ collectorAddress: '' })` will override the default value of `collectorAddress` to empty string, causing errors like `DNS resolution failed`. - -- Use environment variables. - -The supported environment variables are as follows: - -Environment Variable | Description | Default -| :--- | :--- | :--- | -| `SW_AGENT_NAME` | The name of the service | `your-nodejs-service` | -| `SW_AGENT_INSTANCE` | The name of the service instance | Randomly generated | -| `SW_AGENT_COLLECTOR_BACKEND_SERVICES` | The backend OAP server address | `127.0.0.1:11800` | -| `SW_AGENT_SECURE` | Whether to use secure connection to backend OAP server | `false` | -| `SW_AGENT_AUTHENTICATION` | The authentication token to verify that the agent is trusted by the backend OAP, as for how to configure the backend, refer to [the yaml](https://github.com/apache/skywalking/blob/4f0f39ffccdc9b41049903cc540b8904f7c9728e/oap-server/server-bootstrap/src/main/resources/application.yml#L155-L158). | not set | -| `SW_AGENT_LOGGING_LEVEL` | The logging level, could be one of `error`, `warn`, `info`, `debug` | `info` | -| `SW_AGENT_DISABLE_PLUGINS` | Comma-delimited list of plugins to disable in the plugins directory (e.g. "mysql", "express") | `` | -| `SW_COLD_ENDPOINT` | Cold start detection is as follows: First span to run is considered a cold start. This span gets the tag `coldStart` set to 'true'. This span also optionally gets the text '\' appended to the endpoint name if SW_COLD_ENDPOINT is set to 'true'. | `false` | -| `SW_IGNORE_SUFFIX` | The suffices of endpoints that will be ignored (not traced), comma separated | `.jpg,.jpeg,.js,.css,.png,.bmp,.gif,.ico,.mp3,.mp4,.html,.svg` | -| `SW_TRACE_IGNORE_PATH` | The paths of endpoints that will be ignored (not traced), comma separated | `` | -| `SW_HTTP_IGNORE_METHOD` | Comma-delimited list of http methods to ignore (GET, POST, HEAD, OPTIONS, etc...) | `` | -| `SW_SQL_TRACE_PARAMETERS` | If set to 'true' then SQL query parameters will be included | `false` | -| `SW_SQL_PARAMETERS_MAX_LENGTH` | The maximum string length of SQL parameters to log | `512` | -| `SW_MONGO_TRACE_PARAMETERS` | If set to 'true' then mongodb query parameters will be included | `false` | -| `SW_MONGO_PARAMETERS_MAX_LENGTH` | The maximum string length of mongodb parameters to log | `512` | -| `SW_AWS_LAMBDA_FLUSH` | Maximum number of float seconds allowed to pass between invocations before consecutive Lambda function calls flush automatically upon exit, 0 means always flush, -1 means never. | `2` | -| `SW_AWS_LAMBDA_CHAIN` | Pass trace ID to AWS Lambda function in its parameters (to allow linking). Only use if both caller and callee will be instrumented. | `false` | -| `SW_AWS_SQS_CHECK_BODY` | Incoming SQS messages check inside the body for trace ID in order to allow linking outgoing SNS messages to incoming SQS. | `false` | -| `SW_AGENT_MAX_BUFFER_SIZE` | The maximum buffer size before sending the segment data to backend | `'1000'` | -| `SW_AGENT_TRACE_TIMEOUT` | The timeout for trace requests to backend services | `'10000'` | -| `SW_AGENT_NODEJS_RUNTIME_METRICS_REPORTER_ACTIVE` | Whether to report Node.js runtime metrics through MeterReportService (default period 20s) | `true` | -| `SW_AGENT_NODEJS_RUNTIME_METRICS_REPORT_PERIOD` | Runtime metric sample + report interval in milliseconds (aligned with Java `meter.report_interval`) | `20000` | - -Legacy env names `SW_AGENT_RUNTIME_METRICS_*` / `SW_AGENT_NVM_*` for reporter active and report period are still accepted as deprecated aliases. - - -Note that the various ignore options like `SW_IGNORE_SUFFIX`, `SW_TRACE_IGNORE_PATH` and `SW_HTTP_IGNORE_METHOD` as well as endpoints which are not recorded due to exceeding `SW_AGENT_MAX_BUFFER_SIZE` all propagate their ignored status downstream to any other endpoints they may call. If that endpoint is running the Node Skywalking agent then regardless of its ignore settings it will not be recorded since its upstream parent was not recorded. This allows the elimination of entire trees of endpoints you are not interested in as well as eliminating partial traces if a span in the chain is ignored but calls out to other endpoints which are recorded as children of ROOT instead of the actual parent. - -## Node.js Runtime Metrics - -The agent reports twelve process-level meters (`instance_nodejs_*`) via `MeterReportService` by default (sample and report every 20s). Set `SW_AGENT_NODEJS_RUNTIME_METRICS_REPORTER_ACTIVE=false` to disable. Process CPU combines `process.cpuUsage()` user + system, normalized by logical CPU count (0–100%). - -| Node.js source | Meter name | Notes | -| :--- | :--- | :--- | -| `process.cpuUsage()` user + system | `instance_nodejs_process_cpu` | % | -| `process.memoryUsage().heapUsed` | `instance_nodejs_heap_used` | bytes | -| `process.memoryUsage().heapTotal` | `instance_nodejs_heap_total` | bytes | -| `v8.getHeapStatistics().heap_size_limit` | `instance_nodejs_heap_limit` | bytes | -| `process.memoryUsage().rss` | `instance_nodejs_rss` | bytes | -| `process.memoryUsage().external` | `instance_nodejs_external_memory` | bytes | -| `process.memoryUsage().arrayBuffers` | `instance_nodejs_array_buffers` | bytes | -| `process.uptime()` | `instance_nodejs_uptime` | seconds | -| `v8.getHeapStatistics().peak_malloced_memory` | `instance_nodejs_peak_malloced_memory` | bytes | -| `v8.getHeapStatistics().malloced_memory` | `instance_nodejs_malloced_memory` | bytes | -| `v8.getHeapSpaceStatistics()` old_space | `instance_nodejs_old_space_used` | bytes | -| `v8.getHeapSpaceStatistics()` new_space | `instance_nodejs_new_space_used` | bytes | - -Custom business metrics are not available through a public API; use [OpenTelemetry metrics](https://skywalking.apache.org/docs/main/latest/en/setup/backend/opentelemetry-receiver/) if you need those. - -## Supported Libraries - -Some built-in plugins support automatic instrumentation of NodeJS libraries, the complete list is as follows: - -Library | Plugin Name -| :--- | :--- | -| built-in `http` and `https` module | `http` / `https` | -| [`Express`](https://expressjs.com) | `express` | -| [`Axios`](https://github.com/axios/axios) | `axios` | -| [`MySQL`](https://github.com/mysqljs/mysql) | `mysql` | -| [`MySQL`](https://github.com/sidorares/node-mysql2) | `mysql2` | -| [`PostgreSQL`](https://github.com/brianc/node-postgres) | `pg` | -| [`pg-cursor`](https://github.com/brianc/node-postgres) | `pg-cursor` | -| [`MongoDB`](https://github.com/mongodb/node-mongodb-native) | `mongodb` | -| [`Mongoose`](https://github.com/Automattic/mongoose) | `mongoose` | -| [`RabbitMQ`](https://github.com/squaremo/amqp.node) | `amqplib` | -| [`Redis`](https://github.com/luin/ioredis) | `ioredis` | -| [`AWS2DynamoDB`](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html) | `aws-sdk` | -| [`AWS2Lambda`](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html) | `aws-sdk` | -| [`AWS2SNS`](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SNS.html) | `aws-sdk` | -| [`AWS2SQS`](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SQS.html) | `aws-sdk` | - -### Compatible Libraries - -The following are packages that have been tested to some extent and are compatible because they work through the instrumentation of an underlying package: - -Library | Underlying Plugin Name -| :--- | :--- | -| [`request`](https://github.com/request/request) | `http` / `https` | -| [`request-promise`](https://github.com/request/request-promise) | `http` / `https` | -| [`koa`](https://github.com/koajs/koa) | `http` / `https` | - -## Experimental Azure Functions Support - -The plugin `AzureHttpTriggerPlugin` provides a wrapper function for an Azure Functions Javascript HttpTrigger endpoint. This is an http server endpoint and currently must be instrumented manually. So far all other plugins tested work within the HttpTrigger and so a trace can pass through the Function and onto other endpoints called by the function. How much sense it makes to instrument an Azure Function which already lives in the cloud and has robust monitoring incorporated is a good question, but at the least, this plugin will allow those endpoints to show up in a Skywalking trace. - -### Usage: - -```javascript -const {default: agent, AzureHttpTriggerPlugin} = require('skywalking-backend-js'); - -agent.start({ ... }); - -module.exports = AzureHttpTriggerPlugin.wrap(async function (context, req) { - - /* contents of http trigger function */ +The same values can be set with environment variables: -}); +```bash +export SW_AGENT_NAME=checkout-service +export SW_AGENT_COLLECTOR_BACKEND_SERVICES=127.0.0.1:11800 ``` -All that needs to be done is the actual trigger function needs to be wrapped with `azureHttpTriggerPlugin.wrap()`, whether that function is a default export or an explicitly named `entryPoint` or `run` or `index`. - -## Experimental AWS Lambda Functions Support - -The plugins `AWSLambdaTriggerPlugin`, `AWSLambdaGatewayAPIHTTP` and `AWSLambdaGatewayAPIREST` provide a wrapper functions for AWS Lambda Functions endpoints. `AWSLambdaTriggerPlugin` is a generic wrapper plugin which should work with any kind of Lambda trigger but also stores the least amount of information since it does not know anything about the incoming data format. For this reason, this type of trigger also can not link back to the caller, but it can create a new segment that will be propagated to all downstream children, thus starting its own trace. `AWSLambdaGatewayAPIHTTP` and `AWSLambdaGatewayAPIREST` are specific wrappers for Lambda functions triggered by the GatewayAPI HTTP or REST triggers. They have the advantage of knowing the incoming data format and can thus extract existing trace segment information from incoming requests and chain correctly from upstream to any downstream endpoints. - -### Usage: +See [Quick start](docs/en/setup/quick-start.md) for the full setup and a way to load the agent with +Node.js `--require`. -```javascript -const {default: agent, AWSLambdaGatewayAPIHTTP} = require('skywalking-backend-js'); +## Documentation -agent.start({ ... }); +The [documentation index](docs/README.md) includes: -exports.handler = AWSLambdaGatewayAPIHTTP.wrap(async function (event, context, callback) { +- setup and all configuration values; +- tracing and Node.js runtime metrics; +- supported library plugins and AWS SDK v2 behavior; +- serverless and Webpack support; +- build, test, plugin development, and release guides. - /* contents of http trigger function */ +## Main features -}); -``` +- Automatic trace collection for Node.js HTTP, database, messaging, and framework libraries. +- SkyWalking trace context transfer between supported services. +- Twelve process-level Node.js runtime meters. +- Optional wrappers for AWS Lambda and Azure Functions. -This is similar to Azure Functions wrapping, just wrap your handler function with `AWSLambdaTriggerPlugin.wrap()` or `AWSLambdaGatewayAPIHTTP.wrap()` or `AWSLambdaGatewayAPIREST.wrap()`. One thing to note is that AWS freezes processes in between invocations of lambda functions so whether you are doing async or sync handler functions with callbacks, you should make sure everything you need to do finishes before returning control to AWS or calling the synchronous callback. These plugins take this into account and automatically flush the segment buffers before closing a trace span. +## Contributing -## Experimental Webpack Support +Read [Build and test](docs/en/contribution/build-and-test.md) before sending a change. Plugin authors +should also read [Plugin development](docs/en/contribution/plugin-development.md). -Webpack requires that all imports be statically defined at compile-time and so was not compatible with the dynamic search and loading done by the standard `PluginInstaller`. This has been extended to attempt static imports if the application is determined to be running out of a webpack bundle. This requires that any new plugins be manually added to `PluginInstaller.installBundled()`. Only plugins which allow a `require('module/package.json')` will work with this method as `package.json` needs to be loaded to determine the version of the plugin module present. Some modules specifically disallow import of their package.json and so can not be loaded like this. +## Contact -Upon compile with `webpack` it will complain about missing modules for which imports are attempted in the sw agent but which are not present. Simply add these modules to the list of modules to be ignored by webpack, for example by `resolve: {alias: {'module': false}}`. - -## Contact Us -* Submit [an issue](https://github.com/apache/skywalking/issues/new) by using [Nodejs] as title prefix. -* Mail list: **dev@skywalking.apache.org**. Mail to `dev-subscribe@skywalking.apache.org`, follow the reply to subscribe the mail list. -* Join `skywalking` channel at [Apache Slack](http://s.apache.org/slack-invite). If the link is not working, find the latest one at [Apache INFRA WIKI](https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites). -* Twitter, [ASFSkyWalking](https://twitter.com/ASFSkyWalking) +- Report Node.js Agent problems in the + [Apache SkyWalking issue tracker](https://github.com/apache/skywalking/issues/new) with `Nodejs` in + the issue title. +- Join the `dev@skywalking.apache.org` mailing list by sending a message to + `dev-subscribe@skywalking.apache.org`. +- Join the `skywalking` channel on [Apache Slack](https://s.apache.org/slack-invite). ## License -Apache 2.0 + +[Apache License 2.0](LICENSE) diff --git a/docs/How-to-release.md b/docs/How-to-release.md index 240fce2..e47a9a1 100644 --- a/docs/How-to-release.md +++ b/docs/How-to-release.md @@ -1,224 +1,9 @@ -# Apache SkyWalking NodeJS Release Guide +# Release guide moved -This guide releases SkyWalking NodeJS the Apache Way using the two release scripts, and helps -voters check a release. The shell scripts [`scripts/release.sh`](../scripts/release.sh) and -[`scripts/release-finalize.sh`](../scripts/release-finalize.sh) do the mechanical work (versioning, -tagging, signing, svn staging, the GitHub release, npm); this guide covers the human steps around -them (GPG/KEYS, the vote, the announce). +The release guide is now at +[Release the Apache SkyWalking Node.js Agent](https://github.com/apache/skywalking-nodejs/blob/master/docs/en/contribution/release.md). -`master` carries the in-flight dev version (e.g. `0.9.0-dev`), like SkyWalking's `-SNAPSHOT`. You -do **not** edit `package.json` by hand — `scripts/release.sh` strips `-dev` for the release commit -and bumps the branch back to the next `-dev` in the same PR. +## Vote check -## The release scripts - -The whole flow is three commands (steps 1–3 below detail each phase): - -```shell -bash scripts/release.sh --dry-run # rehearse: full local build + sign + verify, NO push/svn/PR -bash scripts/release.sh # cut the RC: tag, sign, svn-stage, open the release PR, print the [VOTE] email -# ... [VOTE] on dev@skywalking.apache.org for >= 72h, >= 3 binding +1 ... -bash scripts/release-finalize.sh # promote svn dev -> release, publish the GitHub release, optional npm publish -``` - -(They are also wired as `npm run release` / `npm run release:finalize` if you prefer npm — but note -the dry-run flag then needs `npm run release -- --dry-run`.) - -Both scripts are **interactive** (every irreversible step asks `y/N`), must run on a single trusted -host (they read your SVN password), and are heavily commented — read the two files above for the -details. Knobs (all optional): - -| Variable / flag | Effect | Default | -| :--- | :--- | :--- | -| `--dry-run` or `SW_RELEASE_DRY_RUN=1` | Run everything locally, perform **no** remote mutation (no tag/branch push, no svn, no PR) | off | -| `SW_RELEASE_REPO_URL` | Git repo to clone + push | `https://github.com/apache/skywalking-nodejs.git` | -| `SW_RELEASE_BRANCH` | Branch to cut from | `master` | -| `SW_RELEASE_GH_REPO` | `owner/repo` for the release PR / GitHub release | `apache/skywalking-nodejs` | -| `SW_GPG_KEY` | Pin the signing key (`release.sh` sets this from your `@apache.org` key automatically) | git/gpg default | -| `NPM_OTP` | npm one-time password for the `release-finalize.sh` publish | prompt | - -> Tip: `unset SW_RELEASE_REPO_URL SW_RELEASE_BRANCH SW_RELEASE_GH_REPO` before a real release — -> a stray override from an earlier test would otherwise retarget the clone/push. - -## Prerequisites (one-time) - -- **Apache GPG key** with an `@apache.org` uid, added to the SkyWalking `KEYS` file: - 1. Upload the public key to a keyserver (e.g. [MIT](http://pgp.mit.edu:11371/)). - 1. Register the fingerprint at [id.apache.org](https://id.apache.org/). - 1. **Append** (never overwrite) your key to the [SkyWalking KEYS](https://dist.apache.org/repos/dist/release/skywalking/KEYS) - file — **PMC only**; ask a PMC member if needed. -- **Tools**: Node >= 20, plus `git`, `svn`, `gh`, `gpg`, `shasum`, `tar` (and `license-eye`, - optional). Run `gh auth login`; for the npm publish, `npm login` as a maintainer of - `skywalking-backend-js`. -- **Milestones**: close/roll the milestone on - [skywalking-nodejs](https://github.com/apache/skywalking-nodejs/milestones) and - [skywalking](https://github.com/apache/skywalking/milestones), and create the next one. - -Run the scripts on a single-user trusted host (they read your SVN password). - -## 1. Cut the release candidate — `bash scripts/release.sh` - -```shell -bash scripts/release.sh --dry-run # rehearse first: full local build + sign + verify, NO push/svn/PR -bash scripts/release.sh # the real cut -``` - -`scripts/release.sh` does, each irreversible step behind a `y/N`: - -1. preflight — the GPG signer is an `@apache.org` key, required tools present, Node >= 20; -1. fresh recursive clone of `master`; cut a `prepare-release-` branch; strip `-dev` and - commit + **tag** the release commit; -1. build + sign + verify `skywalking-nodejs-src-.tgz{,.asc,.sha512}`; -1. push the tag (**only after** verify), add a `Prepare next release -dev` commit, and - **open the release PR**; -1. upload the RC to `dist/dev/skywalking/node-js//`; -1. print the **[VOTE] email** (with the real tag, commit, and sha512 — copy it for step 2). - -> Release notes come from the auto-generated [GitHub Release](https://github.com/apache/skywalking-nodejs/releases) -> notes (`CHANGELOG.md` is a stub). Draft them once the tag exists: -> `gh release create v --draft --generate-notes --verify-tag --notes-start-tag v` - -## 2. Call for vote — `dev@skywalking.apache.org` - -Send the `[VOTE]` email the script printed (template below). Keep it open **>= 72 hours**; it -passes with **>= 3 binding +1** (PMC) and more +1 than -1. - -```text -Subject: [VOTE] Release Apache SkyWalking NodeJS version $VERSION - -Hi the SkyWalking Community: -This is a call for vote to release Apache SkyWalking NodeJS version $VERSION. - -Release notes: - - * https://github.com/apache/skywalking-nodejs/releases/tag/v$VERSION - -Release Candidate: - - * https://dist.apache.org/repos/dist/dev/skywalking/node-js/$VERSION - * sha512 checksums - - sha512xxxxyyyzzz skywalking-nodejs-src-x.x.x.tgz - -Release Tag : - - * (Git Tag) v$VERSION - -Release Commit Hash : - - * https://github.com/apache/skywalking-nodejs/tree/ - -Keys to verify the Release Candidate : - - * https://dist.apache.org/repos/dist/release/skywalking/KEYS - -Guide to build the release from source : - - * https://github.com/apache/skywalking-nodejs/blob/v$VERSION/CONTRIBUTING.md#compiling-and-building - -Voting will start now and will remain open for at least 72 hours. -A release passes with at least 3 binding +1 (PMC) votes and more +1 than -1. - -[ ] +1 Release this package. -[ ] +0 No opinion. -[ ] -1 Do not release this package because.... - -Thanks. - -[1] https://github.com/apache/skywalking-nodejs/blob/master/docs/How-to-release.md#vote-check -``` - -### Vote check - -Before voting +1, verify (all PMC members and committers): - -1. Features test. -1. All artifacts in the staging dir are published with `.asc` and `.sha512` files (no `.md5`). -1. The source package `skywalking-nodejs-src-$VERSION.tgz` is in - `https://dist.apache.org/repos/dist/dev/skywalking/node-js/$VERSION` with its `.asc` + `.sha512`. -1. `LICENSE` and `NOTICE` are present in the source package. -1. `shasum -c skywalking-nodejs-src-$VERSION.tgz.sha512`. -1. `gpg --verify skywalking-nodejs-src-$VERSION.tgz.asc skywalking-nodejs-src-$VERSION.tgz`. -1. Build from the source package following the - [build guide](../CONTRIBUTING.md#compiling-and-building). -1. License-header check via license-eye (`apache/skywalking-eyes`, as run by - `.github/workflows/license.yaml`); style lint via `npm run lint`. - -### Close the vote - -After it passes, send the closing mail, listing the binding and non-binding voters: - -```text -[RESULT][VOTE] Release Apache SkyWalking NodeJS version $VERSION - -72+ hours passed, we’ve got ($NUMBER) +1 bindings (and ... +1 non-bindings): - -(list names) -+1 bindings: -xxx -... - -+1 non-bindings: -xxx -... - -Thank you for voting, I’ll continue the release process. -``` - -## 3. Finalize — `bash scripts/release-finalize.sh` - -```shell -npm login # only if you will publish to npm (maintainer of skywalking-backend-js) -bash scripts/release-finalize.sh -``` - -It does, each irreversible step behind a `y/N` (npm auth is verified **up front**): - -1. promotes the RC on svn: `dev/` -> `release/`, retiring the previous (strictly-older) - release (it auto-archives to archive.apache.org); -1. publishes the **GitHub release** on `v` (auto-notes), attaching the voted artifacts; -1. optionally publishes `skywalking-backend-js@` to npm (skipped if already published). - -Then finish the human steps it reminds you about: - -1. **Merge the release PR** opened in step 1 (`master` returns to the next `-dev`; the `v` tag - stays pinned to the release commit). -1. **Website** ([apache/skywalking-website](https://github.com/apache/skywalking-website)): add the - release event, bump the NodeJS Agent block in `data/releases.yml` and the docs pointer in - `data/docs.yml` (see a prior [PR](https://github.com/apache/skywalking-website/pull/190)). -1. **[ANNOUNCE] email** from your `@apache.org` address to `dev@skywalking.apache.org` and - `announce@apache.org`: - -```text -Subject: [ANNOUNCEMENT] Apache SkyWalking NodeJS $VERSION Released - -Hi the SkyWalking Community - -On behalf of the SkyWalking Team, I’m glad to announce that SkyWalking NodeJS $VERSION is now released. - -SkyWalking NodeJS: The NodeJS Agent for Apache SkyWalking, which provides the native tracing abilities for NodeJS backend project. - -SkyWalking: APM (application performance monitor) tool for distributed systems, especially designed for microservices, cloud native and container-based (Docker, Kubernetes, Mesos) architectures. - -Download Links: http://skywalking.apache.org/downloads/ - -Release Notes : https://github.com/apache/skywalking-nodejs/releases/tag/v$VERSION - -Website: http://skywalking.apache.org/ - -SkyWalking NodeJS Resources: -- Issue: https://github.com/apache/skywalking/issues -- Mailing list: dev@skywalking.apache.org -- Documents: https://github.com/apache/skywalking-nodejs/blob/v$VERSION/README.md - -The Apache SkyWalking Team -``` - -## Manual fallback - -The scripts implement the standard ASF steps and are heavily commented; if you must run a step by -hand, read [`scripts/release.sh`](../scripts/release.sh) and -[`scripts/release-finalize.sh`](../scripts/release-finalize.sh). The essentials: clone with -`--recurse-submodules`; `npm version --no-git-tag-version` to strip `-dev`; `npm install`; -commit; tag locally; `npm run release-src` to build + sign; verify the tarball + signature; -**then** push the tag; `svn` the three artifacts to `dist/dev/.../node-js//`; after the vote -`svn mv` dev -> release; publish the GitHub release; `npm run build && npm publish`. +See the +[current vote checklist](https://github.com/apache/skywalking-nodejs/blob/master/docs/en/contribution/release.md#vote-check). diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..6508e9f --- /dev/null +++ b/docs/README.md @@ -0,0 +1,40 @@ +# Apache SkyWalking Node.js Agent documentation + +The Node.js Agent reports traces and Node.js runtime metrics to Apache SkyWalking OAP. Start with +[Quick start](en/setup/quick-start.md), then use the pages below when you need more detail. + +## Setup + +- [Quick start](en/setup/quick-start.md) — install the package and report the first trace. +- [Configuration](en/setup/configuration.md) — all environment variables and `agent.start()` + options. +- [Start and stop the agent](en/setup/startup-and-shutdown.md) — load order, `flush()`, and + `destroy()`. + +## Features + +- [Tracing](en/features/tracing.md) — trace creation, context transfer, ignored requests, and data + limits. +- [Node.js runtime metrics](en/features/runtime-metrics.md) — the 12 meters reported by the agent. + +## Plugins + +- [Supported libraries](en/plugins/supported-libraries.md) — automatic instrumentation and plugin + names. +- [AWS SDK v2](en/plugins/aws-sdk-v2.md) — DynamoDB, Lambda, SNS, and SQS instrumentation. + +## Advanced use + +- [Serverless](en/advanced/serverless.md) — AWS Lambda and Azure Functions wrappers. +- [Webpack](en/advanced/webpack.md) — current bundle support and its limits. +- [Troubleshooting](en/advanced/troubleshooting.md) — common setup and reporting problems. + +## Development and contribution + +- [Build and test](en/contribution/build-and-test.md) — build the agent and run its checks. +- [Plugin development](en/contribution/plugin-development.md) — add or change library + instrumentation. +- [Release](en/contribution/release.md) — release steps for maintainers and vote checks. + +Published versions are listed on the +[GitHub Releases page](https://github.com/apache/skywalking-nodejs/releases). diff --git a/docs/en/advanced/serverless.md b/docs/en/advanced/serverless.md new file mode 100644 index 0000000..79702e5 --- /dev/null +++ b/docs/en/advanced/serverless.md @@ -0,0 +1,95 @@ +# Serverless + +The package exports wrappers for AWS Lambda and Azure Functions HTTP triggers. These wrappers are +experimental and do not have dedicated current CI suites in this repository. Test them with the +runtime and event form used by your function. + +## AWS Lambda + +Choose the wrapper that matches the event: + +| Export | Use | +| --- | --- | +| `AWSLambdaGatewayAPIHTTP` | API Gateway HTTP API payload version 2.0 | +| `AWSLambdaGatewayAPIREST` | API Gateway REST API payload version 1.0 | +| `AWSLambdaTriggerPlugin` | Other Lambda triggers | + +The two API Gateway wrappers read incoming SkyWalking context and record the HTTP path, method, +URL, source address, and response status when those values are present. The generic wrapper starts +a Lambda span but cannot read HTTP context from an unknown event form. + +```javascript +const { + default: agent, + AWSLambdaGatewayAPIHTTP, +} = require('skywalking-backend-js'); + +agent.start({ + serviceName: 'checkout-lambda', + collectorAddress: 'oap.example.com:11800', +}); + +exports.handler = AWSLambdaGatewayAPIHTTP.wrap(async (event) => { + return { + statusCode: 200, + body: JSON.stringify({ ok: true }), + }; +}); +``` + +The wrapper handles an async return value, the handler callback, and the older `context.done`, +`context.succeed`, and `context.fail` forms. It ends the span only once. + +### Flush before Lambda freezes the process + +AWS can freeze a process after the handler completes. `SW_AWS_LAMBDA_FLUSH` controls when the +wrapper calls `agent.flush()` before it returns: + +- `0` — flush after every invocation. +- A positive number — flush when at least that many seconds passed between wrapped invocations. + The first invocation also flushes. The default is `2`. +- `-1` — never flush in the wrapper. + +A flush adds time to the invocation. Test the setting with your request rate and Lambda timeout. + +### Link direct Lambda invokes + +When one instrumented service calls a wrapped Lambda through AWS SDK v2, set +`SW_AWS_LAMBDA_CHAIN=true` on the caller. The caller adds trace context to the invoke payload, and +the generic wrapper reads and removes it before calling your handler. + +This can change a non-object payload into a JSON object. Read [AWS SDK v2](../plugins/aws-sdk-v2.md) +before enabling it. + +## Azure Functions HTTP trigger + +Wrap a JavaScript HTTP trigger with `AzureHttpTriggerPlugin`: + +```javascript +const { + default: agent, + AzureHttpTriggerPlugin, +} = require('skywalking-backend-js'); + +agent.start({ + serviceName: 'checkout-azure-function', + collectorAddress: 'oap.example.com:11800', +}); + +module.exports = AzureHttpTriggerPlugin.wrap(async function (context, req) { + return { + status: 200, + body: { ok: true }, + }; +}); +``` + +The wrapper reads incoming trace headers, records HTTP values, and supports Promise returns and +`context.done`. Make sure the agent starts before other instrumented modules used by the function. + +## Limits + +- These wrappers are for JavaScript handler functions. Other Azure or AWS host languages are not + handled by this package. +- A platform timeout or forced process stop can still lose data before a flush completes. +- `SW_HTTP_IGNORE_METHOD` also applies to the API Gateway and Azure HTTP wrappers. diff --git a/docs/en/advanced/troubleshooting.md b/docs/en/advanced/troubleshooting.md new file mode 100644 index 0000000..4218ee4 --- /dev/null +++ b/docs/en/advanced/troubleshooting.md @@ -0,0 +1,96 @@ +# Troubleshooting + +Start with the checks below. Set `SW_AGENT_LOGGING_LEVEL=debug` for a short test when you need more +agent detail. Change it back after the test because debug logging adds work and can produce large +logs. + +## No service or traces in SkyWalking + +1. Confirm that Node.js is version 20 or later. +2. Confirm that `agent.start()` runs before Express, HTTP clients, database clients, and other + instrumented modules are loaded. +3. Check `SW_DISABLE`. The exact value `true` keeps the agent stopped. +4. Check `SW_AGENT_NAME` and select that service name in the SkyWalking UI. +5. Send a request that uses a [supported library](../plugins/supported-libraries.md). +6. Check that the path, suffix, and HTTP method are not filtered by trace settings. +7. Check the agent log for plugin version or OAP connection errors. + +If the process is short-lived, call `await agent.flush()` before exit. + +## OAP connection errors + +The collector setting must use `host:port` form: + +```bash +export SW_AGENT_COLLECTOR_BACKEND_SERVICES=oap.example.com:11800 +``` + +Check DNS, network access, firewall rules, and the OAP gRPC port. The current agent uses only the +first item if the value is a comma-separated list. + +An option passed to `agent.start()` replaces the environment value. This includes an empty string: + +```typescript +// Do not do this. It removes the default collector address. +agent.start({ collectorAddress: '' }); +``` + +For a TLS OAP endpoint, set `SW_AGENT_SECURE=true`. The agent uses the system trust store. It has no +configuration for a custom CA or client certificate. Set `SW_AGENT_AUTHENTICATION` if OAP requires +an agent token. + +## Traces are missing during an OAP outage + +The agent keeps finished segments in a memory buffer. When the buffer reaches +`SW_AGENT_MAX_BUFFER_SIZE`, it removes the oldest finished segment. Restore the OAP connection; +increasing the buffer only delays data loss and uses more process memory. + +## A library has no spans + +- Check that the agent started before the library loaded. +- Check the library and plugin in [Supported libraries](../plugins/supported-libraries.md). +- Most plugin version rules are broad, but CI tests exact versions from `package-lock.json`. Test + other versions in your application. +- Webpack uses a smaller static plugin set. See [Webpack](webpack.md). +- AWS SDK for JavaScript v3 is not covered by the AWS SDK v2 plugins. + +## Runtime metrics do not appear + +1. Check that `SW_AGENT_NODEJS_RUNTIME_METRICS_REPORTER_ACTIVE` is not `false`. +2. Wait at least one report period. The default is 20 seconds. +3. Check that your OAP version includes the Node.js runtime meter rules and dashboard. +4. Check the OAP connection and agent log. + +See [Node.js runtime metrics](../features/runtime-metrics.md) for meter names and OAP setup. + +## Agent logs are hard to find + +The default agent log level is `error`. + +- When `NODE_ENV` is not `production`, logs go to the console. +- When `NODE_ENV=production`, logs go to `skywalking.log` in the process working directory. +- Set `SW_LOGGING_TARGET=console` to use the console in production. + +## Data is missing when the process stops + +`agent.destroy()` stops reporters but does not flush them. Use this order: + +```typescript +await agent.flush(); +agent.destroy(); +``` + +Do not use stop and restart as a normal agent update method inside one process. Module patches stay +installed after `destroy()`. + +## SQS receives only one message + +This is current AWS SDK v2 SQS plugin behavior. The plugin removes `MaxNumberOfMessages` so it can +link one message to one entry span. Disable `AWS2SQS` if the application requires batch receives. + +## Ask for help + +If the checks do not solve the problem, open an issue in the +[Apache SkyWalking issue tracker](https://github.com/apache/skywalking/issues/new). Include the +Node.js Agent version, Node.js version, OAP version, target library versions, startup code, and +relevant agent errors. Remove tokens and private data first. diff --git a/docs/en/advanced/webpack.md b/docs/en/advanced/webpack.md new file mode 100644 index 0000000..0677b4c --- /dev/null +++ b/docs/en/advanced/webpack.md @@ -0,0 +1,74 @@ +# Webpack + +Webpack needs module imports to be known at build time. The normal agent loader scans plugin files +and loads target modules at run time, so the agent uses a separate static loader when it detects a +bundle. + +Webpack support is experimental. + +## Plugins in the static loader + +The current static loader includes: + +- Node.js HTTP and HTTPS +- Express +- MySQL +- PostgreSQL +- MongoDB and Mongoose +- ioredis +- amqplib +- AWS SDK v2 DynamoDB, Lambda, SNS, and SQS + +Axios and MySQL2 are not in the static loader because their package metadata cannot be loaded in +the form expected by the current code. They can be instrumented by the normal loader but not by +the current Webpack loader. + +## Missing optional modules at build time + +The static loader contains imports for all modules in the list above. Webpack may report a missing +module even when your application does not use that plugin. Mark each unused optional module as +`false` in `resolve.alias`. + +```javascript +module.exports = { + target: 'node', + resolve: { + alias: { + amqplib: false, + 'aws-sdk': false, + express: false, + ioredis: false, + mongodb: false, + mongoose: false, + mysql: false, + pg: false, + }, + }, +}; +``` + +Keep a module out of this list when your bundle uses it and you want its plugin installed. + +## Disable a bundled plugin + +The bundled loader currently checks its internal plugin file name, including `Plugin`. For +example: + +```bash +export SW_AGENT_DISABLE_PLUGINS='MySQLPlugin,ExpressPlugin' +``` + +This differs from the normal loader, which uses `mysql,express`. Keep the value with the deployment +that needs it. + +## Limits + +- Only plugins listed in the static loader can work in a bundle. +- The loader reads a target module's `package.json` to check its version. A package that blocks this + import cannot use the current static loader. +- A plugin contributor must add a new plugin to both the normal plugin directory and the static + loader when Webpack support is possible. +- Tree shaking and other bundler changes can affect module patching. Test the final production + bundle, not only the source application. + +If bundling is not required, run the normal Node.js output and use the normal plugin loader. diff --git a/docs/en/contribution/build-and-test.md b/docs/en/contribution/build-and-test.md new file mode 100644 index 0000000..586335c --- /dev/null +++ b/docs/en/contribution/build-and-test.md @@ -0,0 +1,94 @@ +# Build and test + +This guide is for contributors building the Node.js Agent from source. + +## Requirements + +- Node.js 20 or later. Node.js 20 is the minimum version, and CI runs on Node.js 20, 22, and 24. +- Git. +- Docker for plugin integration tests and the built-package test. + +## Get the source + +The SkyWalking protocol definitions are a Git submodule. Clone with submodules: + +```bash +git clone --recurse-submodules https://github.com/apache/skywalking-nodejs.git +cd skywalking-nodejs +``` + +If you already cloned the repository: + +```bash +git submodule update --init --recursive +``` + +## Install and build + +```bash +npm install +npm run build +``` + +The `prepare` step generates TypeScript sources from the protocol definitions. The build compiles +the agent into `lib/`. Do not edit generated files in `src/proto/` by hand. + +## Run checks + +Run the TypeScript lint check: + +```bash +npm run lint +``` + +Run unit tests without Docker plugin suites: + +```bash +npx jest --testPathIgnorePatterns '/node_modules/' '/tests/plugins/' --runInBand +``` + +Run all tests, including Docker plugin suites: + +```bash +npm run test +``` + +Run one plugin suite: + +```bash +npm run test tests/plugins/http/ +``` + +Plugin tests start a mock SkyWalking collector and target services with Docker Compose. Make sure +Docker is running and that the test ports are free. + +Test the built package in a container: + +```bash +docker build . -f tests/build/Dockerfile -t skywalking-nodejs:test +docker run --rm skywalking-nodejs:test +``` + +## Main source areas + +| Path | Content | +| --- | --- | +| `src/index.ts` | Public agent start, flush, stop, and wrapper exports | +| `src/config/` | Configuration and environment parsing | +| `src/core/` | Plugin loader and plugin helpers | +| `src/plugins/` | Automatic library instrumentation | +| `src/agent/core/` | Reporting, service registration, meters, and gRPC transport | +| `src/trace/` | Trace context, spans, segments, and component IDs | +| `src/aws/`, `src/azure/` | Serverless wrappers and AWS helpers | +| `tests/plugins/` | Docker-based plugin integration tests | +| `tests/remote/`, `tests/runtime/` | Reporter and runtime meter tests | + +## Before opening a pull request + +1. Add or update tests for the change. +2. Run `npm run lint` and `npm run build`. +3. Run the related unit or plugin tests. +4. Update the user documentation when behavior or configuration changes. +5. Keep Apache license headers on new source and configuration files. + +CI repeats the build, lint, unit, plugin, and built-package checks that apply to the change. diff --git a/docs/en/contribution/plugin-development.md b/docs/en/contribution/plugin-development.md new file mode 100644 index 0000000..689be1f --- /dev/null +++ b/docs/en/contribution/plugin-development.md @@ -0,0 +1,89 @@ +# Plugin development + +A plugin patches a Node.js library so calls made by that library create SkyWalking spans. Read an +existing plugin close to your target library before adding a new one. + +## Plugin contract + +Each automatic plugin in `src/plugins/` implements `SwPlugin` and exports one default instance: + +```typescript +class ExamplePlugin implements SwPlugin { + readonly module = 'example-library'; + readonly versions = '^1.0.0'; + + install(installer: PluginInstaller): void { + const target = installer.require?.(this.module) ?? require(this.module); + // Patch the target API. + } +} + +export default new ExamplePlugin(); +``` + +Use a file name ending in `Plugin.ts`. The normal loader scans compiled files in `lib/plugins/`. + +- `module` is the npm module that must be installed in the application. +- `versions` is a semantic version range checked by the loader. +- Set `isBuiltIn = true` only for a Node.js built-in module such as `http`. +- Use `installer.require` when possible so the plugin loads the application's copy of the module. + +Choose a version range from real API support. Do not use `*` only to avoid checking versions. + +## Instrumentation rules + +Keep library behavior unchanged apart from trace collection: + +1. Save the original function before replacing it. +2. Preserve `this`, arguments, return values, thrown errors, callbacks, events, and Promise results. +3. Create the correct entry, exit, or local span from `ContextManager.current`. +4. Set the component, span layer, operation name, peer, and standard tags when the data exists. +5. Record errors and stop every span on all completion paths. +6. Use `span.async()` and `span.resync()` when work leaves and later returns to the active async + path. +7. Do not record request bodies, database parameters, or other private values by default. + +`src/core/SwPlugin.ts` provides helpers for common callback, Promise, and event-emitter completion +forms. Some libraries need a local helper because their API has different completion rules. + +Use an existing component in `src/trace/Component.ts` when it matches. A new component ID must be +agreed with the SkyWalking project so it does not conflict with another agent or integration. + +## Add tests + +Create `tests/plugins//` by following a similar plugin suite. A normal suite contains: + +- `test.ts` to start Docker Compose and send expected data to the mock collector; +- `docker-compose.yml` for the collector, application, and target service; +- client and server files that exercise the instrumented API; +- `expected.data.yaml` with the expected SkyWalking segment data. + +Add the target package to `devDependencies` and update `package-lock.json` when the test needs it. +The CI plugin matrix finds each directory under `tests/plugins/` except `common`, so a new directory +becomes a CI job without a manual matrix entry. + +Run the suite: + +```bash +npm run test tests/plugins/example-library/ +``` + +Also run unit tests, lint, and build as described in [Build and test](build-and-test.md). + +## Webpack support + +The normal loader finds a compiled plugin automatically. The Webpack loader does not. If the target +package allows its `package.json` to be imported, add the plugin to +`PluginInstaller.installBundled()` and test a production bundle. + +If this is not possible, state the limit in [Webpack](../advanced/webpack.md). Do not claim Webpack +support based only on the normal plugin test. + +## Update documentation + +For a new plugin: + +1. Add the library and current CI state to [Supported libraries](../plugins/supported-libraries.md). +2. Add configuration details if the plugin adds settings. +3. Add important behavior changes, such as changed batching or payload data. +4. Update troubleshooting when the plugin has a common setup limit. diff --git a/docs/en/contribution/release.md b/docs/en/contribution/release.md new file mode 100644 index 0000000..8278534 --- /dev/null +++ b/docs/en/contribution/release.md @@ -0,0 +1,239 @@ +# Release the Apache SkyWalking Node.js Agent + +This guide explains how to release the Node.js Agent and how to check a release candidate. The +shell scripts +[`scripts/release.sh`](https://github.com/apache/skywalking-nodejs/blob/master/scripts/release.sh) +and +[`scripts/release-finalize.sh`](https://github.com/apache/skywalking-nodejs/blob/master/scripts/release-finalize.sh) +update versions, create tags, sign files, stage files in SVN, create the GitHub release, and publish +to npm. This guide covers the GPG key, vote, and announcement steps around those scripts. + +`master` contains the current development version, such as `0.9.0-dev`. Do not edit +`package.json` by hand. `scripts/release.sh` removes `-dev` for the release commit and changes the +branch to the next `-dev` version in the same pull request. + +## Publish the development documentation + +The `Next` documentation should be registered in +[apache/skywalking-website](https://github.com/apache/skywalking-website) as soon as this docs +structure is merged. Do not wait for the next agent release. In the NodeJS Agent entry in +`data/docs.yml`, set `repoUrl` to this repository and add the link +`/docs/skywalking-nodejs/next/readme/`. The website build then imports `docs/README.md` and +`docs/menu.yml`. + +## The release scripts + +The release uses these three commands. The next sections explain each one. + +```shell +bash scripts/release.sh --dry-run # rehearse: full local build + sign + verify, NO push/svn/PR +bash scripts/release.sh # cut the RC: tag, sign, svn-stage, open the release PR, print the [VOTE] email +# ... [VOTE] on dev@skywalking.apache.org for >= 72h, >= 3 binding +1 ... +bash scripts/release-finalize.sh # promote svn dev -> release, publish the GitHub release, optional npm publish +``` + +You can also use `npm run release` and `npm run release:finalize`. With npm, use +`npm run release -- --dry-run` for the dry run. + +Both scripts ask `y/N` before each remote change. Run them on one trusted, single-user host because +they read your SVN password. These settings are optional: + +| Variable / flag | Effect | Default | +| :--- | :--- | :--- | +| `--dry-run` or `SW_RELEASE_DRY_RUN=1` | Run locally with no remote changes: no push, SVN upload, or pull request | off | +| `SW_RELEASE_REPO_URL` | Git repo to clone + push | `https://github.com/apache/skywalking-nodejs.git` | +| `SW_RELEASE_BRANCH` | Branch to cut from | `master` | +| `SW_RELEASE_GH_REPO` | `owner/repo` for the release PR / GitHub release | `apache/skywalking-nodejs` | +| `SW_GPG_KEY` | Pin the signing key (`release.sh` sets this from your `@apache.org` key automatically) | git/gpg default | +| `NPM_OTP` | npm one-time password for the `release-finalize.sh` publish | prompt | + +> Before a real release, run +> `unset SW_RELEASE_REPO_URL SW_RELEASE_BRANCH SW_RELEASE_GH_REPO`. This removes test settings that +> could send release changes to the wrong repository. + +## Prerequisites (one-time) + +- **Apache GPG key** with an `@apache.org` user ID, added to the SkyWalking `KEYS` file: + 1. Upload the public key to a public key server. + 1. Register the fingerprint at [id.apache.org](https://id.apache.org/). + 1. **Append** (never overwrite) your key to the [SkyWalking KEYS](https://dist.apache.org/repos/dist/release/skywalking/KEYS) + file — **PMC only**; ask a PMC member if needed. +- **Tools**: Node >= 20, plus `git`, `svn`, `gh`, `gpg`, `shasum`, `tar` (and `license-eye`, + optional). Run `gh auth login`; for the npm publish, `npm login` as a maintainer of + `skywalking-backend-js`. +- **Milestones**: close the current milestone and create the next one in + [skywalking-nodejs](https://github.com/apache/skywalking-nodejs/milestones) and + [skywalking](https://github.com/apache/skywalking/milestones). + +## 1. Cut the release candidate — `bash scripts/release.sh` + +```shell +bash scripts/release.sh --dry-run # rehearse first: full local build + sign + verify, NO push/svn/PR +bash scripts/release.sh # the real cut +``` + +`scripts/release.sh` performs these steps. It asks for approval before each remote change. + +1. Check the GPG signer, required tools, and Node.js version. +1. Clone `master` with submodules and create a `prepare-release-` branch. +1. Remove `-dev`, commit the release version, and tag the release commit. +1. Build, sign, and check `skywalking-nodejs-src-.tgz{,.asc,.sha512}`. +1. Push the tag only after the checks pass. Add the next `-dev` commit and open the release pull + request. +1. Upload the release candidate to `dist/dev/skywalking/node-js//`. +1. Print the `[VOTE]` email with the tag, commit, and SHA-512 value. + +> Release notes come from the auto-generated [GitHub Release](https://github.com/apache/skywalking-nodejs/releases) +> notes (`CHANGELOG.md` is a stub). Draft them once the tag exists: +> `gh release create v --draft --generate-notes --verify-tag --notes-start-tag v` + +## 2. Call for vote — `dev@skywalking.apache.org` + +Send the `[VOTE]` email printed by the script. The template is below. Keep the vote open for at +least 72 hours. It passes with at least three binding `+1` votes from PMC members and more `+1` +votes than `-1` votes. + +```text +Subject: [VOTE] Release Apache SkyWalking NodeJS version $VERSION + +Hi the SkyWalking Community: +This is a call for vote to release Apache SkyWalking NodeJS version $VERSION. + +Release notes: + + * https://github.com/apache/skywalking-nodejs/releases/tag/v$VERSION + +Release Candidate: + + * https://dist.apache.org/repos/dist/dev/skywalking/node-js/$VERSION + * sha512 checksums + - sha512xxxxyyyzzz skywalking-nodejs-src-x.x.x.tgz + +Release Tag : + + * (Git Tag) v$VERSION + +Release Commit Hash : + + * https://github.com/apache/skywalking-nodejs/tree/ + +Keys to verify the Release Candidate : + + * https://dist.apache.org/repos/dist/release/skywalking/KEYS + +Guide to build the release from source : + + * docs/en/contribution/build-and-test.md (in the source package) + +Voting will start now and will remain open for at least 72 hours. +A release passes with at least 3 binding +1 (PMC) votes and more +1 than -1. + +[ ] +1 Release this package. +[ ] +0 No opinion. +[ ] -1 Do not release this package because.... + +Thanks. + +[1] docs/en/contribution/release.md, "Vote check" section (in the source package) +``` + +### Vote check + +Before voting `+1`, check the following items: + +1. Test the release features. +1. Check that every file in the staging directory has `.asc` and `.sha512` files. Do not use + `.md5`. +1. The source package `skywalking-nodejs-src-$VERSION.tgz` is in + `https://dist.apache.org/repos/dist/dev/skywalking/node-js/$VERSION` with its `.asc` + `.sha512`. +1. `LICENSE` and `NOTICE` are present in the source package. +1. `shasum -c skywalking-nodejs-src-$VERSION.tgz.sha512`. +1. `gpg --verify skywalking-nodejs-src-$VERSION.tgz.asc skywalking-nodejs-src-$VERSION.tgz`. +1. Build from the source package following the + [build and test guide](build-and-test.md). +1. Run the license header check with `apache/skywalking-eyes`, as configured in + `.github/workflows/license.yaml`, and run `npm run lint`. + +### Close the vote + +After the vote passes, send the result email. List the binding and non-binding voters. + +```text +[RESULT][VOTE] Release Apache SkyWalking NodeJS version $VERSION + +72+ hours passed, we’ve got ($NUMBER) +1 bindings (and ... +1 non-bindings): + +(list names) ++1 bindings: +xxx +... + ++1 non-bindings: +xxx +... + +Thank you for voting, I’ll continue the release process. +``` + +## 3. Finalize — `bash scripts/release-finalize.sh` + +```shell +npm login # only if you will publish to npm (maintainer of skywalking-backend-js) +bash scripts/release-finalize.sh +``` + +The script checks npm access first and asks for approval before each remote change. It then: + +1. Moves the release candidate in SVN from `dev/` to `release/` and archives the older + release. +1. Publishes the GitHub release for `v` and attaches the approved files. +1. Optionally publishes `skywalking-backend-js@` to npm. It skips this when the version is + already published. + +Then complete these steps: + +1. **Merge the release pull request** opened in step 1. `master` returns to the next `-dev` + version, while the `v` tag stays on the release commit. +1. **Update the website** in + [apache/skywalking-website](https://github.com/apache/skywalking-website). Update the NodeJS Agent + entries in `data/releases.yml` and `data/docs.yml`. Add the released documentation version and + its commit. The one-time `Next` registration described above should already be present. +1. **[ANNOUNCE] email** from your `@apache.org` address to `dev@skywalking.apache.org` and + `announce@apache.org`: + +```text +Subject: [ANNOUNCEMENT] Apache SkyWalking NodeJS $VERSION Released + +Hi the SkyWalking Community + +On behalf of the SkyWalking Team, I’m glad to announce that Apache SkyWalking NodeJS $VERSION is now released. + +SkyWalking NodeJS is the Node.js Agent for Apache SkyWalking. It reports distributed traces and Node.js runtime metrics. + +Apache SkyWalking is an application performance monitoring system for distributed services. + +Download Links: https://skywalking.apache.org/downloads/ + +Release Notes : https://github.com/apache/skywalking-nodejs/releases/tag/v$VERSION + +Website: https://skywalking.apache.org/ + +SkyWalking NodeJS Resources: +- Issue: https://github.com/apache/skywalking/issues +- Mailing list: dev@skywalking.apache.org +- Documents: https://github.com/apache/skywalking-nodejs/blob/v$VERSION/docs/README.md + +The Apache SkyWalking Team +``` + +## Manual fallback + +The scripts follow the standard ASF release steps. Read +[`scripts/release.sh`](https://github.com/apache/skywalking-nodejs/blob/master/scripts/release.sh) +and +[`scripts/release-finalize.sh`](https://github.com/apache/skywalking-nodejs/blob/master/scripts/release-finalize.sh) +before running any step by hand. Clone with `--recurse-submodules`, remove `-dev` with +`npm version --no-git-tag-version`, and run `npm install`. Commit and tag locally. Run +`npm run release-src`, then check the source file, signature, and checksum before pushing the tag. +Stage the three files in `dist/dev/.../node-js//`. After the vote, move the SVN directory from +`dev` to `release`, publish the GitHub release, and run `npm run build && npm publish`. diff --git a/docs/en/features/runtime-metrics.md b/docs/en/features/runtime-metrics.md new file mode 100644 index 0000000..1ba0304 --- /dev/null +++ b/docs/en/features/runtime-metrics.md @@ -0,0 +1,61 @@ +# Node.js runtime metrics + +The agent reports 12 process-level meters through SkyWalking `MeterReportService`. Runtime metrics +are enabled by default. The default sample and report period is 20 seconds. + +## Reported meters + +| Meter name | Node.js source | Unit or value | +| --- | --- | --- | +| `instance_nodejs_process_cpu` | `process.cpuUsage()` | Percent, user plus system CPU, divided by the logical CPU count | +| `instance_nodejs_heap_used` | `process.memoryUsage().heapUsed` | Bytes | +| `instance_nodejs_heap_total` | `process.memoryUsage().heapTotal` | Bytes | +| `instance_nodejs_heap_limit` | `v8.getHeapStatistics().heap_size_limit` | Bytes | +| `instance_nodejs_rss` | `process.memoryUsage().rss` | Bytes | +| `instance_nodejs_external_memory` | `process.memoryUsage().external` | Bytes | +| `instance_nodejs_array_buffers` | `process.memoryUsage().arrayBuffers` | Bytes | +| `instance_nodejs_uptime` | `process.uptime()` | Seconds | +| `instance_nodejs_peak_malloced_memory` | `v8.getHeapStatistics().peak_malloced_memory` | Bytes | +| `instance_nodejs_malloced_memory` | `v8.getHeapStatistics().malloced_memory` | Bytes | +| `instance_nodejs_old_space_used` | `v8.getHeapSpaceStatistics()` old space | Bytes | +| `instance_nodejs_new_space_used` | `v8.getHeapSpaceStatistics()` new space | Bytes | + +CPU is calculated from the change in user and system CPU time between samples. The result is +divided by the number of logical CPUs, so its normal range is 0 to 100 percent for the whole Node.js +process. + +## Configure reporting + +Disable runtime meters: + +```bash +export SW_AGENT_NODEJS_RUNTIME_METRICS_REPORTER_ACTIVE=false +``` + +Change the period to 30 seconds: + +```bash +export SW_AGENT_NODEJS_RUNTIME_METRICS_REPORT_PERIOD=30000 +``` + +The period must be a positive integer in milliseconds. The same period controls sampling and +reporting. A shorter period creates more work in the agent and OAP. + +The matching `agent.start()` options are `runtimeMetricsReporterActive` and +`runtimeMetricsReportPeriod`. See [Configuration](../setup/configuration.md) for old option names. + +## OAP dashboard + +OAP must include the Node.js runtime meter rules and dashboard. The OAP setup maps the raw +`instance_nodejs_*` meters to stored metrics whose names start with `meter_`. + +See the +[Node.js runtime dashboard setup](https://skywalking.apache.org/docs/main/next/en/setup/backend/dashboards-nodejs-runtime/) +in the main SkyWalking documentation. + +## Flush behavior + +`agent.flush()` takes a current runtime sample and asks the meter reporter to send it. The reporter +does not keep a backlog of old meter values. A sample that cannot be sent is discarded. + +Custom business metrics are not available through a stable public API in this package. diff --git a/docs/en/features/tracing.md b/docs/en/features/tracing.md new file mode 100644 index 0000000..d821787 --- /dev/null +++ b/docs/en/features/tracing.md @@ -0,0 +1,88 @@ +# Tracing + +The agent creates SkyWalking trace segments and sends them to OAP over gRPC. Supported plugins +create spans around HTTP, framework, database, cache, messaging, and AWS SDK calls. + +## How tracing starts + +Start the agent before the modules that it must patch. See +[Start and stop the agent](../setup/startup-and-shutdown.md). + +For an incoming HTTP request, the HTTP plugin reads the SkyWalking `sw8` trace header. A valid +header links the new span to its parent. When the header is absent, the plugin starts a new trace. +A malformed `sw8` header is not recorded as a new trace. For an outgoing HTTP request, the plugin +writes the `sw8` header. + +The HTTP operation name normally contains the method and path. An HTTP status code of 400 or higher +marks the span as an error. Plugins can add more tags, logs, component IDs, and peer information. + +## Filter HTTP traces + +Use these settings to avoid recording requests that do not help you: + +- `SW_IGNORE_SUFFIX` filters a request path by suffix. +- `SW_TRACE_IGNORE_PATH` filters an operation name by a pattern. +- `SW_HTTP_IGNORE_METHOD` filters a request by HTTP method. + +Incoming HTTP and Express operation names use `METHOD:/path`, such as `GET:/orders`. Outgoing HTTP +operation names use `/path`. `SW_TRACE_IGNORE_PATH` accepts comma-separated patterns: + +| Pattern | Meaning | Example match | +| --- | --- | --- | +| `?` | One character except `/` | `GET:/user/?` matches `GET:/user/1`. | +| `*` | Zero or more characters except `/` | `GET:/assets/*` matches `GET:/assets/a.js`. | +| `**` | Zero or more characters across path parts | `**/internal/**` matches incoming and outgoing nested paths. | + +Example: + +```bash +export SW_TRACE_IGNORE_PATH='**/health,**/internal,**/internal/**' +export SW_HTTP_IGNORE_METHOD='OPTIONS,HEAD' +``` + +An ignored trace sends ignored state to supported downstream services. This prevents a later +service from recording a partial trace without its parent. + +## Buffer and timeout + +Finished segments wait in memory until the reporter sends them. `SW_AGENT_MAX_BUFFER_SIZE` controls +the number of finished segments that can wait. When this buffer is full, the oldest finished +segment is removed. + +The same value limits active trace segments. When the active limit is reached, new work uses an +ignored context until the reporter resets the limit. This ignored state is sent to supported +downstream services. + +`SW_AGENT_TRACE_TIMEOUT` sets the gRPC request deadline in milliseconds. It applies to trace and +meter reports and to service registration and heartbeat requests. Reporting errors do not stop +application requests. The agent limits repeated connection error logs to avoid a log storm. + +## Database statements and parameters + +Database plugins can record a statement or command. SQL and MongoDB parameter values are disabled +by default. Enable them only when their data is safe to send to OAP: + +```bash +export SW_SQL_TRACE_PARAMETERS=true +export SW_SQL_PARAMETERS_MAX_LENGTH=512 +export SW_MONGO_TRACE_PARAMETERS=true +export SW_MONGO_PARAMETERS_MAX_LENGTH=512 +``` + +These values may include secrets or personal data. The length settings limit text size but do not +remove private values. + +## Cold start tag + +The first span made by the process gets the tag `coldStart=true`. Set `SW_COLD_ENDPOINT=true` to +also add `` to that operation name. + +## Manual instrumentation status + +The package exports `ContextManager` and `config`, and some applications use `ContextManager` to +create manual spans. This use is not documented as a stable manual instrumentation API and is not +covered by compatibility guarantees. Use the built-in plugins when possible. + +The package does not provide a public API for custom business metrics. For business metrics, +consider the +[SkyWalking OpenTelemetry receiver](https://skywalking.apache.org/docs/main/next/en/setup/backend/opentelemetry-receiver/). diff --git a/docs/en/plugins/aws-sdk-v2.md b/docs/en/plugins/aws-sdk-v2.md new file mode 100644 index 0000000..db7d6ab --- /dev/null +++ b/docs/en/plugins/aws-sdk-v2.md @@ -0,0 +1,96 @@ +# AWS SDK for JavaScript v2 + +The agent instruments selected services in the `aws-sdk` 2.x package. AWS SDK for JavaScript v3 +uses different packages and is not instrumented by these plugins. + +Start the agent before loading `aws-sdk`: + +```javascript +const agent = require('skywalking-backend-js').default; + +agent.start(); + +const AWS = require('aws-sdk'); +``` + +The plugins support normal AWS SDK v2 callbacks, request `.send(callback)`, and request +`.promise()` use. + +## DynamoDB DocumentClient + +The DynamoDB plugin instruments these `AWS.DynamoDB.DocumentClient` methods: + +- `batchGet` and `batchWrite` +- `delete`, `get`, `put`, and `update` +- `query` and `scan` +- `transactGet` and `transactWrite` + +It creates database spans with operation names such as `AWS/DynamoDB/put`. + +## Lambda invoke + +The Lambda plugin instruments `AWS.Lambda.invoke`. It does not trace requests whose +`InvocationType` is `DryRun`. + +Set `SW_AWS_LAMBDA_CHAIN=true` to add SkyWalking trace context to the invoke payload. The plugin may +change a string or buffer payload into a JSON object so it can add the context. Enable this only +when the called Lambda uses a SkyWalking Lambda wrapper and accepts this payload form. + +```bash +export SW_AWS_LAMBDA_CHAIN=true +``` + +The wrapper removes the internal context before it calls your handler. See +[Serverless](../advanced/serverless.md). + +## SNS + +The SNS plugin instruments `publish` and `publishBatch`. For a publish to `TopicArn`, it adds an +internal `__revdTraceId` message attribute. The SQS plugin can use that attribute to link an SNS to +SQS message path. + +The plugin does not add trace context when the SNS destination is only `TargetArn` or +`PhoneNumber`. + +## SQS + +The SQS plugin instruments: + +- `sendMessage` +- `sendMessageBatch` +- `receiveMessage` + +Send operations add an internal `__revdTraceId` message attribute. Receive operations ask SQS for +that attribute, use it as parent trace context, and remove the internal attribute before returning +the message when possible. + +The receive plugin removes `MaxNumberOfMessages` from the request. This makes SQS return at most one +message so the agent can link one received message to one entry span. This changes batch receive +behavior and can reduce receive throughput. Do not use this plugin when your application requires +multi-message receives. + +When SNS wraps message attributes inside the SQS body, set this option to also check the body: + +```bash +export SW_AWS_SQS_CHECK_BODY=true +``` + +## Disable one AWS plugin + +The four normal-loader disable values are: + +```text +AWS2DynamoDB +AWS2Lambda +AWS2SNS +AWS2SQS +``` + +For example: + +```bash +export SW_AGENT_DISABLE_PLUGINS='AWS2SQS' +``` + +AWS SDK plugins do not have a dedicated current CI suite in this repository. Test their behavior +with the exact AWS SDK v2 version and call form used by your service. diff --git a/docs/en/plugins/supported-libraries.md b/docs/en/plugins/supported-libraries.md new file mode 100644 index 0000000..a13bf20 --- /dev/null +++ b/docs/en/plugins/supported-libraries.md @@ -0,0 +1,62 @@ +# Supported libraries + +The agent installs a plugin only when its target module is present and its version matches the +plugin rule. Start the agent before loading these modules. + +## Automatic instrumentation + +| Library | Module | Disable value | Current plugin CI suite | +| --- | --- | --- | --- | +| Node.js HTTP and HTTPS | Built-in `http` and `https` | `http` | Yes | +| [Express](https://expressjs.com/) | `express` | `express` | Yes | +| [Axios](https://axios-http.com/) | `axios` | `axios` | Yes | +| [MySQL](https://github.com/mysqljs/mysql) | `mysql` | `mysql` | Yes | +| [MySQL2](https://github.com/sidorares/node-mysql2) | `mysql2` | `mysql2` | Yes | +| [PostgreSQL](https://node-postgres.com/) | `pg` | `pg` | Yes | +| [pg-cursor](https://github.com/brianc/node-postgres/tree/master/packages/pg-cursor) | `pg-cursor` through the `pg` plugin | `pg` | No current coverage | +| [MongoDB](https://github.com/mongodb/node-mongodb-native) | `mongodb` | `mongodb` | Yes | +| [Mongoose](https://mongoosejs.com/) | `mongoose` | `mongoose` | Yes | +| [RabbitMQ client](https://github.com/amqp-node/amqplib) | `amqplib` | `amqplib` | No dedicated current suite | +| [ioredis](https://github.com/redis/ioredis) | `ioredis` | `ioredis` | Yes | +| [AWS SDK for JavaScript v2](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/welcome.html) | `aws-sdk` 2.x | See below | No dedicated current suite | + +The normal plugin loader uses the plugin file name without its `Plugin` suffix for +`SW_AGENT_DISABLE_PLUGINS`. Values are not case-sensitive. AWS SDK v2 has four plugins, so disable +them separately with `AWS2DynamoDB`, `AWS2Lambda`, `AWS2SNS`, or `AWS2SQS`. + +```bash +export SW_AGENT_DISABLE_PLUGINS='mysql,express,AWS2SQS' +``` + +Webpack uses a separate static plugin loader and currently checks internal names such as +`MySQLPlugin`. Read [Webpack](../advanced/webpack.md) before setting plugin disable values in a +bundle. + +## Version rules + +Most current plugins declare `*` as their module version rule. The AWS plugins declare `2.*`. +`*` means that the loader accepts the installed version; it does not mean that every past or future +module version has been tested. + +Current CI runs plugin suites with the dependency versions in the repository lock file. CI also +runs general unit tests on Node.js 20, 22, and 24. Check the +[test workflow](https://github.com/apache/skywalking-nodejs/blob/master/.github/workflows/test.yaml) +and [package lock](https://github.com/apache/skywalking-nodejs/blob/master/package-lock.json) when an +exact tested version matters. + +## Libraries without a direct plugin + +Libraries built on Node.js HTTP may create spans through the HTTP plugin even when they do not have +a plugin of their own. Earlier project documentation named `request`, `request-promise`, and `koa` +as examples. The current repository does not run a separate CI suite for these packages, so treat +their behavior as indirect HTTP instrumentation. + +## What the plugins record + +- HTTP plugins record request operations, peers, status codes, and trace context. +- Database plugins record database type, instance, peer, and statement or command where available. +- Messaging plugins record broker and queue or topic information where available. +- AWS SDK v2 plugins record supported DynamoDB, Lambda, SNS, and SQS operations. + +Database parameter values are off by default. See [Configuration](../setup/configuration.md) before +enabling them. diff --git a/docs/en/setup/configuration.md b/docs/en/setup/configuration.md new file mode 100644 index 0000000..2660baa --- /dev/null +++ b/docs/en/setup/configuration.md @@ -0,0 +1,109 @@ +# Configuration + +You can configure the agent with environment variables or with options passed to `agent.start()`. +An option passed to `agent.start()` replaces the matching environment value. + +```typescript +import agent from 'skywalking-backend-js'; + +agent.start({ + serviceName: 'checkout-service', + serviceInstance: 'checkout-service-1', + collectorAddress: 'oap.example.com:11800', + secure: true, + authorization: 'token-value', +}); +``` + +Environment variables are read when the package is first loaded. Set them before starting the +Node.js process. `SW_DISABLE` is checked when `start()` is called. + +## Service and OAP connection + +| Environment variable | `agent.start()` option | Default | Description | +| --- | --- | --- | --- | +| `SW_AGENT_NAME` | `serviceName` | `your-nodejs-service` | Service name shown in SkyWalking. | +| `SW_AGENT_INSTANCE` | `serviceInstance` | Host name | Service instance name shown in SkyWalking. | +| `SW_AGENT_COLLECTOR_BACKEND_SERVICES` | `collectorAddress` | `127.0.0.1:11800` | OAP gRPC address in `host:port` form. | +| `SW_AGENT_SECURE` | `secure` | `false` | Use TLS for the OAP gRPC connection. | +| `SW_AGENT_AUTHENTICATION` | `authorization` | Not set | Authentication token sent to OAP. | +| `SW_AGENT_TRACE_TIMEOUT` | `traceTimeout` | `10000` | gRPC deadline in milliseconds for trace and meter reports and service management requests. Must be a positive integer. | + +For token authentication, set the same token in OAP with `SW_AUTHENTICATION`. See +[OAP token authentication](https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-token-auth/). + +When `secure` is enabled, the agent uses the system trust store. It does not provide options for a +custom CA, client certificate, or mutual TLS. + +Use one OAP address. If a comma-separated list is set, the current agent uses only the first entry. + +## Agent control and logging + +| Environment variable | `agent.start()` option | Default | Description | +| --- | --- | --- | --- | +| `SW_DISABLE` | None | Not set | Set the exact value `true` to keep the agent stopped. | +| `SW_AGENT_LOGGING_LEVEL` | None | `error` | Agent log level: `error`, `warn`, `info`, or `debug`. | +| `SW_LOGGING_TARGET` | None | See below | Set to `console` to log to the console in production. | +| `SW_AGENT_MAX_BUFFER_SIZE` | `maxBufferSize` | `1000` | Limit for active and buffered trace segments. Must be a positive integer. | +| `SW_AGENT_DISABLE_PLUGINS` | `disablePlugins` | Empty | Comma-separated plugin file names without the `Plugin` suffix, such as `mysql,express`. | + +Outside production mode, agent logs go to the console. When `NODE_ENV=production`, logs go to +`skywalking.log` in the working directory unless `SW_LOGGING_TARGET=console` is set. + +When the trace buffer is full, the oldest finished segment is removed. Increase the buffer only +after checking process memory and OAP availability. + +## Trace filtering + +| Environment variable | `agent.start()` option | Default | Description | +| --- | --- | --- | --- | +| `SW_IGNORE_SUFFIX` | `ignoreSuffix` | `.jpg,.jpeg,.js,.css,.png,.bmp,.gif,.ico,.mp3,.mp4,.html,.svg` | Comma-separated path suffixes that are not traced. | +| `SW_TRACE_IGNORE_PATH` | `traceIgnorePath` | Empty | Comma-separated operation-name patterns that are not traced. Supports `?`, `*`, and `**`. | +| `SW_HTTP_IGNORE_METHOD` | `httpIgnoreMethod` | Empty | Comma-separated HTTP methods that are not traced, such as `OPTIONS,HEAD`. | +| `SW_COLD_ENDPOINT` | `coldEndpoint` | `false` | Add `` to the first operation name. The first span also gets a `coldStart=true` tag. | + +Ignored trace state is sent to downstream services. This stops a filtered request from creating a +partial trace later in the call chain. See [Tracing](../features/tracing.md) for pattern examples. + +## Database parameters + +| Environment variable | `agent.start()` option | Default | Description | +| --- | --- | --- | --- | +| `SW_SQL_TRACE_PARAMETERS` | `sqlTraceParameters` | `false` | Record SQL parameter values. | +| `SW_SQL_PARAMETERS_MAX_LENGTH` | `sqlParametersMaxLength` | `512` | Maximum SQL parameter text length. | +| `SW_MONGO_TRACE_PARAMETERS` | `mongoTraceParameters` | `false` | Record MongoDB parameter values. | +| `SW_MONGO_PARAMETERS_MAX_LENGTH` | `mongoParametersMaxLength` | `512` | Maximum MongoDB parameter text length. | + +Parameter values can contain passwords, tokens, personal data, or other private data. Keep these +options off unless the data is safe to collect. + +## Runtime metrics + +| Environment variable | `agent.start()` option | Default | Description | +| --- | --- | --- | --- | +| `SW_AGENT_NODEJS_RUNTIME_METRICS_REPORTER_ACTIVE` | `runtimeMetricsReporterActive` | `true` | Enable Node.js runtime meters. Set to `false` to disable them. | +| `SW_AGENT_NODEJS_RUNTIME_METRICS_REPORT_PERIOD` | `runtimeMetricsReportPeriod` | `20000` | Sample and report period in milliseconds. Must be a positive integer. | + +Old runtime metric environment names are still accepted: + +- `SW_AGENT_RUNTIME_METRICS_REPORTER_ACTIVE` +- `SW_AGENT_NVM_METRICS_REPORTER_ACTIVE` +- `SW_AGENT_NVM_JVM_REPORTER_ACTIVE` +- `SW_AGENT_RUNTIME_METRICS_REPORT_PERIOD` +- `SW_AGENT_NVM_METRICS_REPORT_PERIOD` +- `SW_AGENT_NVM_JVM_METRICS_REPORT_PERIOD` + +The old program options `nvmMetricsReporterActive`, `nvmJvmReporterActive`, +`nvmMetricsReportPeriod`, and `nvmJvmMetricsReportPeriod` are also accepted. Use the current names +for new deployments. + +## AWS options + +| Environment variable | `agent.start()` option | Default | Description | +| --- | --- | --- | --- | +| `SW_AWS_LAMBDA_FLUSH` | `awsLambdaFlush` | `2` | Seconds between Lambda calls after which the wrapper flushes on exit. `0` means always; `-1` means never. | +| `SW_AWS_LAMBDA_CHAIN` | `awsLambdaChain` | `false` | Add trace context to an AWS Lambda invoke payload. Use only when the caller and called Lambda are instrumented. | +| `SW_AWS_SQS_CHECK_BODY` | `awsSQSCheckBody` | `false` | Also look in an incoming SQS message body for trace context passed through SNS. | + +Read [AWS SDK v2](../plugins/aws-sdk-v2.md) and [Serverless](../advanced/serverless.md) before +enabling these options. diff --git a/docs/en/setup/quick-start.md b/docs/en/setup/quick-start.md new file mode 100644 index 0000000..d1f4d4f --- /dev/null +++ b/docs/en/setup/quick-start.md @@ -0,0 +1,69 @@ +# Quick start + +This guide installs the Node.js Agent and sends traces and runtime metrics to Apache SkyWalking +OAP. + +## Requirements + +- Node.js 20 or later. +- A compatible Apache SkyWalking OAP server with its gRPC receiver available. The default gRPC + address is `127.0.0.1:11800`. + +See the +[agent and OAP compatibility guide](https://skywalking.apache.org/docs/main/next/en/setup/service-agent/agent-compatibility/) +when you choose an OAP version. + +## Install the package + +```bash +npm install skywalking-backend-js +``` + +## Start the agent in code + +The agent patches supported modules when `agent.start()` runs. Start it before loading your +framework, HTTP client, database client, or other instrumented module. + +```typescript +import agent from 'skywalking-backend-js'; + +agent.start({ + serviceName: 'checkout-service', + serviceInstance: 'checkout-service-1', + collectorAddress: '127.0.0.1:11800', +}); + +// Load the application after agent.start(). +``` + +Options passed to `agent.start()` replace environment values, including empty values. Do not pass +an empty collector address. + +## Start the agent with environment variables + +The package includes a small entry file for Node.js `--require`. This is useful when you cannot +change the first lines of the application. + +```bash +SW_AGENT_NAME=checkout-service \ +SW_AGENT_COLLECTOR_BACKEND_SERVICES=127.0.0.1:11800 \ +node --require skywalking-backend-js/lib/egg app.js +``` + +This entry calls `agent.start()` with environment values. Despite the file name, it can be used as +a general preload entry. + +## Check the result + +1. Send a request to the application. +2. Open the SkyWalking UI and select the value of `SW_AGENT_NAME`. +3. Check that a trace appears. +4. Wait at least 20 seconds and check the Node.js runtime dashboard if it is installed in OAP. + +If no data appears, read [Troubleshooting](../advanced/troubleshooting.md). + +## Next steps + +- Set production values in [Configuration](configuration.md). +- Check your modules in [Supported libraries](../plugins/supported-libraries.md). +- Read [Tracing](../features/tracing.md) to learn which requests are recorded. diff --git a/docs/en/setup/startup-and-shutdown.md b/docs/en/setup/startup-and-shutdown.md new file mode 100644 index 0000000..21f944e --- /dev/null +++ b/docs/en/setup/startup-and-shutdown.md @@ -0,0 +1,61 @@ +# Start and stop the agent + +## Start before application modules + +`agent.start()` installs module patches. Call it before loading any module that must be +instrumented. + +```javascript +const agent = require('skywalking-backend-js').default; + +agent.start(); + +const express = require('express'); +const axios = require('axios'); +``` + +Some forms of early module loading cannot be patched later. For example, a function copied from +Node.js `http` before agent startup may keep the original function. + +You can also use the preload entry: + +```bash +node --require skywalking-backend-js/lib/egg app.js +``` + +Set its configuration through environment variables. + +## Repeated calls to `start()` + +The first successful `start()` call starts the agent. Later calls are ignored, and their options +are not applied. + +Set `SW_DISABLE=true` to make `start()` return without starting the agent. The value must be the +lowercase text `true`. + +## Flush pending data + +`agent.flush()` waits for active spans to finish and asks the trace and meter reporters to send +their current data. + +```typescript +await agent.flush(); +``` + +It returns `null` when the agent is not started. A flush is useful before a short-lived process +exits. Normal long-running services do not need to call it for every request. + +## Stop the agent + +`agent.destroy()` stops the reporting services and timers. + +```typescript +await agent.flush(); +agent.destroy(); +``` + +`destroy()` does not call `flush()` for you. Flush first when pending data is important. + +Do not use repeated `destroy()` and `start()` calls as a normal restart method. Library patches +remain in the process after the reporting services stop. Start one agent for the life of a normal +service process. diff --git a/docs/menu.yml b/docs/menu.yml new file mode 100644 index 0000000..019f77e --- /dev/null +++ b/docs/menu.yml @@ -0,0 +1,56 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +catalog: + - name: "Node.js Agent overview" + path: /readme + - name: "Setup" + catalog: + - name: "Quick start" + path: /en/setup/quick-start + - name: "Configuration" + path: /en/setup/configuration + - name: "Start and stop the agent" + path: /en/setup/startup-and-shutdown + - name: "Features" + catalog: + - name: "Tracing" + path: /en/features/tracing + - name: "Node.js runtime metrics" + path: /en/features/runtime-metrics + - name: "Plugins" + catalog: + - name: "Supported libraries" + path: /en/plugins/supported-libraries + - name: "AWS SDK v2" + path: /en/plugins/aws-sdk-v2 + - name: "Advanced use" + catalog: + - name: "Serverless" + path: /en/advanced/serverless + - name: "Webpack" + path: /en/advanced/webpack + - name: "Troubleshooting" + path: /en/advanced/troubleshooting + - name: "Development and contribution" + catalog: + - name: "Build and test" + path: /en/contribution/build-and-test + - name: "Plugin development" + path: /en/contribution/plugin-development + - name: "Release" + path: /en/contribution/release + - name: "Release notes" + path: https://github.com/apache/skywalking-nodejs/releases diff --git a/scripts/release.sh b/scripts/release.sh index 54e4436..59e57b0 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -409,7 +409,7 @@ Keys to verify the Release Candidate : Guide to build the release from source : - * https://github.com/apache/skywalking-nodejs/blob/${TAG}/CONTRIBUTING.md#compiling-and-building + * docs/en/contribution/build-and-test.md (in the source package) Voting will start now (${VOTE_DATE}) and will remain open for at least 72 hours. A release passes with at least 3 binding +1 (PMC) votes and more +1 than -1. @@ -420,7 +420,7 @@ A release passes with at least 3 binding +1 (PMC) votes and more +1 than -1. Thanks. -[1] https://github.com/apache/skywalking-nodejs/blob/master/docs/How-to-release.md#vote-check +[1] docs/en/contribution/release.md, "Vote check" section (in the source package) ======================================================================== EOF