Skip to content

ref(replay): Extract worker into dedicated package#7139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mydea merged 7 commits into from
Feb 14, 2023
Merged

ref(replay): Extract worker into dedicated package #7139

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8d75149
ref: Extract worker into package
mydea Feb 9, 2023
03c26ed
ref: Use new worker package
mydea Feb 10, 2023
4f0b6e4
small fix worker
mydea Feb 10, 2023
4612c97
fix license
mydea Feb 10, 2023
ce9790d
ref: Extract `getWorkerURL` into worker package
mydea Feb 10, 2023
eb66ee9
fix replay package.json
mydea Feb 10, 2023
201fad3
skip circular dependency check for worker
mydea Feb 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions package.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -52,6 +52,7 @@
"packages/react",
"packages/remix",
"packages/replay",
"packages/replay-worker",
"packages/serverless",
"packages/svelte",
"packages/tracing",
Expand Down
1 change: 1 addition & 0 deletions packages/replay-worker/.eslintignore
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
vendor
17 changes: 17 additions & 0 deletions packages/replay-worker/.eslintrc.js
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
// Note: All paths are relative to the directory in which eslint is being run, rather than the directory where this file
// lives

// ESLint config docs: https://eslint.org/docs/user-guide/configuring/

module.exports = {
extends: ['../../.eslintrc.js'],
overrides: [
{
files: ['src/**/*.ts'],
rules: {
// We cannot use backticks, as that conflicts with the stringified worker
'prefer-template': 'off',
},
},
],
};
5 changes: 5 additions & 0 deletions packages/replay-worker/.gitignore
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
node_modules
/*.tgz
.eslintcache
build
!vendor/*.d.ts
14 changes: 14 additions & 0 deletions packages/replay-worker/LICENSE
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
Copyright (c) 2023 Sentry (https://sentry.io) and individual contributors. All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit
persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
12 changes: 12 additions & 0 deletions packages/replay-worker/README.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
<p align="center">
<a href="https://sentry.io/?utm_source=github&utm_medium=logo" target="_blank">
<img src="https://sentry-brand.storage.googleapis.com/sentry-wordmark-dark-280x84.png" alt="Sentry" width="280" height="84">
</a>
</p>

# Sentry Session Replay Worker

This is an internal package that is used by @sentry/replay.
It generates a web worker and converts it to a string, so that we can process it easier in replay.

By extracting this into a dedicated (private, internal) package, we can streamline the build of replay.
1 change: 1 addition & 0 deletions packages/replay-worker/jest.config.js
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
module.exports = require('../../jest/jest.config.js');
51 changes: 51 additions & 0 deletions packages/replay-worker/package.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
{
"name": "@sentry-internal/replay-worker",
"version": "7.37.2",
"description": "Worker for @sentry/replay",
"main": "build/index.js",
"module": "build/index.js",
"sideEffects": false,
"private": true,
"scripts": {
"build": "yarn build:transpile",
"build:transpile": "rollup -c rollup.worker.config.js",
"build:types": "yarn build:transpile",
"build:dev": "yarn build",
"build:watch": "run-p build:transpile:watch",
"build:dev:watch": "run-p build:watch",
"build:transpile:watch": "yarn build:rollup --watch",
"clean": "rimraf build",
"fix": "run-s fix:eslint fix:prettier",
"fix:eslint": "eslint . --format stylish --fix",
"fix:prettier": "prettier --write \"{src,test}/**/*.ts\"",
"lint": "run-s lint:prettier lint:eslint",
"lint:eslint": "eslint . --format stylish",
"lint:prettier": "prettier --check \"{src,test}/**/*.ts\"",
"test": "jest",
"test:watch": "jest --watch"
},
"repository": {
"type": "git",
"url": "git+https://github.com/getsentry/sentry-javascript.git"
},
"author": "Sentry",
"license": "MIT",
"bugs": {
"url": "https://github.com/getsentry/sentry-javascript/issues"
},
"homepage": "https://docs.sentry.io/platforms/javascript/session-replay/",
"devDependencies": {
"@types/pako": "^2.0.0",
"rollup-plugin-copy": "~3.4.0",
"tslib": "^1.9.3"
},
"dependencies": {
"pako": "^2.1.0"
},
"engines": {
"node": ">=12"
},
"volta": {
"extends": "../../package.json"
}
}
10 changes: 7 additions & 3 deletions packages/replay/rollup.config.worker.js → ...ges/replay-worker/rollup.worker.config.js
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,15 +4,16 @@ import resolve from '@rollup/plugin-node-resolve';
import typescript from '@rollup/plugin-typescript';
import { defineConfig } from 'rollup';
import { terser } from 'rollup-plugin-terser';
import copy from 'rollup-plugin-copy';

const config = defineConfig({
input: ['./worker/src/worker.ts'],
input: ['./src/worker.ts'],
output: {
dir: './src/worker/',
dir: './build/',
format: 'esm',
},
plugins: [
typescript({ tsconfig: './tsconfig.worker.json' }),
typescript({ tsconfig: './tsconfig.json', inlineSourceMap: false, sourceMap: false, inlineSources: false }),
resolve(),
terser({
mangle: {
Expand All@@ -25,6 +26,9 @@ const config = defineConfig({
return `export default \`${code}\`;`;
},
},
copy({
targets: [{ src: 'vendor/*', dest: 'build' }],
}),
],
});

Expand Down
24 changes: 23 additions & 1 deletion packages/replay/worker/src/Compressor.ts → packages/replay-worker/src/Compressor.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
import { constants, Deflate } from 'pako';
import { constants, Deflate, deflate } from 'pako';

/**
* A stateful compressor that can be used to batch compress events.
*/
export class Compressor {
/**
* pako deflator instance
Expand All@@ -15,10 +18,16 @@ export class Compressor {
this._init();
}

/**
* Clear the compressor buffer.
*/
public clear(): void {
this._init();
}

/**
* Add an event to the compressor buffer.
*/
public addEvent(data: string): void {
if (!data) {
throw new Error('Adding invalid event');
Expand All@@ -34,6 +43,9 @@ export class Compressor {
this._hasEvents = true;
}

/**
* Finish compression of the current buffer.
*/
public finish(): Uint8Array {
// We should always have a list, it can be empty
this.deflate.push(']', constants.Z_FINISH);
Expand All@@ -51,6 +63,9 @@ export class Compressor {
return result;
}

/**
* Re-initialize the compressor buffer.
*/
private _init(): void {
this._hasEvents = false;
this.deflate = new Deflate();
Expand All@@ -59,3 +74,10 @@ export class Compressor {
this.deflate.push('[', constants.Z_NO_FLUSH);
}
}

/**
* Compress a string.
*/
export function compress(data: string): Uint8Array {
return deflate(data);
}
13 changes: 10 additions & 3 deletions packages/replay/worker/src/handleMessage.ts → packages/replay-worker/src/handleMessage.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,18 @@
/* eslint-disable @typescript-eslint/no-unsafe-member-access */
import { Compressor } from './Compressor';
import { compress, Compressor } from './Compressor';

const compressor = new Compressor();

interface Handlers {
clear: () => void;
addEvent: (data: string) => void;
finish: () => void;
finish: () => Uint8Array;
compress: (data: string) => Uint8Array;
}

const handlers: Handlers = {
clear: () => {
compressor.clear();
return '';
},

addEvent: (data: string) => {
Expand All@@ -22,8 +22,15 @@ const handlers: Handlers = {
finish: () => {
return compressor.finish();
},

compress: (data: string) => {
return compress(data);
},
};

/**
* Handler for worker messages.
*/
export function handleMessage(e: MessageEvent): void {
const method = e.data.method as string;
const id = e.data.id as number;
Expand Down
0 packages/replay/worker/src/worker.ts → packages/replay-worker/src/worker.ts
View file
Open in desktop
File renamed without changes.
7 changes: 7 additions & 0 deletions packages/replay-worker/test/tsconfig.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
// TODO Once https://github.com/microsoft/TypeScript/issues/33094 is done (if it ever is), this file can disappear, as
// it's purely a placeholder to satisfy VSCode.
{
"extends": "../tsconfig.test.json",

"include": ["./**/*"]
}
4 changes: 2 additions & 2 deletions ...eplay/test/unit/worker/Compressor.test.ts → ...eplay-worker/test/unit/Compressor.test.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
import pako from 'pako';

import { Compressor } from '../../../worker/src/Compressor';
import { Compressor } from '../../src/Compressor';

describe('Unit | worker | Compressor', () => {
describe('Compressor', () => {
it('compresses multiple events', () => {
const compressor = new Compressor();

Expand Down
11 changes: 11 additions & 0 deletions packages/replay-worker/tsconfig.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"module": "esnext",
"lib": ["webworker", "scripthost"],
"esModuleInterop": true,
"target": "es6",
"strictPropertyInitialization": false
},
"include": ["src/**/*.ts"]
}
7 changes: 7 additions & 0 deletions packages/replay-worker/tsconfig.test.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
{
"extends": "./tsconfig.json",
"include": ["test/**/*.ts"],
"compilerOptions": {
"types": ["node", "jest"]
}
}
1 change: 1 addition & 0 deletions packages/replay-worker/vendor/index.d.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
export function getWorkerURL(): string;
6 changes: 6 additions & 0 deletions packages/replay-worker/vendor/index.js
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
import workerString from './worker';

@mydeamydeaFeb 10, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if vendor is the best name, but these are basically pre-built files.
We need this because workerString is built by rollup at build-time, so we can't reference it at dev-time. I figured this is simple enough code to just pipe this through without any build step for now.

Note that this is transparent for the replay package, it doesn't really care about how we build stuff here.

@billyvgbillyvgFeb 10, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

static? not sure if that's any better

@mydeamydeaFeb 14, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll just merge it as vendor for now and we can revisit this later, this is purely internal anyhow.


export function getWorkerURL() {
const workerBlob = new Blob([workerString]);
return URL.createObjectURL(workerBlob);
}
0 packages/replay/src/worker/worker.js.d.ts → packages/replay-worker/vendor/worker.d.ts
View file
Open in desktop
File renamed without changes.
1 change: 0 additions & 1 deletion packages/replay/.eslintrc.js
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -5,7 +5,6 @@

module.exports = {
extends: ['../../.eslintrc.js'],
ignorePatterns: ['rollup.config.worker.js'],
overrides: [
{
files: ['worker/**/*.ts'],
Expand Down
20 changes: 8 additions & 12 deletions packages/replay/package.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -7,25 +7,22 @@
"types": "build/npm/types/index.d.ts",
"sideEffects": false,
"scripts": {
"build": "run-s build:worker && run-p build:core build:types build:bundle",
"build:transpile": "run-s build:worker build:core",
"build": "run-p build:transpile build:types build:bundle",
"build:transpile": "rollup -c rollup.npm.config.js",
"build:bundle": "rollup -c rollup.bundle.config.js",
"build:dev": "run-p build:worker build:transpile build:types",
"build:worker": "rollup -c rollup.config.worker.js",
"build:core": "rollup -c rollup.npm.config.js",
"build:dev": "run-p build:transpile build:types",
"build:types": "tsc -p tsconfig.types.json",
"build:watch": "run-p build:worker:watch build:core:watch build:bundle:watch build:types:watch",
"build:dev:watch": "run-p build:core:watch build:types:watch",
"build:core:watch": "yarn build:core --watch",
"build:worker:watch": "yarn build:worker --watch",
"build:watch": "run-p build:transpile:watch build:bundle:watch build:types:watch",
"build:dev:watch": "run-p build:transpile:watch build:types:watch",
"build:transpile:watch": "yarn build:transpile --watch",
"build:bundle:watch": "yarn build:bundle --watch",
"build:types:watch": "tsc -p tsconfig.types.json --watch",
"build:tarball": "ts-node ../../scripts/prepack.ts --bundles && npm pack ./build/npm",
"circularDepCheck": "madge --circular src/index.ts",
"clean": "rimraf build sentry-replay-*.tgz",
"fix": "run-s fix:eslint fix:prettier",
"fix:eslint": "eslint . --format stylish --fix",
"fix:prettier": "prettier --write \"{src,test,scripts,worker}/**/*.ts\"",
"fix:prettier": "prettier --write \"{src,test,scripts}/**/*.ts\"",
"lint": "run-s lint:prettier lint:eslint",
"lint:eslint": "eslint . --format stylish",
"lint:prettier": "prettier --check \"{src,test,scripts,worker}/**/*.ts\"",
Expand All@@ -46,10 +43,9 @@
"homepage": "https://docs.sentry.io/platforms/javascript/session-replay/",
"devDependencies": {
"@babel/core": "^7.17.5",
"@sentry-internal/replay-worker": "7.37.2",
"@sentry-internal/rrweb": "1.103.0",
"@types/pako": "^2.0.0",
"jsdom-worker": "^0.2.1",
"pako": "^2.0.4",
"tslib": "^1.9.3"
},
"dependencies": {
Expand Down
3 changes: 1 addition & 2 deletions packages/replay/rollup.npm.config.js
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -5,8 +5,7 @@ export default makeNPMConfigVariants(
hasBundles: true,
packageSpecificConfig: {
output: {
// set exports to 'named' or 'auto' so that rollup doesn't warn about
// the default export in `worker/worker.js`
// set exports to 'named' or 'auto' so that rollup doesn't warn
exports: 'named',
// set preserveModules to false because for Replay we actually want
// to bundle everything into one file.
Expand Down
5 changes: 2 additions & 3 deletions packages/replay/src/eventBuffer/index.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
import { getWorkerURL } from '@sentry-internal/replay-worker';
import { logger } from '@sentry/utils';

import type { EventBuffer } from '../types';
import workerString from '../worker/worker.js';
import { EventBufferArray } from './EventBufferArray';
import { EventBufferProxy } from './EventBufferProxy';

Expand All@@ -16,8 +16,7 @@ export function createEventBuffer({ useCompression }: CreateEventBufferParams):
// eslint-disable-next-line no-restricted-globals
if (useCompression && window.Worker) {
try {
const workerBlob = new Blob([workerString]);
const workerUrl = URL.createObjectURL(workerBlob);
const workerUrl = getWorkerURL();

__DEBUG_BUILD__ && logger.log('[Replay] Using compression worker');
const worker = new Worker(workerUrl);
Expand Down
2 changes: 0 additions & 2 deletions packages/replay/src/worker/worker.js
View file
Open in desktop

This file was deleted.

19 changes: 0 additions & 19 deletions packages/replay/tsconfig.worker.json
View file
Open in desktop

This file was deleted.

Loading