Skip to content

Add shared monorepo build setup - #38718

Closed
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330
Closed

Add shared monorepo build setup#38718
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330

Conversation

@huntie

@huntiehuntie commented Aug 1, 2023

Copy link
Copy Markdown
Collaborator

Summary:

NOTE: Replaces #38240

Context

RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641

Changes

To support incoming new React Native packages around debugging (including migrating over @react-native-community/cli-plugin-metro) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the react-native monorepo.

The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:

  • yarn build — Builds all configured packages. Functionally, this:
    • Outputs a dist/ directory with built files.
    • Rewrites package.json "exports" to update every ./src/* reference to ./dist/* (source of truth).
  • scripts/build/babel-register.js — Allows running all Node.js entry points from source, similar to the current setup in facebook/metro. (Example entry point file in this PR: packages/dev-middleware/src/index.js)

Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and packages are opted-in to requiring a build, configured in scripts/build.config.js.

constbuildConfig/*: BuildConfig */={// The packages to include for build and their build optionspackages: {'dev-middleware': {target: 'node'},},};

For now, there is a single target: 'node' option — this is necessary as react-native, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.

Differences from the Metro setup

  • References (and compiles out) repo-local scripts/build/babel-register.js — removing need for an npm-published dependency.

Current integration points

  • CircleCIyarn build is added to the build_npm_package and find_and_publish_bumped_packages jobs.

New Node.js package(s) are not load bearing quite yet: There are not yet any built packages added to the dependencies of packages/react-native/, so this will be further tested in a later PR (and is actively being done in an internal commit stack).

Alternative designs

Per-package config file

Replace scripts/build/config.js with a package-defined key in in package.json, similar to Jest's publishConfig.

"buildConfig": {
"type": "node"
},

This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is enableTypeScriptCodgeen.

Rollup

More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).

Lerna and Nx

  • Most sophisticated setup enabling caching and optimised cloud runs.
  • Probably the most likely thing we'll move towards at a later stage.

Changelog: [Internal]

Differential Revision: D47760330

@facebook-github-botfacebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner fb-exported labels Aug 1, 2023
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

github-actionsBot commented Aug 1, 2023

Copy link
Copy Markdown
Warnings
⚠️🔒 package.json - Changes were made to package.json. This will require a manual import by a Facebook employee.

Generated by 🚫 dangerJS against d9f3a0e

huntie added a commit to huntie/react-native-website that referenced this pull request Aug 1, 2023
Removes mandatory run of `yarn build/watch` when developing Node.js packages — matching the updated build setup in react/react-native#38718 (these packages are now set up to run from source during development).
Partially reverts react#3788 "[docs] Add guidance on repo build scripts"
@analysis-bot

analysis-bot commented Aug 1, 2023

Copy link
Copy Markdown
PlatformEngineArchSize (bytes)Diff
androidhermesarm64-v8a8,894,554-1
androidhermesarmeabi-v7a7,943,106-4
androidhermesx869,292,438-2
androidhermesx86_649,193,963-3
androidjscarm64-v8a9,481,107-1
androidjscarmeabi-v7a8,422,649-1
androidjscx869,465,124-1
androidjscx86_649,779,363-3

Base commit: 0fb7163
Branch: main

@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

Summary:
Pull Request resolved: #38718
> NOTE: Replaces #38240
## Context
RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641
## Changes
To support incoming new React Native packages around debugging (including migrating over [`react-native-community/cli-plugin-metro`](https://github.com/react-native-community/cli/tree/main/packages/cli-plugin-metro)) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the `react-native` monorepo.
The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:
- `yarn build` — Builds all configured packages. Functionally, this:
- Outputs a `dist/` directory with built files.
- Rewrites package.json `"exports"` to update every `./src/*` reference to `./dist/*` (source of truth).
- `scripts/build/babel-register.js` — Allows running all Node.js entry points from source, similar to the current setup in [facebook/metro](https://github.com/facebook/metro). (Example entry point file in this PR: `packages/dev-middleware/src/index.js`)
Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and **packages are opted-in to requiring a build**, configured in `scripts/build.config.js`.
```
const buildConfig /*: BuildConfig */ = {
// The packages to include for build and their build options
packages: {
'dev-middleware': {target: 'node'},
},
};
```
For now, there is a single `target: 'node'` option — this is necessary as `react-native`, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.
{F1043312771}
**Differences from the Metro setup**
- References (and compiles out) repo-local `scripts/build/babel-register.js` — removing need for an npm-published dependency.
## Current integration points
- **CircleCI** — `yarn build` is added to the `build_npm_package` and `find_and_publish_bumped_packages` jobs.
**New Node.js package(s) are not load bearing quite yet**: There are not yet any built packages added to the dependencies of `packages/react-native/`, so this will be further tested in a later PR (and is actively being done in an internal commit stack).
### Alternative designs
**Per-package config file**
Replace `scripts/build/config.js` with a package-defined key in in `package.json`, similar to Jest's [`publishConfig`](https://github.com/jestjs/jest/blob/1f019afdcdfc54a6664908bb45f343db4e3d0848/packages/jest-cli/package.json#L87C3-L89C4).
```
"buildConfig": {
"type": "node"
},
```
This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is `enableTypeScriptCodgeen`.
**Rollup**
More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).
**Lerna and Nx**
- Most sophisticated setup enabling caching and optimised cloud runs.
- Probably the most likely thing we'll move towards at a later stage.
Changelog: [Internal]
Reviewed By: NickGerleman
Differential Revision: D47760330
fbshipit-source-id: 456b506f9e50a43bae9bc902f36910253a0681e0
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

Copy link
Copy Markdown

This pull request was successfully merged by @huntie in cd8f5d1.

When will my fix make it into a release? | Upcoming Releases

@github-actionsgithub-actionsBot added the Merged This PR has been merged. label Aug 3, 2023
facebook-github-bot pushed a commit that referenced this pull request Aug 14, 2023
Summary:
Pull Request resolved: #38990
This PR adds auto-generation of Typescript definitions from Flow source code for packages using the shared monorepo build setup (#38718).
Today, these are the following Node.js packages:
- `packages/community-cli-plugin`
- `packages/dev-middleware` (⬅️ `emitTypeScriptDefs` enabled)
This also improves emitted Flow definitions (`.js.flow`), by using [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator) to strip implementations.
**All changes**
- Include `flow-api-translator` and configure this to emit type definitions as part of `yarn build`.
- Add translation from Flow source to TypeScript definitions (`.d.ts`) adjacent to each built file.
- Improve emitted Flow definitions (`.js.flow`), by using `flow-api-translator` to strip implementations (previously, source files were copied). The Flow and TS defs now mirror each other.
- Add `emitFlowDefs` and `emitTypeScriptDefs` options to build config to configure the above.
- Integrate TypeScript compiler to perform program validation on emitted `.d.ts` files.
- This is based on this guide: https://github.com/microsoft/TypeScript-wiki/blob/main/Using-the-Compiler-API.md#a-minimal-compiler.
- Throw an exception on the `rewritePackageExports` step if a package does not define an `"exports"` field.
- Add minimal `flow-typed` definitions for `typescript` 😄.
**Notes on [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator)**
This project is experimental but is in a more mature state than when we evaluated it earlier in 2023.
- It's now possible to run this tool on our new Node.js packages, since they are exclusively authored using `import`/`export` syntax (a requirement of the tool).
- As a safety net, we run the TypeScript compiler against the generated program, which will fail the build.
Changelog: [Internal]
Reviewed By: robhogan
Differential Revision: D48312463
fbshipit-source-id: 817edb35f911f52fa987946f2d8fc1a319078c9d
@huntie
huntie deleted the export-D47760330 branch August 29, 2023 16:15
facebook-github-bot pushed a commit that referenced this pull request Sep 21, 2023
Summary:
Pull Request resolved: #39566
Fixes running the `build.sh` script for `react-native-codegen` when on [EdenFS](https://github.com/facebook/sapling). This would previously fail due a filesystem "unable to copy extended attributes" error.
This only affects development within the React Native monorepo. Practically, this equates to a workflow improvement for Meta engineers when creating `RNTester` builds.
NOTE: **This is temporary**. The change from `mv` to `cp -R` makes this script more expensive, which will make Android builds via Gradle take slightly longer. I have a WIP cleanup planned which will 1/ move `react-native-codegen` to the [shared monorepo build setup](#38718), and 2/ drop this step entirely from the Android Gradle build.
Changelog: [Internal]
Reviewed By: christophpurrer
Differential Revision: D49468891
fbshipit-source-id: 25d5db81798cf8ab150a135174a45f4d4c2cb5a2
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedMergedThis PR has been merged.p: FacebookPartner: FacebookPartner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@huntie@facebook-github-bot@analysis-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Add shared monorepo build setup by huntie · Pull Request #38718 · react/react-native · GitHub
Skip to content

Add shared monorepo build setup - #38718

Closed
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330
Closed

Add shared monorepo build setup#38718
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330

Conversation

@huntie

@huntiehuntie commented Aug 1, 2023

Copy link
Copy Markdown
Collaborator

Summary:

NOTE: Replaces #38240

Context

RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641

Changes

To support incoming new React Native packages around debugging (including migrating over @react-native-community/cli-plugin-metro) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the react-native monorepo.

The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:

  • yarn build — Builds all configured packages. Functionally, this:
    • Outputs a dist/ directory with built files.
    • Rewrites package.json "exports" to update every ./src/* reference to ./dist/* (source of truth).
  • scripts/build/babel-register.js — Allows running all Node.js entry points from source, similar to the current setup in facebook/metro. (Example entry point file in this PR: packages/dev-middleware/src/index.js)

Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and packages are opted-in to requiring a build, configured in scripts/build.config.js.

constbuildConfig/*: BuildConfig */={// The packages to include for build and their build optionspackages: {'dev-middleware': {target: 'node'},},};

For now, there is a single target: 'node' option — this is necessary as react-native, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.

Differences from the Metro setup

  • References (and compiles out) repo-local scripts/build/babel-register.js — removing need for an npm-published dependency.

Current integration points

  • CircleCIyarn build is added to the build_npm_package and find_and_publish_bumped_packages jobs.

New Node.js package(s) are not load bearing quite yet: There are not yet any built packages added to the dependencies of packages/react-native/, so this will be further tested in a later PR (and is actively being done in an internal commit stack).

Alternative designs

Per-package config file

Replace scripts/build/config.js with a package-defined key in in package.json, similar to Jest's publishConfig.

"buildConfig": {
"type": "node"
},

This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is enableTypeScriptCodgeen.

Rollup

More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).

Lerna and Nx

  • Most sophisticated setup enabling caching and optimised cloud runs.
  • Probably the most likely thing we'll move towards at a later stage.

Changelog: [Internal]

Differential Revision: D47760330

@facebook-github-botfacebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner fb-exported labels Aug 1, 2023
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

github-actionsBot commented Aug 1, 2023

Copy link
Copy Markdown
Warnings
⚠️🔒 package.json - Changes were made to package.json. This will require a manual import by a Facebook employee.

Generated by 🚫 dangerJS against d9f3a0e

huntie added a commit to huntie/react-native-website that referenced this pull request Aug 1, 2023
Removes mandatory run of `yarn build/watch` when developing Node.js packages — matching the updated build setup in react/react-native#38718 (these packages are now set up to run from source during development).
Partially reverts react#3788 "[docs] Add guidance on repo build scripts"
@analysis-bot

analysis-bot commented Aug 1, 2023

Copy link
Copy Markdown
PlatformEngineArchSize (bytes)Diff
androidhermesarm64-v8a8,894,554-1
androidhermesarmeabi-v7a7,943,106-4
androidhermesx869,292,438-2
androidhermesx86_649,193,963-3
androidjscarm64-v8a9,481,107-1
androidjscarmeabi-v7a8,422,649-1
androidjscx869,465,124-1
androidjscx86_649,779,363-3

Base commit: 0fb7163
Branch: main

@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

Summary:
Pull Request resolved: #38718
> NOTE: Replaces #38240
## Context
RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641
## Changes
To support incoming new React Native packages around debugging (including migrating over [`react-native-community/cli-plugin-metro`](https://github.com/react-native-community/cli/tree/main/packages/cli-plugin-metro)) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the `react-native` monorepo.
The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:
- `yarn build` — Builds all configured packages. Functionally, this:
- Outputs a `dist/` directory with built files.
- Rewrites package.json `"exports"` to update every `./src/*` reference to `./dist/*` (source of truth).
- `scripts/build/babel-register.js` — Allows running all Node.js entry points from source, similar to the current setup in [facebook/metro](https://github.com/facebook/metro). (Example entry point file in this PR: `packages/dev-middleware/src/index.js`)
Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and **packages are opted-in to requiring a build**, configured in `scripts/build.config.js`.
```
const buildConfig /*: BuildConfig */ = {
// The packages to include for build and their build options
packages: {
'dev-middleware': {target: 'node'},
},
};
```
For now, there is a single `target: 'node'` option — this is necessary as `react-native`, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.
{F1043312771}
**Differences from the Metro setup**
- References (and compiles out) repo-local `scripts/build/babel-register.js` — removing need for an npm-published dependency.
## Current integration points
- **CircleCI** — `yarn build` is added to the `build_npm_package` and `find_and_publish_bumped_packages` jobs.
**New Node.js package(s) are not load bearing quite yet**: There are not yet any built packages added to the dependencies of `packages/react-native/`, so this will be further tested in a later PR (and is actively being done in an internal commit stack).
### Alternative designs
**Per-package config file**
Replace `scripts/build/config.js` with a package-defined key in in `package.json`, similar to Jest's [`publishConfig`](https://github.com/jestjs/jest/blob/1f019afdcdfc54a6664908bb45f343db4e3d0848/packages/jest-cli/package.json#L87C3-L89C4).
```
"buildConfig": {
"type": "node"
},
```
This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is `enableTypeScriptCodgeen`.
**Rollup**
More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).
**Lerna and Nx**
- Most sophisticated setup enabling caching and optimised cloud runs.
- Probably the most likely thing we'll move towards at a later stage.
Changelog: [Internal]
Reviewed By: NickGerleman
Differential Revision: D47760330
fbshipit-source-id: 456b506f9e50a43bae9bc902f36910253a0681e0
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

Copy link
Copy Markdown

This pull request was successfully merged by @huntie in cd8f5d1.

When will my fix make it into a release? | Upcoming Releases

@github-actionsgithub-actionsBot added the Merged This PR has been merged. label Aug 3, 2023
facebook-github-bot pushed a commit that referenced this pull request Aug 14, 2023
Summary:
Pull Request resolved: #38990
This PR adds auto-generation of Typescript definitions from Flow source code for packages using the shared monorepo build setup (#38718).
Today, these are the following Node.js packages:
- `packages/community-cli-plugin`
- `packages/dev-middleware` (⬅️ `emitTypeScriptDefs` enabled)
This also improves emitted Flow definitions (`.js.flow`), by using [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator) to strip implementations.
**All changes**
- Include `flow-api-translator` and configure this to emit type definitions as part of `yarn build`.
- Add translation from Flow source to TypeScript definitions (`.d.ts`) adjacent to each built file.
- Improve emitted Flow definitions (`.js.flow`), by using `flow-api-translator` to strip implementations (previously, source files were copied). The Flow and TS defs now mirror each other.
- Add `emitFlowDefs` and `emitTypeScriptDefs` options to build config to configure the above.
- Integrate TypeScript compiler to perform program validation on emitted `.d.ts` files.
- This is based on this guide: https://github.com/microsoft/TypeScript-wiki/blob/main/Using-the-Compiler-API.md#a-minimal-compiler.
- Throw an exception on the `rewritePackageExports` step if a package does not define an `"exports"` field.
- Add minimal `flow-typed` definitions for `typescript` 😄.
**Notes on [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator)**
This project is experimental but is in a more mature state than when we evaluated it earlier in 2023.
- It's now possible to run this tool on our new Node.js packages, since they are exclusively authored using `import`/`export` syntax (a requirement of the tool).
- As a safety net, we run the TypeScript compiler against the generated program, which will fail the build.
Changelog: [Internal]
Reviewed By: robhogan
Differential Revision: D48312463
fbshipit-source-id: 817edb35f911f52fa987946f2d8fc1a319078c9d
@huntie
huntie deleted the export-D47760330 branch August 29, 2023 16:15
facebook-github-bot pushed a commit that referenced this pull request Sep 21, 2023
Summary:
Pull Request resolved: #39566
Fixes running the `build.sh` script for `react-native-codegen` when on [EdenFS](https://github.com/facebook/sapling). This would previously fail due a filesystem "unable to copy extended attributes" error.
This only affects development within the React Native monorepo. Practically, this equates to a workflow improvement for Meta engineers when creating `RNTester` builds.
NOTE: **This is temporary**. The change from `mv` to `cp -R` makes this script more expensive, which will make Android builds via Gradle take slightly longer. I have a WIP cleanup planned which will 1/ move `react-native-codegen` to the [shared monorepo build setup](#38718), and 2/ drop this step entirely from the Android Gradle build.
Changelog: [Internal]
Reviewed By: christophpurrer
Differential Revision: D49468891
fbshipit-source-id: 25d5db81798cf8ab150a135174a45f4d4c2cb5a2
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedMergedThis PR has been merged.p: FacebookPartner: FacebookPartner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@huntie@facebook-github-bot@analysis-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Add shared monorepo build setup by huntie · Pull Request #38718 · react/react-native · GitHub
Skip to content

Add shared monorepo build setup - #38718

Closed
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330
Closed

Add shared monorepo build setup#38718
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330

Conversation

@huntie

@huntiehuntie commented Aug 1, 2023

Copy link
Copy Markdown
Collaborator

Summary:

NOTE: Replaces #38240

Context

RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641

Changes

To support incoming new React Native packages around debugging (including migrating over @react-native-community/cli-plugin-metro) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the react-native monorepo.

The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:

  • yarn build — Builds all configured packages. Functionally, this:
    • Outputs a dist/ directory with built files.
    • Rewrites package.json "exports" to update every ./src/* reference to ./dist/* (source of truth).
  • scripts/build/babel-register.js — Allows running all Node.js entry points from source, similar to the current setup in facebook/metro. (Example entry point file in this PR: packages/dev-middleware/src/index.js)

Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and packages are opted-in to requiring a build, configured in scripts/build.config.js.

constbuildConfig/*: BuildConfig */={// The packages to include for build and their build optionspackages: {'dev-middleware': {target: 'node'},},};

For now, there is a single target: 'node' option — this is necessary as react-native, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.

Differences from the Metro setup

  • References (and compiles out) repo-local scripts/build/babel-register.js — removing need for an npm-published dependency.

Current integration points

  • CircleCIyarn build is added to the build_npm_package and find_and_publish_bumped_packages jobs.

New Node.js package(s) are not load bearing quite yet: There are not yet any built packages added to the dependencies of packages/react-native/, so this will be further tested in a later PR (and is actively being done in an internal commit stack).

Alternative designs

Per-package config file

Replace scripts/build/config.js with a package-defined key in in package.json, similar to Jest's publishConfig.

"buildConfig": {
"type": "node"
},

This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is enableTypeScriptCodgeen.

Rollup

More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).

Lerna and Nx

  • Most sophisticated setup enabling caching and optimised cloud runs.
  • Probably the most likely thing we'll move towards at a later stage.

Changelog: [Internal]

Differential Revision: D47760330

@facebook-github-botfacebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner fb-exported labels Aug 1, 2023
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

github-actionsBot commented Aug 1, 2023

Copy link
Copy Markdown
Warnings
⚠️🔒 package.json - Changes were made to package.json. This will require a manual import by a Facebook employee.

Generated by 🚫 dangerJS against d9f3a0e

huntie added a commit to huntie/react-native-website that referenced this pull request Aug 1, 2023
Removes mandatory run of `yarn build/watch` when developing Node.js packages — matching the updated build setup in react/react-native#38718 (these packages are now set up to run from source during development).
Partially reverts react#3788 "[docs] Add guidance on repo build scripts"
@analysis-bot

analysis-bot commented Aug 1, 2023

Copy link
Copy Markdown
PlatformEngineArchSize (bytes)Diff
androidhermesarm64-v8a8,894,554-1
androidhermesarmeabi-v7a7,943,106-4
androidhermesx869,292,438-2
androidhermesx86_649,193,963-3
androidjscarm64-v8a9,481,107-1
androidjscarmeabi-v7a8,422,649-1
androidjscx869,465,124-1
androidjscx86_649,779,363-3

Base commit: 0fb7163
Branch: main

@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

Summary:
Pull Request resolved: #38718
> NOTE: Replaces #38240
## Context
RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641
## Changes
To support incoming new React Native packages around debugging (including migrating over [`react-native-community/cli-plugin-metro`](https://github.com/react-native-community/cli/tree/main/packages/cli-plugin-metro)) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the `react-native` monorepo.
The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:
- `yarn build` — Builds all configured packages. Functionally, this:
- Outputs a `dist/` directory with built files.
- Rewrites package.json `"exports"` to update every `./src/*` reference to `./dist/*` (source of truth).
- `scripts/build/babel-register.js` — Allows running all Node.js entry points from source, similar to the current setup in [facebook/metro](https://github.com/facebook/metro). (Example entry point file in this PR: `packages/dev-middleware/src/index.js`)
Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and **packages are opted-in to requiring a build**, configured in `scripts/build.config.js`.
```
const buildConfig /*: BuildConfig */ = {
// The packages to include for build and their build options
packages: {
'dev-middleware': {target: 'node'},
},
};
```
For now, there is a single `target: 'node'` option — this is necessary as `react-native`, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.
{F1043312771}
**Differences from the Metro setup**
- References (and compiles out) repo-local `scripts/build/babel-register.js` — removing need for an npm-published dependency.
## Current integration points
- **CircleCI** — `yarn build` is added to the `build_npm_package` and `find_and_publish_bumped_packages` jobs.
**New Node.js package(s) are not load bearing quite yet**: There are not yet any built packages added to the dependencies of `packages/react-native/`, so this will be further tested in a later PR (and is actively being done in an internal commit stack).
### Alternative designs
**Per-package config file**
Replace `scripts/build/config.js` with a package-defined key in in `package.json`, similar to Jest's [`publishConfig`](https://github.com/jestjs/jest/blob/1f019afdcdfc54a6664908bb45f343db4e3d0848/packages/jest-cli/package.json#L87C3-L89C4).
```
"buildConfig": {
"type": "node"
},
```
This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is `enableTypeScriptCodgeen`.
**Rollup**
More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).
**Lerna and Nx**
- Most sophisticated setup enabling caching and optimised cloud runs.
- Probably the most likely thing we'll move towards at a later stage.
Changelog: [Internal]
Reviewed By: NickGerleman
Differential Revision: D47760330
fbshipit-source-id: 456b506f9e50a43bae9bc902f36910253a0681e0
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

Copy link
Copy Markdown

This pull request was successfully merged by @huntie in cd8f5d1.

When will my fix make it into a release? | Upcoming Releases

@github-actionsgithub-actionsBot added the Merged This PR has been merged. label Aug 3, 2023
facebook-github-bot pushed a commit that referenced this pull request Aug 14, 2023
Summary:
Pull Request resolved: #38990
This PR adds auto-generation of Typescript definitions from Flow source code for packages using the shared monorepo build setup (#38718).
Today, these are the following Node.js packages:
- `packages/community-cli-plugin`
- `packages/dev-middleware` (⬅️ `emitTypeScriptDefs` enabled)
This also improves emitted Flow definitions (`.js.flow`), by using [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator) to strip implementations.
**All changes**
- Include `flow-api-translator` and configure this to emit type definitions as part of `yarn build`.
- Add translation from Flow source to TypeScript definitions (`.d.ts`) adjacent to each built file.
- Improve emitted Flow definitions (`.js.flow`), by using `flow-api-translator` to strip implementations (previously, source files were copied). The Flow and TS defs now mirror each other.
- Add `emitFlowDefs` and `emitTypeScriptDefs` options to build config to configure the above.
- Integrate TypeScript compiler to perform program validation on emitted `.d.ts` files.
- This is based on this guide: https://github.com/microsoft/TypeScript-wiki/blob/main/Using-the-Compiler-API.md#a-minimal-compiler.
- Throw an exception on the `rewritePackageExports` step if a package does not define an `"exports"` field.
- Add minimal `flow-typed` definitions for `typescript` 😄.
**Notes on [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator)**
This project is experimental but is in a more mature state than when we evaluated it earlier in 2023.
- It's now possible to run this tool on our new Node.js packages, since they are exclusively authored using `import`/`export` syntax (a requirement of the tool).
- As a safety net, we run the TypeScript compiler against the generated program, which will fail the build.
Changelog: [Internal]
Reviewed By: robhogan
Differential Revision: D48312463
fbshipit-source-id: 817edb35f911f52fa987946f2d8fc1a319078c9d
@huntie
huntie deleted the export-D47760330 branch August 29, 2023 16:15
facebook-github-bot pushed a commit that referenced this pull request Sep 21, 2023
Summary:
Pull Request resolved: #39566
Fixes running the `build.sh` script for `react-native-codegen` when on [EdenFS](https://github.com/facebook/sapling). This would previously fail due a filesystem "unable to copy extended attributes" error.
This only affects development within the React Native monorepo. Practically, this equates to a workflow improvement for Meta engineers when creating `RNTester` builds.
NOTE: **This is temporary**. The change from `mv` to `cp -R` makes this script more expensive, which will make Android builds via Gradle take slightly longer. I have a WIP cleanup planned which will 1/ move `react-native-codegen` to the [shared monorepo build setup](#38718), and 2/ drop this step entirely from the Android Gradle build.
Changelog: [Internal]
Reviewed By: christophpurrer
Differential Revision: D49468891
fbshipit-source-id: 25d5db81798cf8ab150a135174a45f4d4c2cb5a2
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedMergedThis PR has been merged.p: FacebookPartner: FacebookPartner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@huntie@facebook-github-bot@analysis-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Add shared monorepo build setup by huntie · Pull Request #38718 · react/react-native · GitHub
Skip to content

Add shared monorepo build setup - #38718

Closed
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330
Closed

Add shared monorepo build setup#38718
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330

Conversation

@huntie

@huntiehuntie commented Aug 1, 2023

Copy link
Copy Markdown
Collaborator

Summary:

NOTE: Replaces #38240

Context

RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641

Changes

To support incoming new React Native packages around debugging (including migrating over @react-native-community/cli-plugin-metro) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the react-native monorepo.

The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:

  • yarn build — Builds all configured packages. Functionally, this:
    • Outputs a dist/ directory with built files.
    • Rewrites package.json "exports" to update every ./src/* reference to ./dist/* (source of truth).
  • scripts/build/babel-register.js — Allows running all Node.js entry points from source, similar to the current setup in facebook/metro. (Example entry point file in this PR: packages/dev-middleware/src/index.js)

Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and packages are opted-in to requiring a build, configured in scripts/build.config.js.

constbuildConfig/*: BuildConfig */={// The packages to include for build and their build optionspackages: {'dev-middleware': {target: 'node'},},};

For now, there is a single target: 'node' option — this is necessary as react-native, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.

Differences from the Metro setup

  • References (and compiles out) repo-local scripts/build/babel-register.js — removing need for an npm-published dependency.

Current integration points

  • CircleCIyarn build is added to the build_npm_package and find_and_publish_bumped_packages jobs.

New Node.js package(s) are not load bearing quite yet: There are not yet any built packages added to the dependencies of packages/react-native/, so this will be further tested in a later PR (and is actively being done in an internal commit stack).

Alternative designs

Per-package config file

Replace scripts/build/config.js with a package-defined key in in package.json, similar to Jest's publishConfig.

"buildConfig": {
"type": "node"
},

This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is enableTypeScriptCodgeen.

Rollup

More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).

Lerna and Nx

  • Most sophisticated setup enabling caching and optimised cloud runs.
  • Probably the most likely thing we'll move towards at a later stage.

Changelog: [Internal]

Differential Revision: D47760330

@facebook-github-botfacebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner fb-exported labels Aug 1, 2023
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

github-actionsBot commented Aug 1, 2023

Copy link
Copy Markdown
Warnings
⚠️🔒 package.json - Changes were made to package.json. This will require a manual import by a Facebook employee.

Generated by 🚫 dangerJS against d9f3a0e

huntie added a commit to huntie/react-native-website that referenced this pull request Aug 1, 2023
Removes mandatory run of `yarn build/watch` when developing Node.js packages — matching the updated build setup in react/react-native#38718 (these packages are now set up to run from source during development).
Partially reverts react#3788 "[docs] Add guidance on repo build scripts"
@analysis-bot

analysis-bot commented Aug 1, 2023

Copy link
Copy Markdown
PlatformEngineArchSize (bytes)Diff
androidhermesarm64-v8a8,894,554-1
androidhermesarmeabi-v7a7,943,106-4
androidhermesx869,292,438-2
androidhermesx86_649,193,963-3
androidjscarm64-v8a9,481,107-1
androidjscarmeabi-v7a8,422,649-1
androidjscx869,465,124-1
androidjscx86_649,779,363-3

Base commit: 0fb7163
Branch: main

@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

Summary:
Pull Request resolved: #38718
> NOTE: Replaces #38240
## Context
RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641
## Changes
To support incoming new React Native packages around debugging (including migrating over [`react-native-community/cli-plugin-metro`](https://github.com/react-native-community/cli/tree/main/packages/cli-plugin-metro)) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the `react-native` monorepo.
The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:
- `yarn build` — Builds all configured packages. Functionally, this:
- Outputs a `dist/` directory with built files.
- Rewrites package.json `"exports"` to update every `./src/*` reference to `./dist/*` (source of truth).
- `scripts/build/babel-register.js` — Allows running all Node.js entry points from source, similar to the current setup in [facebook/metro](https://github.com/facebook/metro). (Example entry point file in this PR: `packages/dev-middleware/src/index.js`)
Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and **packages are opted-in to requiring a build**, configured in `scripts/build.config.js`.
```
const buildConfig /*: BuildConfig */ = {
// The packages to include for build and their build options
packages: {
'dev-middleware': {target: 'node'},
},
};
```
For now, there is a single `target: 'node'` option — this is necessary as `react-native`, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.
{F1043312771}
**Differences from the Metro setup**
- References (and compiles out) repo-local `scripts/build/babel-register.js` — removing need for an npm-published dependency.
## Current integration points
- **CircleCI** — `yarn build` is added to the `build_npm_package` and `find_and_publish_bumped_packages` jobs.
**New Node.js package(s) are not load bearing quite yet**: There are not yet any built packages added to the dependencies of `packages/react-native/`, so this will be further tested in a later PR (and is actively being done in an internal commit stack).
### Alternative designs
**Per-package config file**
Replace `scripts/build/config.js` with a package-defined key in in `package.json`, similar to Jest's [`publishConfig`](https://github.com/jestjs/jest/blob/1f019afdcdfc54a6664908bb45f343db4e3d0848/packages/jest-cli/package.json#L87C3-L89C4).
```
"buildConfig": {
"type": "node"
},
```
This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is `enableTypeScriptCodgeen`.
**Rollup**
More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).
**Lerna and Nx**
- Most sophisticated setup enabling caching and optimised cloud runs.
- Probably the most likely thing we'll move towards at a later stage.
Changelog: [Internal]
Reviewed By: NickGerleman
Differential Revision: D47760330
fbshipit-source-id: 456b506f9e50a43bae9bc902f36910253a0681e0
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

Copy link
Copy Markdown

This pull request was successfully merged by @huntie in cd8f5d1.

When will my fix make it into a release? | Upcoming Releases

@github-actionsgithub-actionsBot added the Merged This PR has been merged. label Aug 3, 2023
facebook-github-bot pushed a commit that referenced this pull request Aug 14, 2023
Summary:
Pull Request resolved: #38990
This PR adds auto-generation of Typescript definitions from Flow source code for packages using the shared monorepo build setup (#38718).
Today, these are the following Node.js packages:
- `packages/community-cli-plugin`
- `packages/dev-middleware` (⬅️ `emitTypeScriptDefs` enabled)
This also improves emitted Flow definitions (`.js.flow`), by using [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator) to strip implementations.
**All changes**
- Include `flow-api-translator` and configure this to emit type definitions as part of `yarn build`.
- Add translation from Flow source to TypeScript definitions (`.d.ts`) adjacent to each built file.
- Improve emitted Flow definitions (`.js.flow`), by using `flow-api-translator` to strip implementations (previously, source files were copied). The Flow and TS defs now mirror each other.
- Add `emitFlowDefs` and `emitTypeScriptDefs` options to build config to configure the above.
- Integrate TypeScript compiler to perform program validation on emitted `.d.ts` files.
- This is based on this guide: https://github.com/microsoft/TypeScript-wiki/blob/main/Using-the-Compiler-API.md#a-minimal-compiler.
- Throw an exception on the `rewritePackageExports` step if a package does not define an `"exports"` field.
- Add minimal `flow-typed` definitions for `typescript` 😄.
**Notes on [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator)**
This project is experimental but is in a more mature state than when we evaluated it earlier in 2023.
- It's now possible to run this tool on our new Node.js packages, since they are exclusively authored using `import`/`export` syntax (a requirement of the tool).
- As a safety net, we run the TypeScript compiler against the generated program, which will fail the build.
Changelog: [Internal]
Reviewed By: robhogan
Differential Revision: D48312463
fbshipit-source-id: 817edb35f911f52fa987946f2d8fc1a319078c9d
@huntie
huntie deleted the export-D47760330 branch August 29, 2023 16:15
facebook-github-bot pushed a commit that referenced this pull request Sep 21, 2023
Summary:
Pull Request resolved: #39566
Fixes running the `build.sh` script for `react-native-codegen` when on [EdenFS](https://github.com/facebook/sapling). This would previously fail due a filesystem "unable to copy extended attributes" error.
This only affects development within the React Native monorepo. Practically, this equates to a workflow improvement for Meta engineers when creating `RNTester` builds.
NOTE: **This is temporary**. The change from `mv` to `cp -R` makes this script more expensive, which will make Android builds via Gradle take slightly longer. I have a WIP cleanup planned which will 1/ move `react-native-codegen` to the [shared monorepo build setup](#38718), and 2/ drop this step entirely from the Android Gradle build.
Changelog: [Internal]
Reviewed By: christophpurrer
Differential Revision: D49468891
fbshipit-source-id: 25d5db81798cf8ab150a135174a45f4d4c2cb5a2
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedMergedThis PR has been merged.p: FacebookPartner: FacebookPartner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@huntie@facebook-github-bot@analysis-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Add shared monorepo build setup by huntie · Pull Request #38718 · react/react-native · GitHub
Skip to content

Add shared monorepo build setup - #38718

Closed
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330
Closed

Add shared monorepo build setup#38718
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330

Conversation

@huntie

@huntiehuntie commented Aug 1, 2023

Copy link
Copy Markdown
Collaborator

Summary:

NOTE: Replaces #38240

Context

RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641

Changes

To support incoming new React Native packages around debugging (including migrating over @react-native-community/cli-plugin-metro) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the react-native monorepo.

The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:

  • yarn build — Builds all configured packages. Functionally, this:
    • Outputs a dist/ directory with built files.
    • Rewrites package.json "exports" to update every ./src/* reference to ./dist/* (source of truth).
  • scripts/build/babel-register.js — Allows running all Node.js entry points from source, similar to the current setup in facebook/metro. (Example entry point file in this PR: packages/dev-middleware/src/index.js)

Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and packages are opted-in to requiring a build, configured in scripts/build.config.js.

constbuildConfig/*: BuildConfig */={// The packages to include for build and their build optionspackages: {'dev-middleware': {target: 'node'},},};

For now, there is a single target: 'node' option — this is necessary as react-native, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.

Differences from the Metro setup

  • References (and compiles out) repo-local scripts/build/babel-register.js — removing need for an npm-published dependency.

Current integration points

  • CircleCIyarn build is added to the build_npm_package and find_and_publish_bumped_packages jobs.

New Node.js package(s) are not load bearing quite yet: There are not yet any built packages added to the dependencies of packages/react-native/, so this will be further tested in a later PR (and is actively being done in an internal commit stack).

Alternative designs

Per-package config file

Replace scripts/build/config.js with a package-defined key in in package.json, similar to Jest's publishConfig.

"buildConfig": {
"type": "node"
},

This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is enableTypeScriptCodgeen.

Rollup

More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).

Lerna and Nx

  • Most sophisticated setup enabling caching and optimised cloud runs.
  • Probably the most likely thing we'll move towards at a later stage.

Changelog: [Internal]

Differential Revision: D47760330

@facebook-github-botfacebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner fb-exported labels Aug 1, 2023
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

github-actionsBot commented Aug 1, 2023

Copy link
Copy Markdown
Warnings
⚠️🔒 package.json - Changes were made to package.json. This will require a manual import by a Facebook employee.

Generated by 🚫 dangerJS against d9f3a0e

huntie added a commit to huntie/react-native-website that referenced this pull request Aug 1, 2023
Removes mandatory run of `yarn build/watch` when developing Node.js packages — matching the updated build setup in react/react-native#38718 (these packages are now set up to run from source during development).
Partially reverts react#3788 "[docs] Add guidance on repo build scripts"
@analysis-bot

analysis-bot commented Aug 1, 2023

Copy link
Copy Markdown
PlatformEngineArchSize (bytes)Diff
androidhermesarm64-v8a8,894,554-1
androidhermesarmeabi-v7a7,943,106-4
androidhermesx869,292,438-2
androidhermesx86_649,193,963-3
androidjscarm64-v8a9,481,107-1
androidjscarmeabi-v7a8,422,649-1
androidjscx869,465,124-1
androidjscx86_649,779,363-3

Base commit: 0fb7163
Branch: main

@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

Summary:
Pull Request resolved: #38718
> NOTE: Replaces #38240
## Context
RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641
## Changes
To support incoming new React Native packages around debugging (including migrating over [`react-native-community/cli-plugin-metro`](https://github.com/react-native-community/cli/tree/main/packages/cli-plugin-metro)) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the `react-native` monorepo.
The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:
- `yarn build` — Builds all configured packages. Functionally, this:
- Outputs a `dist/` directory with built files.
- Rewrites package.json `"exports"` to update every `./src/*` reference to `./dist/*` (source of truth).
- `scripts/build/babel-register.js` — Allows running all Node.js entry points from source, similar to the current setup in [facebook/metro](https://github.com/facebook/metro). (Example entry point file in this PR: `packages/dev-middleware/src/index.js`)
Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and **packages are opted-in to requiring a build**, configured in `scripts/build.config.js`.
```
const buildConfig /*: BuildConfig */ = {
// The packages to include for build and their build options
packages: {
'dev-middleware': {target: 'node'},
},
};
```
For now, there is a single `target: 'node'` option — this is necessary as `react-native`, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.
{F1043312771}
**Differences from the Metro setup**
- References (and compiles out) repo-local `scripts/build/babel-register.js` — removing need for an npm-published dependency.
## Current integration points
- **CircleCI** — `yarn build` is added to the `build_npm_package` and `find_and_publish_bumped_packages` jobs.
**New Node.js package(s) are not load bearing quite yet**: There are not yet any built packages added to the dependencies of `packages/react-native/`, so this will be further tested in a later PR (and is actively being done in an internal commit stack).
### Alternative designs
**Per-package config file**
Replace `scripts/build/config.js` with a package-defined key in in `package.json`, similar to Jest's [`publishConfig`](https://github.com/jestjs/jest/blob/1f019afdcdfc54a6664908bb45f343db4e3d0848/packages/jest-cli/package.json#L87C3-L89C4).
```
"buildConfig": {
"type": "node"
},
```
This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is `enableTypeScriptCodgeen`.
**Rollup**
More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).
**Lerna and Nx**
- Most sophisticated setup enabling caching and optimised cloud runs.
- Probably the most likely thing we'll move towards at a later stage.
Changelog: [Internal]
Reviewed By: NickGerleman
Differential Revision: D47760330
fbshipit-source-id: 456b506f9e50a43bae9bc902f36910253a0681e0
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

Copy link
Copy Markdown

This pull request was successfully merged by @huntie in cd8f5d1.

When will my fix make it into a release? | Upcoming Releases

@github-actionsgithub-actionsBot added the Merged This PR has been merged. label Aug 3, 2023
facebook-github-bot pushed a commit that referenced this pull request Aug 14, 2023
Summary:
Pull Request resolved: #38990
This PR adds auto-generation of Typescript definitions from Flow source code for packages using the shared monorepo build setup (#38718).
Today, these are the following Node.js packages:
- `packages/community-cli-plugin`
- `packages/dev-middleware` (⬅️ `emitTypeScriptDefs` enabled)
This also improves emitted Flow definitions (`.js.flow`), by using [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator) to strip implementations.
**All changes**
- Include `flow-api-translator` and configure this to emit type definitions as part of `yarn build`.
- Add translation from Flow source to TypeScript definitions (`.d.ts`) adjacent to each built file.
- Improve emitted Flow definitions (`.js.flow`), by using `flow-api-translator` to strip implementations (previously, source files were copied). The Flow and TS defs now mirror each other.
- Add `emitFlowDefs` and `emitTypeScriptDefs` options to build config to configure the above.
- Integrate TypeScript compiler to perform program validation on emitted `.d.ts` files.
- This is based on this guide: https://github.com/microsoft/TypeScript-wiki/blob/main/Using-the-Compiler-API.md#a-minimal-compiler.
- Throw an exception on the `rewritePackageExports` step if a package does not define an `"exports"` field.
- Add minimal `flow-typed` definitions for `typescript` 😄.
**Notes on [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator)**
This project is experimental but is in a more mature state than when we evaluated it earlier in 2023.
- It's now possible to run this tool on our new Node.js packages, since they are exclusively authored using `import`/`export` syntax (a requirement of the tool).
- As a safety net, we run the TypeScript compiler against the generated program, which will fail the build.
Changelog: [Internal]
Reviewed By: robhogan
Differential Revision: D48312463
fbshipit-source-id: 817edb35f911f52fa987946f2d8fc1a319078c9d
@huntie
huntie deleted the export-D47760330 branch August 29, 2023 16:15
facebook-github-bot pushed a commit that referenced this pull request Sep 21, 2023
Summary:
Pull Request resolved: #39566
Fixes running the `build.sh` script for `react-native-codegen` when on [EdenFS](https://github.com/facebook/sapling). This would previously fail due a filesystem "unable to copy extended attributes" error.
This only affects development within the React Native monorepo. Practically, this equates to a workflow improvement for Meta engineers when creating `RNTester` builds.
NOTE: **This is temporary**. The change from `mv` to `cp -R` makes this script more expensive, which will make Android builds via Gradle take slightly longer. I have a WIP cleanup planned which will 1/ move `react-native-codegen` to the [shared monorepo build setup](#38718), and 2/ drop this step entirely from the Android Gradle build.
Changelog: [Internal]
Reviewed By: christophpurrer
Differential Revision: D49468891
fbshipit-source-id: 25d5db81798cf8ab150a135174a45f4d4c2cb5a2
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedMergedThis PR has been merged.p: FacebookPartner: FacebookPartner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@huntie@facebook-github-bot@analysis-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Add shared monorepo build setup by huntie · Pull Request #38718 · react/react-native · GitHub
Skip to content

Add shared monorepo build setup - #38718

Closed
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330
Closed

Add shared monorepo build setup#38718
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330

Conversation

@huntie

@huntiehuntie commented Aug 1, 2023

Copy link
Copy Markdown
Collaborator

Summary:

NOTE: Replaces #38240

Context

RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641

Changes

To support incoming new React Native packages around debugging (including migrating over @react-native-community/cli-plugin-metro) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the react-native monorepo.

The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:

  • yarn build — Builds all configured packages. Functionally, this:
    • Outputs a dist/ directory with built files.
    • Rewrites package.json "exports" to update every ./src/* reference to ./dist/* (source of truth).
  • scripts/build/babel-register.js — Allows running all Node.js entry points from source, similar to the current setup in facebook/metro. (Example entry point file in this PR: packages/dev-middleware/src/index.js)

Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and packages are opted-in to requiring a build, configured in scripts/build.config.js.

constbuildConfig/*: BuildConfig */={// The packages to include for build and their build optionspackages: {'dev-middleware': {target: 'node'},},};

For now, there is a single target: 'node' option — this is necessary as react-native, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.

Differences from the Metro setup

  • References (and compiles out) repo-local scripts/build/babel-register.js — removing need for an npm-published dependency.

Current integration points

  • CircleCIyarn build is added to the build_npm_package and find_and_publish_bumped_packages jobs.

New Node.js package(s) are not load bearing quite yet: There are not yet any built packages added to the dependencies of packages/react-native/, so this will be further tested in a later PR (and is actively being done in an internal commit stack).

Alternative designs

Per-package config file

Replace scripts/build/config.js with a package-defined key in in package.json, similar to Jest's publishConfig.

"buildConfig": {
"type": "node"
},

This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is enableTypeScriptCodgeen.

Rollup

More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).

Lerna and Nx

  • Most sophisticated setup enabling caching and optimised cloud runs.
  • Probably the most likely thing we'll move towards at a later stage.

Changelog: [Internal]

Differential Revision: D47760330

@facebook-github-botfacebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner fb-exported labels Aug 1, 2023
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

github-actionsBot commented Aug 1, 2023

Copy link
Copy Markdown
Warnings
⚠️🔒 package.json - Changes were made to package.json. This will require a manual import by a Facebook employee.

Generated by 🚫 dangerJS against d9f3a0e

huntie added a commit to huntie/react-native-website that referenced this pull request Aug 1, 2023
Removes mandatory run of `yarn build/watch` when developing Node.js packages — matching the updated build setup in react/react-native#38718 (these packages are now set up to run from source during development).
Partially reverts react#3788 "[docs] Add guidance on repo build scripts"
@analysis-bot

analysis-bot commented Aug 1, 2023

Copy link
Copy Markdown
PlatformEngineArchSize (bytes)Diff
androidhermesarm64-v8a8,894,554-1
androidhermesarmeabi-v7a7,943,106-4
androidhermesx869,292,438-2
androidhermesx86_649,193,963-3
androidjscarm64-v8a9,481,107-1
androidjscarmeabi-v7a8,422,649-1
androidjscx869,465,124-1
androidjscx86_649,779,363-3

Base commit: 0fb7163
Branch: main

@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

Summary:
Pull Request resolved: #38718
> NOTE: Replaces #38240
## Context
RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641
## Changes
To support incoming new React Native packages around debugging (including migrating over [`react-native-community/cli-plugin-metro`](https://github.com/react-native-community/cli/tree/main/packages/cli-plugin-metro)) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the `react-native` monorepo.
The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:
- `yarn build` — Builds all configured packages. Functionally, this:
- Outputs a `dist/` directory with built files.
- Rewrites package.json `"exports"` to update every `./src/*` reference to `./dist/*` (source of truth).
- `scripts/build/babel-register.js` — Allows running all Node.js entry points from source, similar to the current setup in [facebook/metro](https://github.com/facebook/metro). (Example entry point file in this PR: `packages/dev-middleware/src/index.js`)
Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and **packages are opted-in to requiring a build**, configured in `scripts/build.config.js`.
```
const buildConfig /*: BuildConfig */ = {
// The packages to include for build and their build options
packages: {
'dev-middleware': {target: 'node'},
},
};
```
For now, there is a single `target: 'node'` option — this is necessary as `react-native`, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.
{F1043312771}
**Differences from the Metro setup**
- References (and compiles out) repo-local `scripts/build/babel-register.js` — removing need for an npm-published dependency.
## Current integration points
- **CircleCI** — `yarn build` is added to the `build_npm_package` and `find_and_publish_bumped_packages` jobs.
**New Node.js package(s) are not load bearing quite yet**: There are not yet any built packages added to the dependencies of `packages/react-native/`, so this will be further tested in a later PR (and is actively being done in an internal commit stack).
### Alternative designs
**Per-package config file**
Replace `scripts/build/config.js` with a package-defined key in in `package.json`, similar to Jest's [`publishConfig`](https://github.com/jestjs/jest/blob/1f019afdcdfc54a6664908bb45f343db4e3d0848/packages/jest-cli/package.json#L87C3-L89C4).
```
"buildConfig": {
"type": "node"
},
```
This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is `enableTypeScriptCodgeen`.
**Rollup**
More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).
**Lerna and Nx**
- Most sophisticated setup enabling caching and optimised cloud runs.
- Probably the most likely thing we'll move towards at a later stage.
Changelog: [Internal]
Reviewed By: NickGerleman
Differential Revision: D47760330
fbshipit-source-id: 456b506f9e50a43bae9bc902f36910253a0681e0
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

Copy link
Copy Markdown

This pull request was successfully merged by @huntie in cd8f5d1.

When will my fix make it into a release? | Upcoming Releases

@github-actionsgithub-actionsBot added the Merged This PR has been merged. label Aug 3, 2023
facebook-github-bot pushed a commit that referenced this pull request Aug 14, 2023
Summary:
Pull Request resolved: #38990
This PR adds auto-generation of Typescript definitions from Flow source code for packages using the shared monorepo build setup (#38718).
Today, these are the following Node.js packages:
- `packages/community-cli-plugin`
- `packages/dev-middleware` (⬅️ `emitTypeScriptDefs` enabled)
This also improves emitted Flow definitions (`.js.flow`), by using [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator) to strip implementations.
**All changes**
- Include `flow-api-translator` and configure this to emit type definitions as part of `yarn build`.
- Add translation from Flow source to TypeScript definitions (`.d.ts`) adjacent to each built file.
- Improve emitted Flow definitions (`.js.flow`), by using `flow-api-translator` to strip implementations (previously, source files were copied). The Flow and TS defs now mirror each other.
- Add `emitFlowDefs` and `emitTypeScriptDefs` options to build config to configure the above.
- Integrate TypeScript compiler to perform program validation on emitted `.d.ts` files.
- This is based on this guide: https://github.com/microsoft/TypeScript-wiki/blob/main/Using-the-Compiler-API.md#a-minimal-compiler.
- Throw an exception on the `rewritePackageExports` step if a package does not define an `"exports"` field.
- Add minimal `flow-typed` definitions for `typescript` 😄.
**Notes on [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator)**
This project is experimental but is in a more mature state than when we evaluated it earlier in 2023.
- It's now possible to run this tool on our new Node.js packages, since they are exclusively authored using `import`/`export` syntax (a requirement of the tool).
- As a safety net, we run the TypeScript compiler against the generated program, which will fail the build.
Changelog: [Internal]
Reviewed By: robhogan
Differential Revision: D48312463
fbshipit-source-id: 817edb35f911f52fa987946f2d8fc1a319078c9d
@huntie
huntie deleted the export-D47760330 branch August 29, 2023 16:15
facebook-github-bot pushed a commit that referenced this pull request Sep 21, 2023
Summary:
Pull Request resolved: #39566
Fixes running the `build.sh` script for `react-native-codegen` when on [EdenFS](https://github.com/facebook/sapling). This would previously fail due a filesystem "unable to copy extended attributes" error.
This only affects development within the React Native monorepo. Practically, this equates to a workflow improvement for Meta engineers when creating `RNTester` builds.
NOTE: **This is temporary**. The change from `mv` to `cp -R` makes this script more expensive, which will make Android builds via Gradle take slightly longer. I have a WIP cleanup planned which will 1/ move `react-native-codegen` to the [shared monorepo build setup](#38718), and 2/ drop this step entirely from the Android Gradle build.
Changelog: [Internal]
Reviewed By: christophpurrer
Differential Revision: D49468891
fbshipit-source-id: 25d5db81798cf8ab150a135174a45f4d4c2cb5a2
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedMergedThis PR has been merged.p: FacebookPartner: FacebookPartner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@huntie@facebook-github-bot@analysis-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Add shared monorepo build setup by huntie · Pull Request #38718 · react/react-native · GitHub
Skip to content

Add shared monorepo build setup - #38718

Closed
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330
Closed

Add shared monorepo build setup#38718
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330

Conversation

@huntie

@huntiehuntie commented Aug 1, 2023

Copy link
Copy Markdown
Collaborator

Summary:

NOTE: Replaces #38240

Context

RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641

Changes

To support incoming new React Native packages around debugging (including migrating over @react-native-community/cli-plugin-metro) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the react-native monorepo.

The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:

  • yarn build — Builds all configured packages. Functionally, this:
    • Outputs a dist/ directory with built files.
    • Rewrites package.json "exports" to update every ./src/* reference to ./dist/* (source of truth).
  • scripts/build/babel-register.js — Allows running all Node.js entry points from source, similar to the current setup in facebook/metro. (Example entry point file in this PR: packages/dev-middleware/src/index.js)

Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and packages are opted-in to requiring a build, configured in scripts/build.config.js.

constbuildConfig/*: BuildConfig */={// The packages to include for build and their build optionspackages: {'dev-middleware': {target: 'node'},},};

For now, there is a single target: 'node' option — this is necessary as react-native, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.

Differences from the Metro setup

  • References (and compiles out) repo-local scripts/build/babel-register.js — removing need for an npm-published dependency.

Current integration points

  • CircleCIyarn build is added to the build_npm_package and find_and_publish_bumped_packages jobs.

New Node.js package(s) are not load bearing quite yet: There are not yet any built packages added to the dependencies of packages/react-native/, so this will be further tested in a later PR (and is actively being done in an internal commit stack).

Alternative designs

Per-package config file

Replace scripts/build/config.js with a package-defined key in in package.json, similar to Jest's publishConfig.

"buildConfig": {
"type": "node"
},

This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is enableTypeScriptCodgeen.

Rollup

More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).

Lerna and Nx

  • Most sophisticated setup enabling caching and optimised cloud runs.
  • Probably the most likely thing we'll move towards at a later stage.

Changelog: [Internal]

Differential Revision: D47760330

@facebook-github-botfacebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner fb-exported labels Aug 1, 2023
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

github-actionsBot commented Aug 1, 2023

Copy link
Copy Markdown
Warnings
⚠️🔒 package.json - Changes were made to package.json. This will require a manual import by a Facebook employee.

Generated by 🚫 dangerJS against d9f3a0e

huntie added a commit to huntie/react-native-website that referenced this pull request Aug 1, 2023
Removes mandatory run of `yarn build/watch` when developing Node.js packages — matching the updated build setup in react/react-native#38718 (these packages are now set up to run from source during development).
Partially reverts react#3788 "[docs] Add guidance on repo build scripts"
@analysis-bot

analysis-bot commented Aug 1, 2023

Copy link
Copy Markdown
PlatformEngineArchSize (bytes)Diff
androidhermesarm64-v8a8,894,554-1
androidhermesarmeabi-v7a7,943,106-4
androidhermesx869,292,438-2
androidhermesx86_649,193,963-3
androidjscarm64-v8a9,481,107-1
androidjscarmeabi-v7a8,422,649-1
androidjscx869,465,124-1
androidjscx86_649,779,363-3

Base commit: 0fb7163
Branch: main

@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

Summary:
Pull Request resolved: #38718
> NOTE: Replaces #38240
## Context
RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641
## Changes
To support incoming new React Native packages around debugging (including migrating over [`react-native-community/cli-plugin-metro`](https://github.com/react-native-community/cli/tree/main/packages/cli-plugin-metro)) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the `react-native` monorepo.
The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:
- `yarn build` — Builds all configured packages. Functionally, this:
- Outputs a `dist/` directory with built files.
- Rewrites package.json `"exports"` to update every `./src/*` reference to `./dist/*` (source of truth).
- `scripts/build/babel-register.js` — Allows running all Node.js entry points from source, similar to the current setup in [facebook/metro](https://github.com/facebook/metro). (Example entry point file in this PR: `packages/dev-middleware/src/index.js`)
Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and **packages are opted-in to requiring a build**, configured in `scripts/build.config.js`.
```
const buildConfig /*: BuildConfig */ = {
// The packages to include for build and their build options
packages: {
'dev-middleware': {target: 'node'},
},
};
```
For now, there is a single `target: 'node'` option — this is necessary as `react-native`, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.
{F1043312771}
**Differences from the Metro setup**
- References (and compiles out) repo-local `scripts/build/babel-register.js` — removing need for an npm-published dependency.
## Current integration points
- **CircleCI** — `yarn build` is added to the `build_npm_package` and `find_and_publish_bumped_packages` jobs.
**New Node.js package(s) are not load bearing quite yet**: There are not yet any built packages added to the dependencies of `packages/react-native/`, so this will be further tested in a later PR (and is actively being done in an internal commit stack).
### Alternative designs
**Per-package config file**
Replace `scripts/build/config.js` with a package-defined key in in `package.json`, similar to Jest's [`publishConfig`](https://github.com/jestjs/jest/blob/1f019afdcdfc54a6664908bb45f343db4e3d0848/packages/jest-cli/package.json#L87C3-L89C4).
```
"buildConfig": {
"type": "node"
},
```
This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is `enableTypeScriptCodgeen`.
**Rollup**
More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).
**Lerna and Nx**
- Most sophisticated setup enabling caching and optimised cloud runs.
- Probably the most likely thing we'll move towards at a later stage.
Changelog: [Internal]
Reviewed By: NickGerleman
Differential Revision: D47760330
fbshipit-source-id: 456b506f9e50a43bae9bc902f36910253a0681e0
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

Copy link
Copy Markdown

This pull request was successfully merged by @huntie in cd8f5d1.

When will my fix make it into a release? | Upcoming Releases

@github-actionsgithub-actionsBot added the Merged This PR has been merged. label Aug 3, 2023
facebook-github-bot pushed a commit that referenced this pull request Aug 14, 2023
Summary:
Pull Request resolved: #38990
This PR adds auto-generation of Typescript definitions from Flow source code for packages using the shared monorepo build setup (#38718).
Today, these are the following Node.js packages:
- `packages/community-cli-plugin`
- `packages/dev-middleware` (⬅️ `emitTypeScriptDefs` enabled)
This also improves emitted Flow definitions (`.js.flow`), by using [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator) to strip implementations.
**All changes**
- Include `flow-api-translator` and configure this to emit type definitions as part of `yarn build`.
- Add translation from Flow source to TypeScript definitions (`.d.ts`) adjacent to each built file.
- Improve emitted Flow definitions (`.js.flow`), by using `flow-api-translator` to strip implementations (previously, source files were copied). The Flow and TS defs now mirror each other.
- Add `emitFlowDefs` and `emitTypeScriptDefs` options to build config to configure the above.
- Integrate TypeScript compiler to perform program validation on emitted `.d.ts` files.
- This is based on this guide: https://github.com/microsoft/TypeScript-wiki/blob/main/Using-the-Compiler-API.md#a-minimal-compiler.
- Throw an exception on the `rewritePackageExports` step if a package does not define an `"exports"` field.
- Add minimal `flow-typed` definitions for `typescript` 😄.
**Notes on [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator)**
This project is experimental but is in a more mature state than when we evaluated it earlier in 2023.
- It's now possible to run this tool on our new Node.js packages, since they are exclusively authored using `import`/`export` syntax (a requirement of the tool).
- As a safety net, we run the TypeScript compiler against the generated program, which will fail the build.
Changelog: [Internal]
Reviewed By: robhogan
Differential Revision: D48312463
fbshipit-source-id: 817edb35f911f52fa987946f2d8fc1a319078c9d
@huntie
huntie deleted the export-D47760330 branch August 29, 2023 16:15
facebook-github-bot pushed a commit that referenced this pull request Sep 21, 2023
Summary:
Pull Request resolved: #39566
Fixes running the `build.sh` script for `react-native-codegen` when on [EdenFS](https://github.com/facebook/sapling). This would previously fail due a filesystem "unable to copy extended attributes" error.
This only affects development within the React Native monorepo. Practically, this equates to a workflow improvement for Meta engineers when creating `RNTester` builds.
NOTE: **This is temporary**. The change from `mv` to `cp -R` makes this script more expensive, which will make Android builds via Gradle take slightly longer. I have a WIP cleanup planned which will 1/ move `react-native-codegen` to the [shared monorepo build setup](#38718), and 2/ drop this step entirely from the Android Gradle build.
Changelog: [Internal]
Reviewed By: christophpurrer
Differential Revision: D49468891
fbshipit-source-id: 25d5db81798cf8ab150a135174a45f4d4c2cb5a2
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedMergedThis PR has been merged.p: FacebookPartner: FacebookPartner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@huntie@facebook-github-bot@analysis-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Add shared monorepo build setup by huntie · Pull Request #38718 · react/react-native · GitHub
Skip to content

Add shared monorepo build setup - #38718

Closed
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330
Closed

Add shared monorepo build setup#38718
huntie wants to merge 1 commit into
react:mainfrom
huntie:export-D47760330

Conversation

@huntie

@huntiehuntie commented Aug 1, 2023

Copy link
Copy Markdown
Collaborator

Summary:

NOTE: Replaces #38240

Context

RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641

Changes

To support incoming new React Native packages around debugging (including migrating over @react-native-community/cli-plugin-metro) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the react-native monorepo.

The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:

  • yarn build — Builds all configured packages. Functionally, this:
    • Outputs a dist/ directory with built files.
    • Rewrites package.json "exports" to update every ./src/* reference to ./dist/* (source of truth).
  • scripts/build/babel-register.js — Allows running all Node.js entry points from source, similar to the current setup in facebook/metro. (Example entry point file in this PR: packages/dev-middleware/src/index.js)

Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and packages are opted-in to requiring a build, configured in scripts/build.config.js.

constbuildConfig/*: BuildConfig */={// The packages to include for build and their build optionspackages: {'dev-middleware': {target: 'node'},},};

For now, there is a single target: 'node' option — this is necessary as react-native, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.

Differences from the Metro setup

  • References (and compiles out) repo-local scripts/build/babel-register.js — removing need for an npm-published dependency.

Current integration points

  • CircleCIyarn build is added to the build_npm_package and find_and_publish_bumped_packages jobs.

New Node.js package(s) are not load bearing quite yet: There are not yet any built packages added to the dependencies of packages/react-native/, so this will be further tested in a later PR (and is actively being done in an internal commit stack).

Alternative designs

Per-package config file

Replace scripts/build/config.js with a package-defined key in in package.json, similar to Jest's publishConfig.

"buildConfig": {
"type": "node"
},

This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is enableTypeScriptCodgeen.

Rollup

More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).

Lerna and Nx

  • Most sophisticated setup enabling caching and optimised cloud runs.
  • Probably the most likely thing we'll move towards at a later stage.

Changelog: [Internal]

Differential Revision: D47760330

@facebook-github-botfacebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner fb-exported labels Aug 1, 2023
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

github-actionsBot commented Aug 1, 2023

Copy link
Copy Markdown
Warnings
⚠️🔒 package.json - Changes were made to package.json. This will require a manual import by a Facebook employee.

Generated by 🚫 dangerJS against d9f3a0e

huntie added a commit to huntie/react-native-website that referenced this pull request Aug 1, 2023
Removes mandatory run of `yarn build/watch` when developing Node.js packages — matching the updated build setup in react/react-native#38718 (these packages are now set up to run from source during development).
Partially reverts react#3788 "[docs] Add guidance on repo build scripts"
@analysis-bot

analysis-bot commented Aug 1, 2023

Copy link
Copy Markdown
PlatformEngineArchSize (bytes)Diff
androidhermesarm64-v8a8,894,554-1
androidhermesarmeabi-v7a7,943,106-4
androidhermesx869,292,438-2
androidhermesx86_649,193,963-3
androidjscarm64-v8a9,481,107-1
androidjscarmeabi-v7a8,422,649-1
androidjscx869,465,124-1
androidjscx86_649,779,363-3

Base commit: 0fb7163
Branch: main

@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

Summary:
Pull Request resolved: #38718
> NOTE: Replaces #38240
## Context
RFC: Decoupling Flipper from React Native core: react-native-community/discussions-and-proposals#641
## Changes
To support incoming new React Native packages around debugging (including migrating over [`react-native-community/cli-plugin-metro`](https://github.com/react-native-community/cli/tree/main/packages/cli-plugin-metro)) — which target Node.js and require a build step, this PR adds a minimal shared build setup across the `react-native` monorepo.
The setup is closely inspired/based on the build scripts in Jest, Metro, and React Native CLI — and is a simple set of script wrappers around Babel. These are available as build commands at the root of the repo:
- `yarn build` — Builds all configured packages. Functionally, this:
- Outputs a `dist/` directory with built files.
- Rewrites package.json `"exports"` to update every `./src/*` reference to `./dist/*` (source of truth).
- `scripts/build/babel-register.js` — Allows running all Node.js entry points from source, similar to the current setup in [facebook/metro](https://github.com/facebook/metro). (Example entry point file in this PR: `packages/dev-middleware/src/index.js`)
Build configuration (i.e. Babel config) is shared as a set standard across the monorepo, and **packages are opted-in to requiring a build**, configured in `scripts/build.config.js`.
```
const buildConfig /*: BuildConfig */ = {
// The packages to include for build and their build options
packages: {
'dev-middleware': {target: 'node'},
},
};
```
For now, there is a single `target: 'node'` option — this is necessary as `react-native`, unlike the above other projects, is a repository with packages targeting several runtimes. We may, in future, introduce a build step for other, non-Node, packages — which may be useful for things such as auto-generated TypeScript definitions.
{F1043312771}
**Differences from the Metro setup**
- References (and compiles out) repo-local `scripts/build/babel-register.js` — removing need for an npm-published dependency.
## Current integration points
- **CircleCI** — `yarn build` is added to the `build_npm_package` and `find_and_publish_bumped_packages` jobs.
**New Node.js package(s) are not load bearing quite yet**: There are not yet any built packages added to the dependencies of `packages/react-native/`, so this will be further tested in a later PR (and is actively being done in an internal commit stack).
### Alternative designs
**Per-package config file**
Replace `scripts/build/config.js` with a package-defined key in in `package.json`, similar to Jest's [`publishConfig`](https://github.com/jestjs/jest/blob/1f019afdcdfc54a6664908bb45f343db4e3d0848/packages/jest-cli/package.json#L87C3-L89C4).
```
"buildConfig": {
"type": "node"
},
```
This would be the only customisation required, with a single Babel config still standardised. Another option this might receive in future is `enableTypeScriptCodgeen`.
**Rollup**
More sophisticated build tool for Node.js, used by the React codebase (albeit within a custom script setup as well).
**Lerna and Nx**
- Most sophisticated setup enabling caching and optimised cloud runs.
- Probably the most likely thing we'll move towards at a later stage.
Changelog: [Internal]
Reviewed By: NickGerleman
Differential Revision: D47760330
fbshipit-source-id: 456b506f9e50a43bae9bc902f36910253a0681e0
@facebook-github-bot

Copy link
Copy Markdown
Contributor

This pull request was exported from Phabricator. Differential Revision: D47760330

@github-actions

Copy link
Copy Markdown

This pull request was successfully merged by @huntie in cd8f5d1.

When will my fix make it into a release? | Upcoming Releases

@github-actionsgithub-actionsBot added the Merged This PR has been merged. label Aug 3, 2023
facebook-github-bot pushed a commit that referenced this pull request Aug 14, 2023
Summary:
Pull Request resolved: #38990
This PR adds auto-generation of Typescript definitions from Flow source code for packages using the shared monorepo build setup (#38718).
Today, these are the following Node.js packages:
- `packages/community-cli-plugin`
- `packages/dev-middleware` (⬅️ `emitTypeScriptDefs` enabled)
This also improves emitted Flow definitions (`.js.flow`), by using [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator) to strip implementations.
**All changes**
- Include `flow-api-translator` and configure this to emit type definitions as part of `yarn build`.
- Add translation from Flow source to TypeScript definitions (`.d.ts`) adjacent to each built file.
- Improve emitted Flow definitions (`.js.flow`), by using `flow-api-translator` to strip implementations (previously, source files were copied). The Flow and TS defs now mirror each other.
- Add `emitFlowDefs` and `emitTypeScriptDefs` options to build config to configure the above.
- Integrate TypeScript compiler to perform program validation on emitted `.d.ts` files.
- This is based on this guide: https://github.com/microsoft/TypeScript-wiki/blob/main/Using-the-Compiler-API.md#a-minimal-compiler.
- Throw an exception on the `rewritePackageExports` step if a package does not define an `"exports"` field.
- Add minimal `flow-typed` definitions for `typescript` 😄.
**Notes on [`flow-api-translator`](https://www.npmjs.com/package/flow-api-translator)**
This project is experimental but is in a more mature state than when we evaluated it earlier in 2023.
- It's now possible to run this tool on our new Node.js packages, since they are exclusively authored using `import`/`export` syntax (a requirement of the tool).
- As a safety net, we run the TypeScript compiler against the generated program, which will fail the build.
Changelog: [Internal]
Reviewed By: robhogan
Differential Revision: D48312463
fbshipit-source-id: 817edb35f911f52fa987946f2d8fc1a319078c9d
@huntie
huntie deleted the export-D47760330 branch August 29, 2023 16:15
facebook-github-bot pushed a commit that referenced this pull request Sep 21, 2023
Summary:
Pull Request resolved: #39566
Fixes running the `build.sh` script for `react-native-codegen` when on [EdenFS](https://github.com/facebook/sapling). This would previously fail due a filesystem "unable to copy extended attributes" error.
This only affects development within the React Native monorepo. Practically, this equates to a workflow improvement for Meta engineers when creating `RNTester` builds.
NOTE: **This is temporary**. The change from `mv` to `cp -R` makes this script more expensive, which will make Android builds via Gradle take slightly longer. I have a WIP cleanup planned which will 1/ move `react-native-codegen` to the [shared monorepo build setup](#38718), and 2/ drop this step entirely from the Android Gradle build.
Changelog: [Internal]
Reviewed By: christophpurrer
Differential Revision: D49468891
fbshipit-source-id: 25d5db81798cf8ab150a135174a45f4d4c2cb5a2
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.fb-exportedMergedThis PR has been merged.p: FacebookPartner: FacebookPartner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@huntie@facebook-github-bot@analysis-bot