Uh oh!
There was an error while loading. Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork 1
fix: add global fields FVRs in export#232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Uh oh!
There was an error while loading. Please reload this page.
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
0f5d4c3
fix: add global fields FVRs in export
babff65
Merge branch 'development' into fix/DX-8587
8ab39bd
feat: add global field rule handling to content type
3a15a12
Merge branch 'development' into fix/DX-8587
naman-contentstack 59de86d
feat: enhance field rules audit to include global fields and add corr…
922e81a
fix: typo in test case name
naman-contentstack 08b53a9
fix: update the commented msg
naman-contentstack af8818e
Merge branch 'development' into fix/DX-8587
3537a04
refactor: success logging for content types import
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,74 @@ | ||
| fileignoreconfig: | ||
| - filename: pnpm-lock.yaml | ||
| checksum: 4be4721031793b5ec6957b17778969b715ded2d15a6b6231b6c066735abd7233 | ||
| - filename: pnpm-lock.yaml | ||
| checksum: cb3710ff4c9d3b8f45a8f0b3ff868d49451872ac6d085ae3ccf83d31055cf50d | ||
| - filename: packages/contentstack-bulk-operations/src/messages/index.ts | ||
| checksum: 097f7ed876622aed6edc3f12995a92e219f96c3d053ec078d45c762f7ebe085e | ||
| - filename: packages/contentstack-bulk-operations/test/unit/base-bulk-command.test.ts | ||
| checksum: 522b91a2d4bb6a73ebe5ab92c0111e74d567c4360470beefd6a7ca8d30a6e311 | ||
| - filename: packages/contentstack-import/test/unit/import/modules/assets.test.ts | ||
| checksum: 64e3af1e307d0f001370870267fa4cedd2ae81d0f3c38c2e184e69a9a68b2b30 | ||
| - filename: packages/contentstack-query-export/test/unit/common-helper.test.ts | ||
| checksum: a0aa399a2ec15760fe25f7e952d64792b55d3c38d2281858a48e64f0f9081344 | ||
| - filename: packages/contentstack-import/test/unit/utils/interactive.test.ts | ||
| checksum: dad20e229e02e847b5e939fdaf987b1a6c9ff4e1529b6d96a51130f7293285a9 | ||
| - filename: packages/contentstack-import/test/unit/utils/common-helper.test.ts | ||
| checksum: 12c8161a1cb313fa98b612d72e605f6e5190b9f9561d6e73c269c7d106e1a196 | ||
| - filename: packages/contentstack-apps-cli/src/messages/index.ts | ||
| checksum: 5d7991adb5056a14359c5a43db995460a72234916138dbc5ec14f726faf0054b | ||
| - filename: packages/contentstack-import/test/unit/utils/marketplace-app-helper.test.ts | ||
| checksum: 878be77164311c0ab86629fe18fd3d1be7ff187447e4e5ab65a241b3be0a8d24 | ||
| - filename: packages/contentstack-import/test/unit/import/modules/global-fields.test.ts | ||
| checksum: 4bd86f40b28433470eea876acc9254042c3b24556acdfdb7b15375a948acc094 | ||
| - filename: packages/contentstack-import/test/unit/import/modules/base-class.test.ts | ||
| checksum: 7616f13a4c908eead76c1625cb146320a8a23f31732acb2ea8472e2e0a8b2692 | ||
| - filename: packages/contentstack-import/test/unit/import/modules/entries.test.ts | ||
| checksum: 02332fc203e92039b5fcab6e2e14d14ea5444a4cba2be853009b4c4994692637 | ||
| - filename: packages/contentstack-import/test/unit/import/modules/index.test.ts | ||
| checksum: 6c2bc4bb09dc750532e84a2d6a8198e145bd142bc21ce0d97ae0c8da55dea723 | ||
| - filename: packages/contentstack-import/test/unit/commands/cm/stacks/import.test.ts | ||
| checksum: a52fbd82e4296734d5eb5b06cc9a10d7e2c1fe7b58d15a44d3246f0f2762d5a1 | ||
| - filename: packages/contentstack-migrate-rte/test/utils/index.js | ||
| checksum: f67235e607a446e5b2dd624046966a1ac5af7bce128aa080ec3c420c48b8bddc | ||
| - filename: packages/contentstack-bulk-operations/test/unit/utils/config-builder.test.ts | ||
| checksum: e5891257999e9e940956ca77f5a7834be35d259edccd41c06f77736194ad7454 | ||
| - filename: packages/contentstack-import/test/unit/import/modules/content-types.test.ts | ||
| checksum: cd47b1694cb1edffb11fed0f2024fac631b0279ebf55113510d9121c4b957ab1 | ||
| - filename: packages/contentstack-bulk-operations/test/unit/utils/bulk-operation-log-handler.test.ts | ||
| checksum: 8698f77bbb828df10ddc3a94726076a262fbf47d2468233c7e10e66cac42f97d | ||
| - filename: packages/contentstack-migrate-rte/test/commands/json-migration.test.js | ||
| checksum: 0d18641a32f7cf33d013a5a7c447c1067ba77022513b42304a727f873c741e99 | ||
| - filename: packages/contentstack-export/test/unit/utils/interactive.test.ts | ||
| checksum: 0ea47db305f576c8c9da8cb34ae915f6b01992208ff1c4380cb7ffd93f538e5d | ||
| - filename: packages/contentstack-clone/test/lib/helpers/command-helpers.test.ts | ||
| checksum: 32d385a677a76faa54cb3ddfe92fb1f1f53cfab878a78e2792c823a363956bd2 | ||
| - filename: packages/contentstack-export-to-csv/src/messages/index.ts | ||
| checksum: 54cbfb3fb83d06ce46766897066ace927c190b2383154b45423e668f63fa4392 | ||
| - filename: packages/contentstack-clone/test/commands/stack-clone.test.ts | ||
| checksum: 4547ba010cc3c9954de49e2b2bef45288dd070c4ad000b6ae3e9a8c9bb1ad296 | ||
| - filename: packages/contentstack-content-type/tests/utils/index.test.ts | ||
| checksum: 8510fedf5eb6078c27d6a0d65d8bff4aa2d41b545246aed220d1a6a643188ad8 | ||
| - filename: packages/contentstack-bulk-operations/test/unit/utils/revert-retry-handler.test.ts | ||
| checksum: c34a4bdea186f4ce48171d90b8ab0f1c94162e441acb23e3a0e00cb9f9f9255f | ||
| - filename: packages/contentstack-export-to-csv/test/unit/utils/error-handler.test.ts | ||
| checksum: 7216dd610334e6f4a0e2701712408a273a5a2bd4a5ec30b6d6aae043e2b53cc8 | ||
| - filename: packages/contentstack-export/test/unit/utils/marketplace-app-helper.test.ts | ||
| checksum: 3734c4cf7145ef32ad424ccbc160ea8be9b38157762aa66ade51879ab369cee9 | ||
| - filename: packages/contentstack-import-setup/test/unit/common-helper.test.ts | ||
| checksum: 7ac4645401fd3629347a8a182f8a320f177de545a96b8027422bd68040513068 | ||
| - filename: packages/contentstack-export/test/unit/utils/common-helper.test.ts | ||
| checksum: 928f173ca03855a516ac468dba84092fd5f66d2e41b0100bb548e0af364a4ef5 | ||
| - filename: packages/contentstack-export/test/unit/utils/export-config-handler.test.ts | ||
| checksum: 0e7777f73da4f41f4e01e768dba1f3f62ab36778c4f9cd0b519fd1e3bb5c410f | ||
| - filename: packages/contentstack-clone/test/lib/util/clone-handler.branch.test.ts | ||
| checksum: db8c090aaada21d3952c5bc382228a0a11cebb63926512a5e2581bdaf5f4c2f6 | ||
| - filename: packages/contentstack-import-setup/test/unit/login-handler.test.ts | ||
| checksum: fa7cb2aeb0873e5650beed42096ef4a8edd012a7217818ab0cab2ba6c4611934 | ||
| - filename: packages/contentstack-import/test/unit/import/modules/marketplace-apps.test.ts | ||
| checksum: 42f4d0a15080a69423d4e82b11a6a1933c5f72930cdc4a55b86f27fdc980aeea | ||
| - filename: packages/contentstack-import/test/unit/utils/login-handler.test.ts | ||
| checksum: 49dd751f846f50fe00d60e7f02b5bd237d36480048bf5e06a24f1cf9fe752a1b | ||
| - filename: packages/contentstack-content-type/tests/core/command.test.ts | ||
| checksum: 81da32fc47083a3421f5f1d9c04149ab313cb238d6416d3182f8b6d64dc8827d | ||
| - filename: packages/contentstack-content-type/tests/core/contentstack/client.test.ts | ||
| checksum: da15b04aa0a7ab0e48f70b847e68c05372943279e51c22f8539b01bb48033183 | ||
| version: '1.0' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
47 changes: 47 additions & 0 deletions
47 packages/contentstack-audit/test/unit/modules/field-rules.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
100 changes: 95 additions & 5 deletions
100 packages/contentstack-import/src/import/modules/content-types.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -11,7 +11,7 @@ import { sanitizePath, log, handleAndLogError } from '@contentstack/cli-utilitie | ||
| import { fsUtil, schemaTemplate, lookupExtension, lookUpTaxonomy, fileHelper } from '../../utils'; | ||
| import { ImportConfig, ModuleClassParams } from '../../types'; | ||
| import BaseClass, { ApiOptions } from './base-class'; | ||
| import { updateFieldRules } from '../../utils/content-type-helper'; | ||
| import { updateFieldRules, isGlobalFieldRule } from '../../utils/content-type-helper'; | ||
| export default class ContentTypesImport extends BaseClass { | ||
| private cTsMapperPath: string; | ||
| @@ -34,7 +34,7 @@ export default class ContentTypesImport extends BaseClass { | ||
| private reqConcurrency: number; | ||
| private ignoredFilesInContentTypesFolder: Map<string, string>; | ||
| private titleToUIdMap: Map<string, string>; | ||
| private fieldRules: Array<Record<string, unknown>>; | ||
| private fieldRules: string[]; | ||
| private installedExtensions: Record<string, unknown>; | ||
| private cTsConfig: { | ||
| dirName: string; | ||
| @@ -206,13 +206,103 @@ export default class ContentTypesImport extends BaseClass { | ||
| this.pendingGFs = fsUtil.readFile(this.gFsPendingPath) as any; | ||
| if (!this.pendingGFs || isEmpty(this.pendingGFs)) { | ||
| log.info('No pending global fields found to update.', this.importConfig.context); | ||
| return; | ||
| } else { | ||
| await this.updatePendingGFs().catch((error) => { | ||
| handleAndLogError(error, { ...this.importConfig.context }); | ||
| }); | ||
| log.success('Updated pending global fields with content type with references', this.importConfig.context); | ||
| } | ||
| await this.updatePendingGFs().catch((error) => { | ||
| // Global field rules were skipped during the content type update (see updateFieldRules) because | ||
| // the embedded global field schema was not yet complete on the stack. At this point global | ||
| // fields are expected to be complete (deferred ones via updatePendingGFs above; others already | ||
| // applied in the global-fields module / pre-existing on the stack for module-only imports). | ||
| // Re-apply the global field rules now; if global fields are still incomplete this step may fail | ||
| // and will be reported below. | ||
| const failedGFFieldRuleCTs = await this.updateGFFieldRules().catch((error) => { | ||
| handleAndLogError(error, { ...this.importConfig.context }); | ||
| return [] as string[]; | ||
| }); | ||
| log.success('Updated pending global fields with content type with references', this.importConfig.context); | ||
| if (failedGFFieldRuleCTs.length) { | ||
| // Surface the partial failure instead of claiming an unqualified success. | ||
| log.error( | ||
naman-contentstack marked this conversation as resolved.
Uh oh!There was an error while loading. Please reload this page. | ||
| `Content types imported, but failed to apply global field rules for: ${failedGFFieldRuleCTs.join(', ')}`, | ||
| this.importConfig.context, | ||
| ); | ||
| } | ||
| log.success('Content types have been imported successfully!', this.importConfig.context); | ||
| } | ||
| /** | ||
| * Applies the global field rules that were skipped during the content type update (updateFieldRules | ||
| * strips rules flagged is_global_field_rule, because their paths reference an embedded global field | ||
| * whose schema is not yet complete when the content type is first updated). By the time this runs, | ||
| * every embedded global field is complete, so the rules validate. Runs for deferred, non-deferred | ||
| * and module-only imports alike. | ||
| * @returns the uids of content types whose global field rule update failed. | ||
| */ | ||
| async updateGFFieldRules(): Promise<string[]> { | ||
| const failedCTs: string[] = []; | ||
| if (!this.fieldRules?.length) { | ||
| log.debug('No content types with field rules; skipping global field rules update.', this.importConfig.context); | ||
| return failedCTs; | ||
| } | ||
| const cTs = (fsUtil.readFile(path.join(this.cTsFolderPath, 'schema.json')) || []) as Record<string, any>[]; | ||
| for (const cTUid of this.fieldRules) { | ||
| const contentType: any = find(cTs, { uid: cTUid }); | ||
| if (!contentType?.field_rules?.length) { | ||
| continue; | ||
| } | ||
| // Only content types carrying a global field rule need re-applying; the rest were fully | ||
| // updated (schema + their own rules) in updateCTs. | ||
| const hasGFFieldRule = contentType.field_rules.some((rule: any) => isGlobalFieldRule(rule)); | ||
| if (!hasGFFieldRule) { | ||
| continue; | ||
| } | ||
| log.info(`Re-applying global field rules for content type: ${contentType.uid}`, this.importConfig.context); | ||
| const contentTypeResponse: any = await this.stack | ||
| .contentType(contentType.uid) | ||
| .fetch() | ||
| .catch((error: unknown) => { | ||
| handleAndLogError(error, { ...this.importConfig.context, uid: contentType.uid }); | ||
| }); | ||
| if (!contentTypeResponse) { | ||
| log.debug( | ||
| `Skipping global field rules update for ${contentType.uid} - content type not found`, | ||
| this.importConfig.context, | ||
| ); | ||
| failedCTs.push(contentType.uid); | ||
| continue; | ||
| } | ||
| // Send the global field rules together with the content type's own non-reference rules, | ||
| // NOT the raw on-disk set. updateFieldRules(..., { keepGlobalFieldRules: true }) keeps the | ||
| // now-valid global field rules while still dropping reference-condition rules, which are | ||
| // owned by the entries module (it remaps their entry-uid values post entry-import). Sending | ||
| // the raw set here would resurrect those reference rules prematurely with stale uids. | ||
| // NOTE: field_rules is a whole-array PUT — if any single rule is invalid the API rejects the | ||
| // entire array, so a malformed rule would take the global field rules down with it. | ||
| contentTypeResponse.field_rules = updateFieldRules(contentType, { keepGlobalFieldRules: true }); | ||
| await contentTypeResponse | ||
| .update() | ||
| .then(() => { | ||
| log.success(`Re-applied global field rules for content type: ${contentType.uid}`, this.importConfig.context); | ||
| }) | ||
| .catch((error: Error) => { | ||
| handleAndLogError(error, { ...this.importConfig.context, uid: contentType.uid }); | ||
| failedCTs.push(contentType.uid); | ||
| }); | ||
| } | ||
| return failedCTs; | ||
| } | ||
| async seedCTs(): Promise<any> { | ||
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.