feat: New command `lang:sync` by neznaika0 · Pull Request #9023 · codeigniter4/CodeIgniter4

neznaika0 · 2024-07-05T17:49:58Z

Description
A simple command to copy existing translation files to a new locale.
From the file Language/en/Example.php:

<?phpreturn [
'title' => 'Default title',
'status' => [
'error' => 'Error!',
'done' => 'Done!',
'critical' => 'Critical!',
],
'nullableKey' => null,
'numericKey' => 100500,
'more' => [
'nested' => [
'key' => 'More nested key...',
],
],
];

After the command php spark lang:sync --locale en --target ru we get Language/ru/Example.php .

<?phpreturnarray (
'more' => array (
'nested' => array (
'key' => 'Example.more.nested.key',
),
),
'nullableKey' => 'Example.nullableKey',
'numericKey' => 'Example.numericKey',
'status' => array (
'error' => 'Example.status.error',
'done' => 'Example.status.done',
'critical' => 'Example.status.critical',
),
'title' => 'Example.title',
);

applied cs-fix:

<?phpreturn [
'more' => [
'nested' => [
'key' => 'Example.more.nested.key',
],
],
'nullableKey' => 'Example.nullableKey',
'numericKey' => 'Example.numericKey',
'status' => [
'error' => 'Example.status.error',
'done' => 'Example.status.done',
'critical' => 'Example.status.critical',
],
'title' => 'Example.title',
];

And so it is with all other translations

=====

When there are translations in the target folder Language/ru, their keys are saved only if they exist in the Language/en folder:

Language/ru/Example.php:

<?phpreturn [
// Deleted keys// 'more' => [// 'nested' => [// 'key' => 'Example.more.nested.key',// ],// ],'nullableKey' => null,
'numericKey' => 100000,
'status' => [
'error' => 'ru.status.error',
'done' => 'ru.status.done',
'critical' => 'ru.status.critical',
],
'title' => 'ru.title',
'mistakeKey' => 'ru.mistake_key',
];

Language/ru/Example.php after sync:

<?phpreturn [
'more' => [
'nested' => [
'key' => 'Example.more.nested.key',
],
],
'nullableKey' => null,
'numericKey' => 100000,
'status' => [
'error' => 'ru.status.error',
'done' => 'ru.status.done',
'critical' => 'ru.status.critical',
],
'title' => 'ru.title',
];

Checklist:

Securely signed commits
Component(s) with PHPDoc blocks, only if necessary or adds value
Unit testing, with >80% coverage
User guide updated
Conforms to style guide

neznaika0 · 2024-07-05T17:51:14Z

There are some questions:

Does anyone else need it? 😄
I have indicated the reason for using it as a more precise translation control.
I need help with the documentation. I don't know English
Formatting new arrays requires the use of php-cs-fixer. This was discussed in another PR feat: Language translations finder and update #7896 (comment) . How can we take this out for general use?
phpstan. I don't quite understand what he wants from me. I have specified a common type for the values

neznaika0 · 2024-07-05T18:00:39Z

Userguide:

Changelog:

kenjis · 2024-08-01T02:21:55Z

I don't get the description "When there are translations in the target folder Language/ru, their keys are saved only if they exist in the Language/en folder:". Is that correct?

If app/Language/en/Example.php is:

return [
// Deleted keys// 'more' => [// 'nested' => [// 'key' => 'Example.more.nested.key',// ],// ],'nullableKey' => null,
'numericKey' => 100000,
'status' => [
'error' => 'ru.status.error',
'done' => 'ru.status.done',
'critical' => 'ru.status.critical',
],
'title' => 'ru.title',
'mistakeKey' => 'ru.mistake_key',
];

Then, app/Language/ru/Example.php will be:

return [
'mistakeKey' => 'Example.mistakeKey',
'nullableKey' => 'Example.nullableKey',
'numericKey' => 'Example.numericKey',
'status' => [
'error' => 'Example.status.error',
'done' => 'Example.status.done',
'critical' => 'Example.status.critical',
],
'title' => 'Example.title',
];

That is, deleted keys are gone.

kenjis · 2024-08-01T02:26:50Z

If app/Language/en/Example.php is:

return [
'title' => 'Default title',
'status' => [
'error' => 'Error!',
'done' => 'Done!',
'critical' => 'Critical!',
],
'nullableKey' => null,
'numericKey' => 100500,
'more' => [
'nested' => [
'key' => 'More nested key...',
],
],
];

Then, app/Language/ru/Example.php will be:

return [
'more' => [
'nested' => [
'key' => 'Example.more.nested.key',
],
],
'nullableKey' => 'Example.nullableKey',
'numericKey' => 'Example.numericKey',
'status' => [
'error' => 'Example.status.error',
'done' => 'Example.status.done',
'critical' => 'Example.status.critical',
],
'title' => 'Example.title',
];

That is, the key order changed.
This makes comparison of the two files, original and translated, difficult.

kenjis · 2024-08-01T02:47:17Z

This command would be useful.
However, I would like the order of the keys to match the original.

neznaika0 · 2024-08-01T05:12:09Z

I don't get the description "When there are translations in the target folder Language/ru, their keys are saved only if they exist in the Language/en folder:". Is that correct?
If app/Language/en/Example.php is:
...

No. Look at the order of the files. We from EN combine RU. There are translations in RU ru.title, so the keys are saved and not overwritten Example.title. Old keys remain.

That is, the key order changed.
This makes comparison of the two files, original and translated, difficult.

Yes. The order has changed. I'm sorting a new array. I think it can be fixed. But the keys will be chaotic when combined.

kenjis · 2024-08-02T07:28:29Z

This PR branch is too old. Can you rebase?

kenjis · 2024-08-02T07:34:49Z

No. Look at the order of the files. We from EN combine RU. There are translations in RU ru.title, so the keys are saved and not overwritten Example.title. Old keys remain.

Okay, I got what you say.

app/Language/en/Example.php:

return [
'title' => 'Default title',
'status' => [
'error' => 'Error!',
'done' => 'Done!',
'critical' => 'Critical!',
],
'nullableKey' => null,
'numericKey' => 100500,
'more' => [
'nested' => [
'key' => 'More nested key...',
],
],
];

app/Language/ru/Example.php:

<?phpreturn [
// Deleted keys// 'more' => [// 'nested' => [// 'key' => 'Example.more.nested.key',// ],// ],'nullableKey' => null,
'numericKey' => 100000,
'status' => [
'error' => 'ru.status.error',
'done' => 'ru.status.done',
'critical' => 'ru.status.critical',
],
'title' => 'ru.title',
'mistakeKey' => 'ru.mistake_key',
];

$ php spark lang:sync --locale en --target ru
$ composer cs-fix

app/Language/ru/Example.php:

return [
'more' => [
'nested' => [
'key' => 'Example.more.nested.key',
],
],
'nullableKey' => null,
'numericKey' => 100000,
'status' => [
'error' => 'ru.status.error',
'done' => 'ru.status.done',
'critical' => 'ru.status.critical',
],
'title' => 'ru.title',
];

kenjis · 2024-08-02T08:04:30Z

Yes. The order has changed. I'm sorting a new array. I think it can be fixed. But the keys will be chaotic when combined.

I think the key order should not be changed.

Can't you rewrite with recursive calls?
This is PoC sample code by ChatGPT:

<?phpfunctionsyncTranslationsRecursive($original, $translations) {
// Initialize an array to store the results$syncedTranslations = [];
// Loop through the original message arrayforeach ($originalas$key => $message) {
if (is_array($message)) {
// If the message is an array, recursively synchronize it$syncedTranslations[$key] = syncTranslationsRecursive(
$message,
array_key_exists($key, $translations) ? $translations[$key] : []
);
} else {
// If the message is not an array, check if the translations array contains the same keyif (array_key_exists($key, $translations)) {
$syncedTranslations[$key] = $translations[$key];
} else {
// If the key doesn't exist in the translations array, set an empty string$syncedTranslations[$key] = '';
}
}
}
return$syncedTranslations;
}
// Original message array$original = [
'title' => 'Default title',
'status' => [
'error' => 'Error!',
'done' => 'Done!',
'critical' => 'Critical!',
],
'more' => [
'nested' => [
'key' => 'More nested key...',
],
],
];
// Translated message array (partially translated)$translations = [
'title' => 'Título predeterminado',
'status' => [
'error' => '¡Error!',
'done' => '¡Hecho!',
],
];
// Synchronize the translations array with the original message array$syncedTranslations = syncTranslationsRecursive($original, $translations);
// Print the resultsprint_r($syncedTranslations);

neznaika0 · 2024-08-02T10:14:35Z

What is the disadvantage of the current array merge?

https://github.com/neznaika0/codeigniter-dev/blob/19f9440bd9deb5131079ff427781f6f14de99584/system/Helpers/Array/ArrayHelper.php#L327-L341

I can just remove the sorting and everything looks like you wanted.
https://github.com/neznaika0/codeigniter-dev/blob/19f9440bd9deb5131079ff427781f6f14de99584/system/Commands/Translation/LocalizationSync.php#L136-L137

But, if the target file has keys, then they are added to the end.

kenjis · 2024-08-02T11:05:08Z

I would like the key order is never changed in any case.
If so, I don't care about the implementation.

neznaika0 · 2024-08-05T05:46:47Z

Results based on the data above:

// translation RU is emptyreturn [
'title' => 'Example.title',
'status' => [
'error' => 'Example.status.error',
'done' => 'Example.status.done',
'critical' => 'Example.status.critical',
],
'nullableKey' => 'Example.nullableKey',
'numericKey' => 'Example.numericKey',
'more' => [
'nested' => [
'key' => 'Example.more.nested.key',
],
],
];

// translation RU existreturn [
'nullableKey' => null,
'numericKey' => 100000,
'status' => [
'error' => 'ru.status.error',
'done' => 'ru.status.done',
'critical' => 'ru.status.critical',
],
'title' => 'ru.title',
'more' => [
'nested' => [
'key' => 'Example.more.nested.key',
],
],
];

kenjis · 2024-08-05T07:11:58Z

When translation RU exist, the key order will be different from EN.
In any case, the key order should be the same in all lang files.

neznaika0 · 2024-08-08T14:56:15Z

// Before: app/Language/en/Example.phpreturn [
'title' => 'Default title',
'status' => [
'error' => 'Error!',
'done' => 'Done!',
'critical' => 'Critical!',
],
'nullableKey' => null,
'array' => [],
'numericKey' => 100500,
'more' => [
'nested' => [
'key' => 'More nested key...',
],
],
];

// Before: app/Language/ru/Example.phpreturn [
// Deleted keys// 'more' => [// 'nested' => [// 'key' => 'Example.more.nested.key',// ],// ],'nullableKey' => null,
'status' => [
'critical' => 'ru.status.critical',
'done' => 'ru.status.done',
'error' => 'ru.status.error',
],
'mistakeKey' => 'ru.mistake_key',
'numericKey' => 100000,
'title' => 'ru.title',
];

// After: app/Language/ru/Example.php// 1. Right sort as in EN// 2. Delete 'mistakeKey' (in EN language not exist)// 3. Untranslated keys have a placeholder ''Example.more.nested.key'// 4. Previously translated keys are savedreturn [
'title' => 'ru.title',
'status' => [
'error' => 'ru.status.error',
'done' => 'ru.status.done',
'critical' => 'ru.status.critical',
],
'nullableKey' => null,
'array' => [
],
'numericKey' => 100000,
'more' => [
'nested' => [
'key' => 'Example.more.nested.key',
],
],
];

The array intersection search function ArrayHelper::intersectKeyRecursive is no longer needed.
Combining arrays should now initially not have extra keys (current mistakeKey) from the target array.

datamweb · 2024-08-09T00:29:25Z

I prefer the following result. To make it easier for the developer to translate the file.

// After: app/Language/ru/Example.phpreturn [
'title' => '(To be translated) Default title',
'status' => [
'error' => '(To be translated) Error!',
'done' => '(To be translated) Done!',
'critical' => '(To be translated) Critical!',
],
'nullableKey' => null,
'array' => [
],
'numericKey' => 100000,
'more' => [
'nested' => [
'key' => '(To be translated) More nested key...',
],
],
];

neznaika0 · 2024-08-09T05:46:41Z

@datamweb I think it's wrong. If the current locale is RU and the translations are the same (EN), it is unclear on the page whether the translation was made or not. Because the fallback translation is EN.

Additional text interferes with the preview of the page (too long for some places).

Placeholders clearly show untranslated lines.

It is possible to make it configurable --replace-old

michalsn · 2024-12-21T17:23:43Z

Okay, what is the status of this feature?

It seems like the order of the keys is updated.

Personally, I don't think we need the (To be translated) part as differences in languages are usually quite obvious.

Are there any remaining issues?

neznaika0 · 2024-12-21T18:30:11Z

I'll update the PR in a week. From the corrections: Checking the value is a string. In general, we can abandon PR, since I have a separate repository for lang:find. Synchronization can be moved there.

michalsn · 2024-12-21T18:59:11Z

In general, we can abandon PR, since I have a separate repository for lang:find. Synchronization can be moved there.

So you would rather add this function to the existing lang:find command? Do I understand this correctly?

neznaika0 · 2024-12-22T09:48:03Z

If this command is not needed in CI4, I can add it to the optional package.
The command is separate from lang:find, but in the same group Translation

neznaika0 · 2025-01-10T07:22:21Z

Ready.

updated igroring files - сhecking extension is enough
added EXIT_ and process() returns an int
added new test for checking language folders
ErrorException is added because the bool for mkdir() is unreliable

neznaika0 · 2025-01-11T09:58:34Z

Thanks to everyone who helps.

samsonasik · 2025-01-11T15:02:23Z

Thank you @neznaika0

ddevsr added the 4.6 label Jul 6, 2024

kenjis added the enhancement PRs that improve existing functionalities label Jul 6, 2024

kenjis reviewed Aug 1, 2024
View reviewed changes

Comment threaduser_guide_src/source/outgoing/localization.rst Outdated

datamweb reviewed Aug 1, 2024
View reviewed changes

Comment threaduser_guide_src/source/outgoing/localization.rst Outdated

neznaika0 force-pushed the feat-lang-sync branch from cbd2d0f to 19f9440Compare August 2, 2024 09:49

neznaika0 force-pushed the feat-lang-sync branch from 19f9440 to cf93a62Compare August 5, 2024 05:37

neznaika0 force-pushed the feat-lang-sync branch from cf93a62 to 6bc2c49Compare August 8, 2024 14:55

kenjis reviewed Aug 9, 2024
View reviewed changes

Comment threadtests/system/Commands/Translation/LocalizationSyncTest.php Outdated

neznaika0 force-pushed the feat-lang-sync branch from 6bc2c49 to 435d8d6Compare November 4, 2024 07:09

neznaika0 force-pushed the feat-lang-sync branch from 435d8d6 to 6872fa5Compare December 26, 2024 19:14

michalsn requested a review from paulbalandan January 8, 2025 08:16

samsonasik approved these changes Jan 8, 2025
View reviewed changes

paulbalandan reviewed Jan 10, 2025
View reviewed changes

Comment threadsystem/Commands/Translation/LocalizationSync.php Outdated
Comment threadsystem/Commands/Translation/LocalizationSync.php Outdated
Comment threadsystem/Commands/Translation/LocalizationSync.php Outdated

neznaika0 added 17 commits January 10, 2025 08:33

feat: Add lang:sync command
3fa7394

test: Add tests for lang:sync command
872434a

feat: Add array helper intersectKeyRecursive
49d7a87

docs: Add changelog
4ffb69c

docs: Add description for command
96d640d

fix: Improve condition
7867ecb

fix: docs title underline too short
042dbea

fix: typo
b57def9

fix: Delete debug info
5448d7a

fix: Remove sorting lang keys
8933d3f

fix: key sorting
b29462b

Remove array helper intersectKeyRecursive
d23c879

refactor: Check type translated value
5b90b6f

fix: Update tests to be readable
9c92041

fix: Update changelog
76cd73f

fix: Typo
cd84a6e

refactor: Rework exit codes, skip dots
47a6f8a

neznaika0 force-pushed the feat-lang-sync branch from 20cdcb7 to 47a6f8aCompare January 10, 2025 07:17

paulbalandan reviewed Jan 10, 2025
View reviewed changes

Comment threadsystem/Commands/Translation/LocalizationSync.php Outdated
Comment threadsystem/Commands/Translation/LocalizationSync.php

refactor: Accurate command finishing
1eae834

samsonasik requested a review from paulbalandan January 11, 2025 11:23

paulbalandan approved these changes Jan 11, 2025
View reviewed changes

samsonasik merged commit d2c86f2 into codeigniter4:4.6Jan 11, 2025

neznaika0 deleted the feat-lang-sync branch January 11, 2025 15:27

Uh oh!

Conversation

neznaika0 commented Jul 5, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

neznaika0 commented Jul 5, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

neznaika0 commented Jul 5, 2024

Uh oh!

kenjis commented Aug 1, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kenjis commented Aug 1, 2024

Uh oh!

Uh oh!

Uh oh!

kenjis commented Aug 1, 2024

Uh oh!

neznaika0 commented Aug 1, 2024

Uh oh!

kenjis commented Aug 2, 2024

Uh oh!

kenjis commented Aug 2, 2024

Uh oh!

kenjis commented Aug 2, 2024

Uh oh!

neznaika0 commented Aug 2, 2024

Uh oh!

kenjis commented Aug 2, 2024

Uh oh!

neznaika0 commented Aug 5, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kenjis commented Aug 5, 2024

Uh oh!

neznaika0 commented Aug 8, 2024

Uh oh!

datamweb commented Aug 9, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

neznaika0 commented Aug 9, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

michalsn commented Dec 21, 2024

Uh oh!

neznaika0 commented Dec 21, 2024

Uh oh!

michalsn commented Dec 21, 2024

Uh oh!

neznaika0 commented Dec 22, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

neznaika0 commented Jan 10, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

neznaika0 commented Jan 11, 2025

Uh oh!

samsonasik commented Jan 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

neznaika0 commented Jul 5, 2024•
edited
Loading

neznaika0 commented Jul 5, 2024•
edited
Loading

kenjis commented Aug 1, 2024•
edited
Loading

neznaika0 commented Aug 5, 2024•
edited
Loading

datamweb commented Aug 9, 2024•
edited
Loading

neznaika0 commented Aug 9, 2024•
edited
Loading

neznaika0 commented Jan 10, 2025•
edited
Loading