[4.x] Split global set variables into its own repository and stache store by ryanmitchell · Pull Request #8343 · statamic/cms

ryanmitchell · 2023-06-23T10:22:33Z

There seems to be a lot of people who want to split config from data when using eloquent driver, which is already mostly possible apart from forms and globals. Forms will be possible if/when this merges.

See for reference:
statamic/eloquent-driver#172
statamic/eloquent-driver#128
statamic/eloquent-driver#147
statamic/eloquent-driver#168

This PR splits global variables out into its own repository and store but maintains sharing the same directory as globals, so at a file-level nothing changes. All tests are passing locally which is a good sign that I haven't broken anything, and I was trying not to affect the existing structure in any way so made changes around it rather than simply adding a new directory for variables.

Could you let me know if this is something you would consider merging, and if so I'll spend some more time ensure the test coverage is up to scratch.

[Also, sorry. I know my PRs are always time consuming to review]

jasonvarga · 2023-06-23T15:18:40Z

Also, sorry. I know my PRs are always time consuming to review

The best ones are.

fdeneux · 2023-07-27T13:03:43Z

Thanks @ryanmitchell , our team is missing this feature. @jasonvarga any idea when your team will be able to review (and consider merging) this PR?

We work with multiple environments so we like to split config from data, with config being version controlled and data environment specific. We've managed to so for all stores except globals, meaning that our team is required to replicate config changes for globals on all environments.

ryanmitchell · 2023-07-27T14:18:54Z

If its critical to you then composer patch it in.

jasonvarga · 2023-08-01T15:01:05Z

I'm running into an issue while working on #8505 and I think this PR would help. So I'm working on this now. 👍

…tories

- Globals no longer hold the localizations in a property. They are retrieved on demand from the variables repo. - The GlobalSet class will save and delete localizations rather than the repo. It'll need to happen regardless of the repo implementation. - Make the globals repo dumber - the way it was. Let it just get the global sets from the store. Don't worry about the variables. - Call ->values() in findBySet. Since it filters, the keys could be whack.

…e. On single site, write the sets file.

…e will be one of something returned

ryanmitchell · 2023-08-08T17:59:17Z

Thank you. Did it end up helping fix the other issue (#8505)?

jasonvarga · 2023-08-08T18:57:16Z

Yep it did, and it's what I focused most on with this PR.

But right after merging this I realized this PR totally doesn't work when you have a single site and with the variables directory different from the globals. The variables store doesn't find the base file. Kinda had my multi-site blinders on while reviewing this.

I'm thinking that a simple temporary solution is to throw an exception when you're using the stache and have different directories defined for globals and global-variables.

ryanmitchell · 2023-08-08T19:04:30Z

Yeah that’s a good point. Never thought of trying different directories as the files are merged in that situation.

I think that’s a reasonable solution. Maybe in statamic 5 you always split the files even in single site mode?

ryanmitchell · 2023-06-23T10:22:33Z

There seems to be a lot of people who want to split config from data when using eloquent driver, which is already mostly possible apart from forms and globals. Forms will be possible if/when this merges.

See for reference:
statamic/eloquent-driver#172
statamic/eloquent-driver#128
statamic/eloquent-driver#147
statamic/eloquent-driver#168

This PR splits global variables out into its own repository and store but maintains sharing the same directory as globals, so at a file-level nothing changes. All tests are passing locally which is a good sign that I haven't broken anything, and I was trying not to affect the existing structure in any way so made changes around it rather than simply adding a new directory for variables.

Could you let me know if this is something you would consider merging, and if so I'll spend some more time ensure the test coverage is up to scratch.

[Also, sorry. I know my PRs are always time consuming to review]

jasonvarga · 2023-06-23T15:18:40Z

Also, sorry. I know my PRs are always time consuming to review

The best ones are.

fdeneux · 2023-07-27T13:03:43Z

Thanks @ryanmitchell , our team is missing this feature. @jasonvarga any idea when your team will be able to review (and consider merging) this PR?

We work with multiple environments so we like to split config from data, with config being version controlled and data environment specific. We've managed to so for all stores except globals, meaning that our team is required to replicate config changes for globals on all environments.

ryanmitchell · 2023-07-27T14:18:54Z

If its critical to you then composer patch it in.

jasonvarga · 2023-08-01T15:01:05Z

I'm running into an issue while working on #8505 and I think this PR would help. So I'm working on this now. 👍

…tories

- Globals no longer hold the localizations in a property. They are retrieved on demand from the variables repo. - The GlobalSet class will save and delete localizations rather than the repo. It'll need to happen regardless of the repo implementation. - Make the globals repo dumber - the way it was. Let it just get the global sets from the store. Don't worry about the variables. - Call ->values() in findBySet. Since it filters, the keys could be whack.

…e. On single site, write the sets file.

…e will be one of something returned

ryanmitchell · 2023-08-08T17:59:17Z

Thank you. Did it end up helping fix the other issue (#8505)?

jasonvarga · 2023-08-08T18:57:16Z

Yep it did, and it's what I focused most on with this PR.

But right after merging this I realized this PR totally doesn't work when you have a single site and with the variables directory different from the globals. The variables store doesn't find the base file. Kinda had my multi-site blinders on while reviewing this.

I'm thinking that a simple temporary solution is to throw an exception when you're using the stache and have different directories defined for globals and global-variables.

ryanmitchell · 2023-08-08T19:04:30Z

Yeah that’s a good point. Never thought of trying different directories as the files are merged in that situation.

I think that’s a reasonable solution. Maybe in statamic 5 you always split the files even in single site mode?

ryanmitchell · 2023-06-23T10:22:33Z

There seems to be a lot of people who want to split config from data when using eloquent driver, which is already mostly possible apart from forms and globals. Forms will be possible if/when this merges.

See for reference:
statamic/eloquent-driver#172
statamic/eloquent-driver#128
statamic/eloquent-driver#147
statamic/eloquent-driver#168

This PR splits global variables out into its own repository and store but maintains sharing the same directory as globals, so at a file-level nothing changes. All tests are passing locally which is a good sign that I haven't broken anything, and I was trying not to affect the existing structure in any way so made changes around it rather than simply adding a new directory for variables.

Could you let me know if this is something you would consider merging, and if so I'll spend some more time ensure the test coverage is up to scratch.

[Also, sorry. I know my PRs are always time consuming to review]

jasonvarga · 2023-06-23T15:18:40Z

Also, sorry. I know my PRs are always time consuming to review

The best ones are.

fdeneux · 2023-07-27T13:03:43Z

Thanks @ryanmitchell , our team is missing this feature. @jasonvarga any idea when your team will be able to review (and consider merging) this PR?

We work with multiple environments so we like to split config from data, with config being version controlled and data environment specific. We've managed to so for all stores except globals, meaning that our team is required to replicate config changes for globals on all environments.

ryanmitchell · 2023-07-27T14:18:54Z

If its critical to you then composer patch it in.

jasonvarga · 2023-08-01T15:01:05Z

I'm running into an issue while working on #8505 and I think this PR would help. So I'm working on this now. 👍

…tories

- Globals no longer hold the localizations in a property. They are retrieved on demand from the variables repo. - The GlobalSet class will save and delete localizations rather than the repo. It'll need to happen regardless of the repo implementation. - Make the globals repo dumber - the way it was. Let it just get the global sets from the store. Don't worry about the variables. - Call ->values() in findBySet. Since it filters, the keys could be whack.

…e. On single site, write the sets file.

…e will be one of something returned

ryanmitchell · 2023-08-08T17:59:17Z

Thank you. Did it end up helping fix the other issue (#8505)?

jasonvarga · 2023-08-08T18:57:16Z

Yep it did, and it's what I focused most on with this PR.

But right after merging this I realized this PR totally doesn't work when you have a single site and with the variables directory different from the globals. The variables store doesn't find the base file. Kinda had my multi-site blinders on while reviewing this.

I'm thinking that a simple temporary solution is to throw an exception when you're using the stache and have different directories defined for globals and global-variables.

ryanmitchell · 2023-08-08T19:04:30Z

Yeah that’s a good point. Never thought of trying different directories as the files are merged in that situation.

I think that’s a reasonable solution. Maybe in statamic 5 you always split the files even in single site mode?

ryanmitchell · 2023-06-23T10:22:33Z

There seems to be a lot of people who want to split config from data when using eloquent driver, which is already mostly possible apart from forms and globals. Forms will be possible if/when this merges.

See for reference:
statamic/eloquent-driver#172
statamic/eloquent-driver#128
statamic/eloquent-driver#147
statamic/eloquent-driver#168

This PR splits global variables out into its own repository and store but maintains sharing the same directory as globals, so at a file-level nothing changes. All tests are passing locally which is a good sign that I haven't broken anything, and I was trying not to affect the existing structure in any way so made changes around it rather than simply adding a new directory for variables.

Could you let me know if this is something you would consider merging, and if so I'll spend some more time ensure the test coverage is up to scratch.

[Also, sorry. I know my PRs are always time consuming to review]

jasonvarga · 2023-06-23T15:18:40Z

Also, sorry. I know my PRs are always time consuming to review

The best ones are.

fdeneux · 2023-07-27T13:03:43Z

Thanks @ryanmitchell , our team is missing this feature. @jasonvarga any idea when your team will be able to review (and consider merging) this PR?

We work with multiple environments so we like to split config from data, with config being version controlled and data environment specific. We've managed to so for all stores except globals, meaning that our team is required to replicate config changes for globals on all environments.

ryanmitchell · 2023-07-27T14:18:54Z

If its critical to you then composer patch it in.

jasonvarga · 2023-08-01T15:01:05Z

I'm running into an issue while working on #8505 and I think this PR would help. So I'm working on this now. 👍

…tories

- Globals no longer hold the localizations in a property. They are retrieved on demand from the variables repo. - The GlobalSet class will save and delete localizations rather than the repo. It'll need to happen regardless of the repo implementation. - Make the globals repo dumber - the way it was. Let it just get the global sets from the store. Don't worry about the variables. - Call ->values() in findBySet. Since it filters, the keys could be whack.

…e. On single site, write the sets file.

…e will be one of something returned

ryanmitchell · 2023-08-08T17:59:17Z

Thank you. Did it end up helping fix the other issue (#8505)?

jasonvarga · 2023-08-08T18:57:16Z

Yep it did, and it's what I focused most on with this PR.

But right after merging this I realized this PR totally doesn't work when you have a single site and with the variables directory different from the globals. The variables store doesn't find the base file. Kinda had my multi-site blinders on while reviewing this.

I'm thinking that a simple temporary solution is to throw an exception when you're using the stache and have different directories defined for globals and global-variables.

ryanmitchell · 2023-08-08T19:04:30Z

Yeah that’s a good point. Never thought of trying different directories as the files are merged in that situation.

I think that’s a reasonable solution. Maybe in statamic 5 you always split the files even in single site mode?

ryanmitchell · 2023-06-23T10:22:33Z

There seems to be a lot of people who want to split config from data when using eloquent driver, which is already mostly possible apart from forms and globals. Forms will be possible if/when this merges.

See for reference:
statamic/eloquent-driver#172
statamic/eloquent-driver#128
statamic/eloquent-driver#147
statamic/eloquent-driver#168

This PR splits global variables out into its own repository and store but maintains sharing the same directory as globals, so at a file-level nothing changes. All tests are passing locally which is a good sign that I haven't broken anything, and I was trying not to affect the existing structure in any way so made changes around it rather than simply adding a new directory for variables.

Could you let me know if this is something you would consider merging, and if so I'll spend some more time ensure the test coverage is up to scratch.

[Also, sorry. I know my PRs are always time consuming to review]

jasonvarga · 2023-06-23T15:18:40Z

Also, sorry. I know my PRs are always time consuming to review

The best ones are.

fdeneux · 2023-07-27T13:03:43Z

Thanks @ryanmitchell , our team is missing this feature. @jasonvarga any idea when your team will be able to review (and consider merging) this PR?

We work with multiple environments so we like to split config from data, with config being version controlled and data environment specific. We've managed to so for all stores except globals, meaning that our team is required to replicate config changes for globals on all environments.

ryanmitchell · 2023-07-27T14:18:54Z

If its critical to you then composer patch it in.

jasonvarga · 2023-08-01T15:01:05Z

I'm running into an issue while working on #8505 and I think this PR would help. So I'm working on this now. 👍

…tories

- Globals no longer hold the localizations in a property. They are retrieved on demand from the variables repo. - The GlobalSet class will save and delete localizations rather than the repo. It'll need to happen regardless of the repo implementation. - Make the globals repo dumber - the way it was. Let it just get the global sets from the store. Don't worry about the variables. - Call ->values() in findBySet. Since it filters, the keys could be whack.

…e. On single site, write the sets file.

…e will be one of something returned

ryanmitchell · 2023-08-08T17:59:17Z

Thank you. Did it end up helping fix the other issue (#8505)?

jasonvarga · 2023-08-08T18:57:16Z

Yep it did, and it's what I focused most on with this PR.

But right after merging this I realized this PR totally doesn't work when you have a single site and with the variables directory different from the globals. The variables store doesn't find the base file. Kinda had my multi-site blinders on while reviewing this.

I'm thinking that a simple temporary solution is to throw an exception when you're using the stache and have different directories defined for globals and global-variables.

ryanmitchell · 2023-08-08T19:04:30Z

Yeah that’s a good point. Never thought of trying different directories as the files are merged in that situation.

I think that’s a reasonable solution. Maybe in statamic 5 you always split the files even in single site mode?

ryanmitchell · 2023-06-23T10:22:33Z

There seems to be a lot of people who want to split config from data when using eloquent driver, which is already mostly possible apart from forms and globals. Forms will be possible if/when this merges.

See for reference:
statamic/eloquent-driver#172
statamic/eloquent-driver#128
statamic/eloquent-driver#147
statamic/eloquent-driver#168

This PR splits global variables out into its own repository and store but maintains sharing the same directory as globals, so at a file-level nothing changes. All tests are passing locally which is a good sign that I haven't broken anything, and I was trying not to affect the existing structure in any way so made changes around it rather than simply adding a new directory for variables.

Could you let me know if this is something you would consider merging, and if so I'll spend some more time ensure the test coverage is up to scratch.

[Also, sorry. I know my PRs are always time consuming to review]

jasonvarga · 2023-06-23T15:18:40Z

Also, sorry. I know my PRs are always time consuming to review

The best ones are.

fdeneux · 2023-07-27T13:03:43Z

Thanks @ryanmitchell , our team is missing this feature. @jasonvarga any idea when your team will be able to review (and consider merging) this PR?

We work with multiple environments so we like to split config from data, with config being version controlled and data environment specific. We've managed to so for all stores except globals, meaning that our team is required to replicate config changes for globals on all environments.

ryanmitchell · 2023-07-27T14:18:54Z

If its critical to you then composer patch it in.

jasonvarga · 2023-08-01T15:01:05Z

I'm running into an issue while working on #8505 and I think this PR would help. So I'm working on this now. 👍

…tories

- Globals no longer hold the localizations in a property. They are retrieved on demand from the variables repo. - The GlobalSet class will save and delete localizations rather than the repo. It'll need to happen regardless of the repo implementation. - Make the globals repo dumber - the way it was. Let it just get the global sets from the store. Don't worry about the variables. - Call ->values() in findBySet. Since it filters, the keys could be whack.

…e. On single site, write the sets file.

…e will be one of something returned

ryanmitchell · 2023-08-08T17:59:17Z

Thank you. Did it end up helping fix the other issue (#8505)?

jasonvarga · 2023-08-08T18:57:16Z

Yep it did, and it's what I focused most on with this PR.

But right after merging this I realized this PR totally doesn't work when you have a single site and with the variables directory different from the globals. The variables store doesn't find the base file. Kinda had my multi-site blinders on while reviewing this.

I'm thinking that a simple temporary solution is to throw an exception when you're using the stache and have different directories defined for globals and global-variables.

ryanmitchell · 2023-08-08T19:04:30Z

Yeah that’s a good point. Never thought of trying different directories as the files are merged in that situation.

I think that’s a reasonable solution. Maybe in statamic 5 you always split the files even in single site mode?

ryanmitchell · 2023-06-23T10:22:33Z

There seems to be a lot of people who want to split config from data when using eloquent driver, which is already mostly possible apart from forms and globals. Forms will be possible if/when this merges.

See for reference:
statamic/eloquent-driver#172
statamic/eloquent-driver#128
statamic/eloquent-driver#147
statamic/eloquent-driver#168

This PR splits global variables out into its own repository and store but maintains sharing the same directory as globals, so at a file-level nothing changes. All tests are passing locally which is a good sign that I haven't broken anything, and I was trying not to affect the existing structure in any way so made changes around it rather than simply adding a new directory for variables.

Could you let me know if this is something you would consider merging, and if so I'll spend some more time ensure the test coverage is up to scratch.

[Also, sorry. I know my PRs are always time consuming to review]

jasonvarga · 2023-06-23T15:18:40Z

Also, sorry. I know my PRs are always time consuming to review

The best ones are.

fdeneux · 2023-07-27T13:03:43Z

Thanks @ryanmitchell , our team is missing this feature. @jasonvarga any idea when your team will be able to review (and consider merging) this PR?

We work with multiple environments so we like to split config from data, with config being version controlled and data environment specific. We've managed to so for all stores except globals, meaning that our team is required to replicate config changes for globals on all environments.

ryanmitchell · 2023-07-27T14:18:54Z

If its critical to you then composer patch it in.

jasonvarga · 2023-08-01T15:01:05Z

I'm running into an issue while working on #8505 and I think this PR would help. So I'm working on this now. 👍

…tories

- Globals no longer hold the localizations in a property. They are retrieved on demand from the variables repo. - The GlobalSet class will save and delete localizations rather than the repo. It'll need to happen regardless of the repo implementation. - Make the globals repo dumber - the way it was. Let it just get the global sets from the store. Don't worry about the variables. - Call ->values() in findBySet. Since it filters, the keys could be whack.

…e. On single site, write the sets file.

…e will be one of something returned

ryanmitchell · 2023-08-08T17:59:17Z

Thank you. Did it end up helping fix the other issue (#8505)?

jasonvarga · 2023-08-08T18:57:16Z

Yep it did, and it's what I focused most on with this PR.

But right after merging this I realized this PR totally doesn't work when you have a single site and with the variables directory different from the globals. The variables store doesn't find the base file. Kinda had my multi-site blinders on while reviewing this.

I'm thinking that a simple temporary solution is to throw an exception when you're using the stache and have different directories defined for globals and global-variables.

ryanmitchell · 2023-08-08T19:04:30Z

Yeah that’s a good point. Never thought of trying different directories as the files are merged in that situation.

I think that’s a reasonable solution. Maybe in statamic 5 you always split the files even in single site mode?

ryanmitchell · 2023-06-23T10:22:33Z

There seems to be a lot of people who want to split config from data when using eloquent driver, which is already mostly possible apart from forms and globals. Forms will be possible if/when this merges.

See for reference:
statamic/eloquent-driver#172
statamic/eloquent-driver#128
statamic/eloquent-driver#147
statamic/eloquent-driver#168

This PR splits global variables out into its own repository and store but maintains sharing the same directory as globals, so at a file-level nothing changes. All tests are passing locally which is a good sign that I haven't broken anything, and I was trying not to affect the existing structure in any way so made changes around it rather than simply adding a new directory for variables.

Could you let me know if this is something you would consider merging, and if so I'll spend some more time ensure the test coverage is up to scratch.

[Also, sorry. I know my PRs are always time consuming to review]

jasonvarga · 2023-06-23T15:18:40Z

Also, sorry. I know my PRs are always time consuming to review

The best ones are.

fdeneux · 2023-07-27T13:03:43Z

Thanks @ryanmitchell , our team is missing this feature. @jasonvarga any idea when your team will be able to review (and consider merging) this PR?

We work with multiple environments so we like to split config from data, with config being version controlled and data environment specific. We've managed to so for all stores except globals, meaning that our team is required to replicate config changes for globals on all environments.

ryanmitchell · 2023-07-27T14:18:54Z

If its critical to you then composer patch it in.

jasonvarga · 2023-08-01T15:01:05Z

I'm running into an issue while working on #8505 and I think this PR would help. So I'm working on this now. 👍

…tories

- Globals no longer hold the localizations in a property. They are retrieved on demand from the variables repo. - The GlobalSet class will save and delete localizations rather than the repo. It'll need to happen regardless of the repo implementation. - Make the globals repo dumber - the way it was. Let it just get the global sets from the store. Don't worry about the variables. - Call ->values() in findBySet. Since it filters, the keys could be whack.

…e. On single site, write the sets file.

…e will be one of something returned

ryanmitchell · 2023-08-08T17:59:17Z

Thank you. Did it end up helping fix the other issue (#8505)?

jasonvarga · 2023-08-08T18:57:16Z

Yep it did, and it's what I focused most on with this PR.

But right after merging this I realized this PR totally doesn't work when you have a single site and with the variables directory different from the globals. The variables store doesn't find the base file. Kinda had my multi-site blinders on while reviewing this.

I'm thinking that a simple temporary solution is to throw an exception when you're using the stache and have different directories defined for globals and global-variables.

ryanmitchell · 2023-08-08T19:04:30Z

Yeah that’s a good point. Never thought of trying different directories as the files are merged in that situation.

I think that’s a reasonable solution. Maybe in statamic 5 you always split the files even in single site mode?

Split global set variables into its own repository and stache store
57718cd

ryanmitchell mentioned this pull request Jun 23, 2023
Restructure global variables to use its own table statamic/eloquent-driver#168
Merged

Lets only allow making variables via a GlobalSet to avoid orphaned data
9e1336c

ryanmitchell added 2 commits June 23, 2023 18:37

Refactor to avoid adding a new method
8f0b773

Use :: in id() rather than .
c788e4f

ryanmitchell mentioned this pull request Jun 24, 2023
Split globals and global variables, provide eloquent global variable repository statamic/eloquent-driver#178
Merged

ryanmitchell added 3 commits June 27, 2023 08:28

Add some tests
b271864

Remove saving test, cant get it working (even though it does!)
9f8823d

Yep
1cd1d4e

rename variable to variables
6305bf9

jasonvarga added 3 commits August 1, 2023 11:45

Merge branch '4.x' into feature/allow-split-globals-variables-resposi…
6bcef7d
…tories

Merge branch '4.x' into feature/allow-split-globals-variables-resposi…
b2da5b2
…tories

pint
20b8167

jasonvarga reviewed Aug 4, 2023
View reviewed changes

Comment threadsrc/Stache/Stores/GlobalVariablesStore.php Outdated

jasonvarga added 5 commits August 4, 2023 11:41

add multisite fixtures
67cb37f

Save variables all the time, but only write the file when on multisit…
0db4dfe
…e. On single site, write the sets file.

tests
8c3fc68

forgot each
5099289

ryanmitchell commented Aug 5, 2023
View reviewed changes

Comment threadsrc/Stache/Stores/GlobalVariablesStore.php

jasonvarga added 6 commits August 7, 2023 14:46

rename for consistency
659b722

rename method to whereSet, since find is generally used for when ther…
654c953
…e will be one of something returned

it'll return a variables collection
3b1cbb3

fix saving and deleting
f8b4ef4

plural
f0ca90a

only the "-ing" events should halt on non-nulls
45883b5

jasonvarga added 3 commits August 8, 2023 10:16

setter not necessary if it doesnt do anything special
aa622b3

unused method
3a349ba

nitpick
e3f2888

jasonvarga merged commit 424fc84 into statamic:4.xAug 8, 2023

ryanmitchell deleted the feature/allow-split-globals-variables-respositories branch August 8, 2023 17:59

jasonvarga mentioned this pull request Aug 8, 2023
[4.x] Prevent using different globals and variables directories #8552
Merged

robdekort mentioned this pull request Aug 10, 2023
GlobalSetSaved event doesn't trigger anymore since 4.16 #8559
Closed

jasonvarga mentioned this pull request Aug 10, 2023
[4.x] Fix globals save event #8564
Merged

ryanmitchell mentioned this pull request Aug 11, 2023
Fix console export now that repositories are split statamic/eloquent-driver#196
Merged

duncanmcclean mentioned this pull request Mar 17, 2025
[6.x] Separate globals config and content #11585
Merged