[5.x] Adjust behavior of array fields

[duncanmcclean]

This pull request makes some changes to how Array fields are stored, to fix some long-standing issues.

Currently, Array fields are stored like this, in a simple key/value array:

foo: Foobar: Barbaz: Baz

While this format works well most of the time, everything falls apart when you try to use integers or floats as keys due to how arrays work in YAML. 🫠

To workaround this, we've decided to inflate the key/values like this:

-
key: foovalue: Foo
-
key: barvalue: Bar
-
key: bazvalue: Baz

The Array Fieldtype will continue being able to read from the legacy format, then, when you save the array, it'll convert the field to the new format.

Fixes#3179.
Fixes#2215.
Fixes#5969.

[jasonvarga]

This unfortunately all breaks down when you want to use an array field in your content.

You can no longer do {{ array_field:keyname }} in your templates anymore.

I think maybe we only save the expanded syntax when the fieldtype is being used for config, if possible.

[ryanmitchell]

With the changes will this still close statamic/eloquent-driver#312 ? I was hoping this PR would consistently store everything as key/value but this doesn't seem to be the case any more?

[jasonvarga]

Bummer. I was trying to avoid changing data where possible. It's sort of a breaking change.

If people were doing something like $entry->get('array_field') and expecting it to be ['foo' => 'bar', 'baz' => 'qux'] it will now be [['key' => 'foo', 'value' => 'bar'], ['key' => 'baz', 'value' => 'qux']] if they resave the entry.

I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

[ryanmitchell]

`I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

I was selfishly hoping this would be enough but you're right its breaking so shouldnt be done at this point in the cycle.

I'll get back to work on the eloquent PR.

[jasonvarga]

Maybe we make the new format opt-in per field. Then opt in each config field.

In v6 we could remove the option and make it the only behavior.

[jasonvarga]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

[ryanmitchell]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

Totally, its absolute nonsense!

[jasonvarga]

@ryanmitchell This fixes your issue. Reordering options of a select field will use the order you defined, in MySQL. 👍

[ryanmitchell]

Thank you

[duncanmcclean]

This pull request makes some changes to how Array fields are stored, to fix some long-standing issues.

Currently, Array fields are stored like this, in a simple key/value array:

foo: Foobar: Barbaz: Baz

While this format works well most of the time, everything falls apart when you try to use integers or floats as keys due to how arrays work in YAML. 🫠

To workaround this, we've decided to inflate the key/values like this:

-
key: foovalue: Foo
-
key: barvalue: Bar
-
key: bazvalue: Baz

The Array Fieldtype will continue being able to read from the legacy format, then, when you save the array, it'll convert the field to the new format.

Fixes#3179.
Fixes#2215.
Fixes#5969.

[jasonvarga]

This unfortunately all breaks down when you want to use an array field in your content.

You can no longer do {{ array_field:keyname }} in your templates anymore.

I think maybe we only save the expanded syntax when the fieldtype is being used for config, if possible.

[ryanmitchell]

With the changes will this still close statamic/eloquent-driver#312 ? I was hoping this PR would consistently store everything as key/value but this doesn't seem to be the case any more?

[jasonvarga]

Bummer. I was trying to avoid changing data where possible. It's sort of a breaking change.

If people were doing something like $entry->get('array_field') and expecting it to be ['foo' => 'bar', 'baz' => 'qux'] it will now be [['key' => 'foo', 'value' => 'bar'], ['key' => 'baz', 'value' => 'qux']] if they resave the entry.

I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

[ryanmitchell]

`I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

I was selfishly hoping this would be enough but you're right its breaking so shouldnt be done at this point in the cycle.

I'll get back to work on the eloquent PR.

[jasonvarga]

Maybe we make the new format opt-in per field. Then opt in each config field.

In v6 we could remove the option and make it the only behavior.

[jasonvarga]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

[ryanmitchell]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

Totally, its absolute nonsense!

[jasonvarga]

@ryanmitchell This fixes your issue. Reordering options of a select field will use the order you defined, in MySQL. 👍

[ryanmitchell]

Thank you

[duncanmcclean]

This pull request makes some changes to how Array fields are stored, to fix some long-standing issues.

Currently, Array fields are stored like this, in a simple key/value array:

foo: Foobar: Barbaz: Baz

While this format works well most of the time, everything falls apart when you try to use integers or floats as keys due to how arrays work in YAML. 🫠

To workaround this, we've decided to inflate the key/values like this:

-
key: foovalue: Foo
-
key: barvalue: Bar
-
key: bazvalue: Baz

The Array Fieldtype will continue being able to read from the legacy format, then, when you save the array, it'll convert the field to the new format.

Fixes#3179.
Fixes#2215.
Fixes#5969.

[jasonvarga]

This unfortunately all breaks down when you want to use an array field in your content.

You can no longer do {{ array_field:keyname }} in your templates anymore.

I think maybe we only save the expanded syntax when the fieldtype is being used for config, if possible.

[ryanmitchell]

With the changes will this still close statamic/eloquent-driver#312 ? I was hoping this PR would consistently store everything as key/value but this doesn't seem to be the case any more?

[jasonvarga]

Bummer. I was trying to avoid changing data where possible. It's sort of a breaking change.

If people were doing something like $entry->get('array_field') and expecting it to be ['foo' => 'bar', 'baz' => 'qux'] it will now be [['key' => 'foo', 'value' => 'bar'], ['key' => 'baz', 'value' => 'qux']] if they resave the entry.

I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

[ryanmitchell]

`I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

I was selfishly hoping this would be enough but you're right its breaking so shouldnt be done at this point in the cycle.

I'll get back to work on the eloquent PR.

[jasonvarga]

Maybe we make the new format opt-in per field. Then opt in each config field.

In v6 we could remove the option and make it the only behavior.

[jasonvarga]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

[ryanmitchell]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

Totally, its absolute nonsense!

[jasonvarga]

@ryanmitchell This fixes your issue. Reordering options of a select field will use the order you defined, in MySQL. 👍

[ryanmitchell]

Thank you

[duncanmcclean]

This pull request makes some changes to how Array fields are stored, to fix some long-standing issues.

Currently, Array fields are stored like this, in a simple key/value array:

foo: Foobar: Barbaz: Baz

While this format works well most of the time, everything falls apart when you try to use integers or floats as keys due to how arrays work in YAML. 🫠

To workaround this, we've decided to inflate the key/values like this:

-
key: foovalue: Foo
-
key: barvalue: Bar
-
key: bazvalue: Baz

The Array Fieldtype will continue being able to read from the legacy format, then, when you save the array, it'll convert the field to the new format.

Fixes#3179.
Fixes#2215.
Fixes#5969.

[jasonvarga]

This unfortunately all breaks down when you want to use an array field in your content.

You can no longer do {{ array_field:keyname }} in your templates anymore.

I think maybe we only save the expanded syntax when the fieldtype is being used for config, if possible.

[ryanmitchell]

With the changes will this still close statamic/eloquent-driver#312 ? I was hoping this PR would consistently store everything as key/value but this doesn't seem to be the case any more?

[jasonvarga]

Bummer. I was trying to avoid changing data where possible. It's sort of a breaking change.

If people were doing something like $entry->get('array_field') and expecting it to be ['foo' => 'bar', 'baz' => 'qux'] it will now be [['key' => 'foo', 'value' => 'bar'], ['key' => 'baz', 'value' => 'qux']] if they resave the entry.

I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

[ryanmitchell]

`I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

I was selfishly hoping this would be enough but you're right its breaking so shouldnt be done at this point in the cycle.

I'll get back to work on the eloquent PR.

[jasonvarga]

Maybe we make the new format opt-in per field. Then opt in each config field.

In v6 we could remove the option and make it the only behavior.

[jasonvarga]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

[ryanmitchell]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

Totally, its absolute nonsense!

[jasonvarga]

@ryanmitchell This fixes your issue. Reordering options of a select field will use the order you defined, in MySQL. 👍

[ryanmitchell]

Thank you

[duncanmcclean]

This pull request makes some changes to how Array fields are stored, to fix some long-standing issues.

Currently, Array fields are stored like this, in a simple key/value array:

foo: Foobar: Barbaz: Baz

While this format works well most of the time, everything falls apart when you try to use integers or floats as keys due to how arrays work in YAML. 🫠

To workaround this, we've decided to inflate the key/values like this:

-
key: foovalue: Foo
-
key: barvalue: Bar
-
key: bazvalue: Baz

The Array Fieldtype will continue being able to read from the legacy format, then, when you save the array, it'll convert the field to the new format.

Fixes#3179.
Fixes#2215.
Fixes#5969.

[jasonvarga]

This unfortunately all breaks down when you want to use an array field in your content.

You can no longer do {{ array_field:keyname }} in your templates anymore.

I think maybe we only save the expanded syntax when the fieldtype is being used for config, if possible.

[ryanmitchell]

With the changes will this still close statamic/eloquent-driver#312 ? I was hoping this PR would consistently store everything as key/value but this doesn't seem to be the case any more?

[jasonvarga]

Bummer. I was trying to avoid changing data where possible. It's sort of a breaking change.

If people were doing something like $entry->get('array_field') and expecting it to be ['foo' => 'bar', 'baz' => 'qux'] it will now be [['key' => 'foo', 'value' => 'bar'], ['key' => 'baz', 'value' => 'qux']] if they resave the entry.

I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

[ryanmitchell]

`I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

I was selfishly hoping this would be enough but you're right its breaking so shouldnt be done at this point in the cycle.

I'll get back to work on the eloquent PR.

[jasonvarga]

Maybe we make the new format opt-in per field. Then opt in each config field.

In v6 we could remove the option and make it the only behavior.

[jasonvarga]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

[ryanmitchell]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

Totally, its absolute nonsense!

[jasonvarga]

@ryanmitchell This fixes your issue. Reordering options of a select field will use the order you defined, in MySQL. 👍

[ryanmitchell]

Thank you

[duncanmcclean]

This pull request makes some changes to how Array fields are stored, to fix some long-standing issues.

Currently, Array fields are stored like this, in a simple key/value array:

foo: Foobar: Barbaz: Baz

While this format works well most of the time, everything falls apart when you try to use integers or floats as keys due to how arrays work in YAML. 🫠

To workaround this, we've decided to inflate the key/values like this:

-
key: foovalue: Foo
-
key: barvalue: Bar
-
key: bazvalue: Baz

The Array Fieldtype will continue being able to read from the legacy format, then, when you save the array, it'll convert the field to the new format.

Fixes#3179.
Fixes#2215.
Fixes#5969.

[jasonvarga]

This unfortunately all breaks down when you want to use an array field in your content.

You can no longer do {{ array_field:keyname }} in your templates anymore.

I think maybe we only save the expanded syntax when the fieldtype is being used for config, if possible.

[ryanmitchell]

With the changes will this still close statamic/eloquent-driver#312 ? I was hoping this PR would consistently store everything as key/value but this doesn't seem to be the case any more?

[jasonvarga]

Bummer. I was trying to avoid changing data where possible. It's sort of a breaking change.

If people were doing something like $entry->get('array_field') and expecting it to be ['foo' => 'bar', 'baz' => 'qux'] it will now be [['key' => 'foo', 'value' => 'bar'], ['key' => 'baz', 'value' => 'qux']] if they resave the entry.

I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

[ryanmitchell]

`I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

I was selfishly hoping this would be enough but you're right its breaking so shouldnt be done at this point in the cycle.

I'll get back to work on the eloquent PR.

[jasonvarga]

Maybe we make the new format opt-in per field. Then opt in each config field.

In v6 we could remove the option and make it the only behavior.

[jasonvarga]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

[ryanmitchell]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

Totally, its absolute nonsense!

[jasonvarga]

@ryanmitchell This fixes your issue. Reordering options of a select field will use the order you defined, in MySQL. 👍

[ryanmitchell]

Thank you

[duncanmcclean]

This pull request makes some changes to how Array fields are stored, to fix some long-standing issues.

Currently, Array fields are stored like this, in a simple key/value array:

foo: Foobar: Barbaz: Baz

While this format works well most of the time, everything falls apart when you try to use integers or floats as keys due to how arrays work in YAML. 🫠

To workaround this, we've decided to inflate the key/values like this:

-
key: foovalue: Foo
-
key: barvalue: Bar
-
key: bazvalue: Baz

The Array Fieldtype will continue being able to read from the legacy format, then, when you save the array, it'll convert the field to the new format.

Fixes#3179.
Fixes#2215.
Fixes#5969.

[jasonvarga]

This unfortunately all breaks down when you want to use an array field in your content.

You can no longer do {{ array_field:keyname }} in your templates anymore.

I think maybe we only save the expanded syntax when the fieldtype is being used for config, if possible.

[ryanmitchell]

With the changes will this still close statamic/eloquent-driver#312 ? I was hoping this PR would consistently store everything as key/value but this doesn't seem to be the case any more?

[jasonvarga]

Bummer. I was trying to avoid changing data where possible. It's sort of a breaking change.

If people were doing something like $entry->get('array_field') and expecting it to be ['foo' => 'bar', 'baz' => 'qux'] it will now be [['key' => 'foo', 'value' => 'bar'], ['key' => 'baz', 'value' => 'qux']] if they resave the entry.

I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

[ryanmitchell]

`I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

I was selfishly hoping this would be enough but you're right its breaking so shouldnt be done at this point in the cycle.

I'll get back to work on the eloquent PR.

[jasonvarga]

Maybe we make the new format opt-in per field. Then opt in each config field.

In v6 we could remove the option and make it the only behavior.

[jasonvarga]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

[ryanmitchell]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

Totally, its absolute nonsense!

[jasonvarga]

@ryanmitchell This fixes your issue. Reordering options of a select field will use the order you defined, in MySQL. 👍

[ryanmitchell]

Thank you

[duncanmcclean]

This pull request makes some changes to how Array fields are stored, to fix some long-standing issues.

Currently, Array fields are stored like this, in a simple key/value array:

foo: Foobar: Barbaz: Baz

While this format works well most of the time, everything falls apart when you try to use integers or floats as keys due to how arrays work in YAML. 🫠

To workaround this, we've decided to inflate the key/values like this:

-
key: foovalue: Foo
-
key: barvalue: Bar
-
key: bazvalue: Baz

The Array Fieldtype will continue being able to read from the legacy format, then, when you save the array, it'll convert the field to the new format.

Fixes#3179.
Fixes#2215.
Fixes#5969.

[jasonvarga]

This unfortunately all breaks down when you want to use an array field in your content.

You can no longer do {{ array_field:keyname }} in your templates anymore.

I think maybe we only save the expanded syntax when the fieldtype is being used for config, if possible.

[ryanmitchell]

With the changes will this still close statamic/eloquent-driver#312 ? I was hoping this PR would consistently store everything as key/value but this doesn't seem to be the case any more?

[jasonvarga]

Bummer. I was trying to avoid changing data where possible. It's sort of a breaking change.

If people were doing something like $entry->get('array_field') and expecting it to be ['foo' => 'bar', 'baz' => 'qux'] it will now be [['key' => 'foo', 'value' => 'bar'], ['key' => 'baz', 'value' => 'qux']] if they resave the entry.

I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

[ryanmitchell]

`I can make the augmented version output the old way, that part is fine though. e.g. in templates {{array_field:foo}} would still work.

I was selfishly hoping this would be enough but you're right its breaking so shouldnt be done at this point in the cycle.

I'll get back to work on the eloquent PR.

[jasonvarga]

Maybe we make the new format opt-in per field. Then opt in each config field.

In v6 we could remove the option and make it the only behavior.

[jasonvarga]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

[ryanmitchell]

By the way, it's crazy to me that object key order can't be maintained. It looks like it's maintained in sqlite but not in mysql.

Totally, its absolute nonsense!

[jasonvarga]

@ryanmitchell This fixes your issue. Reordering options of a select field will use the order you defined, in MySQL. 👍

[ryanmitchell]

Thank you