[go-experimental] export required fields without pointer

[shaxbee]

PR checklist

• Read the contribution guidelines.
• Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\. If contributing template-only or documentation-only changes which will change sample output, be sure to build the project first.
• Filed the PR against the correct branch: master, 4.1.x, 5.0.x. Default: master.
• Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

@bkabrda

Required fields are exposed without pointer, as they cannot be set to nil. This reduces boilerplate and GC pressure.
Omitempty tag is only used for fields that are neither required or nullable. This removes need for private member that breaks struct conversions. This also removes need for custom MarshalJSON that creates intermediary map increasing GC pressure.

[bkabrda]

I see two cases where your implementation is problematic:

• How would this work with a field that is required and nullable? AFAICS you'd have
  Foo int ``json:"foo"`` , but there's no way to serialize the explicit null in this scenario AFAICS.
• For scenario where you have a non-required nullable field, how do you distinguish when the field is just null because it was initialized that way versus when the user explicitly set it null?

[shaxbee]

Yes I totally agree, there is no differentiation between undefined and null in this case, but at least we don't introduce private members to what should be data only struct.

In case of required and nullable field since omitempty tag is not there nil value will be emitted as null.

To avoid breaking copying and pasting struct we could model this as interface{} and use helper methods or accept the fact that we cannot differentiate between undefined and null till generics land in Go.

[bkabrda]

As for the required and nullable field, you won't even be able to assign nil, because the type is not pointer - it's Foo int, not Foo *int.

but at least we don't introduce private members to what should be data only struct.

Why should it be a data only structure?

I feel like this is a sacrifice of feature completeness for getting a nicer code, which I'm not in favor of.

[bkabrda]

(Just to add to my previous point, I'd love to be able to simplify this and I'm all for someone doing that. I just don't think that this PR is sufficient, as it sacrifices feature completeness for a bit less boilerplate (which is generated anyway) and a bit of improvement for GC which I honestly don't think would be noticable. Maybe there's a way to modify this PR to make it cover all the cases (a way that I don't see) and I'd be more than happy to give it a thumbsup if you found it.)

[shaxbee]

You are correct, it should be pointer type, I will add it.

Regarding copying, if you copy subset of fields instead of whole object information about field being explicitly null is lost. I also think that gc pressure shouldn't be dismissed here, on every marshaling map is allocated and each value is converted into heap-referenced GC object.

[shaxbee]

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

[bkabrda]

So right now you have

{{^required}}{{^isNullable}}*{{/isNullable}}{{/required}}{{{dataType}}}

but if you want to export all nullable fields as pointers (as your last commit message suggests), you should do

{{#isNullable}}*{{/isNullable}}{{{dataType}}}

Am I missing somethings?

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

I honestly don't know what you mean by that. Could you provide an example of what this would look like?

[shaxbee]

You're absolutely right, required and nullable don't mix well in this case.

Added Nullable types to the templates. I need to probably override dataType in codegen for nullable types. I need to add Get/Set operations for nullable types and SetExplicitNull method.

Nullable types implement their own MarshalJSON/UnmarshalJSON avoiding need for intermediate map.
This approach guarantees that if we copy field value the information about explicit null is not lost, eg.

src:=Foo{}
src.SetBarExplicitNull(true)
dst:=Foo{
Bar: src.Bar
}

[shaxbee]

This is currently broken due to #3988

[etherealjoy]

Apparently the master changed when I merged it.

[shaxbee]

@etherealjoy The build on branch failed and master is broken as well :-(

[etherealjoy]

I regenerated it again after rebase. #4000
The #3988 solved some build issue on the test files.

[shaxbee]

Thanks a lot! :-)

[wing328]

Updates samples via 3c4f512. Let's see how it goes.

[etherealjoy]

a rebase is probably needed right?

[wing328]

rspec ./spec/api_client_spec.rb:158 # Petstore::ApiClient#build_collection_param fails for invalid collection format

We got the same error in the master as well.

[shaxbee]

@wing328 after rebase i'm still getting type generated incorrectly, what command do you use to generate code?

EDIT: fixed itself after mvn package :-)

[bkabrda]

So I think this is starting to look very promising. It may still require couple of tweaks though; I'll try to do a detailed review during today.

[bkabrda]

I left a couple comments inline that I think should be taken care of, but otherwise this looks good. I'll try to test the generated code next, although that will probably take me some more time.

[shaxbee]

@bkabrda is there anything else left to address?

[wing328]

@shaxbee sorry for the delay. Just merged it. Thanks for your contribution.

[wing328]

@shaxbee thanks for the PR, which has been included in v4.2.0 release: https://twitter.com/oas_generator/status/1189824932345069569

[shaxbee]

PR checklist

• Read the contribution guidelines.
• Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\. If contributing template-only or documentation-only changes which will change sample output, be sure to build the project first.
• Filed the PR against the correct branch: master, 4.1.x, 5.0.x. Default: master.
• Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

@bkabrda

Required fields are exposed without pointer, as they cannot be set to nil. This reduces boilerplate and GC pressure.
Omitempty tag is only used for fields that are neither required or nullable. This removes need for private member that breaks struct conversions. This also removes need for custom MarshalJSON that creates intermediary map increasing GC pressure.

[bkabrda]

I see two cases where your implementation is problematic:

• How would this work with a field that is required and nullable? AFAICS you'd have
  Foo int ``json:"foo"`` , but there's no way to serialize the explicit null in this scenario AFAICS.
• For scenario where you have a non-required nullable field, how do you distinguish when the field is just null because it was initialized that way versus when the user explicitly set it null?

[shaxbee]

Yes I totally agree, there is no differentiation between undefined and null in this case, but at least we don't introduce private members to what should be data only struct.

In case of required and nullable field since omitempty tag is not there nil value will be emitted as null.

To avoid breaking copying and pasting struct we could model this as interface{} and use helper methods or accept the fact that we cannot differentiate between undefined and null till generics land in Go.

[bkabrda]

As for the required and nullable field, you won't even be able to assign nil, because the type is not pointer - it's Foo int, not Foo *int.

but at least we don't introduce private members to what should be data only struct.

Why should it be a data only structure?

I feel like this is a sacrifice of feature completeness for getting a nicer code, which I'm not in favor of.

[bkabrda]

(Just to add to my previous point, I'd love to be able to simplify this and I'm all for someone doing that. I just don't think that this PR is sufficient, as it sacrifices feature completeness for a bit less boilerplate (which is generated anyway) and a bit of improvement for GC which I honestly don't think would be noticable. Maybe there's a way to modify this PR to make it cover all the cases (a way that I don't see) and I'd be more than happy to give it a thumbsup if you found it.)

[shaxbee]

You are correct, it should be pointer type, I will add it.

Regarding copying, if you copy subset of fields instead of whole object information about field being explicitly null is lost. I also think that gc pressure shouldn't be dismissed here, on every marshaling map is allocated and each value is converted into heap-referenced GC object.

[shaxbee]

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

[bkabrda]

So right now you have

{{^required}}{{^isNullable}}*{{/isNullable}}{{/required}}{{{dataType}}}

but if you want to export all nullable fields as pointers (as your last commit message suggests), you should do

{{#isNullable}}*{{/isNullable}}{{{dataType}}}

Am I missing somethings?

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

I honestly don't know what you mean by that. Could you provide an example of what this would look like?

[shaxbee]

You're absolutely right, required and nullable don't mix well in this case.

Added Nullable types to the templates. I need to probably override dataType in codegen for nullable types. I need to add Get/Set operations for nullable types and SetExplicitNull method.

Nullable types implement their own MarshalJSON/UnmarshalJSON avoiding need for intermediate map.
This approach guarantees that if we copy field value the information about explicit null is not lost, eg.

src:=Foo{}
src.SetBarExplicitNull(true)
dst:=Foo{
Bar: src.Bar
}

[shaxbee]

This is currently broken due to #3988

[etherealjoy]

Apparently the master changed when I merged it.

[shaxbee]

@etherealjoy The build on branch failed and master is broken as well :-(

[etherealjoy]

I regenerated it again after rebase. #4000
The #3988 solved some build issue on the test files.

[shaxbee]

Thanks a lot! :-)

[wing328]

Updates samples via 3c4f512. Let's see how it goes.

[etherealjoy]

a rebase is probably needed right?

[wing328]

rspec ./spec/api_client_spec.rb:158 # Petstore::ApiClient#build_collection_param fails for invalid collection format

We got the same error in the master as well.

[shaxbee]

@wing328 after rebase i'm still getting type generated incorrectly, what command do you use to generate code?

EDIT: fixed itself after mvn package :-)

[bkabrda]

So I think this is starting to look very promising. It may still require couple of tweaks though; I'll try to do a detailed review during today.

[bkabrda]

I left a couple comments inline that I think should be taken care of, but otherwise this looks good. I'll try to test the generated code next, although that will probably take me some more time.

[shaxbee]

@bkabrda is there anything else left to address?

[wing328]

@shaxbee sorry for the delay. Just merged it. Thanks for your contribution.

[wing328]

@shaxbee thanks for the PR, which has been included in v4.2.0 release: https://twitter.com/oas_generator/status/1189824932345069569

[shaxbee]

PR checklist

• Read the contribution guidelines.
• Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\. If contributing template-only or documentation-only changes which will change sample output, be sure to build the project first.
• Filed the PR against the correct branch: master, 4.1.x, 5.0.x. Default: master.
• Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

@bkabrda

Required fields are exposed without pointer, as they cannot be set to nil. This reduces boilerplate and GC pressure.
Omitempty tag is only used for fields that are neither required or nullable. This removes need for private member that breaks struct conversions. This also removes need for custom MarshalJSON that creates intermediary map increasing GC pressure.

[bkabrda]

I see two cases where your implementation is problematic:

• How would this work with a field that is required and nullable? AFAICS you'd have
  Foo int ``json:"foo"`` , but there's no way to serialize the explicit null in this scenario AFAICS.
• For scenario where you have a non-required nullable field, how do you distinguish when the field is just null because it was initialized that way versus when the user explicitly set it null?

[shaxbee]

Yes I totally agree, there is no differentiation between undefined and null in this case, but at least we don't introduce private members to what should be data only struct.

In case of required and nullable field since omitempty tag is not there nil value will be emitted as null.

To avoid breaking copying and pasting struct we could model this as interface{} and use helper methods or accept the fact that we cannot differentiate between undefined and null till generics land in Go.

[bkabrda]

As for the required and nullable field, you won't even be able to assign nil, because the type is not pointer - it's Foo int, not Foo *int.

but at least we don't introduce private members to what should be data only struct.

Why should it be a data only structure?

I feel like this is a sacrifice of feature completeness for getting a nicer code, which I'm not in favor of.

[bkabrda]

(Just to add to my previous point, I'd love to be able to simplify this and I'm all for someone doing that. I just don't think that this PR is sufficient, as it sacrifices feature completeness for a bit less boilerplate (which is generated anyway) and a bit of improvement for GC which I honestly don't think would be noticable. Maybe there's a way to modify this PR to make it cover all the cases (a way that I don't see) and I'd be more than happy to give it a thumbsup if you found it.)

[shaxbee]

You are correct, it should be pointer type, I will add it.

Regarding copying, if you copy subset of fields instead of whole object information about field being explicitly null is lost. I also think that gc pressure shouldn't be dismissed here, on every marshaling map is allocated and each value is converted into heap-referenced GC object.

[shaxbee]

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

[bkabrda]

So right now you have

{{^required}}{{^isNullable}}*{{/isNullable}}{{/required}}{{{dataType}}}

but if you want to export all nullable fields as pointers (as your last commit message suggests), you should do

{{#isNullable}}*{{/isNullable}}{{{dataType}}}

Am I missing somethings?

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

I honestly don't know what you mean by that. Could you provide an example of what this would look like?

[shaxbee]

You're absolutely right, required and nullable don't mix well in this case.

Added Nullable types to the templates. I need to probably override dataType in codegen for nullable types. I need to add Get/Set operations for nullable types and SetExplicitNull method.

Nullable types implement their own MarshalJSON/UnmarshalJSON avoiding need for intermediate map.
This approach guarantees that if we copy field value the information about explicit null is not lost, eg.

src:=Foo{}
src.SetBarExplicitNull(true)
dst:=Foo{
Bar: src.Bar
}

[shaxbee]

This is currently broken due to #3988

[etherealjoy]

Apparently the master changed when I merged it.

[shaxbee]

@etherealjoy The build on branch failed and master is broken as well :-(

[etherealjoy]

I regenerated it again after rebase. #4000
The #3988 solved some build issue on the test files.

[shaxbee]

Thanks a lot! :-)

[wing328]

Updates samples via 3c4f512. Let's see how it goes.

[etherealjoy]

a rebase is probably needed right?

[wing328]

rspec ./spec/api_client_spec.rb:158 # Petstore::ApiClient#build_collection_param fails for invalid collection format

We got the same error in the master as well.

[shaxbee]

@wing328 after rebase i'm still getting type generated incorrectly, what command do you use to generate code?

EDIT: fixed itself after mvn package :-)

[bkabrda]

So I think this is starting to look very promising. It may still require couple of tweaks though; I'll try to do a detailed review during today.

[bkabrda]

I left a couple comments inline that I think should be taken care of, but otherwise this looks good. I'll try to test the generated code next, although that will probably take me some more time.

[shaxbee]

@bkabrda is there anything else left to address?

[wing328]

@shaxbee sorry for the delay. Just merged it. Thanks for your contribution.

[wing328]

@shaxbee thanks for the PR, which has been included in v4.2.0 release: https://twitter.com/oas_generator/status/1189824932345069569

[shaxbee]

PR checklist

• Read the contribution guidelines.
• Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\. If contributing template-only or documentation-only changes which will change sample output, be sure to build the project first.
• Filed the PR against the correct branch: master, 4.1.x, 5.0.x. Default: master.
• Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

@bkabrda

Required fields are exposed without pointer, as they cannot be set to nil. This reduces boilerplate and GC pressure.
Omitempty tag is only used for fields that are neither required or nullable. This removes need for private member that breaks struct conversions. This also removes need for custom MarshalJSON that creates intermediary map increasing GC pressure.

[bkabrda]

I see two cases where your implementation is problematic:

• How would this work with a field that is required and nullable? AFAICS you'd have
  Foo int ``json:"foo"`` , but there's no way to serialize the explicit null in this scenario AFAICS.
• For scenario where you have a non-required nullable field, how do you distinguish when the field is just null because it was initialized that way versus when the user explicitly set it null?

[shaxbee]

Yes I totally agree, there is no differentiation between undefined and null in this case, but at least we don't introduce private members to what should be data only struct.

In case of required and nullable field since omitempty tag is not there nil value will be emitted as null.

To avoid breaking copying and pasting struct we could model this as interface{} and use helper methods or accept the fact that we cannot differentiate between undefined and null till generics land in Go.

[bkabrda]

As for the required and nullable field, you won't even be able to assign nil, because the type is not pointer - it's Foo int, not Foo *int.

but at least we don't introduce private members to what should be data only struct.

Why should it be a data only structure?

I feel like this is a sacrifice of feature completeness for getting a nicer code, which I'm not in favor of.

[bkabrda]

(Just to add to my previous point, I'd love to be able to simplify this and I'm all for someone doing that. I just don't think that this PR is sufficient, as it sacrifices feature completeness for a bit less boilerplate (which is generated anyway) and a bit of improvement for GC which I honestly don't think would be noticable. Maybe there's a way to modify this PR to make it cover all the cases (a way that I don't see) and I'd be more than happy to give it a thumbsup if you found it.)

[shaxbee]

You are correct, it should be pointer type, I will add it.

Regarding copying, if you copy subset of fields instead of whole object information about field being explicitly null is lost. I also think that gc pressure shouldn't be dismissed here, on every marshaling map is allocated and each value is converted into heap-referenced GC object.

[shaxbee]

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

[bkabrda]

So right now you have

{{^required}}{{^isNullable}}*{{/isNullable}}{{/required}}{{{dataType}}}

but if you want to export all nullable fields as pointers (as your last commit message suggests), you should do

{{#isNullable}}*{{/isNullable}}{{{dataType}}}

Am I missing somethings?

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

I honestly don't know what you mean by that. Could you provide an example of what this would look like?

[shaxbee]

You're absolutely right, required and nullable don't mix well in this case.

Added Nullable types to the templates. I need to probably override dataType in codegen for nullable types. I need to add Get/Set operations for nullable types and SetExplicitNull method.

Nullable types implement their own MarshalJSON/UnmarshalJSON avoiding need for intermediate map.
This approach guarantees that if we copy field value the information about explicit null is not lost, eg.

src:=Foo{}
src.SetBarExplicitNull(true)
dst:=Foo{
Bar: src.Bar
}

[shaxbee]

This is currently broken due to #3988

[etherealjoy]

Apparently the master changed when I merged it.

[shaxbee]

@etherealjoy The build on branch failed and master is broken as well :-(

[etherealjoy]

I regenerated it again after rebase. #4000
The #3988 solved some build issue on the test files.

[shaxbee]

Thanks a lot! :-)

[wing328]

Updates samples via 3c4f512. Let's see how it goes.

[etherealjoy]

a rebase is probably needed right?

[wing328]

rspec ./spec/api_client_spec.rb:158 # Petstore::ApiClient#build_collection_param fails for invalid collection format

We got the same error in the master as well.

[shaxbee]

@wing328 after rebase i'm still getting type generated incorrectly, what command do you use to generate code?

EDIT: fixed itself after mvn package :-)

[bkabrda]

So I think this is starting to look very promising. It may still require couple of tweaks though; I'll try to do a detailed review during today.

[bkabrda]

I left a couple comments inline that I think should be taken care of, but otherwise this looks good. I'll try to test the generated code next, although that will probably take me some more time.

[shaxbee]

@bkabrda is there anything else left to address?

[wing328]

@shaxbee sorry for the delay. Just merged it. Thanks for your contribution.

[wing328]

@shaxbee thanks for the PR, which has been included in v4.2.0 release: https://twitter.com/oas_generator/status/1189824932345069569

[shaxbee]

PR checklist

• Read the contribution guidelines.
• Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\. If contributing template-only or documentation-only changes which will change sample output, be sure to build the project first.
• Filed the PR against the correct branch: master, 4.1.x, 5.0.x. Default: master.
• Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

@bkabrda

Required fields are exposed without pointer, as they cannot be set to nil. This reduces boilerplate and GC pressure.
Omitempty tag is only used for fields that are neither required or nullable. This removes need for private member that breaks struct conversions. This also removes need for custom MarshalJSON that creates intermediary map increasing GC pressure.

[bkabrda]

I see two cases where your implementation is problematic:

• How would this work with a field that is required and nullable? AFAICS you'd have
  Foo int ``json:"foo"`` , but there's no way to serialize the explicit null in this scenario AFAICS.
• For scenario where you have a non-required nullable field, how do you distinguish when the field is just null because it was initialized that way versus when the user explicitly set it null?

[shaxbee]

Yes I totally agree, there is no differentiation between undefined and null in this case, but at least we don't introduce private members to what should be data only struct.

In case of required and nullable field since omitempty tag is not there nil value will be emitted as null.

To avoid breaking copying and pasting struct we could model this as interface{} and use helper methods or accept the fact that we cannot differentiate between undefined and null till generics land in Go.

[bkabrda]

As for the required and nullable field, you won't even be able to assign nil, because the type is not pointer - it's Foo int, not Foo *int.

but at least we don't introduce private members to what should be data only struct.

Why should it be a data only structure?

I feel like this is a sacrifice of feature completeness for getting a nicer code, which I'm not in favor of.

[bkabrda]

(Just to add to my previous point, I'd love to be able to simplify this and I'm all for someone doing that. I just don't think that this PR is sufficient, as it sacrifices feature completeness for a bit less boilerplate (which is generated anyway) and a bit of improvement for GC which I honestly don't think would be noticable. Maybe there's a way to modify this PR to make it cover all the cases (a way that I don't see) and I'd be more than happy to give it a thumbsup if you found it.)

[shaxbee]

You are correct, it should be pointer type, I will add it.

Regarding copying, if you copy subset of fields instead of whole object information about field being explicitly null is lost. I also think that gc pressure shouldn't be dismissed here, on every marshaling map is allocated and each value is converted into heap-referenced GC object.

[shaxbee]

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

[bkabrda]

So right now you have

{{^required}}{{^isNullable}}*{{/isNullable}}{{/required}}{{{dataType}}}

but if you want to export all nullable fields as pointers (as your last commit message suggests), you should do

{{#isNullable}}*{{/isNullable}}{{{dataType}}}

Am I missing somethings?

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

I honestly don't know what you mean by that. Could you provide an example of what this would look like?

[shaxbee]

You're absolutely right, required and nullable don't mix well in this case.

Added Nullable types to the templates. I need to probably override dataType in codegen for nullable types. I need to add Get/Set operations for nullable types and SetExplicitNull method.

Nullable types implement their own MarshalJSON/UnmarshalJSON avoiding need for intermediate map.
This approach guarantees that if we copy field value the information about explicit null is not lost, eg.

src:=Foo{}
src.SetBarExplicitNull(true)
dst:=Foo{
Bar: src.Bar
}

[shaxbee]

This is currently broken due to #3988

[etherealjoy]

Apparently the master changed when I merged it.

[shaxbee]

@etherealjoy The build on branch failed and master is broken as well :-(

[etherealjoy]

I regenerated it again after rebase. #4000
The #3988 solved some build issue on the test files.

[shaxbee]

Thanks a lot! :-)

[wing328]

Updates samples via 3c4f512. Let's see how it goes.

[etherealjoy]

a rebase is probably needed right?

[wing328]

rspec ./spec/api_client_spec.rb:158 # Petstore::ApiClient#build_collection_param fails for invalid collection format

We got the same error in the master as well.

[shaxbee]

@wing328 after rebase i'm still getting type generated incorrectly, what command do you use to generate code?

EDIT: fixed itself after mvn package :-)

[bkabrda]

So I think this is starting to look very promising. It may still require couple of tweaks though; I'll try to do a detailed review during today.

[bkabrda]

I left a couple comments inline that I think should be taken care of, but otherwise this looks good. I'll try to test the generated code next, although that will probably take me some more time.

[shaxbee]

@bkabrda is there anything else left to address?

[wing328]

@shaxbee sorry for the delay. Just merged it. Thanks for your contribution.

[wing328]

@shaxbee thanks for the PR, which has been included in v4.2.0 release: https://twitter.com/oas_generator/status/1189824932345069569

[shaxbee]

PR checklist

• Read the contribution guidelines.
• Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\. If contributing template-only or documentation-only changes which will change sample output, be sure to build the project first.
• Filed the PR against the correct branch: master, 4.1.x, 5.0.x. Default: master.
• Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

@bkabrda

Required fields are exposed without pointer, as they cannot be set to nil. This reduces boilerplate and GC pressure.
Omitempty tag is only used for fields that are neither required or nullable. This removes need for private member that breaks struct conversions. This also removes need for custom MarshalJSON that creates intermediary map increasing GC pressure.

[bkabrda]

I see two cases where your implementation is problematic:

• How would this work with a field that is required and nullable? AFAICS you'd have
  Foo int ``json:"foo"`` , but there's no way to serialize the explicit null in this scenario AFAICS.
• For scenario where you have a non-required nullable field, how do you distinguish when the field is just null because it was initialized that way versus when the user explicitly set it null?

[shaxbee]

Yes I totally agree, there is no differentiation between undefined and null in this case, but at least we don't introduce private members to what should be data only struct.

In case of required and nullable field since omitempty tag is not there nil value will be emitted as null.

To avoid breaking copying and pasting struct we could model this as interface{} and use helper methods or accept the fact that we cannot differentiate between undefined and null till generics land in Go.

[bkabrda]

As for the required and nullable field, you won't even be able to assign nil, because the type is not pointer - it's Foo int, not Foo *int.

but at least we don't introduce private members to what should be data only struct.

Why should it be a data only structure?

I feel like this is a sacrifice of feature completeness for getting a nicer code, which I'm not in favor of.

[bkabrda]

(Just to add to my previous point, I'd love to be able to simplify this and I'm all for someone doing that. I just don't think that this PR is sufficient, as it sacrifices feature completeness for a bit less boilerplate (which is generated anyway) and a bit of improvement for GC which I honestly don't think would be noticable. Maybe there's a way to modify this PR to make it cover all the cases (a way that I don't see) and I'd be more than happy to give it a thumbsup if you found it.)

[shaxbee]

You are correct, it should be pointer type, I will add it.

Regarding copying, if you copy subset of fields instead of whole object information about field being explicitly null is lost. I also think that gc pressure shouldn't be dismissed here, on every marshaling map is allocated and each value is converted into heap-referenced GC object.

[shaxbee]

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

[bkabrda]

So right now you have

{{^required}}{{^isNullable}}*{{/isNullable}}{{/required}}{{{dataType}}}

but if you want to export all nullable fields as pointers (as your last commit message suggests), you should do

{{#isNullable}}*{{/isNullable}}{{{dataType}}}

Am I missing somethings?

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

I honestly don't know what you mean by that. Could you provide an example of what this would look like?

[shaxbee]

You're absolutely right, required and nullable don't mix well in this case.

Added Nullable types to the templates. I need to probably override dataType in codegen for nullable types. I need to add Get/Set operations for nullable types and SetExplicitNull method.

Nullable types implement their own MarshalJSON/UnmarshalJSON avoiding need for intermediate map.
This approach guarantees that if we copy field value the information about explicit null is not lost, eg.

src:=Foo{}
src.SetBarExplicitNull(true)
dst:=Foo{
Bar: src.Bar
}

[shaxbee]

This is currently broken due to #3988

[etherealjoy]

Apparently the master changed when I merged it.

[shaxbee]

@etherealjoy The build on branch failed and master is broken as well :-(

[etherealjoy]

I regenerated it again after rebase. #4000
The #3988 solved some build issue on the test files.

[shaxbee]

Thanks a lot! :-)

[wing328]

Updates samples via 3c4f512. Let's see how it goes.

[etherealjoy]

a rebase is probably needed right?

[wing328]

rspec ./spec/api_client_spec.rb:158 # Petstore::ApiClient#build_collection_param fails for invalid collection format

We got the same error in the master as well.

[shaxbee]

@wing328 after rebase i'm still getting type generated incorrectly, what command do you use to generate code?

EDIT: fixed itself after mvn package :-)

[bkabrda]

So I think this is starting to look very promising. It may still require couple of tweaks though; I'll try to do a detailed review during today.

[bkabrda]

I left a couple comments inline that I think should be taken care of, but otherwise this looks good. I'll try to test the generated code next, although that will probably take me some more time.

[shaxbee]

@bkabrda is there anything else left to address?

[wing328]

@shaxbee sorry for the delay. Just merged it. Thanks for your contribution.

[wing328]

@shaxbee thanks for the PR, which has been included in v4.2.0 release: https://twitter.com/oas_generator/status/1189824932345069569

[shaxbee]

PR checklist

• Read the contribution guidelines.
• Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\. If contributing template-only or documentation-only changes which will change sample output, be sure to build the project first.
• Filed the PR against the correct branch: master, 4.1.x, 5.0.x. Default: master.
• Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

@bkabrda

Required fields are exposed without pointer, as they cannot be set to nil. This reduces boilerplate and GC pressure.
Omitempty tag is only used for fields that are neither required or nullable. This removes need for private member that breaks struct conversions. This also removes need for custom MarshalJSON that creates intermediary map increasing GC pressure.

[bkabrda]

I see two cases where your implementation is problematic:

• How would this work with a field that is required and nullable? AFAICS you'd have
  Foo int ``json:"foo"`` , but there's no way to serialize the explicit null in this scenario AFAICS.
• For scenario where you have a non-required nullable field, how do you distinguish when the field is just null because it was initialized that way versus when the user explicitly set it null?

[shaxbee]

Yes I totally agree, there is no differentiation between undefined and null in this case, but at least we don't introduce private members to what should be data only struct.

In case of required and nullable field since omitempty tag is not there nil value will be emitted as null.

To avoid breaking copying and pasting struct we could model this as interface{} and use helper methods or accept the fact that we cannot differentiate between undefined and null till generics land in Go.

[bkabrda]

As for the required and nullable field, you won't even be able to assign nil, because the type is not pointer - it's Foo int, not Foo *int.

but at least we don't introduce private members to what should be data only struct.

Why should it be a data only structure?

I feel like this is a sacrifice of feature completeness for getting a nicer code, which I'm not in favor of.

[bkabrda]

(Just to add to my previous point, I'd love to be able to simplify this and I'm all for someone doing that. I just don't think that this PR is sufficient, as it sacrifices feature completeness for a bit less boilerplate (which is generated anyway) and a bit of improvement for GC which I honestly don't think would be noticable. Maybe there's a way to modify this PR to make it cover all the cases (a way that I don't see) and I'd be more than happy to give it a thumbsup if you found it.)

[shaxbee]

You are correct, it should be pointer type, I will add it.

Regarding copying, if you copy subset of fields instead of whole object information about field being explicitly null is lost. I also think that gc pressure shouldn't be dismissed here, on every marshaling map is allocated and each value is converted into heap-referenced GC object.

[shaxbee]

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

[bkabrda]

So right now you have

{{^required}}{{^isNullable}}*{{/isNullable}}{{/required}}{{{dataType}}}

but if you want to export all nullable fields as pointers (as your last commit message suggests), you should do

{{#isNullable}}*{{/isNullable}}{{{dataType}}}

Am I missing somethings?

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

I honestly don't know what you mean by that. Could you provide an example of what this would look like?

[shaxbee]

You're absolutely right, required and nullable don't mix well in this case.

Added Nullable types to the templates. I need to probably override dataType in codegen for nullable types. I need to add Get/Set operations for nullable types and SetExplicitNull method.

Nullable types implement their own MarshalJSON/UnmarshalJSON avoiding need for intermediate map.
This approach guarantees that if we copy field value the information about explicit null is not lost, eg.

src:=Foo{}
src.SetBarExplicitNull(true)
dst:=Foo{
Bar: src.Bar
}

[shaxbee]

This is currently broken due to #3988

[etherealjoy]

Apparently the master changed when I merged it.

[shaxbee]

@etherealjoy The build on branch failed and master is broken as well :-(

[etherealjoy]

I regenerated it again after rebase. #4000
The #3988 solved some build issue on the test files.

[shaxbee]

Thanks a lot! :-)

[wing328]

Updates samples via 3c4f512. Let's see how it goes.

[etherealjoy]

a rebase is probably needed right?

[wing328]

rspec ./spec/api_client_spec.rb:158 # Petstore::ApiClient#build_collection_param fails for invalid collection format

We got the same error in the master as well.

[shaxbee]

@wing328 after rebase i'm still getting type generated incorrectly, what command do you use to generate code?

EDIT: fixed itself after mvn package :-)

[bkabrda]

So I think this is starting to look very promising. It may still require couple of tweaks though; I'll try to do a detailed review during today.

[bkabrda]

I left a couple comments inline that I think should be taken care of, but otherwise this looks good. I'll try to test the generated code next, although that will probably take me some more time.

[shaxbee]

@bkabrda is there anything else left to address?

[wing328]

@shaxbee sorry for the delay. Just merged it. Thanks for your contribution.

[wing328]

@shaxbee thanks for the PR, which has been included in v4.2.0 release: https://twitter.com/oas_generator/status/1189824932345069569

[shaxbee]

PR checklist

• Read the contribution guidelines.
• Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\. If contributing template-only or documentation-only changes which will change sample output, be sure to build the project first.
• Filed the PR against the correct branch: master, 4.1.x, 5.0.x. Default: master.
• Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

@bkabrda

Required fields are exposed without pointer, as they cannot be set to nil. This reduces boilerplate and GC pressure.
Omitempty tag is only used for fields that are neither required or nullable. This removes need for private member that breaks struct conversions. This also removes need for custom MarshalJSON that creates intermediary map increasing GC pressure.

[bkabrda]

I see two cases where your implementation is problematic:

• How would this work with a field that is required and nullable? AFAICS you'd have
  Foo int ``json:"foo"`` , but there's no way to serialize the explicit null in this scenario AFAICS.
• For scenario where you have a non-required nullable field, how do you distinguish when the field is just null because it was initialized that way versus when the user explicitly set it null?

[shaxbee]

Yes I totally agree, there is no differentiation between undefined and null in this case, but at least we don't introduce private members to what should be data only struct.

In case of required and nullable field since omitempty tag is not there nil value will be emitted as null.

To avoid breaking copying and pasting struct we could model this as interface{} and use helper methods or accept the fact that we cannot differentiate between undefined and null till generics land in Go.

[bkabrda]

As for the required and nullable field, you won't even be able to assign nil, because the type is not pointer - it's Foo int, not Foo *int.

but at least we don't introduce private members to what should be data only struct.

Why should it be a data only structure?

I feel like this is a sacrifice of feature completeness for getting a nicer code, which I'm not in favor of.

[bkabrda]

(Just to add to my previous point, I'd love to be able to simplify this and I'm all for someone doing that. I just don't think that this PR is sufficient, as it sacrifices feature completeness for a bit less boilerplate (which is generated anyway) and a bit of improvement for GC which I honestly don't think would be noticable. Maybe there's a way to modify this PR to make it cover all the cases (a way that I don't see) and I'd be more than happy to give it a thumbsup if you found it.)

[shaxbee]

You are correct, it should be pointer type, I will add it.

Regarding copying, if you copy subset of fields instead of whole object information about field being explicitly null is lost. I also think that gc pressure shouldn't be dismissed here, on every marshaling map is allocated and each value is converted into heap-referenced GC object.

[shaxbee]

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

[bkabrda]

So right now you have

{{^required}}{{^isNullable}}*{{/isNullable}}{{/required}}{{{dataType}}}

but if you want to export all nullable fields as pointers (as your last commit message suggests), you should do

{{#isNullable}}*{{/isNullable}}{{{dataType}}}

Am I missing somethings?

Coming to think about it - we could simply generate Nullable types and Ptr methods for object schemas and this way support explicit null.

I honestly don't know what you mean by that. Could you provide an example of what this would look like?

[shaxbee]

You're absolutely right, required and nullable don't mix well in this case.

Added Nullable types to the templates. I need to probably override dataType in codegen for nullable types. I need to add Get/Set operations for nullable types and SetExplicitNull method.

Nullable types implement their own MarshalJSON/UnmarshalJSON avoiding need for intermediate map.
This approach guarantees that if we copy field value the information about explicit null is not lost, eg.

src:=Foo{}
src.SetBarExplicitNull(true)
dst:=Foo{
Bar: src.Bar
}

[shaxbee]

This is currently broken due to #3988

[etherealjoy]

Apparently the master changed when I merged it.

[shaxbee]

@etherealjoy The build on branch failed and master is broken as well :-(

[etherealjoy]

I regenerated it again after rebase. #4000
The #3988 solved some build issue on the test files.

[shaxbee]

Thanks a lot! :-)

[wing328]

Updates samples via 3c4f512. Let's see how it goes.

[etherealjoy]

a rebase is probably needed right?

[wing328]

rspec ./spec/api_client_spec.rb:158 # Petstore::ApiClient#build_collection_param fails for invalid collection format

We got the same error in the master as well.

[shaxbee]

@wing328 after rebase i'm still getting type generated incorrectly, what command do you use to generate code?

EDIT: fixed itself after mvn package :-)

[bkabrda]

So I think this is starting to look very promising. It may still require couple of tweaks though; I'll try to do a detailed review during today.

[bkabrda]

I left a couple comments inline that I think should be taken care of, but otherwise this looks good. I'll try to test the generated code next, although that will probably take me some more time.

[shaxbee]

@bkabrda is there anything else left to address?

[wing328]

@shaxbee sorry for the delay. Just merged it. Thanks for your contribution.

[wing328]

@shaxbee thanks for the PR, which has been included in v4.2.0 release: https://twitter.com/oas_generator/status/1189824932345069569