Add in Optional handling for SpringBoot POJOs

[welshm]

Modified pojo.mustache to have Optional be used on POJOs

• Added spring-boot-oas3-nullable.yaml to generate nullable examples to validate against
• Added spring-boot-oas3-nullable-disabled.yaml to generate optional behaviour when nullable is disabled
• [BUG] [JAVA] [SPRING] Optional values aren't optional in POJOs, only in path params #11383

NOTE: This is a breaking change for anyone with useOptional enabled.

PR checklist

• Read the contribution guidelines.
• Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
• Run the following to build the project and update samples:

  ./mvnw clean package ./bin/generate-samples.sh
  ./bin/utils/export_docs_generators.sh
  

  Commit all changed files.
  This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
  These must match the expectations made by your contribution.
  You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
• File the PR against the correct branch: master (5.3.0), 6.0.x
• If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Spec that shows the use case

ApiDefault:
type: object
required:
- requiredMessage
- requiredNullableMessage
properties:
message:
type: string
description: A message
nullableMessage:
type: string
description: A nullable message
nullable: true
requiredMessage:
type: string
description: A required message
requiredNullableMessage:
type: string
description: A required nullable message
nullable: true

Output

useOptional: false
openApiNullable: false
private String message;
private String nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: true
openApiNullable: false
private Optional<String> message;
private Optional<String> nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: false
openApiNullable: true
private String message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

useOptional: true
openApiNullable: true
private Optional<String> message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

[welshm]

Note: This validates against 3_0

[welshm]

@wing328@cachescrubber

@bbdouglas (2017/07) @sreeshas (2017/08) @jfiala (2017/08) @lukoyanov (2017/09) @cbornet (2017/09) @jeff9finger (2018/01) @karismann (2019/03) @Zomzog (2019/04) @lwlee2608 (2019/10)

[wing328]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

[welshm]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

That is my opinion on how useOptional should function, yes. I will post the issue to the Slack to see if anyone has any differing opinions

[welshm]

@MelleD I tried to cover all relevant cases I could think of here

[welshm]

A quick test shows this to be working with both enabled:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JsonNullableModule());
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

However, I don't see nullability being enforced - if I don't set a required field (such as requiredNumber) this will still parse to/from JSON no problem. This is unrelated to Optional work here.

[wing328]

About the maven plug-in test failure, I think you will also need to update modules/openapi-generator-maven-plugin/examples/spring.xml

Test: https://github.com/OpenAPITools/openapi-generator/blob/master/.github/workflows/openapi-generator.yaml#L188

[cachescrubber]

@welshm You do not need to provide the jackson-datatype-jdk8 dependency. It is provided by spring-boot-starter-json which in turn is a transitive dependency of spring-boot-starter-web . The reason it failed on the maven-plugin test is the outdated version of spring-boot used there (2.2.1.RELEASE). Just update the spring-boot version instead.

The Jackson Jdk8Module is configured by default in an recent version of spring-boot. No need to provide it as an @bean.

[MelleD]

You should initialize the member with Optional.empty

A null optional field should never exists.

[welshm]

Good catch - fixed in next patch

[MelleD]

Bean validation is duplicated also in the getter. It should add to the getter

[welshm]

Hmm... that becomes tricky because the mustache generates both as {{>optionalDataTypeWithEnum}} which internally uses {{>optionalDataType}} which has {{>beanValidationCore}} as part of the type.

While it is redundant, I am fine with the duplication for how it makes the use of the mustache simpler.

Alernatively, we could have optionalDataTypeWithEnum and annotatedOptionalDataTypeWithEnum - but would require making the same distinction for the inner usage of optionalDataType.

[MelleD]

I see.
So far, optionals have only been used as parameters. I think for member variables we need a new mustache without bean validation. Because the beanvalidation is in <> always makes it awkward.

[welshm]

Created optionalDataTypeWithoutAnnotations - I did leave it on the setter but I'm not that familiar with how the validation is applied. I can convert it to getter only if that's more correct.

[MelleD]

Bean validation for DTOs/Pojos have to be set on getter or fields. Currently the template set it on getter

[welshm]

Will remove from setter then

[MelleD]

Is this really working with Optional?

[welshm]

It seems to be from my test:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JavaTimeModule())
.registerModule(new JsonNullableModule())
;
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.optionalDate(LocalDate.of(2020, 10, 2))
.optionalDateTime(OffsetDateTime.of(LocalDateTime.of(2020, 10, 2, 12, 30), ZoneOffset.UTC))
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

I can see the date time and read it back - it matches. JSON output looks fine too.

[cachescrubber]

I would prefer to use an include here. The code is difficult to read and hard to understand. Isn't it possible to use optionalDataType.mustache?

[welshm]

This comment seems old - but I need to fix the bean annotations anyway so this might change to an include

[cachescrubber]

Old, but still valid IMO

[welshm]

Moved it's own include - I had trouble coming up with a good name so it's dataTypeDeclaration - which is kind of misleading because these are all data type declarations.

Open to suggestions of better names - and I'm taking a look at your wrapper type change now

[cachescrubber]

Have you considered to override AbstractJavaCodegen#toDefaultValue() in SpringCodegen? I would prefer to avoid logic in the templates if possible.

[welshm]

Sure I can look into that. I did it this way because that's how it was already done for JsonNullable but I'll try tk improve it

[cachescrubber]

Not as straight forward as one might think (but doable) I currently doing a proof of concept with JsonNullable.

[welshm]

Might make sense to do it as a separate PR then? Consolidate the defaultValue decision and generation?

[cachescrubber]

See #11666

[cachescrubber]

Looks good!

[welshm]

I think this can still be improved, but it'll be a lot easier with https://github.com/OpenAPITools/openapi-generator/pull/11666/files so I'll iterate on it after that

[welshm]

Did you want to get #11666 in first? This change is good to go otherwise, but might be easier to deal with merge conflicts and resolve any unit tests changes if I update this after yours is in first.

[welshm]

Closing - since this is way out of date

[welshm]

Modified pojo.mustache to have Optional be used on POJOs

• Added spring-boot-oas3-nullable.yaml to generate nullable examples to validate against
• Added spring-boot-oas3-nullable-disabled.yaml to generate optional behaviour when nullable is disabled
• [BUG] [JAVA] [SPRING] Optional values aren't optional in POJOs, only in path params #11383

NOTE: This is a breaking change for anyone with useOptional enabled.

PR checklist

• Read the contribution guidelines.
• Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
• Run the following to build the project and update samples:

  ./mvnw clean package ./bin/generate-samples.sh
  ./bin/utils/export_docs_generators.sh
  

  Commit all changed files.
  This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
  These must match the expectations made by your contribution.
  You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
• File the PR against the correct branch: master (5.3.0), 6.0.x
• If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Spec that shows the use case

ApiDefault:
type: object
required:
- requiredMessage
- requiredNullableMessage
properties:
message:
type: string
description: A message
nullableMessage:
type: string
description: A nullable message
nullable: true
requiredMessage:
type: string
description: A required message
requiredNullableMessage:
type: string
description: A required nullable message
nullable: true

Output

useOptional: false
openApiNullable: false
private String message;
private String nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: true
openApiNullable: false
private Optional<String> message;
private Optional<String> nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: false
openApiNullable: true
private String message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

useOptional: true
openApiNullable: true
private Optional<String> message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

[welshm]

Note: This validates against 3_0

[welshm]

@wing328@cachescrubber

@bbdouglas (2017/07) @sreeshas (2017/08) @jfiala (2017/08) @lukoyanov (2017/09) @cbornet (2017/09) @jeff9finger (2018/01) @karismann (2019/03) @Zomzog (2019/04) @lwlee2608 (2019/10)

[wing328]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

[welshm]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

That is my opinion on how useOptional should function, yes. I will post the issue to the Slack to see if anyone has any differing opinions

[welshm]

@MelleD I tried to cover all relevant cases I could think of here

[welshm]

A quick test shows this to be working with both enabled:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JsonNullableModule());
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

However, I don't see nullability being enforced - if I don't set a required field (such as requiredNumber) this will still parse to/from JSON no problem. This is unrelated to Optional work here.

[wing328]

About the maven plug-in test failure, I think you will also need to update modules/openapi-generator-maven-plugin/examples/spring.xml

Test: https://github.com/OpenAPITools/openapi-generator/blob/master/.github/workflows/openapi-generator.yaml#L188

[cachescrubber]

@welshm You do not need to provide the jackson-datatype-jdk8 dependency. It is provided by spring-boot-starter-json which in turn is a transitive dependency of spring-boot-starter-web . The reason it failed on the maven-plugin test is the outdated version of spring-boot used there (2.2.1.RELEASE). Just update the spring-boot version instead.

The Jackson Jdk8Module is configured by default in an recent version of spring-boot. No need to provide it as an @bean.

[MelleD]

You should initialize the member with Optional.empty

A null optional field should never exists.

[welshm]

Good catch - fixed in next patch

[MelleD]

Bean validation is duplicated also in the getter. It should add to the getter

[welshm]

Hmm... that becomes tricky because the mustache generates both as {{>optionalDataTypeWithEnum}} which internally uses {{>optionalDataType}} which has {{>beanValidationCore}} as part of the type.

While it is redundant, I am fine with the duplication for how it makes the use of the mustache simpler.

Alernatively, we could have optionalDataTypeWithEnum and annotatedOptionalDataTypeWithEnum - but would require making the same distinction for the inner usage of optionalDataType.

[MelleD]

I see.
So far, optionals have only been used as parameters. I think for member variables we need a new mustache without bean validation. Because the beanvalidation is in <> always makes it awkward.

[welshm]

Created optionalDataTypeWithoutAnnotations - I did leave it on the setter but I'm not that familiar with how the validation is applied. I can convert it to getter only if that's more correct.

[MelleD]

Bean validation for DTOs/Pojos have to be set on getter or fields. Currently the template set it on getter

[welshm]

Will remove from setter then

[MelleD]

Is this really working with Optional?

[welshm]

It seems to be from my test:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JavaTimeModule())
.registerModule(new JsonNullableModule())
;
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.optionalDate(LocalDate.of(2020, 10, 2))
.optionalDateTime(OffsetDateTime.of(LocalDateTime.of(2020, 10, 2, 12, 30), ZoneOffset.UTC))
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

I can see the date time and read it back - it matches. JSON output looks fine too.

[cachescrubber]

I would prefer to use an include here. The code is difficult to read and hard to understand. Isn't it possible to use optionalDataType.mustache?

[welshm]

This comment seems old - but I need to fix the bean annotations anyway so this might change to an include

[cachescrubber]

Old, but still valid IMO

[welshm]

Moved it's own include - I had trouble coming up with a good name so it's dataTypeDeclaration - which is kind of misleading because these are all data type declarations.

Open to suggestions of better names - and I'm taking a look at your wrapper type change now

[cachescrubber]

Have you considered to override AbstractJavaCodegen#toDefaultValue() in SpringCodegen? I would prefer to avoid logic in the templates if possible.

[welshm]

Sure I can look into that. I did it this way because that's how it was already done for JsonNullable but I'll try tk improve it

[cachescrubber]

Not as straight forward as one might think (but doable) I currently doing a proof of concept with JsonNullable.

[welshm]

Might make sense to do it as a separate PR then? Consolidate the defaultValue decision and generation?

[cachescrubber]

See #11666

[cachescrubber]

Looks good!

[welshm]

I think this can still be improved, but it'll be a lot easier with https://github.com/OpenAPITools/openapi-generator/pull/11666/files so I'll iterate on it after that

[welshm]

Did you want to get #11666 in first? This change is good to go otherwise, but might be easier to deal with merge conflicts and resolve any unit tests changes if I update this after yours is in first.

[welshm]

Closing - since this is way out of date

[welshm]

Modified pojo.mustache to have Optional be used on POJOs

• Added spring-boot-oas3-nullable.yaml to generate nullable examples to validate against
• Added spring-boot-oas3-nullable-disabled.yaml to generate optional behaviour when nullable is disabled
• [BUG] [JAVA] [SPRING] Optional values aren't optional in POJOs, only in path params #11383

NOTE: This is a breaking change for anyone with useOptional enabled.

PR checklist

• Read the contribution guidelines.
• Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
• Run the following to build the project and update samples:

  ./mvnw clean package ./bin/generate-samples.sh
  ./bin/utils/export_docs_generators.sh
  

  Commit all changed files.
  This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
  These must match the expectations made by your contribution.
  You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
• File the PR against the correct branch: master (5.3.0), 6.0.x
• If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Spec that shows the use case

ApiDefault:
type: object
required:
- requiredMessage
- requiredNullableMessage
properties:
message:
type: string
description: A message
nullableMessage:
type: string
description: A nullable message
nullable: true
requiredMessage:
type: string
description: A required message
requiredNullableMessage:
type: string
description: A required nullable message
nullable: true

Output

useOptional: false
openApiNullable: false
private String message;
private String nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: true
openApiNullable: false
private Optional<String> message;
private Optional<String> nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: false
openApiNullable: true
private String message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

useOptional: true
openApiNullable: true
private Optional<String> message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

[welshm]

Note: This validates against 3_0

[welshm]

@wing328@cachescrubber

@bbdouglas (2017/07) @sreeshas (2017/08) @jfiala (2017/08) @lukoyanov (2017/09) @cbornet (2017/09) @jeff9finger (2018/01) @karismann (2019/03) @Zomzog (2019/04) @lwlee2608 (2019/10)

[wing328]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

[welshm]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

That is my opinion on how useOptional should function, yes. I will post the issue to the Slack to see if anyone has any differing opinions

[welshm]

@MelleD I tried to cover all relevant cases I could think of here

[welshm]

A quick test shows this to be working with both enabled:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JsonNullableModule());
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

However, I don't see nullability being enforced - if I don't set a required field (such as requiredNumber) this will still parse to/from JSON no problem. This is unrelated to Optional work here.

[wing328]

About the maven plug-in test failure, I think you will also need to update modules/openapi-generator-maven-plugin/examples/spring.xml

Test: https://github.com/OpenAPITools/openapi-generator/blob/master/.github/workflows/openapi-generator.yaml#L188

[cachescrubber]

@welshm You do not need to provide the jackson-datatype-jdk8 dependency. It is provided by spring-boot-starter-json which in turn is a transitive dependency of spring-boot-starter-web . The reason it failed on the maven-plugin test is the outdated version of spring-boot used there (2.2.1.RELEASE). Just update the spring-boot version instead.

The Jackson Jdk8Module is configured by default in an recent version of spring-boot. No need to provide it as an @bean.

[MelleD]

You should initialize the member with Optional.empty

A null optional field should never exists.

[welshm]

Good catch - fixed in next patch

[MelleD]

Bean validation is duplicated also in the getter. It should add to the getter

[welshm]

Hmm... that becomes tricky because the mustache generates both as {{>optionalDataTypeWithEnum}} which internally uses {{>optionalDataType}} which has {{>beanValidationCore}} as part of the type.

While it is redundant, I am fine with the duplication for how it makes the use of the mustache simpler.

Alernatively, we could have optionalDataTypeWithEnum and annotatedOptionalDataTypeWithEnum - but would require making the same distinction for the inner usage of optionalDataType.

[MelleD]

I see.
So far, optionals have only been used as parameters. I think for member variables we need a new mustache without bean validation. Because the beanvalidation is in <> always makes it awkward.

[welshm]

Created optionalDataTypeWithoutAnnotations - I did leave it on the setter but I'm not that familiar with how the validation is applied. I can convert it to getter only if that's more correct.

[MelleD]

Bean validation for DTOs/Pojos have to be set on getter or fields. Currently the template set it on getter

[welshm]

Will remove from setter then

[MelleD]

Is this really working with Optional?

[welshm]

It seems to be from my test:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JavaTimeModule())
.registerModule(new JsonNullableModule())
;
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.optionalDate(LocalDate.of(2020, 10, 2))
.optionalDateTime(OffsetDateTime.of(LocalDateTime.of(2020, 10, 2, 12, 30), ZoneOffset.UTC))
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

I can see the date time and read it back - it matches. JSON output looks fine too.

[cachescrubber]

I would prefer to use an include here. The code is difficult to read and hard to understand. Isn't it possible to use optionalDataType.mustache?

[welshm]

This comment seems old - but I need to fix the bean annotations anyway so this might change to an include

[cachescrubber]

Old, but still valid IMO

[welshm]

Moved it's own include - I had trouble coming up with a good name so it's dataTypeDeclaration - which is kind of misleading because these are all data type declarations.

Open to suggestions of better names - and I'm taking a look at your wrapper type change now

[cachescrubber]

Have you considered to override AbstractJavaCodegen#toDefaultValue() in SpringCodegen? I would prefer to avoid logic in the templates if possible.

[welshm]

Sure I can look into that. I did it this way because that's how it was already done for JsonNullable but I'll try tk improve it

[cachescrubber]

Not as straight forward as one might think (but doable) I currently doing a proof of concept with JsonNullable.

[welshm]

Might make sense to do it as a separate PR then? Consolidate the defaultValue decision and generation?

[cachescrubber]

See #11666

[cachescrubber]

Looks good!

[welshm]

I think this can still be improved, but it'll be a lot easier with https://github.com/OpenAPITools/openapi-generator/pull/11666/files so I'll iterate on it after that

[welshm]

Did you want to get #11666 in first? This change is good to go otherwise, but might be easier to deal with merge conflicts and resolve any unit tests changes if I update this after yours is in first.

[welshm]

Closing - since this is way out of date

[welshm]

Modified pojo.mustache to have Optional be used on POJOs

• Added spring-boot-oas3-nullable.yaml to generate nullable examples to validate against
• Added spring-boot-oas3-nullable-disabled.yaml to generate optional behaviour when nullable is disabled
• [BUG] [JAVA] [SPRING] Optional values aren't optional in POJOs, only in path params #11383

NOTE: This is a breaking change for anyone with useOptional enabled.

PR checklist

• Read the contribution guidelines.
• Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
• Run the following to build the project and update samples:

  ./mvnw clean package ./bin/generate-samples.sh
  ./bin/utils/export_docs_generators.sh
  

  Commit all changed files.
  This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
  These must match the expectations made by your contribution.
  You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
• File the PR against the correct branch: master (5.3.0), 6.0.x
• If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Spec that shows the use case

ApiDefault:
type: object
required:
- requiredMessage
- requiredNullableMessage
properties:
message:
type: string
description: A message
nullableMessage:
type: string
description: A nullable message
nullable: true
requiredMessage:
type: string
description: A required message
requiredNullableMessage:
type: string
description: A required nullable message
nullable: true

Output

useOptional: false
openApiNullable: false
private String message;
private String nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: true
openApiNullable: false
private Optional<String> message;
private Optional<String> nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: false
openApiNullable: true
private String message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

useOptional: true
openApiNullable: true
private Optional<String> message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

[welshm]

Note: This validates against 3_0

[welshm]

@wing328@cachescrubber

@bbdouglas (2017/07) @sreeshas (2017/08) @jfiala (2017/08) @lukoyanov (2017/09) @cbornet (2017/09) @jeff9finger (2018/01) @karismann (2019/03) @Zomzog (2019/04) @lwlee2608 (2019/10)

[wing328]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

[welshm]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

That is my opinion on how useOptional should function, yes. I will post the issue to the Slack to see if anyone has any differing opinions

[welshm]

@MelleD I tried to cover all relevant cases I could think of here

[welshm]

A quick test shows this to be working with both enabled:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JsonNullableModule());
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

However, I don't see nullability being enforced - if I don't set a required field (such as requiredNumber) this will still parse to/from JSON no problem. This is unrelated to Optional work here.

[wing328]

About the maven plug-in test failure, I think you will also need to update modules/openapi-generator-maven-plugin/examples/spring.xml

Test: https://github.com/OpenAPITools/openapi-generator/blob/master/.github/workflows/openapi-generator.yaml#L188

[cachescrubber]

@welshm You do not need to provide the jackson-datatype-jdk8 dependency. It is provided by spring-boot-starter-json which in turn is a transitive dependency of spring-boot-starter-web . The reason it failed on the maven-plugin test is the outdated version of spring-boot used there (2.2.1.RELEASE). Just update the spring-boot version instead.

The Jackson Jdk8Module is configured by default in an recent version of spring-boot. No need to provide it as an @bean.

[MelleD]

You should initialize the member with Optional.empty

A null optional field should never exists.

[welshm]

Good catch - fixed in next patch

[MelleD]

Bean validation is duplicated also in the getter. It should add to the getter

[welshm]

Hmm... that becomes tricky because the mustache generates both as {{>optionalDataTypeWithEnum}} which internally uses {{>optionalDataType}} which has {{>beanValidationCore}} as part of the type.

While it is redundant, I am fine with the duplication for how it makes the use of the mustache simpler.

Alernatively, we could have optionalDataTypeWithEnum and annotatedOptionalDataTypeWithEnum - but would require making the same distinction for the inner usage of optionalDataType.

[MelleD]

I see.
So far, optionals have only been used as parameters. I think for member variables we need a new mustache without bean validation. Because the beanvalidation is in <> always makes it awkward.

[welshm]

Created optionalDataTypeWithoutAnnotations - I did leave it on the setter but I'm not that familiar with how the validation is applied. I can convert it to getter only if that's more correct.

[MelleD]

Bean validation for DTOs/Pojos have to be set on getter or fields. Currently the template set it on getter

[welshm]

Will remove from setter then

[MelleD]

Is this really working with Optional?

[welshm]

It seems to be from my test:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JavaTimeModule())
.registerModule(new JsonNullableModule())
;
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.optionalDate(LocalDate.of(2020, 10, 2))
.optionalDateTime(OffsetDateTime.of(LocalDateTime.of(2020, 10, 2, 12, 30), ZoneOffset.UTC))
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

I can see the date time and read it back - it matches. JSON output looks fine too.

[cachescrubber]

I would prefer to use an include here. The code is difficult to read and hard to understand. Isn't it possible to use optionalDataType.mustache?

[welshm]

This comment seems old - but I need to fix the bean annotations anyway so this might change to an include

[cachescrubber]

Old, but still valid IMO

[welshm]

Moved it's own include - I had trouble coming up with a good name so it's dataTypeDeclaration - which is kind of misleading because these are all data type declarations.

Open to suggestions of better names - and I'm taking a look at your wrapper type change now

[cachescrubber]

Have you considered to override AbstractJavaCodegen#toDefaultValue() in SpringCodegen? I would prefer to avoid logic in the templates if possible.

[welshm]

Sure I can look into that. I did it this way because that's how it was already done for JsonNullable but I'll try tk improve it

[cachescrubber]

Not as straight forward as one might think (but doable) I currently doing a proof of concept with JsonNullable.

[welshm]

Might make sense to do it as a separate PR then? Consolidate the defaultValue decision and generation?

[cachescrubber]

See #11666

[cachescrubber]

Looks good!

[welshm]

I think this can still be improved, but it'll be a lot easier with https://github.com/OpenAPITools/openapi-generator/pull/11666/files so I'll iterate on it after that

[welshm]

Did you want to get #11666 in first? This change is good to go otherwise, but might be easier to deal with merge conflicts and resolve any unit tests changes if I update this after yours is in first.

[welshm]

Closing - since this is way out of date

[welshm]

Modified pojo.mustache to have Optional be used on POJOs

• Added spring-boot-oas3-nullable.yaml to generate nullable examples to validate against
• Added spring-boot-oas3-nullable-disabled.yaml to generate optional behaviour when nullable is disabled
• [BUG] [JAVA] [SPRING] Optional values aren't optional in POJOs, only in path params #11383

NOTE: This is a breaking change for anyone with useOptional enabled.

PR checklist

• Read the contribution guidelines.
• Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
• Run the following to build the project and update samples:

  ./mvnw clean package ./bin/generate-samples.sh
  ./bin/utils/export_docs_generators.sh
  

  Commit all changed files.
  This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
  These must match the expectations made by your contribution.
  You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
• File the PR against the correct branch: master (5.3.0), 6.0.x
• If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Spec that shows the use case

ApiDefault:
type: object
required:
- requiredMessage
- requiredNullableMessage
properties:
message:
type: string
description: A message
nullableMessage:
type: string
description: A nullable message
nullable: true
requiredMessage:
type: string
description: A required message
requiredNullableMessage:
type: string
description: A required nullable message
nullable: true

Output

useOptional: false
openApiNullable: false
private String message;
private String nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: true
openApiNullable: false
private Optional<String> message;
private Optional<String> nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: false
openApiNullable: true
private String message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

useOptional: true
openApiNullable: true
private Optional<String> message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

[welshm]

Note: This validates against 3_0

[welshm]

@wing328@cachescrubber

@bbdouglas (2017/07) @sreeshas (2017/08) @jfiala (2017/08) @lukoyanov (2017/09) @cbornet (2017/09) @jeff9finger (2018/01) @karismann (2019/03) @Zomzog (2019/04) @lwlee2608 (2019/10)

[wing328]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

[welshm]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

That is my opinion on how useOptional should function, yes. I will post the issue to the Slack to see if anyone has any differing opinions

[welshm]

@MelleD I tried to cover all relevant cases I could think of here

[welshm]

A quick test shows this to be working with both enabled:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JsonNullableModule());
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

However, I don't see nullability being enforced - if I don't set a required field (such as requiredNumber) this will still parse to/from JSON no problem. This is unrelated to Optional work here.

[wing328]

About the maven plug-in test failure, I think you will also need to update modules/openapi-generator-maven-plugin/examples/spring.xml

Test: https://github.com/OpenAPITools/openapi-generator/blob/master/.github/workflows/openapi-generator.yaml#L188

[cachescrubber]

@welshm You do not need to provide the jackson-datatype-jdk8 dependency. It is provided by spring-boot-starter-json which in turn is a transitive dependency of spring-boot-starter-web . The reason it failed on the maven-plugin test is the outdated version of spring-boot used there (2.2.1.RELEASE). Just update the spring-boot version instead.

The Jackson Jdk8Module is configured by default in an recent version of spring-boot. No need to provide it as an @bean.

[MelleD]

You should initialize the member with Optional.empty

A null optional field should never exists.

[welshm]

Good catch - fixed in next patch

[MelleD]

Bean validation is duplicated also in the getter. It should add to the getter

[welshm]

Hmm... that becomes tricky because the mustache generates both as {{>optionalDataTypeWithEnum}} which internally uses {{>optionalDataType}} which has {{>beanValidationCore}} as part of the type.

While it is redundant, I am fine with the duplication for how it makes the use of the mustache simpler.

Alernatively, we could have optionalDataTypeWithEnum and annotatedOptionalDataTypeWithEnum - but would require making the same distinction for the inner usage of optionalDataType.

[MelleD]

I see.
So far, optionals have only been used as parameters. I think for member variables we need a new mustache without bean validation. Because the beanvalidation is in <> always makes it awkward.

[welshm]

Created optionalDataTypeWithoutAnnotations - I did leave it on the setter but I'm not that familiar with how the validation is applied. I can convert it to getter only if that's more correct.

[MelleD]

Bean validation for DTOs/Pojos have to be set on getter or fields. Currently the template set it on getter

[welshm]

Will remove from setter then

[MelleD]

Is this really working with Optional?

[welshm]

It seems to be from my test:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JavaTimeModule())
.registerModule(new JsonNullableModule())
;
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.optionalDate(LocalDate.of(2020, 10, 2))
.optionalDateTime(OffsetDateTime.of(LocalDateTime.of(2020, 10, 2, 12, 30), ZoneOffset.UTC))
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

I can see the date time and read it back - it matches. JSON output looks fine too.

[cachescrubber]

I would prefer to use an include here. The code is difficult to read and hard to understand. Isn't it possible to use optionalDataType.mustache?

[welshm]

This comment seems old - but I need to fix the bean annotations anyway so this might change to an include

[cachescrubber]

Old, but still valid IMO

[welshm]

Moved it's own include - I had trouble coming up with a good name so it's dataTypeDeclaration - which is kind of misleading because these are all data type declarations.

Open to suggestions of better names - and I'm taking a look at your wrapper type change now

[cachescrubber]

Have you considered to override AbstractJavaCodegen#toDefaultValue() in SpringCodegen? I would prefer to avoid logic in the templates if possible.

[welshm]

Sure I can look into that. I did it this way because that's how it was already done for JsonNullable but I'll try tk improve it

[cachescrubber]

Not as straight forward as one might think (but doable) I currently doing a proof of concept with JsonNullable.

[welshm]

Might make sense to do it as a separate PR then? Consolidate the defaultValue decision and generation?

[cachescrubber]

See #11666

[cachescrubber]

Looks good!

[welshm]

I think this can still be improved, but it'll be a lot easier with https://github.com/OpenAPITools/openapi-generator/pull/11666/files so I'll iterate on it after that

[welshm]

Did you want to get #11666 in first? This change is good to go otherwise, but might be easier to deal with merge conflicts and resolve any unit tests changes if I update this after yours is in first.

[welshm]

Closing - since this is way out of date

[welshm]

Modified pojo.mustache to have Optional be used on POJOs

• Added spring-boot-oas3-nullable.yaml to generate nullable examples to validate against
• Added spring-boot-oas3-nullable-disabled.yaml to generate optional behaviour when nullable is disabled
• [BUG] [JAVA] [SPRING] Optional values aren't optional in POJOs, only in path params #11383

NOTE: This is a breaking change for anyone with useOptional enabled.

PR checklist

• Read the contribution guidelines.
• Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
• Run the following to build the project and update samples:

  ./mvnw clean package ./bin/generate-samples.sh
  ./bin/utils/export_docs_generators.sh
  

  Commit all changed files.
  This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
  These must match the expectations made by your contribution.
  You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
• File the PR against the correct branch: master (5.3.0), 6.0.x
• If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Spec that shows the use case

ApiDefault:
type: object
required:
- requiredMessage
- requiredNullableMessage
properties:
message:
type: string
description: A message
nullableMessage:
type: string
description: A nullable message
nullable: true
requiredMessage:
type: string
description: A required message
requiredNullableMessage:
type: string
description: A required nullable message
nullable: true

Output

useOptional: false
openApiNullable: false
private String message;
private String nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: true
openApiNullable: false
private Optional<String> message;
private Optional<String> nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: false
openApiNullable: true
private String message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

useOptional: true
openApiNullable: true
private Optional<String> message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

[welshm]

Note: This validates against 3_0

[welshm]

@wing328@cachescrubber

@bbdouglas (2017/07) @sreeshas (2017/08) @jfiala (2017/08) @lukoyanov (2017/09) @cbornet (2017/09) @jeff9finger (2018/01) @karismann (2019/03) @Zomzog (2019/04) @lwlee2608 (2019/10)

[wing328]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

[welshm]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

That is my opinion on how useOptional should function, yes. I will post the issue to the Slack to see if anyone has any differing opinions

[welshm]

@MelleD I tried to cover all relevant cases I could think of here

[welshm]

A quick test shows this to be working with both enabled:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JsonNullableModule());
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

However, I don't see nullability being enforced - if I don't set a required field (such as requiredNumber) this will still parse to/from JSON no problem. This is unrelated to Optional work here.

[wing328]

About the maven plug-in test failure, I think you will also need to update modules/openapi-generator-maven-plugin/examples/spring.xml

Test: https://github.com/OpenAPITools/openapi-generator/blob/master/.github/workflows/openapi-generator.yaml#L188

[cachescrubber]

@welshm You do not need to provide the jackson-datatype-jdk8 dependency. It is provided by spring-boot-starter-json which in turn is a transitive dependency of spring-boot-starter-web . The reason it failed on the maven-plugin test is the outdated version of spring-boot used there (2.2.1.RELEASE). Just update the spring-boot version instead.

The Jackson Jdk8Module is configured by default in an recent version of spring-boot. No need to provide it as an @bean.

[MelleD]

You should initialize the member with Optional.empty

A null optional field should never exists.

[welshm]

Good catch - fixed in next patch

[MelleD]

Bean validation is duplicated also in the getter. It should add to the getter

[welshm]

Hmm... that becomes tricky because the mustache generates both as {{>optionalDataTypeWithEnum}} which internally uses {{>optionalDataType}} which has {{>beanValidationCore}} as part of the type.

While it is redundant, I am fine with the duplication for how it makes the use of the mustache simpler.

Alernatively, we could have optionalDataTypeWithEnum and annotatedOptionalDataTypeWithEnum - but would require making the same distinction for the inner usage of optionalDataType.

[MelleD]

I see.
So far, optionals have only been used as parameters. I think for member variables we need a new mustache without bean validation. Because the beanvalidation is in <> always makes it awkward.

[welshm]

Created optionalDataTypeWithoutAnnotations - I did leave it on the setter but I'm not that familiar with how the validation is applied. I can convert it to getter only if that's more correct.

[MelleD]

Bean validation for DTOs/Pojos have to be set on getter or fields. Currently the template set it on getter

[welshm]

Will remove from setter then

[MelleD]

Is this really working with Optional?

[welshm]

It seems to be from my test:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JavaTimeModule())
.registerModule(new JsonNullableModule())
;
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.optionalDate(LocalDate.of(2020, 10, 2))
.optionalDateTime(OffsetDateTime.of(LocalDateTime.of(2020, 10, 2, 12, 30), ZoneOffset.UTC))
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

I can see the date time and read it back - it matches. JSON output looks fine too.

[cachescrubber]

I would prefer to use an include here. The code is difficult to read and hard to understand. Isn't it possible to use optionalDataType.mustache?

[welshm]

This comment seems old - but I need to fix the bean annotations anyway so this might change to an include

[cachescrubber]

Old, but still valid IMO

[welshm]

Moved it's own include - I had trouble coming up with a good name so it's dataTypeDeclaration - which is kind of misleading because these are all data type declarations.

Open to suggestions of better names - and I'm taking a look at your wrapper type change now

[cachescrubber]

Have you considered to override AbstractJavaCodegen#toDefaultValue() in SpringCodegen? I would prefer to avoid logic in the templates if possible.

[welshm]

Sure I can look into that. I did it this way because that's how it was already done for JsonNullable but I'll try tk improve it

[cachescrubber]

Not as straight forward as one might think (but doable) I currently doing a proof of concept with JsonNullable.

[welshm]

Might make sense to do it as a separate PR then? Consolidate the defaultValue decision and generation?

[cachescrubber]

See #11666

[cachescrubber]

Looks good!

[welshm]

I think this can still be improved, but it'll be a lot easier with https://github.com/OpenAPITools/openapi-generator/pull/11666/files so I'll iterate on it after that

[welshm]

Did you want to get #11666 in first? This change is good to go otherwise, but might be easier to deal with merge conflicts and resolve any unit tests changes if I update this after yours is in first.

[welshm]

Closing - since this is way out of date

[welshm]

Modified pojo.mustache to have Optional be used on POJOs

• Added spring-boot-oas3-nullable.yaml to generate nullable examples to validate against
• Added spring-boot-oas3-nullable-disabled.yaml to generate optional behaviour when nullable is disabled
• [BUG] [JAVA] [SPRING] Optional values aren't optional in POJOs, only in path params #11383

NOTE: This is a breaking change for anyone with useOptional enabled.

PR checklist

• Read the contribution guidelines.
• Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
• Run the following to build the project and update samples:

  ./mvnw clean package ./bin/generate-samples.sh
  ./bin/utils/export_docs_generators.sh
  

  Commit all changed files.
  This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
  These must match the expectations made by your contribution.
  You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
• File the PR against the correct branch: master (5.3.0), 6.0.x
• If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Spec that shows the use case

ApiDefault:
type: object
required:
- requiredMessage
- requiredNullableMessage
properties:
message:
type: string
description: A message
nullableMessage:
type: string
description: A nullable message
nullable: true
requiredMessage:
type: string
description: A required message
requiredNullableMessage:
type: string
description: A required nullable message
nullable: true

Output

useOptional: false
openApiNullable: false
private String message;
private String nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: true
openApiNullable: false
private Optional<String> message;
private Optional<String> nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: false
openApiNullable: true
private String message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

useOptional: true
openApiNullable: true
private Optional<String> message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

[welshm]

Note: This validates against 3_0

[welshm]

@wing328@cachescrubber

@bbdouglas (2017/07) @sreeshas (2017/08) @jfiala (2017/08) @lukoyanov (2017/09) @cbornet (2017/09) @jeff9finger (2018/01) @karismann (2019/03) @Zomzog (2019/04) @lwlee2608 (2019/10)

[wing328]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

[welshm]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

That is my opinion on how useOptional should function, yes. I will post the issue to the Slack to see if anyone has any differing opinions

[welshm]

@MelleD I tried to cover all relevant cases I could think of here

[welshm]

A quick test shows this to be working with both enabled:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JsonNullableModule());
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

However, I don't see nullability being enforced - if I don't set a required field (such as requiredNumber) this will still parse to/from JSON no problem. This is unrelated to Optional work here.

[wing328]

About the maven plug-in test failure, I think you will also need to update modules/openapi-generator-maven-plugin/examples/spring.xml

Test: https://github.com/OpenAPITools/openapi-generator/blob/master/.github/workflows/openapi-generator.yaml#L188

[cachescrubber]

@welshm You do not need to provide the jackson-datatype-jdk8 dependency. It is provided by spring-boot-starter-json which in turn is a transitive dependency of spring-boot-starter-web . The reason it failed on the maven-plugin test is the outdated version of spring-boot used there (2.2.1.RELEASE). Just update the spring-boot version instead.

The Jackson Jdk8Module is configured by default in an recent version of spring-boot. No need to provide it as an @bean.

[MelleD]

You should initialize the member with Optional.empty

A null optional field should never exists.

[welshm]

Good catch - fixed in next patch

[MelleD]

Bean validation is duplicated also in the getter. It should add to the getter

[welshm]

Hmm... that becomes tricky because the mustache generates both as {{>optionalDataTypeWithEnum}} which internally uses {{>optionalDataType}} which has {{>beanValidationCore}} as part of the type.

While it is redundant, I am fine with the duplication for how it makes the use of the mustache simpler.

Alernatively, we could have optionalDataTypeWithEnum and annotatedOptionalDataTypeWithEnum - but would require making the same distinction for the inner usage of optionalDataType.

[MelleD]

I see.
So far, optionals have only been used as parameters. I think for member variables we need a new mustache without bean validation. Because the beanvalidation is in <> always makes it awkward.

[welshm]

Created optionalDataTypeWithoutAnnotations - I did leave it on the setter but I'm not that familiar with how the validation is applied. I can convert it to getter only if that's more correct.

[MelleD]

Bean validation for DTOs/Pojos have to be set on getter or fields. Currently the template set it on getter

[welshm]

Will remove from setter then

[MelleD]

Is this really working with Optional?

[welshm]

It seems to be from my test:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JavaTimeModule())
.registerModule(new JsonNullableModule())
;
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.optionalDate(LocalDate.of(2020, 10, 2))
.optionalDateTime(OffsetDateTime.of(LocalDateTime.of(2020, 10, 2, 12, 30), ZoneOffset.UTC))
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

I can see the date time and read it back - it matches. JSON output looks fine too.

[cachescrubber]

I would prefer to use an include here. The code is difficult to read and hard to understand. Isn't it possible to use optionalDataType.mustache?

[welshm]

This comment seems old - but I need to fix the bean annotations anyway so this might change to an include

[cachescrubber]

Old, but still valid IMO

[welshm]

Moved it's own include - I had trouble coming up with a good name so it's dataTypeDeclaration - which is kind of misleading because these are all data type declarations.

Open to suggestions of better names - and I'm taking a look at your wrapper type change now

[cachescrubber]

Have you considered to override AbstractJavaCodegen#toDefaultValue() in SpringCodegen? I would prefer to avoid logic in the templates if possible.

[welshm]

Sure I can look into that. I did it this way because that's how it was already done for JsonNullable but I'll try tk improve it

[cachescrubber]

Not as straight forward as one might think (but doable) I currently doing a proof of concept with JsonNullable.

[welshm]

Might make sense to do it as a separate PR then? Consolidate the defaultValue decision and generation?

[cachescrubber]

See #11666

[cachescrubber]

Looks good!

[welshm]

I think this can still be improved, but it'll be a lot easier with https://github.com/OpenAPITools/openapi-generator/pull/11666/files so I'll iterate on it after that

[welshm]

Did you want to get #11666 in first? This change is good to go otherwise, but might be easier to deal with merge conflicts and resolve any unit tests changes if I update this after yours is in first.

[welshm]

Closing - since this is way out of date

[welshm]

Modified pojo.mustache to have Optional be used on POJOs

• Added spring-boot-oas3-nullable.yaml to generate nullable examples to validate against
• Added spring-boot-oas3-nullable-disabled.yaml to generate optional behaviour when nullable is disabled
• [BUG] [JAVA] [SPRING] Optional values aren't optional in POJOs, only in path params #11383

NOTE: This is a breaking change for anyone with useOptional enabled.

PR checklist

• Read the contribution guidelines.
• Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
• Run the following to build the project and update samples:

  ./mvnw clean package ./bin/generate-samples.sh
  ./bin/utils/export_docs_generators.sh
  

  Commit all changed files.
  This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
  These must match the expectations made by your contribution.
  You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
• File the PR against the correct branch: master (5.3.0), 6.0.x
• If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

Spec that shows the use case

ApiDefault:
type: object
required:
- requiredMessage
- requiredNullableMessage
properties:
message:
type: string
description: A message
nullableMessage:
type: string
description: A nullable message
nullable: true
requiredMessage:
type: string
description: A required message
requiredNullableMessage:
type: string
description: A required nullable message
nullable: true

Output

useOptional: false
openApiNullable: false
private String message;
private String nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: true
openApiNullable: false
private Optional<String> message;
private Optional<String> nullableMessage
private String requiredMessage;
private String requiredNullableMessage;

useOptional: false
openApiNullable: true
private String message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

useOptional: true
openApiNullable: true
private Optional<String> message;
private JsonNullable<String> nullableMessage
private String requiredMessage;
private JsonNullable<String> requiredNullableMessage;

[welshm]

Note: This validates against 3_0

[welshm]

@wing328@cachescrubber

@bbdouglas (2017/07) @sreeshas (2017/08) @jfiala (2017/08) @lukoyanov (2017/09) @cbornet (2017/09) @jeff9finger (2018/01) @karismann (2019/03) @Zomzog (2019/04) @lwlee2608 (2019/10)

[wing328]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

[welshm]

NOTE: This is a breaking change for anyone with useOptional enabled.

Thanks for the PR. Is it correct to say that the uesOptional option will work as expected after this PR gets merged? If other words, this PR aims to fix issues when the useOptional option is set to true, right?

That is my opinion on how useOptional should function, yes. I will post the issue to the Slack to see if anyone has any differing opinions

[welshm]

@MelleD I tried to cover all relevant cases I could think of here

[welshm]

A quick test shows this to be working with both enabled:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JsonNullableModule());
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

However, I don't see nullability being enforced - if I don't set a required field (such as requiredNumber) this will still parse to/from JSON no problem. This is unrelated to Optional work here.

[wing328]

About the maven plug-in test failure, I think you will also need to update modules/openapi-generator-maven-plugin/examples/spring.xml

Test: https://github.com/OpenAPITools/openapi-generator/blob/master/.github/workflows/openapi-generator.yaml#L188

[cachescrubber]

@welshm You do not need to provide the jackson-datatype-jdk8 dependency. It is provided by spring-boot-starter-json which in turn is a transitive dependency of spring-boot-starter-web . The reason it failed on the maven-plugin test is the outdated version of spring-boot used there (2.2.1.RELEASE). Just update the spring-boot version instead.

The Jackson Jdk8Module is configured by default in an recent version of spring-boot. No need to provide it as an @bean.

[MelleD]

You should initialize the member with Optional.empty

A null optional field should never exists.

[welshm]

Good catch - fixed in next patch

[MelleD]

Bean validation is duplicated also in the getter. It should add to the getter

[welshm]

Hmm... that becomes tricky because the mustache generates both as {{>optionalDataTypeWithEnum}} which internally uses {{>optionalDataType}} which has {{>beanValidationCore}} as part of the type.

While it is redundant, I am fine with the duplication for how it makes the use of the mustache simpler.

Alernatively, we could have optionalDataTypeWithEnum and annotatedOptionalDataTypeWithEnum - but would require making the same distinction for the inner usage of optionalDataType.

[MelleD]

I see.
So far, optionals have only been used as parameters. I think for member variables we need a new mustache without bean validation. Because the beanvalidation is in <> always makes it awkward.

[welshm]

Created optionalDataTypeWithoutAnnotations - I did leave it on the setter but I'm not that familiar with how the validation is applied. I can convert it to getter only if that's more correct.

[MelleD]

Bean validation for DTOs/Pojos have to be set on getter or fields. Currently the template set it on getter

[welshm]

Will remove from setter then

[MelleD]

Is this really working with Optional?

[welshm]

It seems to be from my test:

 @Test
void jdk8ModuleObjectMapper() throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module().configureAbsentsAsNulls(true))
.registerModule(new JavaTimeModule())
.registerModule(new JsonNullableModule())
;
final Params params = new Params()
.optionalString("OPTIONAL STRING")
.requiredString("REQUIRED STRING")
.requiredNullableString(null)
.requiredEnum(RequiredEnumEnum.ONE)
.optionalDate(LocalDate.of(2020, 10, 2))
.optionalDateTime(OffsetDateTime.of(LocalDateTime.of(2020, 10, 2, 12, 30), ZoneOffset.UTC))
.requiredNumber(123L);
final String jsonResult = objectMapper.writeValueAsString(params);
final Params readBack = objectMapper.readValue(jsonResult, Params.class);
System.console().printf(readBack.toString());
}

I can see the date time and read it back - it matches. JSON output looks fine too.

[cachescrubber]

I would prefer to use an include here. The code is difficult to read and hard to understand. Isn't it possible to use optionalDataType.mustache?

[welshm]

This comment seems old - but I need to fix the bean annotations anyway so this might change to an include

[cachescrubber]

Old, but still valid IMO

[welshm]

Moved it's own include - I had trouble coming up with a good name so it's dataTypeDeclaration - which is kind of misleading because these are all data type declarations.

Open to suggestions of better names - and I'm taking a look at your wrapper type change now

[cachescrubber]

Have you considered to override AbstractJavaCodegen#toDefaultValue() in SpringCodegen? I would prefer to avoid logic in the templates if possible.

[welshm]

Sure I can look into that. I did it this way because that's how it was already done for JsonNullable but I'll try tk improve it

[cachescrubber]

Not as straight forward as one might think (but doable) I currently doing a proof of concept with JsonNullable.

[welshm]

Might make sense to do it as a separate PR then? Consolidate the defaultValue decision and generation?

[cachescrubber]

See #11666

[cachescrubber]

Looks good!

[welshm]

I think this can still be improved, but it'll be a lot easier with https://github.com/OpenAPITools/openapi-generator/pull/11666/files so I'll iterate on it after that

[welshm]

Did you want to get #11666 in first? This change is good to go otherwise, but might be easier to deal with merge conflicts and resolve any unit tests changes if I update this after yours is in first.

[welshm]

Closing - since this is way out of date