[BUG] [JavaScript] java.lang.OutOfMemoryError on simple OpenAPI file

[advance512]

Description

Using the following OpenAPI file below, the following error occurs:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOfRange(Arrays.java:3664)
	at java.lang.String.<init>(String.java:207)
	at java.lang.StringBuilder.toString(StringBuilder.java:407)
	at com.fasterxml.jackson.core.util.TextBuffer.contentsAsString(TextBuffer.java:404)
	at com.fasterxml.jackson.core.io.SegmentedStringWriter.getAndClear(SegmentedStringWriter.java:83)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsString(ObjectWriter.java:999)
	at io.swagger.v3.core.util.Json.pretty(Json.java:24)
	at org.openapitools.codegen.DefaultCodegen.fromModel(DefaultCodegen.java:1627)
	at org.openapitools.codegen.languages.JavascriptClientCodegen.fromModel(JavascriptClientCodegen.java:842)
	at org.openapitools.codegen.DefaultCodegen.fromOperation(DefaultCodegen.java:2437)
	at org.openapitools.codegen.DefaultGenerator.processOperation(DefaultGenerator.java:1021)
	at org.openapitools.codegen.DefaultGenerator.processPaths(DefaultGenerator.java:944)
	at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:516)
	at org.openapitools.codegen.DefaultGenerator.generate(DefaultGenerator.java:902)
	at org.openapitools.codegen.cmd.Generate.run(Generate.java:368)
	at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:60)

openapi-generator version

This happens both in openapi-generator-cli-4.0.0-20190325.151629-451.jar and openapi-generator-cli-3.3.4.jar.

OpenAPI declaration file content or url

swagger: '2.0'
info:
 version: "1.0"
 title: DEV-2369

# ===============================================================================
# Paths
# ===============================================================================

paths:

 /users/responders:

 get:
 operationId: getUserResponderList
 summary: Gets a list of Responder objects
 responses:
 '200':
 description: A list of Responder objects
 schema:
 $ref: '#/definitions/ResponderList'
 '500':
 $ref: '#/responses/InternalServerError'
 default:
 $ref: '#/responses/TotallyUnexpectedResponse'

 /responders:

 post:
 operationId: createResponder
 parameters:
 - name: responder
 in: body
 required: true
 schema:
 $ref: '#/definitions/ResponderCreate'
 responses:
 '201':
 description: A Responder object
 schema:
 $ref: '#/definitions/Responder'
 '400':
 $ref: '#/responses/BadRequest'
 '500':
 $ref: '#/responses/InternalServerError'
 default:
 $ref: '#/responses/TotallyUnexpectedResponse'


# ===============================================================================
# Definitions
# ===============================================================================

definitions:

 ID:
 type: integer
 format: int64
 readOnly: true

 Reviewer:
 type: object
 properties:
 name:
 type: string
 maxLength: 100

 SuccessfulBugFix:
 type: object
 properties:
 reviewers:
 type: array
 minItems: 0
 maxItems: 100
 uniqueItems: true
 items:
 $ref: '#/definitions/Reviewer'

 ResponderProfileCreate:
 type: object
 properties:
 successfulBugFixes:
 type: array
 minItems: 0
 maxItems: 100
 uniqueItems: true
 items:
 $ref: '#/definitions/SuccessfulBugFix'

 ResponderProfile:
 type: object
 properties:
 successfulBugFixes:
 type: array
 minItems: 0
 maxItems: 100
 uniqueItems: true
 items:
 $ref: '#/definitions/SuccessfulBugFix'

 Error:
 type: object
 required:
 - message
 properties:
 message:
 type: string


 User:
 type: object
 properties:
 id:
 $ref: '#/definitions/ID'

 Responder:
 type: object
 properties:
 id:
 $ref: '#/definitions/ID'
 responderProfile:
 $ref: '#/definitions/ResponderProfile'

 # Used only for the creation of a new Responder
 ResponderCreate:
 type: object
 properties:
 responderProfile:
 $ref: '#/definitions/ResponderProfileCreate'

 ResponderList:
 type: object
 required:
 - items
 properties:
 items:
 type: array
 minItems: 0
 maxItems: 1000
 uniqueItems: true
 items:
 $ref: '#/definitions/Responder'



# ===============================================================================
# Responses
# ===============================================================================

responses:

 InternalServerError:
 description: An unexpected error occured.
 schema:
 $ref: '#/definitions/Error'
 BadRequest:
 description: Bad request; could not perform requested operation.
 EntityDoesNotExist:
 description: Entity does not exist.
 TotallyUnexpectedResponse:
 description: A totally unexpected response

Command line used for generation

java -jar ./openapi-generator-cli-3.3.4.jar generate -i ../openapi.yaml -l javascript --additional-properties usePromises=true --additional-properties useES6=false -o ./javascript/

also:
java -jar ./openapi-generator-cli-4.0.0-20190325.151629-451.jar generate -i ../openapi.yaml -g javascript -o ./javascript/

also:
java -jar ./openapi-generator-cli-4.0.0-20190325.151629-451.jar generate -i ../openapi.yaml -g javascript --additional-properties usePromises=true --additional-properties useES6=false -o ./javascript/

Steps to reproduce

1. Save the OpenAPI spec into a file called openapi.yaml.
2. Run either of the commands above.

Related issues/PRs

None that I am aware of.

Suggest a fix

Not sure.

[auto-labeler]

👍 Thanks for opening this issue!
🏷 I have applied any labels matching special text in your issue.

The team will review the labels and make any necessary changes.

[macjohnny]

by changing

definitions:
 Reviewer:
 type: string

the issue does not occur.
somehow the memory issue occurs due to the deep-nesting of objects

[advance512]

Super strange, ain't it? :)

[macjohnny]

It is. Maybe you could investigate the problem using the stacktrace?

[advance512]

We side-stepped the problem for now.
Unfortunately, I am not a Java developer and don't have time to go deep into this, as I am terribly busy with my own startup... but we will offer a bounty for fixing this, as soon as BountySource wake up and pay our previous bounties to those claiming them.

[macjohnny]

it seems that DefaultCodegen::fromModel is called twice for name = "ResponderList"
the first call works and schema in

openapi-generator/modules/openapi-generator/src/main/java/org/openapitools/codegen/DefaultCodegen.java

Line 1627 in 46e8ccb

m.modelJson = Json.pretty(schema);

contains the following object:

schema

class ObjectSchema {
 class Schema {
 title: null
 multipleOf: null
 maximum: null
 exclusiveMaximum: null
 minimum: null
 exclusiveMinimum: null
 maxLength: null
 minLength: null
 pattern: null
 maxItems: null
 minItems: null
 uniqueItems: null
 maxProperties: null
 minProperties: null
 required: [items]
 type: null
 not: null
 properties: {items=class ArraySchema {
 class Schema {
 title: null
 multipleOf: null
 maximum: null
 exclusiveMaximum: null
 minimum: null
 exclusiveMinimum: null
 maxLength: null
 minLength: null
 pattern: null
 maxItems: 1000
 minItems: 0
 uniqueItems: true
 maxProperties: null
 minProperties: null
 required: null
 type: null
 not: null
 properties: null
 additionalProperties: null
 description: null
 format: null
 $ref: null
 nullable: null
 readOnly: null
 writeOnly: null
 example: null
 externalDocs: null
 deprecated: null
 discriminator: null
 xml: null
 }
 type: array
 items: class Schema {
 title: null
 multipleOf: null
 maximum: null
 exclusiveMaximum: null
 minimum: null
 exclusiveMinimum: null
 maxLength: null
 minLength: null
 pattern: null
 maxItems: null
 minItems: null
 uniqueItems: null
 maxProperties: null
 minProperties: null
 required: null
 type: null
 not: null
 properties: null
 additionalProperties: null
 description: null
 format: null
 $ref: #/components/schemas/Responder
 nullable: null
 readOnly: null
 writeOnly: null
 example: null
 externalDocs: null
 deprecated: null
 discriminator: null
 xml: null
 }
 }}
 additionalProperties: null
 description: null
 format: null
 $ref: null
 nullable: null
 readOnly: null
 writeOnly: null
 example: null
 externalDocs: null
 deprecated: null
 discriminator: null
 xml: null
 }
 type: object
 defaultObject: null
}

but in the second call, schema contains the following object:

schema

class ObjectSchema {
 class Schema {
 title: null
 multipleOf: null
 maximum: null
 exclusiveMaximum: null
 minimum: null
 exclusiveMinimum: null
 maxLength: null
 minLength: null
 pattern: null
 maxItems: null
 minItems: null
 uniqueItems: null
 maxProperties: null
 minProperties: null
 required: [items]
 type: null
 not: null
 properties: {items=class ArraySchema {
 class Schema {
 title: null
 multipleOf: null
 maximum: null
 exclusiveMaximum: null
 minimum: null
 exclusiveMinimum: null
 maxLength: null
 minLength: null
 pattern: null
 maxItems: 1000
 minItems: 0
 uniqueItems: true
 maxProperties: null
 minProperties: null
 required: null
 type: null
 not: null
 properties: null
 additionalProperties: null
 description: null
 format: null
 $ref: null
 nullable: null
 readOnly: null
 writeOnly: null
 example: null
 externalDocs: null
 deprecated: null
 discriminator: null
 xml: null
 }
 type: array
 items: class Schema {
 title: null
 multipleOf: null
 maximum: null
 exclusiveMaximum: null
 minimum: null
 exclusiveMinimum: null
 maxLength: null
 minLength: null
 pattern: null
 maxItems: null
 minItems: null
 uniqueItems: null
 maxProperties: null
 minProperties: null
 required: null
 type: null
 not: null
 properties: null
 additionalProperties: null
 description: null
 format: null
 $ref: #/components/schemas/Responder
 nullable: null
 readOnly: null
 writeOnly: null
 example: null
 externalDocs: null
 deprecated: null
 discriminator: null
 xml: null
 }
 }}
 additionalProperties: null
 description: null
 format: null
 $ref: null
 nullable: null
 readOnly: null
 writeOnly: null
 example: {items=[Ljava.lang.Object;@1cfd1875}
 externalDocs: null
 deprecated: null
 discriminator: null
 xml: null
 }
 type: object
 defaultObject: null
}

The only difference seems to be

example: {items=[Ljava.lang.Object;@1cfd1875}

where items is an object with 1000 entries similar to

{responderProfile={successfulBugFixes=[Ljava.lang.Object;@1af7f54a}, id=0}

it seems that the example-generation with 1000 entries is clearly an overkill and leads to a memory issue for many nested objects.

[macjohnny]

the relevant code that generates the examples is located here:

openapi-generator/modules/openapi-generator/src/main/java/org/openapitools/codegen/examples/ExampleGenerator.java

Line 234 in 46e8ccb

int arrayLength = null == ((ArraySchema) property).getMaxItems() ? 2 : ((ArraySchema) property).getMaxItems();

it creates arrays of length maxItems.

@advance512 a simple workaround for you:
explicitly define an example in the models where you use type: array properties in combination with maxItems, e.g.:

definitions:
 ResponderList:
 type: object
 required:
 - items
 properties:
 items:
 type: array
 minItems: 0
 maxItems: 1000
 uniqueItems: true
 items:
 $ref: '#/definitions/Responder'
 example:
 items: []

so this is your adapted spec where the issue does not occur anymore:

adapted spec

swagger: '2.0'
info:
 version: "1.0"
 title: DEV-2369

# ===============================================================================
# Paths
# ===============================================================================

paths:

 /users/responders:

 get:
 operationId: getUserResponderList
 summary: Gets a list of Responder objects
 responses:
 '200':
 description: A list of Responder objects
 schema:
 $ref: '#/definitions/ResponderList'
 '500':
 $ref: '#/responses/InternalServerError'
 default:
 $ref: '#/responses/TotallyUnexpectedResponse'

 /responders:

 post:
 operationId: createResponder
 parameters:
 - name: responder
 in: body
 required: true
 schema:
 $ref: '#/definitions/ResponderCreate'
 responses:
 '201':
 description: A Responder object
 schema:
 $ref: '#/definitions/Responder'
 '400':
 $ref: '#/responses/BadRequest'
 '500':
 $ref: '#/responses/InternalServerError'
 default:
 $ref: '#/responses/TotallyUnexpectedResponse'


# ===============================================================================
# Definitions
# ===============================================================================

definitions:

 ID:
 type: integer
 format: int64
 readOnly: true

 Reviewer:
 type: object
 properties:
 name:
 type: string
 maxLength: 100

 SuccessfulBugFix:
 type: object
 properties:
 reviewers:
 type: array
 minItems: 0
 maxItems: 100
 uniqueItems: true
 items:
 $ref: '#/definitions/Reviewer'
 example:
 reviewers: []

 ResponderProfileCreate:
 type: object
 properties:
 successfulBugFixes:
 type: array
 minItems: 0
 maxItems: 100
 uniqueItems: true
 items:
 $ref: '#/definitions/SuccessfulBugFix'

 ResponderProfile:
 type: object
 properties:
 successfulBugFixes:
 type: array
 minItems: 0
 maxItems: 100
 uniqueItems: true
 items:
 $ref: '#/definitions/SuccessfulBugFix'
 example:
 successfulBugFixes: []

 Error:
 type: object
 required:
 - message
 properties:
 message:
 type: string


 User:
 type: object
 properties:
 id:
 $ref: '#/definitions/ID'

 Responder:
 type: object
 properties:
 id:
 $ref: '#/definitions/ID'
 responderProfile:
 $ref: '#/definitions/ResponderProfile'

 # Used only for the creation of a new Responder
 ResponderCreate:
 type: object
 properties:
 responderProfile:
 $ref: '#/definitions/ResponderProfileCreate'

 ResponderList:
 type: object
 required:
 - items
 properties:
 items:
 type: array
 minItems: 0
 maxItems: 1000
 uniqueItems: true
 items:
 $ref: '#/definitions/Responder'
 example:
 items: []



# ===============================================================================
# Responses
# ===============================================================================

responses:

 InternalServerError:
 description: An unexpected error occured.
 schema:
 $ref: '#/definitions/Error'
 BadRequest:
 description: Bad request; could not perform requested operation.
 EntityDoesNotExist:
 description: Entity does not exist.
 TotallyUnexpectedResponse:
 description: A totally unexpected response

your maxItems: 1000 is just a little below the value 1024 that would trigger a warning

openapi-generator/modules/openapi-generator/src/main/java/org/openapitools/codegen/examples/ExampleGenerator.java

Lines 234 to 244 in 46e8ccb

	int arrayLength = null == ((ArraySchema) property).getMaxItems() ? 2 : ((ArraySchema) property).getMaxItems();
	if (arrayLength == Integer.MAX_VALUE) {
	// swagger-jersey2-jaxrs generated spec may contain maxItem = 2147483647
	// semantically this means there is no upper limit
	// treating this as if the property was not present at all
	LOGGER.warn("The max items allowed in property {} of {} equals Integer.MAX_VALUE. Treating this as if no max items has been specified.", property, arrayLength);
	arrayLength = 2;
	} else if (arrayLength > 1024) {
	LOGGER.warn("The max items allowed in property {} is too large ({} items), restricting it to 1024 items", property, arrayLength);
	arrayLength = 1024;
	}

Anyway, even the trigger value of 1024 is not useful if the spec contains many nested objects.

If you feel like this answer solves the problem for you, feel free to contribute a bounty here: https://github.com/bithost-gmbh/ngx-mat-select-search#support-development

I think this issue should be closed as it is not really a bug that can be prevented in every possible situation.

[advance512]

Thanks for your help, @macjohnny.

Say, why does the generator instantiate a thousand objects of responderProfile? What is the point?
If we're talking about an example object that is being generated - first of all, where is it used? Second, why create the maximum items in an array, and not just.. 3 items, for example? Or 5? Is it not much more confusing, having an example with 1000 objects?

I think this also makes the generator itself slower than it should be.

Letting the generator crash is definitely a bug that should be fixed. It doesn't communicate anything of value to the person using the generator - there is no clear route of "how to fix the problem". A proper fix would be to cap the amount of items created in the example, I think.

We will definitely contribute a bounty. Could you cap the items in the example to 3 or 5?

[macjohnny]

I think you are right, limiting the number of objects to 5 does make sense.
The generated example is used here

openapi-generator/modules/openapi-generator/src/main/resources/JavaSpring/methodBody.mustache

Lines 10 to 13 in 3a0d520

	{{#async}} {{/async}}{{^async}}{{#jdk8}} {{/jdk8}}{{/async}} if (mediaType.isCompatibleWith(MediaType.valueOf("{{{contentType}}}"))) {
	{{#async}} {{/async}}{{^async}}{{#jdk8}} {{/jdk8}}{{/async}} ApiUtil.setExampleResponse(request, "{{{contentType}}}", "{{#lambdaRemoveLineBreak}}{{#lambdaEscapeDoubleQuote}}{{{example}}}{{/lambdaEscapeDoubleQuote}}{{/lambdaRemoveLineBreak}}");
	{{#async}} {{/async}}{{^async}}{{#jdk8}} {{/jdk8}}{{/async}} break;
	{{#async}} {{/async}}{{^async}}{{#jdk8}} {{/jdk8}}{{/async}} }

and results in some server api implementations in an example response, e.g.

openapi-generator/samples/server/petstore/springboot/src/main/java/org/openapitools/api/PetApi.java

Lines 85 to 89 in 3a0d520

	default ResponseEntity<List<Pet>> findPetsByStatus(@NotNull @ApiParam(value = "Status values that need to be considered for filter", required = true, allowableValues = "available, pending, sold") @Valid @RequestParam(value = "status", required = true) List<String> status) {
	getRequest().ifPresent(request -> {
	for (MediaType mediaType: MediaType.parseMediaTypes(request.getHeader("Accept"))) {
	if (mediaType.isCompatibleWith(MediaType.valueOf("application/json"))) {
	ApiUtil.setExampleResponse(request, "application/json", "{ \"photoUrls\" : [ \"photoUrls\", \"photoUrls\" ], \"name\" : \"doggie\", \"id\" : 0, \"category\" : { \"name\" : \"default-name\", \"id\" : 6 }, \"tags\" : [ { \"name\" : \"name\", \"id\" : 1 }, { \"name\" : \"name\", \"id\" : 1 } ], \"status\" : \"available\"}");

I opened PR #2536 to limit the number of example items in an array to max. 5, which fixes the issue (tested with your original spec).

[advance512]

Thanks for your help, @macjohnny.