Skip to content

Non-UTF-8 and truncated multipart bodies raise uncaught exceptions instead of returning responses #3392

Description

@adamtheturtle

What happens

Two more inputs make a validator raise an exception which is not a ValidatorError, so no backend catches it and the mock produces no HTTP response.

A body which is not UTF-8.validate_json and most other services validators call request_body.decode() with no error handling. A client which encodes its JSON as latin-1 — a name containing an accented character is enough — gets:

VWS latin-1 encoded name -> UNCAUGHT UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 13

The body sent was '{"name": "café", "width": 1, "image": ""}'.encode("latin-1").

A truncated multipart body on the Query API.MultiPartParser.parse raises when the body ends before the closing boundary:

query truncated multipart -> UNCAUGHT ValueError: Invalid form-data cannot parse beyond State.DATA
query nested multipart no final boundary -> UNCAUGHT ValueError: Invalid form-data cannot parse beyond State.DATA

That is what a client which is cut off mid-upload sends, which is a thing that happens.

Other malformed multipart inputs are handled correctly and return 400: an empty body, a missing boundary parameter, a missing Content-Type header, a boundary which does not match the body, and an image field which is not an image.

The pattern

These are the fourth and fifth known instances of one failure mode: a validator raises something other than ValidatorError, and because both backends only catch ValidatorError, the mock fails to respond at all. Through requests and httpx the exception surfaces inside the caller's request call; through Flask and Docker it is a 500.

The known instances:

InputExceptionIssue
Signed request to an unrouted pathValueError#3368
Image with large dimensionsDecompressionBombError#3378
Valid JSON which is not an objectAttributeError#3391
Body which is not UTF-8UnicodeDecodeErrorthis issue
Truncated multipart bodyValueErrorthis issue

Each was found by trying a handful of inputs by hand, so the list is a lower bound rather than a complete one.

Why it matters

The specific inputs vary in how likely they are. A latin-1 body and a truncated upload are both realistic; a JSON number as a request body is mostly a fuzzer. What they share is the failure mode, and the failure mode is the worst one available to a mock: the caller's code under test never sees a response, so whatever error handling it has is not exercised, and the traceback points into mock_vws rather than at the request that caused it.

Suggested resolution

Fix the two above where they occur — decode with an explicit error strategy and return whatever Vuforia returns for a non-UTF-8 body, and wrap the multipart parse the same way. Both need checking against a real database to pick the right result code.

For the class as a whole, the tempting fix is a catch-all in each backend converting unexpected exceptions to a 500. I would argue against that on its own: it would turn every one of these into a quiet, plausible-looking response and remove the signal that found them.

A regression harness is the better shape. A test which feeds a corpus of malformed requests — non-UTF-8 bodies, truncated multipart, non-object JSON, unrouted paths, oversized images — through every backend and asserts each produces an HTTP response with a status under 500 would pin all five and catch the sixth. The corpus can start as the inputs listed above.

If a catch-all is added as well, it should probably log loudly, so that a mock bug is still visible to whoever is running the tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions