Skip to content

Reject vertex/triangle counts whose allocation size would wrap 32 bits #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Fedr merged 6 commits into from
Sep 8, 2026
Merged

Reject vertex/triangle counts whose allocation size would wrap 32 bits #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c6a5721
Reject vertex/triangle counts whose allocation size would overflow si…
Fedr Sep 8, 2026
8e70fe6
Free partial mesh on mVertices allocation failure
Fedr Sep 8, 2026
ae76f66
Bound count guard by 32-bit UINT_MAX and compute float-map size in si…
Fedr Sep 8, 2026
9544585
Tighten count guard to the exact 16-byte stride bound
Fedr Sep 8, 2026
de43e4d
Free MG1 temporary indices when the packed-int read fails
Fedr Sep 8, 2026
2fd8f34
Explain why the count guard is bounded by UINT_MAX and not SIZE_MAX
Fedr Sep 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions lib/compressMG1.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,10 @@ int _ctmUncompressMesh_MG1(_CTMcontext * self)
return CTM_FALSE;
}
if(!_ctmStreamReadPackedInts(self, (CTMint *) indices, self->mTriangleCount, 3, CTM_FALSE))
{
free(indices);
return CTM_FALSE;
}

// Restore indices
_ctmRestoreIndices(self, indices);
Expand Down
17 changes: 16 additions & 1 deletion lib/openctm.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1149,7 +1149,8 @@ static CTMuint _ctmAllocateFloatMaps(_CTMcontext * self,
_CTMfloatmap ** aMapListPtr, CTMuint aCount, CTMuint aChannels)
{
_CTMfloatmap ** mapListPtr;
CTMuint i, size;
CTMuint i;
size_t size;

mapListPtr = aMapListPtr;
for(i = 0; i < aCount; ++ i)
Expand Down Expand Up @@ -1234,6 +1235,19 @@ CTMEXPORT void CTMCALL ctmLoadCustom(CTMcontext aContext, CTMreadfn aReadFn,
self->mUVMapCount = _ctmStreamReadUINT(self);
self->mAttribMapCount = _ctmStreamReadUINT(self);
flags = _ctmStreamReadUINT(self);

// Reject counts whose count * stride could wrap an allocation to an
// undersized buffer. The largest stride is 16 bytes (4 floats per vertex).
// UINT_MAX rather than SIZE_MAX because count * 3, count * 4 and the temp
// buffer sizes in stream.c are still computed in 32-bit CTMuint everywhere.
if(self->mVertexCount > 0xFFFFFFFFu / 16 ||
self->mTriangleCount > 0xFFFFFFFFu / 16)
{
_ctmClearMesh(self);
self->mError = CTM_BAD_FORMAT;
return;
}

_ctmStreamReadSTRING(self, &self->mFileComment);
if(self->mError != CTM_NONE)
{
Expand All @@ -1245,6 +1259,7 @@ CTMEXPORT void CTMCALL ctmLoadCustom(CTMcontext aContext, CTMreadfn aReadFn,
self->mVertices = (CTMfloat *) malloc(self->mVertexCount * sizeof(CTMfloat) * 3);
if(!self->mVertices)
{
_ctmClearMesh(self);
self->mError = CTM_OUT_OF_MEMORY;
return;
}
Expand Down