Skip to content

[3.0] Split up ExtractNestedTyping and add tests for TransformHandles - #2587

Merged
Exanite merged 45 commits into
feature/generator-docsfrom
feature/generator-fixes-and-polish
Jun 12, 2026
Merged

[3.0] Split up ExtractNestedTyping and add tests for TransformHandles#2587
Exanite merged 45 commits into
feature/generator-docsfrom
feature/generator-fixes-and-polish

Conversation

@Exanite

@ExaniteExanite commented Jun 7, 2026•
edited
Loading

Copy link
Copy Markdown
Member

Summary of the PR

This will be merged into #2578 and focuses on polishing up the generator, specifically focusing on the issues that I discovered while writing the generator documentation and any other minor generator improvements I decide to include into this PR.

Related issues, Discord discussions, or proposals

Generator documentation PR: #2578

Further Comments

Tasks

  • Split ExtractNestedTyping into multiple mods (ExtractNestedTypes, ExtractEnumConstants, ExtractFunctionPointers)
    • This was an old task that I decided to handle in this PR. Reason is that the tasks below require understanding and modifying this mod. I found it easier to refactor first before making the changes.
    • Verify no bindings changes were caused by the refactor.
      • Looks like I did end up causing one change, which is that some missing namespaces are now properly generated. Not precisely sure how I fixed this, but I'm not going to investigate since I did clean up some of the namespacing code.
  • Issues to address from generator documentation PR
    • ExtractNestedTyping constant extraction does not handle nor rewrite fields that are typed as a predefined type, but are natively typed as an enum (test case: SuccessfullyExtractsCStyleEnumConstants_Field)
    • ExtractNestedTyping constant extraction does not handle nor rewrite return types that are typed as a predefined type, but are natively typed as an enum (test case: SuccessfullyExtractsCStyleEnumConstants_ReturnType)
    • ExtractNestedTyping does not annotate extracted enums with the [NativeName] attribute. Eg: SDL_BlendMode.
    • Consider making it so that handle types implement the IEquatable<T> interface.
  • Update docs
    • TransformHandles - Should mention test cases
    • ExtractNestedTyping - Should be replaced with 3 sections corresponding to the 3 new mods
  • Update test cases
  • Regenerate using Windows for consistency and to ensure no unwanted changes

@Exanite
Exanite changed the base branch from develop/3.0 to feature/generator-docsJune 7, 2026 23:51
@ExaniteExanite changed the title [3.0] Fix and polish bindings generator[3.0] Improve and polish bindings generatorJun 8, 2026
Exanite added 24 commits June 8, 2026 00:08
@Exanite
Add stubs for ExtractEnumConstants and ExtractFunctionPointers
4e39057
@Exanite
Fix invalid interface implementation
a4f0a98
@Exanite
Work on splitting ExtractNestedTyping into 3 separate mods
2a622cb
@Exanite
Continue to move code to their new locations
298c3b0
@Exanite
Further cleanup
e9b67f9
@Exanite
Rename ExtractNestedTyping to ExtractNestedTypes
d569b7a
This is be more consistent with C#'s terminology now that ExtractNestedTypes strictly only handles NestedTypes and not other forms of "nested typing".
@Exanite
Split ExtractNestedTypes tests into 3 sets of tests and ensure projec…
88b225e
…t is compilable (if not correct)
This is so I can start testing for expected behavior.
@Exanite
Get ExtractNestedTypes working
afa8792
@Exanite
Update test snapshot names
a969aeb
@Exanite
Get ExtractFunctionPointers working
9f120c6
@Exanite
Fix subtly incorrect behavior and remove dependency on Mod for Extrac…
bd48a68
…tFunctionPointers/NestedTypes
@Exanite
Move GetNativeTypeNameForPredefinedType to ExtractEnumConstants and c…
de0a8ca
…leanup
@Exanite
Fix incorrect file name
cd3a9f8
@Exanite
Prefer IMod over Mod where possible
06056fc
This also means free performance since Mod sneakily creates a compilation and walks through it
@Exanite
Document where ExtractFunctionPointers and ExtractNestedTypes output …
33437dc
…their extracted files
Also add todo for doing the same in ExtractEnumConstants
@Exanite
Get ExtractEnumConstants working
725edda
@Exanite
Update config
95d88c8
@Exanite
Add remark comment on ExtractNestedTypes only handling ClangScraper's…
425f575
… special structs
@Exanite
Generate on Windows (apparently accidentally fixed a namespace bug)
18f2e21
@Exanite
Remove unused code
626fb61
@Exanite
Cleanup GetNativeTypeNameForPredefinedType
6b9108d
@Exanite
Use constructor and avoid nullable properties
02ce5e3
We always allocate these so this is strictly a readability and performance improvement.
@Exanite
Refactor code to use record structs
0531153
@Exanite
Work on deciphering tuple usages
1f62ae0
@ExaniteExanite mentioned this pull request Jun 9, 2026
Merged
13 tasks
@github-actions

github-actionsBot commented Jun 9, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

FileCoverage
All files20%✅

Minimum allowed coverage is 0%

Generated by 🐒 cobertura-action against b34ee89

@Exanite

Exanite commented Jun 9, 2026

Copy link
Copy Markdown
MemberAuthor

It's satisfying seeing the coverage percent increase with every PR 😁

@ExaniteExanite mentioned this pull request Jun 10, 2026
Open
71 tasks
@Exanite
Change Sdl in ExtractEnumConstantsTests to be a class, not a struct
7b71e87
This is more representative of the actual bindings even though it doesn't affect the tests.
@Exanite
Fix incorrect variable name and type name
cca521e
Exanite
Exanite commented Jun 10, 2026
View reviewed changes

@ExaniteExanite left a comment

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self code review completed.

Comment threadsources/SDL/SDL/SDL3/BlendMode.gen.cs
Comment threadsources/SilkTouch/SilkTouch/Mods/Common/AttributeUtils.cs
Comment threadsources/SilkTouch/SilkTouch/Mods/Common/Mod.cs
Comment threadsources/SilkTouch/SilkTouch/Mods/Common/ModLoader.cs
Comment threadsources/SilkTouch/SilkTouch/Mods/AddApiProfiles.cs
Comment threadsources/SilkTouch/SilkTouch/Mods/ExtractEnumConstants.cs
Comment threadsources/SilkTouch/SilkTouch/Mods/ExtractFunctionPointers.cs
Comment threadsources/SilkTouch/SilkTouch/Mods/ExtractNestedTypes.cs
Comment threadsources/SilkTouch/SilkTouch/Mods/TransformHandles.cs
Comment threadgenerator.json
@Exanite
Exanite marked this pull request as ready for review June 10, 2026 06:43
@Exanite
Exanite requested a review from a team as a code ownerJune 10, 2026 06:43
@ExaniteExanite changed the title [3.0] Improve and polish bindings generator[3.0] Split up ExtractNestedTyping and add tests for TransformHandlesJun 11, 2026
@Exanite
Exanite merged commit 3b72894 into feature/generator-docsJun 12, 2026
17 checks passed
@Exanite
Exanite deleted the feature/generator-fixes-and-polish branch June 12, 2026 06:27
Exanite added a commit that referenced this pull request Jun 12, 2026
@Exanite
[3.0] Add generator docs (#2578)
89841e7
* Add WIP documentation for using the Silk 3 generator
* Add for-contributors/Generator folder
* Work on an outline for two new generator docs pages
* Work on high-level overview section
* Edit the overview section and begin writing about PrettifyNames
* Work on PrettifyNames section
* Omit the name processor parameters since they don't provide much additional info
* Edit wording
* Work on Name Splitting and Name Prettification sections
* Revise the name processing docs
* More editing
* Write the Acronym Indeterminate Inputs section
* Finish writing the Name Prettification section
* Work on name affixes section
* Add PrettifyNames - Notable Decisions section
* Write the Referenced Affixes section
* Add Name Affixes - Metadata Format section
* Add Referenced Affixes - Metadata Format section
* Remove completed todo
* Add test cases to cover ID3D12Device shared prefix identification behavior
* Write the Symbol-based Renamer section
* Edit name processing docs
* Reflow the text
* Add note on where the referenced affix example comes from
* Add example where only one of the two consecutive acronyms is preserved
* Begin working on generator mods docs
* Write the Mod Configuration section
* Add empty sections for each mod
* Clarify what information will go in the Available Mods section
* List out the name affix categories and work on outlining the rest of the docs
* Add information about usage recommendations section
* Work on docs and write the AddApiProfiles section
* Work on docs and attempt to write AddVTables usage recommendation
* Remove the WIP usage recommendations for AddVTables since I don't know enough to explain it
* Work on mods docs
* Add light docs for ClangScraper mod
* Add docs for ExtractHandles and add ExtractHandlesTests matching the example given in the docs
* Document that the example has a matching test case
* Change ModUtils.RelativePath() to work when no project path is available
This is because project path is null during unit tests and I can't figure out if it is possible to set the project path of an AdhocWorkspace project.
* Add TestUtils.VerifyDocumentsAsync
* Use Single instead of First since we expect one matching doc
* Work on ExtractNestedTyping docs and add SuccessfullyExtractsNestedInlineArray test case
* Work on docs and add SuccessfullyExtractsFunctionPointer test case
* Add SuccessfullyExtractsCStyleEnumConstants test case
* Add passing SuccessfullyExtractsCStyleEnumConstants_Pointer and failing SuccessfullyExtractsCStyleEnumConstants_ReturnType test case
* Add SuccessfullyExtractsCStyleEnumConstants_Field test case
* Rename snapshot to match renamed test
* Write the IdentifySharedPrefixes section
* Add note about IdentifySharedPrefixes's history
* Fix incorrect test document name
* Add more information to the IdentifySharedPrefixes section
* Fix and edit wording
* Add InterceptNativeFunctionsTests
* Write the InterceptNativeFunctions section
* Work on docs for MarkNativeNames
* Work on MixKhronosData docs
* Work on documenting the name affixes used by MixKhronosData
* Write docs for KhronosNonExclusiveVendor
* Finish writing the name affix docs for MixKhronosData
* Finish up the MixKhronosData section
* Write the PrettifyNames section
* Write the StripAttributes section
* Write the TransformEnums section and add note about ClangScraper platform differences
* Edit wording
* Attempt to write some docs for TransformFunctions
* Work on TransformHandles docs
* Write docs for TransformProperties and finish the first pass of the generator mod docs
Probably not going to too much of a second pass since I'm tired of writing docs and the idea is that these docs will improve over time. Going from no docs to almost everything documented is already a really good step forward.
* Adjust wording
* Add OpenAL and Vulkan to changelog
* Remove todos relating to PR #2574
* Document ClangScraper platform-specific differences and edit generator mod docs
* Add Library-Specific Notes doc
* Add link for more information
* Add link from ClangScraper section to Library-Specific Notes doc
* Fix incorrect title for Static vs Instance Bindings doc
* Add link from AddVTables section to Static vs Instance Bindings doc
* Prefer referring to native APIs using the term "API" rather than "library"
I saw that static-vs-instance-bindings.md used the terms "API" and "native API" to describe the APIs being bound to using SilkTouch.
I'm deciding to keep this pattern moving forward for consistency.
* Add links to Using the Generator and remap-stdint.rsp to generator-mods.md
* Work on Using the Generator docs
* Continue editing the Using the Generator docs
* Don't surround ClangSharpPInvokeGenerator in backticks
* Cleanup section on general eng/silktouch folder structure
* Finish editing the Using the Generator docs
* Cleanup
* Update test cases to use TestUtils.VerifyDocumentsAsync
* Cleanup
* Edit the intro to the Using the Generator docs
* Edit docs and avoid using inline code snippets in headers
* Add <see> reference to ClangScraper
(cherry picked from commit 3192da8be03dffedae9319636018ee901f959716)
* Change ExtractHandlesTests to not filter the verified documents list
* Add some spacing to the KhronosNonExclusiveVendor section
* Edit Generator Mods docs
* Edit MarkNativeNames to clarify who is responsible for adding NativeName attributes
* [3.0] Split up ExtractNestedTyping and add tests for TransformHandles (#2587)
* Start of generator fixes and polish branch
* Add stubs for ExtractEnumConstants and ExtractFunctionPointers
* Fix invalid interface implementation
* Work on splitting ExtractNestedTyping into 3 separate mods
* Continue to move code to their new locations
* Further cleanup
* Rename ExtractNestedTyping to ExtractNestedTypes
This is be more consistent with C#'s terminology now that ExtractNestedTypes strictly only handles NestedTypes and not other forms of "nested typing".
* Split ExtractNestedTypes tests into 3 sets of tests and ensure project is compilable (if not correct)
This is so I can start testing for expected behavior.
* Get ExtractNestedTypes working
* Update test snapshot names
* Get ExtractFunctionPointers working
* Fix subtly incorrect behavior and remove dependency on Mod for ExtractFunctionPointers/NestedTypes
* Move GetNativeTypeNameForPredefinedType to ExtractEnumConstants and cleanup
* Fix incorrect file name
* Prefer IMod over Mod where possible
This also means free performance since Mod sneakily creates a compilation and walks through it
* Document where ExtractFunctionPointers and ExtractNestedTypes output their extracted files
Also add todo for doing the same in ExtractEnumConstants
* Get ExtractEnumConstants working
* Update config
* Add remark comment on ExtractNestedTypes only handling ClangScraper's special structs
* Generate on Windows (apparently accidentally fixed a namespace bug)
* Remove unused code
* Cleanup GetNativeTypeNameForPredefinedType
* Use constructor and avoid nullable properties
We always allocate these so this is strictly a readability and performance improvement.
* Refactor code to use record structs
* Work on deciphering tuple usages
* Figure out what _numericTypeNames does
* Add support for requiredTargetSpecifier to TryParseNativeTypeName
* Handle method return types properly and fix SuccessfullyExtractsCStyleEnumConstants_ReturnType
* Cleanup todos
* Handle fields properly and fix SuccessfullyExtractsCStyleEnumConstants_Field
* Generate on Windows
* Cleanup
* Annotate extracted enums with [NativeName]
* Generate on Windows
* Add IEquatable to transformed handle types
These already implemented all the required members for IEquatable, so I found it weird that they didn't also implement the interface.
* Generate on Windows for IEquatable handles change
* Add TransformHandlesTests
* Reorder code for clarity
* Update docs with sections for the 3 mods ExtractNestedTyping was split into
* Change Sdl in ExtractEnumConstantsTests to be a class, not a struct
This is more representative of the actual bindings even though it doesn't affect the tests.
* Fix incorrect variable name and type name
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@curincurincurin approved these changes

Assignees

No one assigned

Labels

None yet

Projects

Silk.NET Development Tracker
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Exanite@curin