Replace the Java code generator with a Python implementation

[jonathan343]

Summary

Smithy Python currently generates Python clients with the Java-based code generator in codegen. Contributing to it requires a JDK, Gradle, and Java. This issue tracks replacing it with smithy-python, a code generator written in Python and distributed as a Python package. It follows the discussion in aws/aws-sdk-python#8.

Background: The smithy-ruby team converted their generator from Java to Ruby, so this is not a new idea for a Smithy generator.

Approach

Smithy's run plugin invokes an external program during a build, writing the projected model to it as a JSON AST on standard input. This allows us to write the code generator in Python and still integrate with a standard smithy build. Design documentation lives under designs/codegen, starting with an overview and the CLI contract proposed in #744. Follow-up PRs add a page for each area they implement.

Important

The Java generator stays authoritative for the whole migration. Before cutting any published artifact over, we will re-evaluate the Java generator's feature set as it stands then and confirm the Python generator covers all of it, so the cutover does not regress a customer.

Customer Experience

Installing the generator is a Python tool install. No JDK, Gradle, or Maven coordinates.

uv tool install smithy-python

Generating a client from a model on disk:

smithy-python generate client \
  --package weather \
  --package-version 0.0.1 \
  --model weather.json \
  --output ./build/weather-client

In smithy-build.json it is a run plugin. Smithy sends the model on standard input and the generator writes to the plugin's output directory, so --model and --output are not needed:

{
  "version": "1.0",
  "sources": ["model"],
  "projections": {
    "client": {
      "plugins": {
        "run::python-client": {
          "command": [
            "smithy-python", "generate", "client",
            "--package", "weather",
            "--package-version", "0.0.1"
          ]
        }
      }
    }
  }
}

Customizing generation becomes a Python task too. Protocols and platform behavior are contributed through the smithy_python.codegen.plugins entry-point group instead of a Java PythonIntegration on the Gradle classpath.

Required Changes

• Add experimental Python codegen CLI scaffold #744 adds the smithy-python package and its CLI: generate client and generate types, run plugin and direct invocation, and an error hierarchy separating invocation errors from I/O and generation failures.
• Load Smithy models and select the shapes to generate #795 adds model loading: the JSON AST becomes an immutable shape index with mixins and apply statements resolved, and generation is scoped to the resolved service's closure.
• Symbol provider mapping shape IDs to Python modules and names, with prelude collisions and aliasing.
• Writer with import management, CommonMark to docstring conversion, deterministic ordering, and atomic writes.
• Schema generation, including recursive schemas.
• Structures, unions, enums, intEnums, lists, and maps, with defaults, nullability, and error correction.
• Config and client generation, with dedicated input and output structures per operation.
• Plugin system: entry-point discovery, topological ordering, model preprocessing, symbol provider decoration, and code sections.
• Protocol generators for restJson1, awsJson1_0, awsJson1_1, and awsQuery, plus generated HTTP protocol tests.
• Auth scheme resolver generation and httpApiKeyAuth.
• AWS integrations for service ID, user agent, identity, and regional endpoints, plus the DynamoDB retry and API Gateway customizations.
• Package setup generation for pyproject.toml, README, and runtime dependency constraints.
• Post-processing with ruff format and opt-in ruff check.
• Settings parity with PythonSettings and the types plugin settings.
• Publishing smithy-python to PyPI.

The python-native-codegen-poc branch covers most of that end to end and implements the AWS customizations as Python plugins. It is a reference for the end state, not a merge candidate. Merged work is being split into PRs stacked on #744.

Completion Criteria

• A parity review confirms the Python generator covers the Java generator's features at cutover, with any gap closed or accepted in writing.
• smithy-python generates the clients published in aws/aws-sdk-python.
• Generated packages pass the same protocol and integration tests as the Java path.
• Contributing a codegen change requires only a Python toolchain.
• codegen is removed.

If a specific protocol, trait, or customization matters to you, please comment so it gets ordered correctly.