fix: scope Codecov report to product code, excluding test files

[jam-jee]

Issue

CI uploads unit-test coverage to Codecov, but the coverage is not scoped to src/ product code, so test code and test-support scripts dominate the report.

Root cause (confirmed): the CI unit-test build runs, from inside each sub-package directory:

cd $CODEBUILD_SRC_DIR/$SUBMODULE
tox -e py310,py311,py312 -- tests/unit -v --cov=. --cov-append --cov-report=xml

--cov=. measures the entire submodule directory, which contains both src/ and tests/. So executed test files and test-data scripts (e.g. tests/data/*.py training entry-scripts and model definitions) are recorded and uploaded alongside product code. (The buildspec lives in the internal CDK infra package, not this repo.)

Inspecting the Codecov API for recent commits confirms the effect:

Commit	Reported	Files in report	Test files	Product files
3f810c67 (Release 3.16.0, master)	88.06%	290	285	5
PR #5969 head (d4493939)	90.26%	479	473	6

~98% of the measured files/lines in the uploaded report are test files, not shippable product code. Consequences:

• Reported project coverage (~88–90%) is far above true product-code coverage and largely reflects test files executing themselves.
• The number is unstable across commits (88% / 90% / 93%) depending on which sub-package uploads land, with no gate to wait for all of them.

Note: integration tests contribute no coverage (the integ build runs plain pytest with IGNORE_COVERAGE=- and no --cov), so this is purely the unit-coverage upload.

Verified real product coverage

A verification run in CI (all four unit suites unioned with a corrected --cov=sagemaker, test files excluded) shows the true product-code coverage:

Package	Coverage
sagemaker-mlops	80.9%
sagemaker-serve	76.0%
sagemaker-train	72.2%
sagemaker-core	67.4%
Overall	70.7% (561 product files, 0 test files)

Fix

Add a repo-root codecov.yml that:

• Scopes the Codecov report to product code via an ignore: list (tests, conftest.py, tests/data scripts, test_script.py, codegen and workflow_helper utilities, docs, examples). This corrects the report server-side, independent of the CI --cov flag.
• Sets status checks calibrated to the verified baseline: projecttarget 65% (an explicit floor rather than auto, so the one-time recalibration from the inflated ~90% down to the true ~71% is not read as a regression, while still guarding against real drops) and patchtarget 70% on changed lines.
• Sets codecov.notify.after_n_builds: 4 so the project % is computed once all four sub-package uploads are in.

Validated with the Codecov config validator (curl --data-binary @codecov.yml https://codecov.io/validate → Valid!).

Recommended companion change (separate, in the CI infra package)

The cleanest upstream fix is to change the CI unit command from --cov=. to --cov=sagemaker (match by import package name, which follows the installed package and excludes tests) so the generated coverage.xml never contains test files. That change lives in the internal CodeBuild infra package (SageMakerMLFPySDKInfraCDK/lib/buildspecs.ts), not this repo, so it is intentionally not part of this PR. The codecov.yml here fixes the reported number regardless, and both changes are complementary.

Note on the tox --fail-under gate

Each tox.ini defines coverage report --fail-under=86 under the runcoverage env, but CI does not invoke that env (it runs tox -e py310,py311,py312), so this change does not affect any CI gate. If a future coverage gate is wired up, the threshold should reflect the real ~71% baseline rather than the old inflated number.

Testing

• python -c "import yaml; yaml.safe_load(open('codecov.yml'))" → parses.
• Codecov validator returns Valid! and compiles the expected ignore regexes.
• No product code, tox config, or CI workflow is modified; only a new codecov.yml is added.

[jam-jee]

Issue

CI uploads unit-test coverage to Codecov, but the coverage is not scoped to src/ product code, so test code and test-support scripts dominate the report.

Root cause (confirmed): the CI unit-test build runs, from inside each sub-package directory:

cd $CODEBUILD_SRC_DIR/$SUBMODULE
tox -e py310,py311,py312 -- tests/unit -v --cov=. --cov-append --cov-report=xml

--cov=. measures the entire submodule directory, which contains both src/ and tests/. So executed test files and test-data scripts (e.g. tests/data/*.py training entry-scripts and model definitions) are recorded and uploaded alongside product code. (The buildspec lives in the internal CDK infra package, not this repo.)

Inspecting the Codecov API for recent commits confirms the effect:

Commit	Reported	Files in report	Test files	Product files
3f810c67 (Release 3.16.0, master)	88.06%	290	285	5
PR #5969 head (d4493939)	90.26%	479	473	6

~98% of the measured files/lines in the uploaded report are test files, not shippable product code. Consequences:

• Reported project coverage (~88–90%) is far above true product-code coverage and largely reflects test files executing themselves.
• The number is unstable across commits (88% / 90% / 93%) depending on which sub-package uploads land, with no gate to wait for all of them.

Note: integration tests contribute no coverage (the integ build runs plain pytest with IGNORE_COVERAGE=- and no --cov), so this is purely the unit-coverage upload.

Verified real product coverage

A verification run in CI (all four unit suites unioned with a corrected --cov=sagemaker, test files excluded) shows the true product-code coverage:

Package	Coverage
sagemaker-mlops	80.9%
sagemaker-serve	76.0%
sagemaker-train	72.2%
sagemaker-core	67.4%
Overall	70.7% (561 product files, 0 test files)

Fix

Add a repo-root codecov.yml that:

• Scopes the Codecov report to product code via an ignore: list (tests, conftest.py, tests/data scripts, test_script.py, codegen and workflow_helper utilities, docs, examples). This corrects the report server-side, independent of the CI --cov flag.
• Sets status checks calibrated to the verified baseline: projecttarget 65% (an explicit floor rather than auto, so the one-time recalibration from the inflated ~90% down to the true ~71% is not read as a regression, while still guarding against real drops) and patchtarget 70% on changed lines.
• Sets codecov.notify.after_n_builds: 4 so the project % is computed once all four sub-package uploads are in.

Validated with the Codecov config validator (curl --data-binary @codecov.yml https://codecov.io/validate → Valid!).

Recommended companion change (separate, in the CI infra package)

The cleanest upstream fix is to change the CI unit command from --cov=. to --cov=sagemaker (match by import package name, which follows the installed package and excludes tests) so the generated coverage.xml never contains test files. That change lives in the internal CodeBuild infra package (SageMakerMLFPySDKInfraCDK/lib/buildspecs.ts), not this repo, so it is intentionally not part of this PR. The codecov.yml here fixes the reported number regardless, and both changes are complementary.

Note on the tox --fail-under gate

Each tox.ini defines coverage report --fail-under=86 under the runcoverage env, but CI does not invoke that env (it runs tox -e py310,py311,py312), so this change does not affect any CI gate. If a future coverage gate is wired up, the threshold should reflect the real ~71% baseline rather than the old inflated number.

Testing

• python -c "import yaml; yaml.safe_load(open('codecov.yml'))" → parses.
• Codecov validator returns Valid! and compiles the expected ignore regexes.
• No product code, tox config, or CI workflow is modified; only a new codecov.yml is added.

[jam-jee]

Issue

CI uploads unit-test coverage to Codecov, but the coverage is not scoped to src/ product code, so test code and test-support scripts dominate the report.

Root cause (confirmed): the CI unit-test build runs, from inside each sub-package directory:

cd $CODEBUILD_SRC_DIR/$SUBMODULE
tox -e py310,py311,py312 -- tests/unit -v --cov=. --cov-append --cov-report=xml

--cov=. measures the entire submodule directory, which contains both src/ and tests/. So executed test files and test-data scripts (e.g. tests/data/*.py training entry-scripts and model definitions) are recorded and uploaded alongside product code. (The buildspec lives in the internal CDK infra package, not this repo.)

Inspecting the Codecov API for recent commits confirms the effect:

Commit	Reported	Files in report	Test files	Product files
3f810c67 (Release 3.16.0, master)	88.06%	290	285	5
PR #5969 head (d4493939)	90.26%	479	473	6

~98% of the measured files/lines in the uploaded report are test files, not shippable product code. Consequences:

• Reported project coverage (~88–90%) is far above true product-code coverage and largely reflects test files executing themselves.
• The number is unstable across commits (88% / 90% / 93%) depending on which sub-package uploads land, with no gate to wait for all of them.

Note: integration tests contribute no coverage (the integ build runs plain pytest with IGNORE_COVERAGE=- and no --cov), so this is purely the unit-coverage upload.

Verified real product coverage

A verification run in CI (all four unit suites unioned with a corrected --cov=sagemaker, test files excluded) shows the true product-code coverage:

Package	Coverage
sagemaker-mlops	80.9%
sagemaker-serve	76.0%
sagemaker-train	72.2%
sagemaker-core	67.4%
Overall	70.7% (561 product files, 0 test files)

Fix

Add a repo-root codecov.yml that:

• Scopes the Codecov report to product code via an ignore: list (tests, conftest.py, tests/data scripts, test_script.py, codegen and workflow_helper utilities, docs, examples). This corrects the report server-side, independent of the CI --cov flag.
• Sets status checks calibrated to the verified baseline: projecttarget 65% (an explicit floor rather than auto, so the one-time recalibration from the inflated ~90% down to the true ~71% is not read as a regression, while still guarding against real drops) and patchtarget 70% on changed lines.
• Sets codecov.notify.after_n_builds: 4 so the project % is computed once all four sub-package uploads are in.

Validated with the Codecov config validator (curl --data-binary @codecov.yml https://codecov.io/validate → Valid!).

Recommended companion change (separate, in the CI infra package)

The cleanest upstream fix is to change the CI unit command from --cov=. to --cov=sagemaker (match by import package name, which follows the installed package and excludes tests) so the generated coverage.xml never contains test files. That change lives in the internal CodeBuild infra package (SageMakerMLFPySDKInfraCDK/lib/buildspecs.ts), not this repo, so it is intentionally not part of this PR. The codecov.yml here fixes the reported number regardless, and both changes are complementary.

Note on the tox --fail-under gate

Each tox.ini defines coverage report --fail-under=86 under the runcoverage env, but CI does not invoke that env (it runs tox -e py310,py311,py312), so this change does not affect any CI gate. If a future coverage gate is wired up, the threshold should reflect the real ~71% baseline rather than the old inflated number.

Testing

• python -c "import yaml; yaml.safe_load(open('codecov.yml'))" → parses.
• Codecov validator returns Valid! and compiles the expected ignore regexes.
• No product code, tox config, or CI workflow is modified; only a new codecov.yml is added.

[jam-jee]

Issue

CI uploads unit-test coverage to Codecov, but the coverage is not scoped to src/ product code, so test code and test-support scripts dominate the report.

Root cause (confirmed): the CI unit-test build runs, from inside each sub-package directory:

cd $CODEBUILD_SRC_DIR/$SUBMODULE
tox -e py310,py311,py312 -- tests/unit -v --cov=. --cov-append --cov-report=xml

--cov=. measures the entire submodule directory, which contains both src/ and tests/. So executed test files and test-data scripts (e.g. tests/data/*.py training entry-scripts and model definitions) are recorded and uploaded alongside product code. (The buildspec lives in the internal CDK infra package, not this repo.)

Inspecting the Codecov API for recent commits confirms the effect:

Commit	Reported	Files in report	Test files	Product files
3f810c67 (Release 3.16.0, master)	88.06%	290	285	5
PR #5969 head (d4493939)	90.26%	479	473	6

~98% of the measured files/lines in the uploaded report are test files, not shippable product code. Consequences:

• Reported project coverage (~88–90%) is far above true product-code coverage and largely reflects test files executing themselves.
• The number is unstable across commits (88% / 90% / 93%) depending on which sub-package uploads land, with no gate to wait for all of them.

Note: integration tests contribute no coverage (the integ build runs plain pytest with IGNORE_COVERAGE=- and no --cov), so this is purely the unit-coverage upload.

Verified real product coverage

A verification run in CI (all four unit suites unioned with a corrected --cov=sagemaker, test files excluded) shows the true product-code coverage:

Package	Coverage
sagemaker-mlops	80.9%
sagemaker-serve	76.0%
sagemaker-train	72.2%
sagemaker-core	67.4%
Overall	70.7% (561 product files, 0 test files)

Fix

Add a repo-root codecov.yml that:

• Scopes the Codecov report to product code via an ignore: list (tests, conftest.py, tests/data scripts, test_script.py, codegen and workflow_helper utilities, docs, examples). This corrects the report server-side, independent of the CI --cov flag.
• Sets status checks calibrated to the verified baseline: projecttarget 65% (an explicit floor rather than auto, so the one-time recalibration from the inflated ~90% down to the true ~71% is not read as a regression, while still guarding against real drops) and patchtarget 70% on changed lines.
• Sets codecov.notify.after_n_builds: 4 so the project % is computed once all four sub-package uploads are in.

Validated with the Codecov config validator (curl --data-binary @codecov.yml https://codecov.io/validate → Valid!).

Recommended companion change (separate, in the CI infra package)

The cleanest upstream fix is to change the CI unit command from --cov=. to --cov=sagemaker (match by import package name, which follows the installed package and excludes tests) so the generated coverage.xml never contains test files. That change lives in the internal CodeBuild infra package (SageMakerMLFPySDKInfraCDK/lib/buildspecs.ts), not this repo, so it is intentionally not part of this PR. The codecov.yml here fixes the reported number regardless, and both changes are complementary.

Note on the tox --fail-under gate

Each tox.ini defines coverage report --fail-under=86 under the runcoverage env, but CI does not invoke that env (it runs tox -e py310,py311,py312), so this change does not affect any CI gate. If a future coverage gate is wired up, the threshold should reflect the real ~71% baseline rather than the old inflated number.

Testing

• python -c "import yaml; yaml.safe_load(open('codecov.yml'))" → parses.
• Codecov validator returns Valid! and compiles the expected ignore regexes.
• No product code, tox config, or CI workflow is modified; only a new codecov.yml is added.

[jam-jee]

Issue

CI uploads unit-test coverage to Codecov, but the coverage is not scoped to src/ product code, so test code and test-support scripts dominate the report.

Root cause (confirmed): the CI unit-test build runs, from inside each sub-package directory:

cd $CODEBUILD_SRC_DIR/$SUBMODULE
tox -e py310,py311,py312 -- tests/unit -v --cov=. --cov-append --cov-report=xml

--cov=. measures the entire submodule directory, which contains both src/ and tests/. So executed test files and test-data scripts (e.g. tests/data/*.py training entry-scripts and model definitions) are recorded and uploaded alongside product code. (The buildspec lives in the internal CDK infra package, not this repo.)

Inspecting the Codecov API for recent commits confirms the effect:

Commit	Reported	Files in report	Test files	Product files
3f810c67 (Release 3.16.0, master)	88.06%	290	285	5
PR #5969 head (d4493939)	90.26%	479	473	6

~98% of the measured files/lines in the uploaded report are test files, not shippable product code. Consequences:

• Reported project coverage (~88–90%) is far above true product-code coverage and largely reflects test files executing themselves.
• The number is unstable across commits (88% / 90% / 93%) depending on which sub-package uploads land, with no gate to wait for all of them.

Note: integration tests contribute no coverage (the integ build runs plain pytest with IGNORE_COVERAGE=- and no --cov), so this is purely the unit-coverage upload.

Verified real product coverage

A verification run in CI (all four unit suites unioned with a corrected --cov=sagemaker, test files excluded) shows the true product-code coverage:

Package	Coverage
sagemaker-mlops	80.9%
sagemaker-serve	76.0%
sagemaker-train	72.2%
sagemaker-core	67.4%
Overall	70.7% (561 product files, 0 test files)

Fix

Add a repo-root codecov.yml that:

• Scopes the Codecov report to product code via an ignore: list (tests, conftest.py, tests/data scripts, test_script.py, codegen and workflow_helper utilities, docs, examples). This corrects the report server-side, independent of the CI --cov flag.
• Sets status checks calibrated to the verified baseline: projecttarget 65% (an explicit floor rather than auto, so the one-time recalibration from the inflated ~90% down to the true ~71% is not read as a regression, while still guarding against real drops) and patchtarget 70% on changed lines.
• Sets codecov.notify.after_n_builds: 4 so the project % is computed once all four sub-package uploads are in.

Validated with the Codecov config validator (curl --data-binary @codecov.yml https://codecov.io/validate → Valid!).

Recommended companion change (separate, in the CI infra package)

The cleanest upstream fix is to change the CI unit command from --cov=. to --cov=sagemaker (match by import package name, which follows the installed package and excludes tests) so the generated coverage.xml never contains test files. That change lives in the internal CodeBuild infra package (SageMakerMLFPySDKInfraCDK/lib/buildspecs.ts), not this repo, so it is intentionally not part of this PR. The codecov.yml here fixes the reported number regardless, and both changes are complementary.

Note on the tox --fail-under gate

Each tox.ini defines coverage report --fail-under=86 under the runcoverage env, but CI does not invoke that env (it runs tox -e py310,py311,py312), so this change does not affect any CI gate. If a future coverage gate is wired up, the threshold should reflect the real ~71% baseline rather than the old inflated number.

Testing

• python -c "import yaml; yaml.safe_load(open('codecov.yml'))" → parses.
• Codecov validator returns Valid! and compiles the expected ignore regexes.
• No product code, tox config, or CI workflow is modified; only a new codecov.yml is added.

[jam-jee]

Issue

CI uploads unit-test coverage to Codecov, but the coverage is not scoped to src/ product code, so test code and test-support scripts dominate the report.

Root cause (confirmed): the CI unit-test build runs, from inside each sub-package directory:

cd $CODEBUILD_SRC_DIR/$SUBMODULE
tox -e py310,py311,py312 -- tests/unit -v --cov=. --cov-append --cov-report=xml

--cov=. measures the entire submodule directory, which contains both src/ and tests/. So executed test files and test-data scripts (e.g. tests/data/*.py training entry-scripts and model definitions) are recorded and uploaded alongside product code. (The buildspec lives in the internal CDK infra package, not this repo.)

Inspecting the Codecov API for recent commits confirms the effect:

Commit	Reported	Files in report	Test files	Product files
3f810c67 (Release 3.16.0, master)	88.06%	290	285	5
PR #5969 head (d4493939)	90.26%	479	473	6

~98% of the measured files/lines in the uploaded report are test files, not shippable product code. Consequences:

• Reported project coverage (~88–90%) is far above true product-code coverage and largely reflects test files executing themselves.
• The number is unstable across commits (88% / 90% / 93%) depending on which sub-package uploads land, with no gate to wait for all of them.

Note: integration tests contribute no coverage (the integ build runs plain pytest with IGNORE_COVERAGE=- and no --cov), so this is purely the unit-coverage upload.

Verified real product coverage

A verification run in CI (all four unit suites unioned with a corrected --cov=sagemaker, test files excluded) shows the true product-code coverage:

Package	Coverage
sagemaker-mlops	80.9%
sagemaker-serve	76.0%
sagemaker-train	72.2%
sagemaker-core	67.4%
Overall	70.7% (561 product files, 0 test files)

Fix

Add a repo-root codecov.yml that:

• Scopes the Codecov report to product code via an ignore: list (tests, conftest.py, tests/data scripts, test_script.py, codegen and workflow_helper utilities, docs, examples). This corrects the report server-side, independent of the CI --cov flag.
• Sets status checks calibrated to the verified baseline: projecttarget 65% (an explicit floor rather than auto, so the one-time recalibration from the inflated ~90% down to the true ~71% is not read as a regression, while still guarding against real drops) and patchtarget 70% on changed lines.
• Sets codecov.notify.after_n_builds: 4 so the project % is computed once all four sub-package uploads are in.

Validated with the Codecov config validator (curl --data-binary @codecov.yml https://codecov.io/validate → Valid!).

Recommended companion change (separate, in the CI infra package)

The cleanest upstream fix is to change the CI unit command from --cov=. to --cov=sagemaker (match by import package name, which follows the installed package and excludes tests) so the generated coverage.xml never contains test files. That change lives in the internal CodeBuild infra package (SageMakerMLFPySDKInfraCDK/lib/buildspecs.ts), not this repo, so it is intentionally not part of this PR. The codecov.yml here fixes the reported number regardless, and both changes are complementary.

Note on the tox --fail-under gate

Each tox.ini defines coverage report --fail-under=86 under the runcoverage env, but CI does not invoke that env (it runs tox -e py310,py311,py312), so this change does not affect any CI gate. If a future coverage gate is wired up, the threshold should reflect the real ~71% baseline rather than the old inflated number.

Testing

• python -c "import yaml; yaml.safe_load(open('codecov.yml'))" → parses.
• Codecov validator returns Valid! and compiles the expected ignore regexes.
• No product code, tox config, or CI workflow is modified; only a new codecov.yml is added.

[jam-jee]

Issue

CI uploads unit-test coverage to Codecov, but the coverage is not scoped to src/ product code, so test code and test-support scripts dominate the report.

Root cause (confirmed): the CI unit-test build runs, from inside each sub-package directory:

cd $CODEBUILD_SRC_DIR/$SUBMODULE
tox -e py310,py311,py312 -- tests/unit -v --cov=. --cov-append --cov-report=xml

--cov=. measures the entire submodule directory, which contains both src/ and tests/. So executed test files and test-data scripts (e.g. tests/data/*.py training entry-scripts and model definitions) are recorded and uploaded alongside product code. (The buildspec lives in the internal CDK infra package, not this repo.)

Inspecting the Codecov API for recent commits confirms the effect:

Commit	Reported	Files in report	Test files	Product files
3f810c67 (Release 3.16.0, master)	88.06%	290	285	5
PR #5969 head (d4493939)	90.26%	479	473	6

~98% of the measured files/lines in the uploaded report are test files, not shippable product code. Consequences:

• Reported project coverage (~88–90%) is far above true product-code coverage and largely reflects test files executing themselves.
• The number is unstable across commits (88% / 90% / 93%) depending on which sub-package uploads land, with no gate to wait for all of them.

Note: integration tests contribute no coverage (the integ build runs plain pytest with IGNORE_COVERAGE=- and no --cov), so this is purely the unit-coverage upload.

Verified real product coverage

A verification run in CI (all four unit suites unioned with a corrected --cov=sagemaker, test files excluded) shows the true product-code coverage:

Package	Coverage
sagemaker-mlops	80.9%
sagemaker-serve	76.0%
sagemaker-train	72.2%
sagemaker-core	67.4%
Overall	70.7% (561 product files, 0 test files)

Fix

Add a repo-root codecov.yml that:

• Scopes the Codecov report to product code via an ignore: list (tests, conftest.py, tests/data scripts, test_script.py, codegen and workflow_helper utilities, docs, examples). This corrects the report server-side, independent of the CI --cov flag.
• Sets status checks calibrated to the verified baseline: projecttarget 65% (an explicit floor rather than auto, so the one-time recalibration from the inflated ~90% down to the true ~71% is not read as a regression, while still guarding against real drops) and patchtarget 70% on changed lines.
• Sets codecov.notify.after_n_builds: 4 so the project % is computed once all four sub-package uploads are in.

Validated with the Codecov config validator (curl --data-binary @codecov.yml https://codecov.io/validate → Valid!).

Recommended companion change (separate, in the CI infra package)

The cleanest upstream fix is to change the CI unit command from --cov=. to --cov=sagemaker (match by import package name, which follows the installed package and excludes tests) so the generated coverage.xml never contains test files. That change lives in the internal CodeBuild infra package (SageMakerMLFPySDKInfraCDK/lib/buildspecs.ts), not this repo, so it is intentionally not part of this PR. The codecov.yml here fixes the reported number regardless, and both changes are complementary.

Note on the tox --fail-under gate

Each tox.ini defines coverage report --fail-under=86 under the runcoverage env, but CI does not invoke that env (it runs tox -e py310,py311,py312), so this change does not affect any CI gate. If a future coverage gate is wired up, the threshold should reflect the real ~71% baseline rather than the old inflated number.

Testing

• python -c "import yaml; yaml.safe_load(open('codecov.yml'))" → parses.
• Codecov validator returns Valid! and compiles the expected ignore regexes.
• No product code, tox config, or CI workflow is modified; only a new codecov.yml is added.

[jam-jee]

Issue

CI uploads unit-test coverage to Codecov, but the coverage is not scoped to src/ product code, so test code and test-support scripts dominate the report.

Root cause (confirmed): the CI unit-test build runs, from inside each sub-package directory:

cd $CODEBUILD_SRC_DIR/$SUBMODULE
tox -e py310,py311,py312 -- tests/unit -v --cov=. --cov-append --cov-report=xml

--cov=. measures the entire submodule directory, which contains both src/ and tests/. So executed test files and test-data scripts (e.g. tests/data/*.py training entry-scripts and model definitions) are recorded and uploaded alongside product code. (The buildspec lives in the internal CDK infra package, not this repo.)

Inspecting the Codecov API for recent commits confirms the effect:

Commit	Reported	Files in report	Test files	Product files
3f810c67 (Release 3.16.0, master)	88.06%	290	285	5
PR #5969 head (d4493939)	90.26%	479	473	6

~98% of the measured files/lines in the uploaded report are test files, not shippable product code. Consequences:

• Reported project coverage (~88–90%) is far above true product-code coverage and largely reflects test files executing themselves.
• The number is unstable across commits (88% / 90% / 93%) depending on which sub-package uploads land, with no gate to wait for all of them.

Note: integration tests contribute no coverage (the integ build runs plain pytest with IGNORE_COVERAGE=- and no --cov), so this is purely the unit-coverage upload.

Verified real product coverage

A verification run in CI (all four unit suites unioned with a corrected --cov=sagemaker, test files excluded) shows the true product-code coverage:

Package	Coverage
sagemaker-mlops	80.9%
sagemaker-serve	76.0%
sagemaker-train	72.2%
sagemaker-core	67.4%
Overall	70.7% (561 product files, 0 test files)

Fix

Add a repo-root codecov.yml that:

• Scopes the Codecov report to product code via an ignore: list (tests, conftest.py, tests/data scripts, test_script.py, codegen and workflow_helper utilities, docs, examples). This corrects the report server-side, independent of the CI --cov flag.
• Sets status checks calibrated to the verified baseline: projecttarget 65% (an explicit floor rather than auto, so the one-time recalibration from the inflated ~90% down to the true ~71% is not read as a regression, while still guarding against real drops) and patchtarget 70% on changed lines.
• Sets codecov.notify.after_n_builds: 4 so the project % is computed once all four sub-package uploads are in.

Validated with the Codecov config validator (curl --data-binary @codecov.yml https://codecov.io/validate → Valid!).

Recommended companion change (separate, in the CI infra package)

The cleanest upstream fix is to change the CI unit command from --cov=. to --cov=sagemaker (match by import package name, which follows the installed package and excludes tests) so the generated coverage.xml never contains test files. That change lives in the internal CodeBuild infra package (SageMakerMLFPySDKInfraCDK/lib/buildspecs.ts), not this repo, so it is intentionally not part of this PR. The codecov.yml here fixes the reported number regardless, and both changes are complementary.

Note on the tox --fail-under gate

Each tox.ini defines coverage report --fail-under=86 under the runcoverage env, but CI does not invoke that env (it runs tox -e py310,py311,py312), so this change does not affect any CI gate. If a future coverage gate is wired up, the threshold should reflect the real ~71% baseline rather than the old inflated number.

Testing

• python -c "import yaml; yaml.safe_load(open('codecov.yml'))" → parses.
• Codecov validator returns Valid! and compiles the expected ignore regexes.
• No product code, tox config, or CI workflow is modified; only a new codecov.yml is added.