[PyTorch] Prune L0 unit test

[ksivaman]

Description

Reduce L0 unit testing runtime without removing significant coverage.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

• Print detailed logs only for failed tests to reduce logging overheads.
• Initialize supported recipes beforehand instead of launching multiple tests and then skipping unsupported options.
• Remove unused imports.
• Remove redundant/duplicate tests. E.g. remove cpu offloading and CG tests from sanity as they are individually tested more exhaustively in their own L0 scripts.
• For slow tests such as ONNX export, refactor the tests to split the @parametrized options into individual tests instead of combining them all together, which leads to exponentiation. This way, one feature is tested at a time while setting reasonable defaults for others. For the recipe specifically, the order of recipes in the list ensures different defaults on different devices based on available support to maximize coverage.

Maybe in the future

• Distributed/other tests.
• Apply the split test refactor to other tests if they grow; currently most tests run in under 3 mins without the logging overhead.
• Test one recipe only per device?
• In case of rapid growth, have 2 levels of sanity, L0 and L1.

Local testing times (mins)

• H100: 34 → 14
• B100: 39 → 13

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[ksivaman]

/te-ci pytorch

[ksivaman]

/te-ci pytorch

[ksivaman]

Description

Reduce L0 unit testing runtime without removing significant coverage.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

• Print detailed logs only for failed tests to reduce logging overheads.
• Initialize supported recipes beforehand instead of launching multiple tests and then skipping unsupported options.
• Remove unused imports.
• Remove redundant/duplicate tests. E.g. remove cpu offloading and CG tests from sanity as they are individually tested more exhaustively in their own L0 scripts.
• For slow tests such as ONNX export, refactor the tests to split the @parametrized options into individual tests instead of combining them all together, which leads to exponentiation. This way, one feature is tested at a time while setting reasonable defaults for others. For the recipe specifically, the order of recipes in the list ensures different defaults on different devices based on available support to maximize coverage.

Maybe in the future

• Distributed/other tests.
• Apply the split test refactor to other tests if they grow; currently most tests run in under 3 mins without the logging overhead.
• Test one recipe only per device?
• In case of rapid growth, have 2 levels of sanity, L0 and L1.

Local testing times (mins)

• H100: 34 → 14
• B100: 39 → 13

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[ksivaman]

/te-ci pytorch

[ksivaman]

/te-ci pytorch

[ksivaman]

Description

Reduce L0 unit testing runtime without removing significant coverage.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

• Print detailed logs only for failed tests to reduce logging overheads.
• Initialize supported recipes beforehand instead of launching multiple tests and then skipping unsupported options.
• Remove unused imports.
• Remove redundant/duplicate tests. E.g. remove cpu offloading and CG tests from sanity as they are individually tested more exhaustively in their own L0 scripts.
• For slow tests such as ONNX export, refactor the tests to split the @parametrized options into individual tests instead of combining them all together, which leads to exponentiation. This way, one feature is tested at a time while setting reasonable defaults for others. For the recipe specifically, the order of recipes in the list ensures different defaults on different devices based on available support to maximize coverage.

Maybe in the future

• Distributed/other tests.
• Apply the split test refactor to other tests if they grow; currently most tests run in under 3 mins without the logging overhead.
• Test one recipe only per device?
• In case of rapid growth, have 2 levels of sanity, L0 and L1.

Local testing times (mins)

• H100: 34 → 14
• B100: 39 → 13

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[ksivaman]

/te-ci pytorch

[ksivaman]

/te-ci pytorch

[ksivaman]

Description

Reduce L0 unit testing runtime without removing significant coverage.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

• Print detailed logs only for failed tests to reduce logging overheads.
• Initialize supported recipes beforehand instead of launching multiple tests and then skipping unsupported options.
• Remove unused imports.
• Remove redundant/duplicate tests. E.g. remove cpu offloading and CG tests from sanity as they are individually tested more exhaustively in their own L0 scripts.
• For slow tests such as ONNX export, refactor the tests to split the @parametrized options into individual tests instead of combining them all together, which leads to exponentiation. This way, one feature is tested at a time while setting reasonable defaults for others. For the recipe specifically, the order of recipes in the list ensures different defaults on different devices based on available support to maximize coverage.

Maybe in the future

• Distributed/other tests.
• Apply the split test refactor to other tests if they grow; currently most tests run in under 3 mins without the logging overhead.
• Test one recipe only per device?
• In case of rapid growth, have 2 levels of sanity, L0 and L1.

Local testing times (mins)

• H100: 34 → 14
• B100: 39 → 13

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[ksivaman]

/te-ci pytorch

[ksivaman]

/te-ci pytorch

[ksivaman]

Description

Reduce L0 unit testing runtime without removing significant coverage.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

• Print detailed logs only for failed tests to reduce logging overheads.
• Initialize supported recipes beforehand instead of launching multiple tests and then skipping unsupported options.
• Remove unused imports.
• Remove redundant/duplicate tests. E.g. remove cpu offloading and CG tests from sanity as they are individually tested more exhaustively in their own L0 scripts.
• For slow tests such as ONNX export, refactor the tests to split the @parametrized options into individual tests instead of combining them all together, which leads to exponentiation. This way, one feature is tested at a time while setting reasonable defaults for others. For the recipe specifically, the order of recipes in the list ensures different defaults on different devices based on available support to maximize coverage.

Maybe in the future

• Distributed/other tests.
• Apply the split test refactor to other tests if they grow; currently most tests run in under 3 mins without the logging overhead.
• Test one recipe only per device?
• In case of rapid growth, have 2 levels of sanity, L0 and L1.

Local testing times (mins)

• H100: 34 → 14
• B100: 39 → 13

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[ksivaman]

/te-ci pytorch

[ksivaman]

/te-ci pytorch

[ksivaman]

Description

Reduce L0 unit testing runtime without removing significant coverage.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

• Print detailed logs only for failed tests to reduce logging overheads.
• Initialize supported recipes beforehand instead of launching multiple tests and then skipping unsupported options.
• Remove unused imports.
• Remove redundant/duplicate tests. E.g. remove cpu offloading and CG tests from sanity as they are individually tested more exhaustively in their own L0 scripts.
• For slow tests such as ONNX export, refactor the tests to split the @parametrized options into individual tests instead of combining them all together, which leads to exponentiation. This way, one feature is tested at a time while setting reasonable defaults for others. For the recipe specifically, the order of recipes in the list ensures different defaults on different devices based on available support to maximize coverage.

Maybe in the future

• Distributed/other tests.
• Apply the split test refactor to other tests if they grow; currently most tests run in under 3 mins without the logging overhead.
• Test one recipe only per device?
• In case of rapid growth, have 2 levels of sanity, L0 and L1.

Local testing times (mins)

• H100: 34 → 14
• B100: 39 → 13

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[ksivaman]

/te-ci pytorch

[ksivaman]

/te-ci pytorch

[ksivaman]

Description

Reduce L0 unit testing runtime without removing significant coverage.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

• Print detailed logs only for failed tests to reduce logging overheads.
• Initialize supported recipes beforehand instead of launching multiple tests and then skipping unsupported options.
• Remove unused imports.
• Remove redundant/duplicate tests. E.g. remove cpu offloading and CG tests from sanity as they are individually tested more exhaustively in their own L0 scripts.
• For slow tests such as ONNX export, refactor the tests to split the @parametrized options into individual tests instead of combining them all together, which leads to exponentiation. This way, one feature is tested at a time while setting reasonable defaults for others. For the recipe specifically, the order of recipes in the list ensures different defaults on different devices based on available support to maximize coverage.

Maybe in the future

• Distributed/other tests.
• Apply the split test refactor to other tests if they grow; currently most tests run in under 3 mins without the logging overhead.
• Test one recipe only per device?
• In case of rapid growth, have 2 levels of sanity, L0 and L1.

Local testing times (mins)

• H100: 34 → 14
• B100: 39 → 13

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[ksivaman]

/te-ci pytorch

[ksivaman]

/te-ci pytorch