Qualcomm AI Engine Direct - Quantizer refine for qat by chunit-quic · Pull Request #6747 · pytorch/executorch

chunit-quic · 2024-11-11T03:59:54Z

Follow the instruction to resubmit the PR after PR6513 is reverted.

pytorch-bot · 2024-11-11T03:59:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6747

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[DomainsOnly] Jobs fail with GLIBC version not found

✅ No Failures

As of commit 844acda with merge base ecdc007 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-11-11T04:34:30Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-12T00:41:06Z

 get_16a4w_qnn_ptq_config,
+ get_16a4w_qnn_qat_config,
 get_16a8w_qnn_ptq_config,
- get_default_16bit_qnn_ptq_config,


Removing get_default_16bit_qnn_ptq_config causing internal failure...can we do it in a the following way
This PR includes both new config and the old config
I submit a PR internally to remove the old config call site
You submit a new PR to remove the old config
Then we can land it safely...

Hi Chen,
Thanks for pointing out what to fix.
It seems to fail again. Shuold I also add get_default_8bit_qnn_ptq_config back ?

Sorry what is fail again?

Two checks in CI chcek list below

Oh hmm I feel like the Meta Internal-Only check isn't very accurate...

No problem. Let me know what should be fix if any. :D

Let me import and check CI again...

facebook-github-bot · 2024-11-13T04:09:12Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-13T05:49:32Z

Oh looks like some internal failure is resolved! There is one more left but I think it's already there. I'll submit a PR to fix that and then merge this change.

facebook-github-bot · 2024-11-14T00:58:47Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-14T01:25:57Z

Hey can you rebase this change? I landed the internal PR...

kimishpatel · 2024-11-14T04:00:24Z

+
+
+# TODO move to torch/ao/quantization/observer.py.
+class PerChannelParamObserver(UniformQuantizationObserverBase):


Can you add some comments here as to why chose the min/max ranges in the way being done here?

Hi @kimishpatel, just add the comment. If it's not clear enought please feel free to let me know. Thanks

facebook-github-bot · 2024-11-18T02:53:07Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-18T03:03:34Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

 The Pull Request could not be imported cleanly from GitHub. This usually happens for one of two reasons:
1. The Pull Request is out of date, if you rebase the PR on the latest commit on the GitHub branch and find merge conflicts this is probably the reason
2. ShipIt is broken for this repo, check bunnylol `oss org/repo`, click on the ShipIt tab, and see if there are any alerts. Fix those and retry the import.

- Reorginize qualcomm/quantizer - Split quantizer/utils.py to -- qconfig -- annotators -- observers directory - Change coresponding callees - Rename get_default_Nbit_qnn_ptq_config to get_NaNw_qnn_ptq_config - Add 16a4w conv test* (It is not compared with original model)

- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

- Add todo for per_channel_param_observer.py

chunit-quic · 2024-11-18T03:24:43Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

Just rebased. Thanks for pointing out.

cccclai · 2024-11-18T03:44:45Z

Thanks! There is still a small lintrunner error in the CI...

facebook-github-bot · 2024-11-18T03:45:13Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot · 2024-11-18T03:51:51Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

chunit-quic · 2024-11-11T03:59:54Z

Follow the instruction to resubmit the PR after PR6513 is reverted.

pytorch-bot · 2024-11-11T03:59:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6747

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[DomainsOnly] Jobs fail with GLIBC version not found

✅ No Failures

As of commit 844acda with merge base ecdc007 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-11-11T04:34:30Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-12T00:41:06Z

 get_16a4w_qnn_ptq_config,
+ get_16a4w_qnn_qat_config,
 get_16a8w_qnn_ptq_config,
- get_default_16bit_qnn_ptq_config,


Removing get_default_16bit_qnn_ptq_config causing internal failure...can we do it in a the following way
This PR includes both new config and the old config
I submit a PR internally to remove the old config call site
You submit a new PR to remove the old config
Then we can land it safely...

Hi Chen,
Thanks for pointing out what to fix.
It seems to fail again. Shuold I also add get_default_8bit_qnn_ptq_config back ?

Sorry what is fail again?

Two checks in CI chcek list below

Oh hmm I feel like the Meta Internal-Only check isn't very accurate...

No problem. Let me know what should be fix if any. :D

Let me import and check CI again...

facebook-github-bot · 2024-11-13T04:09:12Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-13T05:49:32Z

Oh looks like some internal failure is resolved! There is one more left but I think it's already there. I'll submit a PR to fix that and then merge this change.

facebook-github-bot · 2024-11-14T00:58:47Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-14T01:25:57Z

Hey can you rebase this change? I landed the internal PR...

kimishpatel · 2024-11-14T04:00:24Z

+
+
+# TODO move to torch/ao/quantization/observer.py.
+class PerChannelParamObserver(UniformQuantizationObserverBase):


Can you add some comments here as to why chose the min/max ranges in the way being done here?

Hi @kimishpatel, just add the comment. If it's not clear enought please feel free to let me know. Thanks

facebook-github-bot · 2024-11-18T02:53:07Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-18T03:03:34Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

 The Pull Request could not be imported cleanly from GitHub. This usually happens for one of two reasons:
1. The Pull Request is out of date, if you rebase the PR on the latest commit on the GitHub branch and find merge conflicts this is probably the reason
2. ShipIt is broken for this repo, check bunnylol `oss org/repo`, click on the ShipIt tab, and see if there are any alerts. Fix those and retry the import.

- Reorginize qualcomm/quantizer - Split quantizer/utils.py to -- qconfig -- annotators -- observers directory - Change coresponding callees - Rename get_default_Nbit_qnn_ptq_config to get_NaNw_qnn_ptq_config - Add 16a4w conv test* (It is not compared with original model)

- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

- Add todo for per_channel_param_observer.py

chunit-quic · 2024-11-18T03:24:43Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

Just rebased. Thanks for pointing out.

cccclai · 2024-11-18T03:44:45Z

Thanks! There is still a small lintrunner error in the CI...

facebook-github-bot · 2024-11-18T03:45:13Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot · 2024-11-18T03:51:51Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

chunit-quic · 2024-11-11T03:59:54Z

Follow the instruction to resubmit the PR after PR6513 is reverted.

pytorch-bot · 2024-11-11T03:59:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6747

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[DomainsOnly] Jobs fail with GLIBC version not found

✅ No Failures

As of commit 844acda with merge base ecdc007 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-11-11T04:34:30Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-12T00:41:06Z

 get_16a4w_qnn_ptq_config,
+ get_16a4w_qnn_qat_config,
 get_16a8w_qnn_ptq_config,
- get_default_16bit_qnn_ptq_config,


Removing get_default_16bit_qnn_ptq_config causing internal failure...can we do it in a the following way
This PR includes both new config and the old config
I submit a PR internally to remove the old config call site
You submit a new PR to remove the old config
Then we can land it safely...

Hi Chen,
Thanks for pointing out what to fix.
It seems to fail again. Shuold I also add get_default_8bit_qnn_ptq_config back ?

Sorry what is fail again?

Two checks in CI chcek list below

Oh hmm I feel like the Meta Internal-Only check isn't very accurate...

No problem. Let me know what should be fix if any. :D

Let me import and check CI again...

facebook-github-bot · 2024-11-13T04:09:12Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-13T05:49:32Z

Oh looks like some internal failure is resolved! There is one more left but I think it's already there. I'll submit a PR to fix that and then merge this change.

facebook-github-bot · 2024-11-14T00:58:47Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-14T01:25:57Z

Hey can you rebase this change? I landed the internal PR...

kimishpatel · 2024-11-14T04:00:24Z

+
+
+# TODO move to torch/ao/quantization/observer.py.
+class PerChannelParamObserver(UniformQuantizationObserverBase):


Can you add some comments here as to why chose the min/max ranges in the way being done here?

Hi @kimishpatel, just add the comment. If it's not clear enought please feel free to let me know. Thanks

facebook-github-bot · 2024-11-18T02:53:07Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-18T03:03:34Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

 The Pull Request could not be imported cleanly from GitHub. This usually happens for one of two reasons:
1. The Pull Request is out of date, if you rebase the PR on the latest commit on the GitHub branch and find merge conflicts this is probably the reason
2. ShipIt is broken for this repo, check bunnylol `oss org/repo`, click on the ShipIt tab, and see if there are any alerts. Fix those and retry the import.

- Reorginize qualcomm/quantizer - Split quantizer/utils.py to -- qconfig -- annotators -- observers directory - Change coresponding callees - Rename get_default_Nbit_qnn_ptq_config to get_NaNw_qnn_ptq_config - Add 16a4w conv test* (It is not compared with original model)

- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

- Add todo for per_channel_param_observer.py

chunit-quic · 2024-11-18T03:24:43Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

Just rebased. Thanks for pointing out.

cccclai · 2024-11-18T03:44:45Z

Thanks! There is still a small lintrunner error in the CI...

facebook-github-bot · 2024-11-18T03:45:13Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot · 2024-11-18T03:51:51Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

chunit-quic · 2024-11-11T03:59:54Z

Follow the instruction to resubmit the PR after PR6513 is reverted.

pytorch-bot · 2024-11-11T03:59:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6747

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[DomainsOnly] Jobs fail with GLIBC version not found

✅ No Failures

As of commit 844acda with merge base ecdc007 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-11-11T04:34:30Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-12T00:41:06Z

 get_16a4w_qnn_ptq_config,
+ get_16a4w_qnn_qat_config,
 get_16a8w_qnn_ptq_config,
- get_default_16bit_qnn_ptq_config,


Removing get_default_16bit_qnn_ptq_config causing internal failure...can we do it in a the following way
This PR includes both new config and the old config
I submit a PR internally to remove the old config call site
You submit a new PR to remove the old config
Then we can land it safely...

Hi Chen,
Thanks for pointing out what to fix.
It seems to fail again. Shuold I also add get_default_8bit_qnn_ptq_config back ?

Sorry what is fail again?

Two checks in CI chcek list below

Oh hmm I feel like the Meta Internal-Only check isn't very accurate...

No problem. Let me know what should be fix if any. :D

Let me import and check CI again...

facebook-github-bot · 2024-11-13T04:09:12Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-13T05:49:32Z

Oh looks like some internal failure is resolved! There is one more left but I think it's already there. I'll submit a PR to fix that and then merge this change.

facebook-github-bot · 2024-11-14T00:58:47Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-14T01:25:57Z

Hey can you rebase this change? I landed the internal PR...

kimishpatel · 2024-11-14T04:00:24Z

+
+
+# TODO move to torch/ao/quantization/observer.py.
+class PerChannelParamObserver(UniformQuantizationObserverBase):


Can you add some comments here as to why chose the min/max ranges in the way being done here?

Hi @kimishpatel, just add the comment. If it's not clear enought please feel free to let me know. Thanks

facebook-github-bot · 2024-11-18T02:53:07Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-18T03:03:34Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

 The Pull Request could not be imported cleanly from GitHub. This usually happens for one of two reasons:
1. The Pull Request is out of date, if you rebase the PR on the latest commit on the GitHub branch and find merge conflicts this is probably the reason
2. ShipIt is broken for this repo, check bunnylol `oss org/repo`, click on the ShipIt tab, and see if there are any alerts. Fix those and retry the import.

- Reorginize qualcomm/quantizer - Split quantizer/utils.py to -- qconfig -- annotators -- observers directory - Change coresponding callees - Rename get_default_Nbit_qnn_ptq_config to get_NaNw_qnn_ptq_config - Add 16a4w conv test* (It is not compared with original model)

- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

- Add todo for per_channel_param_observer.py

chunit-quic · 2024-11-18T03:24:43Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

Just rebased. Thanks for pointing out.

cccclai · 2024-11-18T03:44:45Z

Thanks! There is still a small lintrunner error in the CI...

facebook-github-bot · 2024-11-18T03:45:13Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot · 2024-11-18T03:51:51Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

chunit-quic · 2024-11-11T03:59:54Z

Follow the instruction to resubmit the PR after PR6513 is reverted.

pytorch-bot · 2024-11-11T03:59:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6747

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[DomainsOnly] Jobs fail with GLIBC version not found

✅ No Failures

As of commit 844acda with merge base ecdc007 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-11-11T04:34:30Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-12T00:41:06Z

 get_16a4w_qnn_ptq_config,
+ get_16a4w_qnn_qat_config,
 get_16a8w_qnn_ptq_config,
- get_default_16bit_qnn_ptq_config,


Removing get_default_16bit_qnn_ptq_config causing internal failure...can we do it in a the following way
This PR includes both new config and the old config
I submit a PR internally to remove the old config call site
You submit a new PR to remove the old config
Then we can land it safely...

Hi Chen,
Thanks for pointing out what to fix.
It seems to fail again. Shuold I also add get_default_8bit_qnn_ptq_config back ?

Sorry what is fail again?

Two checks in CI chcek list below

Oh hmm I feel like the Meta Internal-Only check isn't very accurate...

No problem. Let me know what should be fix if any. :D

Let me import and check CI again...

facebook-github-bot · 2024-11-13T04:09:12Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-13T05:49:32Z

Oh looks like some internal failure is resolved! There is one more left but I think it's already there. I'll submit a PR to fix that and then merge this change.

facebook-github-bot · 2024-11-14T00:58:47Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-14T01:25:57Z

Hey can you rebase this change? I landed the internal PR...

kimishpatel · 2024-11-14T04:00:24Z

+
+
+# TODO move to torch/ao/quantization/observer.py.
+class PerChannelParamObserver(UniformQuantizationObserverBase):


Can you add some comments here as to why chose the min/max ranges in the way being done here?

Hi @kimishpatel, just add the comment. If it's not clear enought please feel free to let me know. Thanks

facebook-github-bot · 2024-11-18T02:53:07Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-18T03:03:34Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

 The Pull Request could not be imported cleanly from GitHub. This usually happens for one of two reasons:
1. The Pull Request is out of date, if you rebase the PR on the latest commit on the GitHub branch and find merge conflicts this is probably the reason
2. ShipIt is broken for this repo, check bunnylol `oss org/repo`, click on the ShipIt tab, and see if there are any alerts. Fix those and retry the import.

- Reorginize qualcomm/quantizer - Split quantizer/utils.py to -- qconfig -- annotators -- observers directory - Change coresponding callees - Rename get_default_Nbit_qnn_ptq_config to get_NaNw_qnn_ptq_config - Add 16a4w conv test* (It is not compared with original model)

- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

- Add todo for per_channel_param_observer.py

chunit-quic · 2024-11-18T03:24:43Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

Just rebased. Thanks for pointing out.

cccclai · 2024-11-18T03:44:45Z

Thanks! There is still a small lintrunner error in the CI...

facebook-github-bot · 2024-11-18T03:45:13Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot · 2024-11-18T03:51:51Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

chunit-quic · 2024-11-11T03:59:54Z

Follow the instruction to resubmit the PR after PR6513 is reverted.

pytorch-bot · 2024-11-11T03:59:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6747

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[DomainsOnly] Jobs fail with GLIBC version not found

✅ No Failures

As of commit 844acda with merge base ecdc007 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-11-11T04:34:30Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-12T00:41:06Z

 get_16a4w_qnn_ptq_config,
+ get_16a4w_qnn_qat_config,
 get_16a8w_qnn_ptq_config,
- get_default_16bit_qnn_ptq_config,


Removing get_default_16bit_qnn_ptq_config causing internal failure...can we do it in a the following way
This PR includes both new config and the old config
I submit a PR internally to remove the old config call site
You submit a new PR to remove the old config
Then we can land it safely...

Hi Chen,
Thanks for pointing out what to fix.
It seems to fail again. Shuold I also add get_default_8bit_qnn_ptq_config back ?

Sorry what is fail again?

Two checks in CI chcek list below

Oh hmm I feel like the Meta Internal-Only check isn't very accurate...

No problem. Let me know what should be fix if any. :D

Let me import and check CI again...

facebook-github-bot · 2024-11-13T04:09:12Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-13T05:49:32Z

Oh looks like some internal failure is resolved! There is one more left but I think it's already there. I'll submit a PR to fix that and then merge this change.

facebook-github-bot · 2024-11-14T00:58:47Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-14T01:25:57Z

Hey can you rebase this change? I landed the internal PR...

kimishpatel · 2024-11-14T04:00:24Z

+
+
+# TODO move to torch/ao/quantization/observer.py.
+class PerChannelParamObserver(UniformQuantizationObserverBase):


Can you add some comments here as to why chose the min/max ranges in the way being done here?

Hi @kimishpatel, just add the comment. If it's not clear enought please feel free to let me know. Thanks

facebook-github-bot · 2024-11-18T02:53:07Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-18T03:03:34Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

 The Pull Request could not be imported cleanly from GitHub. This usually happens for one of two reasons:
1. The Pull Request is out of date, if you rebase the PR on the latest commit on the GitHub branch and find merge conflicts this is probably the reason
2. ShipIt is broken for this repo, check bunnylol `oss org/repo`, click on the ShipIt tab, and see if there are any alerts. Fix those and retry the import.

- Reorginize qualcomm/quantizer - Split quantizer/utils.py to -- qconfig -- annotators -- observers directory - Change coresponding callees - Rename get_default_Nbit_qnn_ptq_config to get_NaNw_qnn_ptq_config - Add 16a4w conv test* (It is not compared with original model)

- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

- Add todo for per_channel_param_observer.py

chunit-quic · 2024-11-18T03:24:43Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

Just rebased. Thanks for pointing out.

cccclai · 2024-11-18T03:44:45Z

Thanks! There is still a small lintrunner error in the CI...

facebook-github-bot · 2024-11-18T03:45:13Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot · 2024-11-18T03:51:51Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

chunit-quic · 2024-11-11T03:59:54Z

Follow the instruction to resubmit the PR after PR6513 is reverted.

pytorch-bot · 2024-11-11T03:59:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6747

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[DomainsOnly] Jobs fail with GLIBC version not found

✅ No Failures

As of commit 844acda with merge base ecdc007 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-11-11T04:34:30Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-12T00:41:06Z

 get_16a4w_qnn_ptq_config,
+ get_16a4w_qnn_qat_config,
 get_16a8w_qnn_ptq_config,
- get_default_16bit_qnn_ptq_config,


Removing get_default_16bit_qnn_ptq_config causing internal failure...can we do it in a the following way
This PR includes both new config and the old config
I submit a PR internally to remove the old config call site
You submit a new PR to remove the old config
Then we can land it safely...

Hi Chen,
Thanks for pointing out what to fix.
It seems to fail again. Shuold I also add get_default_8bit_qnn_ptq_config back ?

Sorry what is fail again?

Two checks in CI chcek list below

Oh hmm I feel like the Meta Internal-Only check isn't very accurate...

No problem. Let me know what should be fix if any. :D

Let me import and check CI again...

facebook-github-bot · 2024-11-13T04:09:12Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-13T05:49:32Z

Oh looks like some internal failure is resolved! There is one more left but I think it's already there. I'll submit a PR to fix that and then merge this change.

facebook-github-bot · 2024-11-14T00:58:47Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-14T01:25:57Z

Hey can you rebase this change? I landed the internal PR...

kimishpatel · 2024-11-14T04:00:24Z

+
+
+# TODO move to torch/ao/quantization/observer.py.
+class PerChannelParamObserver(UniformQuantizationObserverBase):


Can you add some comments here as to why chose the min/max ranges in the way being done here?

Hi @kimishpatel, just add the comment. If it's not clear enought please feel free to let me know. Thanks

facebook-github-bot · 2024-11-18T02:53:07Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-18T03:03:34Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

 The Pull Request could not be imported cleanly from GitHub. This usually happens for one of two reasons:
1. The Pull Request is out of date, if you rebase the PR on the latest commit on the GitHub branch and find merge conflicts this is probably the reason
2. ShipIt is broken for this repo, check bunnylol `oss org/repo`, click on the ShipIt tab, and see if there are any alerts. Fix those and retry the import.

- Reorginize qualcomm/quantizer - Split quantizer/utils.py to -- qconfig -- annotators -- observers directory - Change coresponding callees - Rename get_default_Nbit_qnn_ptq_config to get_NaNw_qnn_ptq_config - Add 16a4w conv test* (It is not compared with original model)

- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

- Add todo for per_channel_param_observer.py

chunit-quic · 2024-11-18T03:24:43Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

Just rebased. Thanks for pointing out.

cccclai · 2024-11-18T03:44:45Z

Thanks! There is still a small lintrunner error in the CI...

facebook-github-bot · 2024-11-18T03:45:13Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot · 2024-11-18T03:51:51Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

chunit-quic · 2024-11-11T03:59:54Z

Follow the instruction to resubmit the PR after PR6513 is reverted.

pytorch-bot · 2024-11-11T03:59:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6747

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[DomainsOnly] Jobs fail with GLIBC version not found

✅ No Failures

As of commit 844acda with merge base ecdc007 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-11-11T04:34:30Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-12T00:41:06Z

 get_16a4w_qnn_ptq_config,
+ get_16a4w_qnn_qat_config,
 get_16a8w_qnn_ptq_config,
- get_default_16bit_qnn_ptq_config,


Removing get_default_16bit_qnn_ptq_config causing internal failure...can we do it in a the following way
This PR includes both new config and the old config
I submit a PR internally to remove the old config call site
You submit a new PR to remove the old config
Then we can land it safely...

Hi Chen,
Thanks for pointing out what to fix.
It seems to fail again. Shuold I also add get_default_8bit_qnn_ptq_config back ?

Sorry what is fail again?

Two checks in CI chcek list below

Oh hmm I feel like the Meta Internal-Only check isn't very accurate...

No problem. Let me know what should be fix if any. :D

Let me import and check CI again...

facebook-github-bot · 2024-11-13T04:09:12Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-13T05:49:32Z

Oh looks like some internal failure is resolved! There is one more left but I think it's already there. I'll submit a PR to fix that and then merge this change.

facebook-github-bot · 2024-11-14T00:58:47Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-14T01:25:57Z

Hey can you rebase this change? I landed the internal PR...

kimishpatel · 2024-11-14T04:00:24Z

+
+
+# TODO move to torch/ao/quantization/observer.py.
+class PerChannelParamObserver(UniformQuantizationObserverBase):


Can you add some comments here as to why chose the min/max ranges in the way being done here?

Hi @kimishpatel, just add the comment. If it's not clear enought please feel free to let me know. Thanks

facebook-github-bot · 2024-11-18T02:53:07Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

cccclai · 2024-11-18T03:03:34Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

 The Pull Request could not be imported cleanly from GitHub. This usually happens for one of two reasons:
1. The Pull Request is out of date, if you rebase the PR on the latest commit on the GitHub branch and find merge conflicts this is probably the reason
2. ShipIt is broken for this repo, check bunnylol `oss org/repo`, click on the ShipIt tab, and see if there are any alerts. Fix those and retry the import.

- Reorginize qualcomm/quantizer - Split quantizer/utils.py to -- qconfig -- annotators -- observers directory - Change coresponding callees - Rename get_default_Nbit_qnn_ptq_config to get_NaNw_qnn_ptq_config - Add 16a4w conv test* (It is not compared with original model)

- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

- Add todo for per_channel_param_observer.py

chunit-quic · 2024-11-18T03:24:43Z

Hey wonder if you can rebase this PR, I've been failing to import this PR and the error is following

Just rebased. Thanks for pointing out.

cccclai · 2024-11-18T03:44:45Z

Thanks! There is still a small lintrunner error in the CI...

facebook-github-bot · 2024-11-18T03:45:13Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot · 2024-11-18T03:51:51Z

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 11, 2024

chunit-quic mentioned this pull request Nov 11, 2024
Qualcomm AI Engine Direct - Quantizer refine for qat #6513
Merged

cccclai reviewed Nov 12, 2024
View reviewed changes

tarun292 approved these changes Nov 13, 2024
View reviewed changes

kimishpatel reviewed Nov 14, 2024
View reviewed changes

Joey Tsai added 6 commits November 18, 2024 11:23

Fix baed on comments
602e9c3
- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

Add a comment
7d92616
- Add todo for per_channel_param_observer.py

[Fix lint]
b28c346

Bypass the meta internal test error
6879421

[Add comment and reference of param observer]
b353d4c

chunit-quic force-pushed the dev1/chunit/qat_quantizer_refine branch from dd19c66 to b353d4cCompare November 18, 2024 03:23

Fix linting error
844acda
- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot merged commit e95f171 into pytorch:mainNov 18, 2024

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 11, 2024

chunit-quic mentioned this pull request Nov 11, 2024
Qualcomm AI Engine Direct - Quantizer refine for qat #6513
Merged

cccclai reviewed Nov 12, 2024
View reviewed changes

tarun292 approved these changes Nov 13, 2024
View reviewed changes

kimishpatel reviewed Nov 14, 2024
View reviewed changes

Joey Tsai added 6 commits November 18, 2024 11:23

Fix baed on comments
602e9c3
- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

Add a comment
7d92616
- Add todo for per_channel_param_observer.py

[Fix lint]
b28c346

Bypass the meta internal test error
6879421

[Add comment and reference of param observer]
b353d4c

chunit-quic force-pushed the dev1/chunit/qat_quantizer_refine branch from dd19c66 to b353d4cCompare November 18, 2024 03:23

Fix linting error
844acda
- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot merged commit e95f171 into pytorch:mainNov 18, 2024

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 11, 2024

chunit-quic mentioned this pull request Nov 11, 2024
Qualcomm AI Engine Direct - Quantizer refine for qat #6513
Merged

cccclai reviewed Nov 12, 2024
View reviewed changes

tarun292 approved these changes Nov 13, 2024
View reviewed changes

kimishpatel reviewed Nov 14, 2024
View reviewed changes

Joey Tsai added 6 commits November 18, 2024 11:23

Fix baed on comments
602e9c3
- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

Add a comment
7d92616
- Add todo for per_channel_param_observer.py

[Fix lint]
b28c346

Bypass the meta internal test error
6879421

[Add comment and reference of param observer]
b353d4c

chunit-quic force-pushed the dev1/chunit/qat_quantizer_refine branch from dd19c66 to b353d4cCompare November 18, 2024 03:23

Fix linting error
844acda
- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot merged commit e95f171 into pytorch:mainNov 18, 2024

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 11, 2024

chunit-quic mentioned this pull request Nov 11, 2024
Qualcomm AI Engine Direct - Quantizer refine for qat #6513
Merged

cccclai reviewed Nov 12, 2024
View reviewed changes

tarun292 approved these changes Nov 13, 2024
View reviewed changes

kimishpatel reviewed Nov 14, 2024
View reviewed changes

Joey Tsai added 6 commits November 18, 2024 11:23

Fix baed on comments
602e9c3
- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

Add a comment
7d92616
- Add todo for per_channel_param_observer.py

[Fix lint]
b28c346

Bypass the meta internal test error
6879421

[Add comment and reference of param observer]
b353d4c

chunit-quic force-pushed the dev1/chunit/qat_quantizer_refine branch from dd19c66 to b353d4cCompare November 18, 2024 03:23

Fix linting error
844acda
- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot merged commit e95f171 into pytorch:mainNov 18, 2024

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 11, 2024

chunit-quic mentioned this pull request Nov 11, 2024
Qualcomm AI Engine Direct - Quantizer refine for qat #6513
Merged

cccclai reviewed Nov 12, 2024
View reviewed changes

tarun292 approved these changes Nov 13, 2024
View reviewed changes

kimishpatel reviewed Nov 14, 2024
View reviewed changes

Joey Tsai added 6 commits November 18, 2024 11:23

Fix baed on comments
602e9c3
- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

Add a comment
7d92616
- Add todo for per_channel_param_observer.py

[Fix lint]
b28c346

Bypass the meta internal test error
6879421

[Add comment and reference of param observer]
b353d4c

chunit-quic force-pushed the dev1/chunit/qat_quantizer_refine branch from dd19c66 to b353d4cCompare November 18, 2024 03:23

Fix linting error
844acda
- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot merged commit e95f171 into pytorch:mainNov 18, 2024

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 11, 2024

chunit-quic mentioned this pull request Nov 11, 2024
Qualcomm AI Engine Direct - Quantizer refine for qat #6513
Merged

cccclai reviewed Nov 12, 2024
View reviewed changes

tarun292 approved these changes Nov 13, 2024
View reviewed changes

kimishpatel reviewed Nov 14, 2024
View reviewed changes

Joey Tsai added 6 commits November 18, 2024 11:23

Fix baed on comments
602e9c3
- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

Add a comment
7d92616
- Add todo for per_channel_param_observer.py

[Fix lint]
b28c346

Bypass the meta internal test error
6879421

[Add comment and reference of param observer]
b353d4c

chunit-quic force-pushed the dev1/chunit/qat_quantizer_refine branch from dd19c66 to b353d4cCompare November 18, 2024 03:23

Fix linting error
844acda
- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot merged commit e95f171 into pytorch:mainNov 18, 2024

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 11, 2024

chunit-quic mentioned this pull request Nov 11, 2024
Qualcomm AI Engine Direct - Quantizer refine for qat #6513
Merged

cccclai reviewed Nov 12, 2024
View reviewed changes

tarun292 approved these changes Nov 13, 2024
View reviewed changes

kimishpatel reviewed Nov 14, 2024
View reviewed changes

Joey Tsai added 6 commits November 18, 2024 11:23

Fix baed on comments
602e9c3
- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

Add a comment
7d92616
- Add todo for per_channel_param_observer.py

[Fix lint]
b28c346

Bypass the meta internal test error
6879421

[Add comment and reference of param observer]
b353d4c

chunit-quic force-pushed the dev1/chunit/qat_quantizer_refine branch from dd19c66 to b353d4cCompare November 18, 2024 03:23

Fix linting error
844acda
- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot merged commit e95f171 into pytorch:mainNov 18, 2024

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 11, 2024

chunit-quic mentioned this pull request Nov 11, 2024
Qualcomm AI Engine Direct - Quantizer refine for qat #6513
Merged

cccclai reviewed Nov 12, 2024
View reviewed changes

tarun292 approved these changes Nov 13, 2024
View reviewed changes

kimishpatel reviewed Nov 14, 2024
View reviewed changes

Joey Tsai added 6 commits November 18, 2024 11:23

Fix baed on comments
602e9c3
- Move and rename param_observer.py to per_channel_param_observer.py - Add todo to merge qconfig

Add a comment
7d92616
- Add todo for per_channel_param_observer.py

[Fix lint]
b28c346

Bypass the meta internal test error
6879421

[Add comment and reference of param observer]
b353d4c

chunit-quic force-pushed the dev1/chunit/qat_quantizer_refine branch from dd19c66 to b353d4cCompare November 18, 2024 03:23

Fix linting error
844acda
- Fix get_ptq_per_channel_quant_config not founded error

facebook-github-bot merged commit e95f171 into pytorch:mainNov 18, 2024



		# TODO move to torch/ao/quantization/observer.py.
		class PerChannelParamObserver(UniformQuantizationObserverBase):

Uh oh!

Conversation

chunit-quic commented Nov 11, 2024

Uh oh!

pytorch-botBot commented Nov 11, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6747

❗ 1 Active SEVs

✅ No Failures

Uh oh!

facebook-github-bot commented Nov 11, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chunit-quicNov 13, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Nov 13, 2024

Uh oh!

cccclai commented Nov 13, 2024

Uh oh!

facebook-github-bot commented Nov 14, 2024

Uh oh!

cccclai commented Nov 14, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Nov 18, 2024

Uh oh!

cccclai commented Nov 18, 2024

Uh oh!

chunit-quic commented Nov 18, 2024

Uh oh!

cccclai commented Nov 18, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Nov 18, 2024

Uh oh!

facebook-github-bot commented Nov 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

chunit-quic commented Nov 11, 2024

Uh oh!

pytorch-botBot commented Nov 11, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6747

❗ 1 Active SEVs

✅ No Failures

Uh oh!

facebook-github-bot commented Nov 11, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chunit-quicNov 13, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

pytorch-botBot commented Nov 11, 2024•
edited
Loading

chunit-quicNov 13, 2024•
edited
Loading

cccclai commented Nov 18, 2024•
edited
Loading

pytorch-botBot commented Nov 11, 2024•
edited
Loading

chunit-quicNov 13, 2024•
edited
Loading

cccclai commented Nov 18, 2024•
edited
Loading