[microNPU] Move optimization passes to be a module pass and ensure they are running by lhutton1 · Pull Request #9831 · apache/tvm

lhutton1 · 2022-01-04T15:17:10Z

Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU.

cc @ekalda @manupa-arm

ekalda

Thanks @lhutton1 for spotting and fixing this, it is a very subtle problem! As discussed offline, the fact that these passes silently ceased running as a part of a pipeline (they certainly did use to run when I was implementing the LUT pass) is quite worrying since it means there has been an underlying change to how a relay.transform.function_pass works (or maybe even a bug since the docstring of the implementation implies that it should work on the modules), so there could be other optimization passes that are silently not running now. I think @lhutton1 mentioned though that he didn't grep out many use cases of that function_pass outside of tutorials, but I think in that case we should properly retire it and update the tutorials. Re-tagging @manupa-arm for opinion...

ekalda · 2022-01-11T11:16:24Z

(Forgot to say that dealing with function_pass is a stuff for other PRs, this one looks good to me!)

lhutton1 · 2022-01-11T11:52:31Z

Thanks @ekalda for taking a look, yes it seems quite strange to me. Just wanted to note that function_pass is used outside of tutorials, I was foolishly searching for relay.transform.function_pass rather than just function_pass :)

I did a little bit more digging and it seems like the function_pass doesn't get run on a function when there is a "Compiler" attribute associated with it. I wonder if this is intended behavior and if/when this changed?

manupak · 2022-01-11T12:55:16Z

kCompiler (a.k.a. "Compiler") attribute generally blocks most passes from visiting the function body. Are we saying there was a time where the pass worked even when the kCompiler is set ?

ekalda · 2022-01-11T13:12:09Z

I saw the LUT pass working as a part of a pipeline when I was developing it (I first developed it by running an end to end codegen test with a breakpoint before the pass and checked that the Relay had changed as expected after stepping over it). Out of interest, why does the kCompiler attribute block the passes visiting the function body?

manupak · 2022-01-11T14:24:22Z

It is used to annotate relay functions meant to be compiled with BYOC. Therefore, the compilation passes of core compiler ignores it. IIRC, that was the reason.

I believe this affects function pass because (and not module pass) the pass it self does not choose which functions to mutate as opposed to a module pass.

lhutton1 · 2022-01-11T17:38:28Z

Thanks @manupa-arm, in that case it sounds like migrating to a module pass is the correct thing to do here. It's still a bit of a mystery as to why @ekalda and I believe this used to work, although its probably worth not dwelling on it for now

manupak

The implementation LGTM!

I think we need to be a specific with the title and/or commit message to say what this PR enables. I believe that should be :

Moves LUTOptimizer to be a module pass
Ensure LUTOptimizer Pass is run

rather than generically adding the feature to ensure all optimization passes are run.

lhutton1 · 2022-01-18T11:05:15Z

Agreed, that makes more sense :)

are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-20T17:43:37Z

friendly ping for approval/comments

manupak

LGTM!

manupak · 2022-01-20T18:19:04Z

Thanks @lhutton1 @ekalda !

…ey (apache#9831) are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-04T15:17:10Z

Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU.

cc @ekalda @manupa-arm

ekalda

Thanks @lhutton1 for spotting and fixing this, it is a very subtle problem! As discussed offline, the fact that these passes silently ceased running as a part of a pipeline (they certainly did use to run when I was implementing the LUT pass) is quite worrying since it means there has been an underlying change to how a relay.transform.function_pass works (or maybe even a bug since the docstring of the implementation implies that it should work on the modules), so there could be other optimization passes that are silently not running now. I think @lhutton1 mentioned though that he didn't grep out many use cases of that function_pass outside of tutorials, but I think in that case we should properly retire it and update the tutorials. Re-tagging @manupa-arm for opinion...

ekalda · 2022-01-11T11:16:24Z

(Forgot to say that dealing with function_pass is a stuff for other PRs, this one looks good to me!)

lhutton1 · 2022-01-11T11:52:31Z

Thanks @ekalda for taking a look, yes it seems quite strange to me. Just wanted to note that function_pass is used outside of tutorials, I was foolishly searching for relay.transform.function_pass rather than just function_pass :)

I did a little bit more digging and it seems like the function_pass doesn't get run on a function when there is a "Compiler" attribute associated with it. I wonder if this is intended behavior and if/when this changed?

manupak · 2022-01-11T12:55:16Z

kCompiler (a.k.a. "Compiler") attribute generally blocks most passes from visiting the function body. Are we saying there was a time where the pass worked even when the kCompiler is set ?

ekalda · 2022-01-11T13:12:09Z

I saw the LUT pass working as a part of a pipeline when I was developing it (I first developed it by running an end to end codegen test with a breakpoint before the pass and checked that the Relay had changed as expected after stepping over it). Out of interest, why does the kCompiler attribute block the passes visiting the function body?

manupak · 2022-01-11T14:24:22Z

It is used to annotate relay functions meant to be compiled with BYOC. Therefore, the compilation passes of core compiler ignores it. IIRC, that was the reason.

I believe this affects function pass because (and not module pass) the pass it self does not choose which functions to mutate as opposed to a module pass.

lhutton1 · 2022-01-11T17:38:28Z

Thanks @manupa-arm, in that case it sounds like migrating to a module pass is the correct thing to do here. It's still a bit of a mystery as to why @ekalda and I believe this used to work, although its probably worth not dwelling on it for now

manupak

The implementation LGTM!

I think we need to be a specific with the title and/or commit message to say what this PR enables. I believe that should be :

Moves LUTOptimizer to be a module pass
Ensure LUTOptimizer Pass is run

rather than generically adding the feature to ensure all optimization passes are run.

lhutton1 · 2022-01-18T11:05:15Z

Agreed, that makes more sense :)

are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-20T17:43:37Z

friendly ping for approval/comments

manupak

LGTM!

manupak · 2022-01-20T18:19:04Z

Thanks @lhutton1 @ekalda !

…ey (apache#9831) are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-04T15:17:10Z

Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU.

cc @ekalda @manupa-arm

ekalda

Thanks @lhutton1 for spotting and fixing this, it is a very subtle problem! As discussed offline, the fact that these passes silently ceased running as a part of a pipeline (they certainly did use to run when I was implementing the LUT pass) is quite worrying since it means there has been an underlying change to how a relay.transform.function_pass works (or maybe even a bug since the docstring of the implementation implies that it should work on the modules), so there could be other optimization passes that are silently not running now. I think @lhutton1 mentioned though that he didn't grep out many use cases of that function_pass outside of tutorials, but I think in that case we should properly retire it and update the tutorials. Re-tagging @manupa-arm for opinion...

ekalda · 2022-01-11T11:16:24Z

(Forgot to say that dealing with function_pass is a stuff for other PRs, this one looks good to me!)

lhutton1 · 2022-01-11T11:52:31Z

Thanks @ekalda for taking a look, yes it seems quite strange to me. Just wanted to note that function_pass is used outside of tutorials, I was foolishly searching for relay.transform.function_pass rather than just function_pass :)

I did a little bit more digging and it seems like the function_pass doesn't get run on a function when there is a "Compiler" attribute associated with it. I wonder if this is intended behavior and if/when this changed?

manupak · 2022-01-11T12:55:16Z

kCompiler (a.k.a. "Compiler") attribute generally blocks most passes from visiting the function body. Are we saying there was a time where the pass worked even when the kCompiler is set ?

ekalda · 2022-01-11T13:12:09Z

I saw the LUT pass working as a part of a pipeline when I was developing it (I first developed it by running an end to end codegen test with a breakpoint before the pass and checked that the Relay had changed as expected after stepping over it). Out of interest, why does the kCompiler attribute block the passes visiting the function body?

manupak · 2022-01-11T14:24:22Z

It is used to annotate relay functions meant to be compiled with BYOC. Therefore, the compilation passes of core compiler ignores it. IIRC, that was the reason.

I believe this affects function pass because (and not module pass) the pass it self does not choose which functions to mutate as opposed to a module pass.

lhutton1 · 2022-01-11T17:38:28Z

Thanks @manupa-arm, in that case it sounds like migrating to a module pass is the correct thing to do here. It's still a bit of a mystery as to why @ekalda and I believe this used to work, although its probably worth not dwelling on it for now

manupak

The implementation LGTM!

I think we need to be a specific with the title and/or commit message to say what this PR enables. I believe that should be :

Moves LUTOptimizer to be a module pass
Ensure LUTOptimizer Pass is run

rather than generically adding the feature to ensure all optimization passes are run.

lhutton1 · 2022-01-18T11:05:15Z

Agreed, that makes more sense :)

are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-20T17:43:37Z

friendly ping for approval/comments

manupak

LGTM!

manupak · 2022-01-20T18:19:04Z

Thanks @lhutton1 @ekalda !

…ey (apache#9831) are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-04T15:17:10Z

Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU.

cc @ekalda @manupa-arm

ekalda

Thanks @lhutton1 for spotting and fixing this, it is a very subtle problem! As discussed offline, the fact that these passes silently ceased running as a part of a pipeline (they certainly did use to run when I was implementing the LUT pass) is quite worrying since it means there has been an underlying change to how a relay.transform.function_pass works (or maybe even a bug since the docstring of the implementation implies that it should work on the modules), so there could be other optimization passes that are silently not running now. I think @lhutton1 mentioned though that he didn't grep out many use cases of that function_pass outside of tutorials, but I think in that case we should properly retire it and update the tutorials. Re-tagging @manupa-arm for opinion...

ekalda · 2022-01-11T11:16:24Z

(Forgot to say that dealing with function_pass is a stuff for other PRs, this one looks good to me!)

lhutton1 · 2022-01-11T11:52:31Z

Thanks @ekalda for taking a look, yes it seems quite strange to me. Just wanted to note that function_pass is used outside of tutorials, I was foolishly searching for relay.transform.function_pass rather than just function_pass :)

I did a little bit more digging and it seems like the function_pass doesn't get run on a function when there is a "Compiler" attribute associated with it. I wonder if this is intended behavior and if/when this changed?

manupak · 2022-01-11T12:55:16Z

kCompiler (a.k.a. "Compiler") attribute generally blocks most passes from visiting the function body. Are we saying there was a time where the pass worked even when the kCompiler is set ?

ekalda · 2022-01-11T13:12:09Z

I saw the LUT pass working as a part of a pipeline when I was developing it (I first developed it by running an end to end codegen test with a breakpoint before the pass and checked that the Relay had changed as expected after stepping over it). Out of interest, why does the kCompiler attribute block the passes visiting the function body?

manupak · 2022-01-11T14:24:22Z

It is used to annotate relay functions meant to be compiled with BYOC. Therefore, the compilation passes of core compiler ignores it. IIRC, that was the reason.

I believe this affects function pass because (and not module pass) the pass it self does not choose which functions to mutate as opposed to a module pass.

lhutton1 · 2022-01-11T17:38:28Z

Thanks @manupa-arm, in that case it sounds like migrating to a module pass is the correct thing to do here. It's still a bit of a mystery as to why @ekalda and I believe this used to work, although its probably worth not dwelling on it for now

manupak

The implementation LGTM!

I think we need to be a specific with the title and/or commit message to say what this PR enables. I believe that should be :

Moves LUTOptimizer to be a module pass
Ensure LUTOptimizer Pass is run

rather than generically adding the feature to ensure all optimization passes are run.

lhutton1 · 2022-01-18T11:05:15Z

Agreed, that makes more sense :)

are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-20T17:43:37Z

friendly ping for approval/comments

manupak

LGTM!

manupak · 2022-01-20T18:19:04Z

Thanks @lhutton1 @ekalda !

…ey (apache#9831) are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-04T15:17:10Z

Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU.

cc @ekalda @manupa-arm

ekalda

Thanks @lhutton1 for spotting and fixing this, it is a very subtle problem! As discussed offline, the fact that these passes silently ceased running as a part of a pipeline (they certainly did use to run when I was implementing the LUT pass) is quite worrying since it means there has been an underlying change to how a relay.transform.function_pass works (or maybe even a bug since the docstring of the implementation implies that it should work on the modules), so there could be other optimization passes that are silently not running now. I think @lhutton1 mentioned though that he didn't grep out many use cases of that function_pass outside of tutorials, but I think in that case we should properly retire it and update the tutorials. Re-tagging @manupa-arm for opinion...

ekalda · 2022-01-11T11:16:24Z

(Forgot to say that dealing with function_pass is a stuff for other PRs, this one looks good to me!)

lhutton1 · 2022-01-11T11:52:31Z

Thanks @ekalda for taking a look, yes it seems quite strange to me. Just wanted to note that function_pass is used outside of tutorials, I was foolishly searching for relay.transform.function_pass rather than just function_pass :)

I did a little bit more digging and it seems like the function_pass doesn't get run on a function when there is a "Compiler" attribute associated with it. I wonder if this is intended behavior and if/when this changed?

manupak · 2022-01-11T12:55:16Z

kCompiler (a.k.a. "Compiler") attribute generally blocks most passes from visiting the function body. Are we saying there was a time where the pass worked even when the kCompiler is set ?

ekalda · 2022-01-11T13:12:09Z

I saw the LUT pass working as a part of a pipeline when I was developing it (I first developed it by running an end to end codegen test with a breakpoint before the pass and checked that the Relay had changed as expected after stepping over it). Out of interest, why does the kCompiler attribute block the passes visiting the function body?

manupak · 2022-01-11T14:24:22Z

It is used to annotate relay functions meant to be compiled with BYOC. Therefore, the compilation passes of core compiler ignores it. IIRC, that was the reason.

I believe this affects function pass because (and not module pass) the pass it self does not choose which functions to mutate as opposed to a module pass.

lhutton1 · 2022-01-11T17:38:28Z

Thanks @manupa-arm, in that case it sounds like migrating to a module pass is the correct thing to do here. It's still a bit of a mystery as to why @ekalda and I believe this used to work, although its probably worth not dwelling on it for now

manupak

The implementation LGTM!

I think we need to be a specific with the title and/or commit message to say what this PR enables. I believe that should be :

Moves LUTOptimizer to be a module pass
Ensure LUTOptimizer Pass is run

rather than generically adding the feature to ensure all optimization passes are run.

lhutton1 · 2022-01-18T11:05:15Z

Agreed, that makes more sense :)

are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-20T17:43:37Z

friendly ping for approval/comments

manupak

LGTM!

manupak · 2022-01-20T18:19:04Z

Thanks @lhutton1 @ekalda !

…ey (apache#9831) are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-04T15:17:10Z

Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU.

cc @ekalda @manupa-arm

ekalda

Thanks @lhutton1 for spotting and fixing this, it is a very subtle problem! As discussed offline, the fact that these passes silently ceased running as a part of a pipeline (they certainly did use to run when I was implementing the LUT pass) is quite worrying since it means there has been an underlying change to how a relay.transform.function_pass works (or maybe even a bug since the docstring of the implementation implies that it should work on the modules), so there could be other optimization passes that are silently not running now. I think @lhutton1 mentioned though that he didn't grep out many use cases of that function_pass outside of tutorials, but I think in that case we should properly retire it and update the tutorials. Re-tagging @manupa-arm for opinion...

ekalda · 2022-01-11T11:16:24Z

(Forgot to say that dealing with function_pass is a stuff for other PRs, this one looks good to me!)

lhutton1 · 2022-01-11T11:52:31Z

Thanks @ekalda for taking a look, yes it seems quite strange to me. Just wanted to note that function_pass is used outside of tutorials, I was foolishly searching for relay.transform.function_pass rather than just function_pass :)

I did a little bit more digging and it seems like the function_pass doesn't get run on a function when there is a "Compiler" attribute associated with it. I wonder if this is intended behavior and if/when this changed?

manupak · 2022-01-11T12:55:16Z

kCompiler (a.k.a. "Compiler") attribute generally blocks most passes from visiting the function body. Are we saying there was a time where the pass worked even when the kCompiler is set ?

ekalda · 2022-01-11T13:12:09Z

I saw the LUT pass working as a part of a pipeline when I was developing it (I first developed it by running an end to end codegen test with a breakpoint before the pass and checked that the Relay had changed as expected after stepping over it). Out of interest, why does the kCompiler attribute block the passes visiting the function body?

manupak · 2022-01-11T14:24:22Z

It is used to annotate relay functions meant to be compiled with BYOC. Therefore, the compilation passes of core compiler ignores it. IIRC, that was the reason.

I believe this affects function pass because (and not module pass) the pass it self does not choose which functions to mutate as opposed to a module pass.

lhutton1 · 2022-01-11T17:38:28Z

Thanks @manupa-arm, in that case it sounds like migrating to a module pass is the correct thing to do here. It's still a bit of a mystery as to why @ekalda and I believe this used to work, although its probably worth not dwelling on it for now

manupak

The implementation LGTM!

I think we need to be a specific with the title and/or commit message to say what this PR enables. I believe that should be :

Moves LUTOptimizer to be a module pass
Ensure LUTOptimizer Pass is run

rather than generically adding the feature to ensure all optimization passes are run.

lhutton1 · 2022-01-18T11:05:15Z

Agreed, that makes more sense :)

are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-20T17:43:37Z

friendly ping for approval/comments

manupak

LGTM!

manupak · 2022-01-20T18:19:04Z

Thanks @lhutton1 @ekalda !

…ey (apache#9831) are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-04T15:17:10Z

Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU.

cc @ekalda @manupa-arm

ekalda

Thanks @lhutton1 for spotting and fixing this, it is a very subtle problem! As discussed offline, the fact that these passes silently ceased running as a part of a pipeline (they certainly did use to run when I was implementing the LUT pass) is quite worrying since it means there has been an underlying change to how a relay.transform.function_pass works (or maybe even a bug since the docstring of the implementation implies that it should work on the modules), so there could be other optimization passes that are silently not running now. I think @lhutton1 mentioned though that he didn't grep out many use cases of that function_pass outside of tutorials, but I think in that case we should properly retire it and update the tutorials. Re-tagging @manupa-arm for opinion...

ekalda · 2022-01-11T11:16:24Z

(Forgot to say that dealing with function_pass is a stuff for other PRs, this one looks good to me!)

lhutton1 · 2022-01-11T11:52:31Z

Thanks @ekalda for taking a look, yes it seems quite strange to me. Just wanted to note that function_pass is used outside of tutorials, I was foolishly searching for relay.transform.function_pass rather than just function_pass :)

I did a little bit more digging and it seems like the function_pass doesn't get run on a function when there is a "Compiler" attribute associated with it. I wonder if this is intended behavior and if/when this changed?

manupak · 2022-01-11T12:55:16Z

kCompiler (a.k.a. "Compiler") attribute generally blocks most passes from visiting the function body. Are we saying there was a time where the pass worked even when the kCompiler is set ?

ekalda · 2022-01-11T13:12:09Z

I saw the LUT pass working as a part of a pipeline when I was developing it (I first developed it by running an end to end codegen test with a breakpoint before the pass and checked that the Relay had changed as expected after stepping over it). Out of interest, why does the kCompiler attribute block the passes visiting the function body?

manupak · 2022-01-11T14:24:22Z

It is used to annotate relay functions meant to be compiled with BYOC. Therefore, the compilation passes of core compiler ignores it. IIRC, that was the reason.

I believe this affects function pass because (and not module pass) the pass it self does not choose which functions to mutate as opposed to a module pass.

lhutton1 · 2022-01-11T17:38:28Z

Thanks @manupa-arm, in that case it sounds like migrating to a module pass is the correct thing to do here. It's still a bit of a mystery as to why @ekalda and I believe this used to work, although its probably worth not dwelling on it for now

manupak

The implementation LGTM!

I think we need to be a specific with the title and/or commit message to say what this PR enables. I believe that should be :

Moves LUTOptimizer to be a module pass
Ensure LUTOptimizer Pass is run

rather than generically adding the feature to ensure all optimization passes are run.

lhutton1 · 2022-01-18T11:05:15Z

Agreed, that makes more sense :)

are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-20T17:43:37Z

friendly ping for approval/comments

manupak

LGTM!

manupak · 2022-01-20T18:19:04Z

Thanks @lhutton1 @ekalda !

…ey (apache#9831) are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-04T15:17:10Z

Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU.

cc @ekalda @manupa-arm

ekalda

Thanks @lhutton1 for spotting and fixing this, it is a very subtle problem! As discussed offline, the fact that these passes silently ceased running as a part of a pipeline (they certainly did use to run when I was implementing the LUT pass) is quite worrying since it means there has been an underlying change to how a relay.transform.function_pass works (or maybe even a bug since the docstring of the implementation implies that it should work on the modules), so there could be other optimization passes that are silently not running now. I think @lhutton1 mentioned though that he didn't grep out many use cases of that function_pass outside of tutorials, but I think in that case we should properly retire it and update the tutorials. Re-tagging @manupa-arm for opinion...

ekalda · 2022-01-11T11:16:24Z

(Forgot to say that dealing with function_pass is a stuff for other PRs, this one looks good to me!)

lhutton1 · 2022-01-11T11:52:31Z

Thanks @ekalda for taking a look, yes it seems quite strange to me. Just wanted to note that function_pass is used outside of tutorials, I was foolishly searching for relay.transform.function_pass rather than just function_pass :)

I did a little bit more digging and it seems like the function_pass doesn't get run on a function when there is a "Compiler" attribute associated with it. I wonder if this is intended behavior and if/when this changed?

manupak · 2022-01-11T12:55:16Z

kCompiler (a.k.a. "Compiler") attribute generally blocks most passes from visiting the function body. Are we saying there was a time where the pass worked even when the kCompiler is set ?

ekalda · 2022-01-11T13:12:09Z

I saw the LUT pass working as a part of a pipeline when I was developing it (I first developed it by running an end to end codegen test with a breakpoint before the pass and checked that the Relay had changed as expected after stepping over it). Out of interest, why does the kCompiler attribute block the passes visiting the function body?

manupak · 2022-01-11T14:24:22Z

It is used to annotate relay functions meant to be compiled with BYOC. Therefore, the compilation passes of core compiler ignores it. IIRC, that was the reason.

I believe this affects function pass because (and not module pass) the pass it self does not choose which functions to mutate as opposed to a module pass.

lhutton1 · 2022-01-11T17:38:28Z

Thanks @manupa-arm, in that case it sounds like migrating to a module pass is the correct thing to do here. It's still a bit of a mystery as to why @ekalda and I believe this used to work, although its probably worth not dwelling on it for now

manupak

The implementation LGTM!

I think we need to be a specific with the title and/or commit message to say what this PR enables. I believe that should be :

Moves LUTOptimizer to be a module pass
Ensure LUTOptimizer Pass is run

rather than generically adding the feature to ensure all optimization passes are run.

lhutton1 · 2022-01-18T11:05:15Z

Agreed, that makes more sense :)

are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 · 2022-01-20T17:43:37Z

friendly ping for approval/comments

manupak

LGTM!

manupak · 2022-01-20T18:19:04Z

Thanks @lhutton1 @ekalda !

…ey (apache#9831) are running Moves LayoutOptimizer and LUTOptimizer passes to be a module pass, rather than a function pass. This is because it was found that these passes were not running in the NPU compilation flow. In addition, a test for both LayoutOptimizer and LUTOptimizer has been added to check that the passes are running in the compilation pipeline of the NPU. Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

lhutton1 requested review from MarisaKirisame, ZihengJiang, anijain2305, areusch, comaniac, icemelon, jroesch, junrushao, jwfromm, mbrookhart, merrymercy, slyubomirsky, tqchen, vinx13, wweic, yzhliu and zhiics as code owners January 4, 2022 15:17

ekalda approved these changes Jan 11, 2022
View reviewed changes

manupak reviewed Jan 18, 2022
View reviewed changes

lhutton1 force-pushed the check-optimize branch from 9858432 to 9301f72Compare January 18, 2022 11:16

lhutton1 changed the title ~~[microNPU] Ensure optimization passes are running in compilation pipeline~~[microNPU] Move optimization passes to be a module pass and ensure they are runningJan 18, 2022

manupak approved these changes Jan 20, 2022
View reviewed changes

manupak merged commit e390d9e into apache:mainJan 20, 2022

lhutton1 deleted the check-optimize branch January 20, 2022 18:51

driazati mentioned this pull request Jul 14, 2022
TVM v0.9.0.rc0 Release Candidate Notes #12102
Closed