[microNPU][2b] Create CascaderGraphs from TE graphs by mbaret · Pull Request #9471 · apache/tvm

mbaret · 2021-11-08T17:04:57Z

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators.

In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested. Change-Id: I36bbbdbafd14f09e6d1a8abf32591d6755192915

Change-Id: I34e48e4ce4fd14e6c92601412f5029aa08efe8fc

Change-Id: Idcb92c26863837b06fa68f9122a1a79df79ee242

Change-Id: I0aad319b7722a56b5c7f116adcc791951223baa3

Change-Id: Ib1633341ed84e83c29c4220dd9bc4a7b6d17a5d0

Change-Id: I5223da462a8dadfda1798e1ea1c9b30b4e804217

Change-Id: Ib614c21e97723571aa002a2a4a3edda3afafb9d4

jacobbohlin

LGTM!

mbaret · 2022-01-06T17:37:08Z

cc @Mousius could you take a look?

Mousius

Overall I think the code is non-controversial but test coverage needs a bit of work, could you look at adding the additional tests? It'd give confidence that as this evolves we don't lose some of the more specific clauses 😸

Mousius · 2022-01-07T15:27:02Z

+ write = output_tensor
+ if write.op.name != "ethosu_write":
+ return None
+ convert_to_nhcwb16 = write.op.input_tensors[0]
+ if convert_to_nhcwb16.op.name != "ethosu_convert_to_nhcwb16":
+ return None
+ conv2d = convert_to_nhcwb16.op.input_tensors[0]
+ if conv2d.op.name != "ethosu_conv2d":
+ return None
+ pad = conv2d.op.input_tensors[0]
+ if pad.op.name != "ethosu_pad":
+ return None
+ convert_to_nhwc = pad.op.input_tensors[0]
+ if convert_to_nhwc.op.name != "ethosu_convert_to_nhwc":
+ return None
+ read = convert_to_nhwc.op.input_tensors[0]
+ if read.op.name != "ethosu_read":
+ return None


I can't see test cases for all of these clauses, so we're missing most of the logic of this function? Can you add some?

Mousius · 2022-01-07T15:35:34Z

+ part = match_ethosu_conv2d(out)
+
+ assert isinstance(part, cs.EthosuPart)
+ assert len(part.propagators) == 3


Can we test the other properties of the part returned?

Mousius · 2022-01-07T15:36:17Z

+ part = match_ethosu_inline(out)
+
+ assert isinstance(part, cs.InlinePart)
+ assert len(part.propagators) == 1


Can we test the other properties of the part ?

Mousius · 2022-01-07T15:37:12Z

-)
+pytest.importorskip("ethosu.vela")
+
+import tvm.contrib.ethosu.cascader as cs


As much as I appreciate using my initials here, it'd be really great to use something I could immediately associate with the cascader; potentially cascader rather than cs ?

Mousius · 2022-01-07T15:38:31Z

+ assert part.propagators[0].transform == ifm_transform
+ assert part.propagators[0].offset == ifm_offset


It would be good to test at least a few cases just to ensure the function doesn't always return the same thing 😸

Mousius · 2022-01-07T15:38:49Z

+ if output_tensor.op.name not in INLINE_OPS:
+ return None


Quick test for this please!

Mousius · 2022-01-07T15:40:08Z

+ input_tensors = part.subgraph.input_tensors
+ break
+
+ assert part is not None, f"The tensor {tensor} doesn't match any part."


Would be good to ensure this fires when we expect it to with pytest.raises.

Mousius

Had a chat with @mbaret, because this patch is blocking a bunch of other patches I'll commit this to enable forward progress and ensure that the test coverage is followed up on.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

mbaret · 2021-11-08T17:04:57Z

RFC: apache/tvm-rfcs#37
Issue: #9429

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators.

In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested. Change-Id: I36bbbdbafd14f09e6d1a8abf32591d6755192915

Change-Id: I34e48e4ce4fd14e6c92601412f5029aa08efe8fc

Change-Id: Idcb92c26863837b06fa68f9122a1a79df79ee242

Change-Id: I0aad319b7722a56b5c7f116adcc791951223baa3

Change-Id: Ib1633341ed84e83c29c4220dd9bc4a7b6d17a5d0

Change-Id: I5223da462a8dadfda1798e1ea1c9b30b4e804217

Change-Id: Ib614c21e97723571aa002a2a4a3edda3afafb9d4

jacobbohlin

LGTM!

mbaret · 2022-01-06T17:37:08Z

cc @Mousius could you take a look?

Mousius

Overall I think the code is non-controversial but test coverage needs a bit of work, could you look at adding the additional tests? It'd give confidence that as this evolves we don't lose some of the more specific clauses 😸

Mousius · 2022-01-07T15:27:02Z

+ write = output_tensor
+ if write.op.name != "ethosu_write":
+ return None
+ convert_to_nhcwb16 = write.op.input_tensors[0]
+ if convert_to_nhcwb16.op.name != "ethosu_convert_to_nhcwb16":
+ return None
+ conv2d = convert_to_nhcwb16.op.input_tensors[0]
+ if conv2d.op.name != "ethosu_conv2d":
+ return None
+ pad = conv2d.op.input_tensors[0]
+ if pad.op.name != "ethosu_pad":
+ return None
+ convert_to_nhwc = pad.op.input_tensors[0]
+ if convert_to_nhwc.op.name != "ethosu_convert_to_nhwc":
+ return None
+ read = convert_to_nhwc.op.input_tensors[0]
+ if read.op.name != "ethosu_read":
+ return None


I can't see test cases for all of these clauses, so we're missing most of the logic of this function? Can you add some?

Mousius · 2022-01-07T15:35:34Z

+ part = match_ethosu_conv2d(out)
+
+ assert isinstance(part, cs.EthosuPart)
+ assert len(part.propagators) == 3


Can we test the other properties of the part returned?

Mousius · 2022-01-07T15:36:17Z

+ part = match_ethosu_inline(out)
+
+ assert isinstance(part, cs.InlinePart)
+ assert len(part.propagators) == 1


Can we test the other properties of the part ?

Mousius · 2022-01-07T15:37:12Z

-)
+pytest.importorskip("ethosu.vela")
+
+import tvm.contrib.ethosu.cascader as cs


As much as I appreciate using my initials here, it'd be really great to use something I could immediately associate with the cascader; potentially cascader rather than cs ?

Mousius · 2022-01-07T15:38:31Z

+ assert part.propagators[0].transform == ifm_transform
+ assert part.propagators[0].offset == ifm_offset


It would be good to test at least a few cases just to ensure the function doesn't always return the same thing 😸

Mousius · 2022-01-07T15:38:49Z

+ if output_tensor.op.name not in INLINE_OPS:
+ return None


Quick test for this please!

Mousius · 2022-01-07T15:40:08Z

+ input_tensors = part.subgraph.input_tensors
+ break
+
+ assert part is not None, f"The tensor {tensor} doesn't match any part."


Would be good to ensure this fires when we expect it to with pytest.raises.

Mousius

Had a chat with @mbaret, because this patch is blocking a bunch of other patches I'll commit this to enable forward progress and ensure that the test coverage is followed up on.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

mbaret · 2021-11-08T17:04:57Z

RFC: apache/tvm-rfcs#37
Issue: #9429

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators.

In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested. Change-Id: I36bbbdbafd14f09e6d1a8abf32591d6755192915

Change-Id: I34e48e4ce4fd14e6c92601412f5029aa08efe8fc

Change-Id: Idcb92c26863837b06fa68f9122a1a79df79ee242

Change-Id: I0aad319b7722a56b5c7f116adcc791951223baa3

Change-Id: Ib1633341ed84e83c29c4220dd9bc4a7b6d17a5d0

Change-Id: I5223da462a8dadfda1798e1ea1c9b30b4e804217

Change-Id: Ib614c21e97723571aa002a2a4a3edda3afafb9d4

jacobbohlin

LGTM!

mbaret · 2022-01-06T17:37:08Z

cc @Mousius could you take a look?

Mousius

Overall I think the code is non-controversial but test coverage needs a bit of work, could you look at adding the additional tests? It'd give confidence that as this evolves we don't lose some of the more specific clauses 😸

Mousius · 2022-01-07T15:27:02Z

+ write = output_tensor
+ if write.op.name != "ethosu_write":
+ return None
+ convert_to_nhcwb16 = write.op.input_tensors[0]
+ if convert_to_nhcwb16.op.name != "ethosu_convert_to_nhcwb16":
+ return None
+ conv2d = convert_to_nhcwb16.op.input_tensors[0]
+ if conv2d.op.name != "ethosu_conv2d":
+ return None
+ pad = conv2d.op.input_tensors[0]
+ if pad.op.name != "ethosu_pad":
+ return None
+ convert_to_nhwc = pad.op.input_tensors[0]
+ if convert_to_nhwc.op.name != "ethosu_convert_to_nhwc":
+ return None
+ read = convert_to_nhwc.op.input_tensors[0]
+ if read.op.name != "ethosu_read":
+ return None


I can't see test cases for all of these clauses, so we're missing most of the logic of this function? Can you add some?

Mousius · 2022-01-07T15:35:34Z

+ part = match_ethosu_conv2d(out)
+
+ assert isinstance(part, cs.EthosuPart)
+ assert len(part.propagators) == 3


Can we test the other properties of the part returned?

Mousius · 2022-01-07T15:36:17Z

+ part = match_ethosu_inline(out)
+
+ assert isinstance(part, cs.InlinePart)
+ assert len(part.propagators) == 1


Can we test the other properties of the part ?

Mousius · 2022-01-07T15:37:12Z

-)
+pytest.importorskip("ethosu.vela")
+
+import tvm.contrib.ethosu.cascader as cs


As much as I appreciate using my initials here, it'd be really great to use something I could immediately associate with the cascader; potentially cascader rather than cs ?

Mousius · 2022-01-07T15:38:31Z

+ assert part.propagators[0].transform == ifm_transform
+ assert part.propagators[0].offset == ifm_offset


It would be good to test at least a few cases just to ensure the function doesn't always return the same thing 😸

Mousius · 2022-01-07T15:38:49Z

+ if output_tensor.op.name not in INLINE_OPS:
+ return None


Quick test for this please!

Mousius · 2022-01-07T15:40:08Z

+ input_tensors = part.subgraph.input_tensors
+ break
+
+ assert part is not None, f"The tensor {tensor} doesn't match any part."


Would be good to ensure this fires when we expect it to with pytest.raises.

Mousius

Had a chat with @mbaret, because this patch is blocking a bunch of other patches I'll commit this to enable forward progress and ensure that the test coverage is followed up on.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

mbaret · 2021-11-08T17:04:57Z

RFC: apache/tvm-rfcs#37
Issue: #9429

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators.

In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested. Change-Id: I36bbbdbafd14f09e6d1a8abf32591d6755192915

Change-Id: I34e48e4ce4fd14e6c92601412f5029aa08efe8fc

Change-Id: Idcb92c26863837b06fa68f9122a1a79df79ee242

Change-Id: I0aad319b7722a56b5c7f116adcc791951223baa3

Change-Id: Ib1633341ed84e83c29c4220dd9bc4a7b6d17a5d0

Change-Id: I5223da462a8dadfda1798e1ea1c9b30b4e804217

Change-Id: Ib614c21e97723571aa002a2a4a3edda3afafb9d4

jacobbohlin

LGTM!

mbaret · 2022-01-06T17:37:08Z

cc @Mousius could you take a look?

Mousius

Overall I think the code is non-controversial but test coverage needs a bit of work, could you look at adding the additional tests? It'd give confidence that as this evolves we don't lose some of the more specific clauses 😸

Mousius · 2022-01-07T15:27:02Z

+ write = output_tensor
+ if write.op.name != "ethosu_write":
+ return None
+ convert_to_nhcwb16 = write.op.input_tensors[0]
+ if convert_to_nhcwb16.op.name != "ethosu_convert_to_nhcwb16":
+ return None
+ conv2d = convert_to_nhcwb16.op.input_tensors[0]
+ if conv2d.op.name != "ethosu_conv2d":
+ return None
+ pad = conv2d.op.input_tensors[0]
+ if pad.op.name != "ethosu_pad":
+ return None
+ convert_to_nhwc = pad.op.input_tensors[0]
+ if convert_to_nhwc.op.name != "ethosu_convert_to_nhwc":
+ return None
+ read = convert_to_nhwc.op.input_tensors[0]
+ if read.op.name != "ethosu_read":
+ return None


I can't see test cases for all of these clauses, so we're missing most of the logic of this function? Can you add some?

Mousius · 2022-01-07T15:35:34Z

+ part = match_ethosu_conv2d(out)
+
+ assert isinstance(part, cs.EthosuPart)
+ assert len(part.propagators) == 3


Can we test the other properties of the part returned?

Mousius · 2022-01-07T15:36:17Z

+ part = match_ethosu_inline(out)
+
+ assert isinstance(part, cs.InlinePart)
+ assert len(part.propagators) == 1


Can we test the other properties of the part ?

Mousius · 2022-01-07T15:37:12Z

-)
+pytest.importorskip("ethosu.vela")
+
+import tvm.contrib.ethosu.cascader as cs


As much as I appreciate using my initials here, it'd be really great to use something I could immediately associate with the cascader; potentially cascader rather than cs ?

Mousius · 2022-01-07T15:38:31Z

+ assert part.propagators[0].transform == ifm_transform
+ assert part.propagators[0].offset == ifm_offset


It would be good to test at least a few cases just to ensure the function doesn't always return the same thing 😸

Mousius · 2022-01-07T15:38:49Z

+ if output_tensor.op.name not in INLINE_OPS:
+ return None


Quick test for this please!

Mousius · 2022-01-07T15:40:08Z

+ input_tensors = part.subgraph.input_tensors
+ break
+
+ assert part is not None, f"The tensor {tensor} doesn't match any part."


Would be good to ensure this fires when we expect it to with pytest.raises.

Mousius

Had a chat with @mbaret, because this patch is blocking a bunch of other patches I'll commit this to enable forward progress and ensure that the test coverage is followed up on.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

mbaret · 2021-11-08T17:04:57Z

RFC: apache/tvm-rfcs#37
Issue: #9429

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators.

In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested. Change-Id: I36bbbdbafd14f09e6d1a8abf32591d6755192915

Change-Id: I34e48e4ce4fd14e6c92601412f5029aa08efe8fc

Change-Id: Idcb92c26863837b06fa68f9122a1a79df79ee242

Change-Id: I0aad319b7722a56b5c7f116adcc791951223baa3

Change-Id: Ib1633341ed84e83c29c4220dd9bc4a7b6d17a5d0

Change-Id: I5223da462a8dadfda1798e1ea1c9b30b4e804217

Change-Id: Ib614c21e97723571aa002a2a4a3edda3afafb9d4

jacobbohlin

LGTM!

mbaret · 2022-01-06T17:37:08Z

cc @Mousius could you take a look?

Mousius

Overall I think the code is non-controversial but test coverage needs a bit of work, could you look at adding the additional tests? It'd give confidence that as this evolves we don't lose some of the more specific clauses 😸

Mousius · 2022-01-07T15:27:02Z

+ write = output_tensor
+ if write.op.name != "ethosu_write":
+ return None
+ convert_to_nhcwb16 = write.op.input_tensors[0]
+ if convert_to_nhcwb16.op.name != "ethosu_convert_to_nhcwb16":
+ return None
+ conv2d = convert_to_nhcwb16.op.input_tensors[0]
+ if conv2d.op.name != "ethosu_conv2d":
+ return None
+ pad = conv2d.op.input_tensors[0]
+ if pad.op.name != "ethosu_pad":
+ return None
+ convert_to_nhwc = pad.op.input_tensors[0]
+ if convert_to_nhwc.op.name != "ethosu_convert_to_nhwc":
+ return None
+ read = convert_to_nhwc.op.input_tensors[0]
+ if read.op.name != "ethosu_read":
+ return None


I can't see test cases for all of these clauses, so we're missing most of the logic of this function? Can you add some?

Mousius · 2022-01-07T15:35:34Z

+ part = match_ethosu_conv2d(out)
+
+ assert isinstance(part, cs.EthosuPart)
+ assert len(part.propagators) == 3


Can we test the other properties of the part returned?

Mousius · 2022-01-07T15:36:17Z

+ part = match_ethosu_inline(out)
+
+ assert isinstance(part, cs.InlinePart)
+ assert len(part.propagators) == 1


Can we test the other properties of the part ?

Mousius · 2022-01-07T15:37:12Z

-)
+pytest.importorskip("ethosu.vela")
+
+import tvm.contrib.ethosu.cascader as cs


As much as I appreciate using my initials here, it'd be really great to use something I could immediately associate with the cascader; potentially cascader rather than cs ?

Mousius · 2022-01-07T15:38:31Z

+ assert part.propagators[0].transform == ifm_transform
+ assert part.propagators[0].offset == ifm_offset


It would be good to test at least a few cases just to ensure the function doesn't always return the same thing 😸

Mousius · 2022-01-07T15:38:49Z

+ if output_tensor.op.name not in INLINE_OPS:
+ return None


Quick test for this please!

Mousius · 2022-01-07T15:40:08Z

+ input_tensors = part.subgraph.input_tensors
+ break
+
+ assert part is not None, f"The tensor {tensor} doesn't match any part."


Would be good to ensure this fires when we expect it to with pytest.raises.

Mousius

Had a chat with @mbaret, because this patch is blocking a bunch of other patches I'll commit this to enable forward progress and ensure that the test coverage is followed up on.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

mbaret · 2021-11-08T17:04:57Z

RFC: apache/tvm-rfcs#37
Issue: #9429

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators.

In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested. Change-Id: I36bbbdbafd14f09e6d1a8abf32591d6755192915

Change-Id: I34e48e4ce4fd14e6c92601412f5029aa08efe8fc

Change-Id: Idcb92c26863837b06fa68f9122a1a79df79ee242

Change-Id: I0aad319b7722a56b5c7f116adcc791951223baa3

Change-Id: Ib1633341ed84e83c29c4220dd9bc4a7b6d17a5d0

Change-Id: I5223da462a8dadfda1798e1ea1c9b30b4e804217

Change-Id: Ib614c21e97723571aa002a2a4a3edda3afafb9d4

jacobbohlin

LGTM!

mbaret · 2022-01-06T17:37:08Z

cc @Mousius could you take a look?

Mousius

Overall I think the code is non-controversial but test coverage needs a bit of work, could you look at adding the additional tests? It'd give confidence that as this evolves we don't lose some of the more specific clauses 😸

Mousius · 2022-01-07T15:27:02Z

+ write = output_tensor
+ if write.op.name != "ethosu_write":
+ return None
+ convert_to_nhcwb16 = write.op.input_tensors[0]
+ if convert_to_nhcwb16.op.name != "ethosu_convert_to_nhcwb16":
+ return None
+ conv2d = convert_to_nhcwb16.op.input_tensors[0]
+ if conv2d.op.name != "ethosu_conv2d":
+ return None
+ pad = conv2d.op.input_tensors[0]
+ if pad.op.name != "ethosu_pad":
+ return None
+ convert_to_nhwc = pad.op.input_tensors[0]
+ if convert_to_nhwc.op.name != "ethosu_convert_to_nhwc":
+ return None
+ read = convert_to_nhwc.op.input_tensors[0]
+ if read.op.name != "ethosu_read":
+ return None


I can't see test cases for all of these clauses, so we're missing most of the logic of this function? Can you add some?

Mousius · 2022-01-07T15:35:34Z

+ part = match_ethosu_conv2d(out)
+
+ assert isinstance(part, cs.EthosuPart)
+ assert len(part.propagators) == 3


Can we test the other properties of the part returned?

Mousius · 2022-01-07T15:36:17Z

+ part = match_ethosu_inline(out)
+
+ assert isinstance(part, cs.InlinePart)
+ assert len(part.propagators) == 1


Can we test the other properties of the part ?

Mousius · 2022-01-07T15:37:12Z

-)
+pytest.importorskip("ethosu.vela")
+
+import tvm.contrib.ethosu.cascader as cs


As much as I appreciate using my initials here, it'd be really great to use something I could immediately associate with the cascader; potentially cascader rather than cs ?

Mousius · 2022-01-07T15:38:31Z

+ assert part.propagators[0].transform == ifm_transform
+ assert part.propagators[0].offset == ifm_offset


It would be good to test at least a few cases just to ensure the function doesn't always return the same thing 😸

Mousius · 2022-01-07T15:38:49Z

+ if output_tensor.op.name not in INLINE_OPS:
+ return None


Quick test for this please!

Mousius · 2022-01-07T15:40:08Z

+ input_tensors = part.subgraph.input_tensors
+ break
+
+ assert part is not None, f"The tensor {tensor} doesn't match any part."


Would be good to ensure this fires when we expect it to with pytest.raises.

Mousius

Had a chat with @mbaret, because this patch is blocking a bunch of other patches I'll commit this to enable forward progress and ensure that the test coverage is followed up on.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

mbaret · 2021-11-08T17:04:57Z

RFC: apache/tvm-rfcs#37
Issue: #9429

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators.

In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested. Change-Id: I36bbbdbafd14f09e6d1a8abf32591d6755192915

Change-Id: I34e48e4ce4fd14e6c92601412f5029aa08efe8fc

Change-Id: Idcb92c26863837b06fa68f9122a1a79df79ee242

Change-Id: I0aad319b7722a56b5c7f116adcc791951223baa3

Change-Id: Ib1633341ed84e83c29c4220dd9bc4a7b6d17a5d0

Change-Id: I5223da462a8dadfda1798e1ea1c9b30b4e804217

Change-Id: Ib614c21e97723571aa002a2a4a3edda3afafb9d4

jacobbohlin

LGTM!

mbaret · 2022-01-06T17:37:08Z

cc @Mousius could you take a look?

Mousius

Overall I think the code is non-controversial but test coverage needs a bit of work, could you look at adding the additional tests? It'd give confidence that as this evolves we don't lose some of the more specific clauses 😸

Mousius · 2022-01-07T15:27:02Z

+ write = output_tensor
+ if write.op.name != "ethosu_write":
+ return None
+ convert_to_nhcwb16 = write.op.input_tensors[0]
+ if convert_to_nhcwb16.op.name != "ethosu_convert_to_nhcwb16":
+ return None
+ conv2d = convert_to_nhcwb16.op.input_tensors[0]
+ if conv2d.op.name != "ethosu_conv2d":
+ return None
+ pad = conv2d.op.input_tensors[0]
+ if pad.op.name != "ethosu_pad":
+ return None
+ convert_to_nhwc = pad.op.input_tensors[0]
+ if convert_to_nhwc.op.name != "ethosu_convert_to_nhwc":
+ return None
+ read = convert_to_nhwc.op.input_tensors[0]
+ if read.op.name != "ethosu_read":
+ return None


I can't see test cases for all of these clauses, so we're missing most of the logic of this function? Can you add some?

Mousius · 2022-01-07T15:35:34Z

+ part = match_ethosu_conv2d(out)
+
+ assert isinstance(part, cs.EthosuPart)
+ assert len(part.propagators) == 3


Can we test the other properties of the part returned?

Mousius · 2022-01-07T15:36:17Z

+ part = match_ethosu_inline(out)
+
+ assert isinstance(part, cs.InlinePart)
+ assert len(part.propagators) == 1


Can we test the other properties of the part ?

Mousius · 2022-01-07T15:37:12Z

-)
+pytest.importorskip("ethosu.vela")
+
+import tvm.contrib.ethosu.cascader as cs


As much as I appreciate using my initials here, it'd be really great to use something I could immediately associate with the cascader; potentially cascader rather than cs ?

Mousius · 2022-01-07T15:38:31Z

+ assert part.propagators[0].transform == ifm_transform
+ assert part.propagators[0].offset == ifm_offset


It would be good to test at least a few cases just to ensure the function doesn't always return the same thing 😸

Mousius · 2022-01-07T15:38:49Z

+ if output_tensor.op.name not in INLINE_OPS:
+ return None


Quick test for this please!

Mousius · 2022-01-07T15:40:08Z

+ input_tensors = part.subgraph.input_tensors
+ break
+
+ assert part is not None, f"The tensor {tensor} doesn't match any part."


Would be good to ensure this fires when we expect it to with pytest.raises.

Mousius

Had a chat with @mbaret, because this patch is blocking a bunch of other patches I'll commit this to enable forward progress and ensure that the test coverage is followed up on.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

mbaret · 2021-11-08T17:04:57Z

RFC: apache/tvm-rfcs#37
Issue: #9429

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators.

In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested. Change-Id: I36bbbdbafd14f09e6d1a8abf32591d6755192915

Change-Id: I34e48e4ce4fd14e6c92601412f5029aa08efe8fc

Change-Id: Idcb92c26863837b06fa68f9122a1a79df79ee242

Change-Id: I0aad319b7722a56b5c7f116adcc791951223baa3

Change-Id: Ib1633341ed84e83c29c4220dd9bc4a7b6d17a5d0

Change-Id: I5223da462a8dadfda1798e1ea1c9b30b4e804217

Change-Id: Ib614c21e97723571aa002a2a4a3edda3afafb9d4

jacobbohlin

LGTM!

mbaret · 2022-01-06T17:37:08Z

cc @Mousius could you take a look?

Mousius

Overall I think the code is non-controversial but test coverage needs a bit of work, could you look at adding the additional tests? It'd give confidence that as this evolves we don't lose some of the more specific clauses 😸

Mousius · 2022-01-07T15:27:02Z

+ write = output_tensor
+ if write.op.name != "ethosu_write":
+ return None
+ convert_to_nhcwb16 = write.op.input_tensors[0]
+ if convert_to_nhcwb16.op.name != "ethosu_convert_to_nhcwb16":
+ return None
+ conv2d = convert_to_nhcwb16.op.input_tensors[0]
+ if conv2d.op.name != "ethosu_conv2d":
+ return None
+ pad = conv2d.op.input_tensors[0]
+ if pad.op.name != "ethosu_pad":
+ return None
+ convert_to_nhwc = pad.op.input_tensors[0]
+ if convert_to_nhwc.op.name != "ethosu_convert_to_nhwc":
+ return None
+ read = convert_to_nhwc.op.input_tensors[0]
+ if read.op.name != "ethosu_read":
+ return None


I can't see test cases for all of these clauses, so we're missing most of the logic of this function? Can you add some?

Mousius · 2022-01-07T15:35:34Z

+ part = match_ethosu_conv2d(out)
+
+ assert isinstance(part, cs.EthosuPart)
+ assert len(part.propagators) == 3


Can we test the other properties of the part returned?

Mousius · 2022-01-07T15:36:17Z

+ part = match_ethosu_inline(out)
+
+ assert isinstance(part, cs.InlinePart)
+ assert len(part.propagators) == 1


Can we test the other properties of the part ?

Mousius · 2022-01-07T15:37:12Z

-)
+pytest.importorskip("ethosu.vela")
+
+import tvm.contrib.ethosu.cascader as cs


As much as I appreciate using my initials here, it'd be really great to use something I could immediately associate with the cascader; potentially cascader rather than cs ?

Mousius · 2022-01-07T15:38:31Z

+ assert part.propagators[0].transform == ifm_transform
+ assert part.propagators[0].offset == ifm_offset


It would be good to test at least a few cases just to ensure the function doesn't always return the same thing 😸

Mousius · 2022-01-07T15:38:49Z

+ if output_tensor.op.name not in INLINE_OPS:
+ return None


Quick test for this please!

Mousius · 2022-01-07T15:40:08Z

+ input_tensors = part.subgraph.input_tensors
+ break
+
+ assert part is not None, f"The tensor {tensor} doesn't match any part."


Would be good to ensure this fires when we expect it to with pytest.raises.

Mousius

Had a chat with @mbaret, because this patch is blocking a bunch of other patches I'll commit this to enable forward progress and ensure that the test coverage is followed up on.

The first step in the cascader is to create a CascaderGraph from a TE graph. To do this, every operator in the TE graph must get 'matched' by a Part matcher. This converts TE operations into cascader Parts by augmenting them with Propagators. In this initial commit, we include basic Part matchers for ethosu_conv2d and some transform operators so that the graph creation can be tested.

mbaret requested review from MarisaKirisame, ZihengJiang, anijain2305, areusch, comaniac, icemelon, jroesch, junrushao, jwfromm, mbrookhart, merrymercy, slyubomirsky, tqchen, vinx13, wweic, yzhliu and zhiics as code owners November 8, 2021 17:04

mbaret mentioned this pull request Nov 8, 2021
[Tracking Issue] Arm(R) Ethos(TM)-U Cascading Scheduler #9429
Closed
12 tasks

mbaret removed request for MarisaKirisame, areusch, comaniac, icemelon, jroesch, mbrookhart, slyubomirsky, tqchen, vinx13, wweic, yzhliu and zhiics November 8, 2021 17:41

mbaret removed request for ZihengJiang, junrushao and merrymercy November 8, 2021 17:41

mbaret marked this pull request as draft November 8, 2021 17:42

mbaret force-pushed the ethosu-cascader-3 branch from b92a2a2 to 6e1b3d4Compare November 25, 2021 11:39

mbaret force-pushed the ethosu-cascader-3 branch from 6e1b3d4 to 1279da0Compare December 20, 2021 11:17

jacobbohlin mentioned this pull request Dec 20, 2021
[microNPU][2c] Add performance modelling to cascader #9778
Merged

mbaret force-pushed the ethosu-cascader-3 branch 2 times, most recently from aba0a8e to 950f280Compare December 22, 2021 10:54

mbaret force-pushed the ethosu-cascader-3 branch 3 times, most recently from 867ef3a to 7a0cfb9Compare January 5, 2022 16:00

mbaret marked this pull request as ready for review January 5, 2022 16:00

jacobbohlin reviewed Jan 5, 2022
View reviewed changes

Comment threadtests/python/contrib/test_ethosu/cascader/conftest.py Outdated
Comment threadtests/python/contrib/test_ethosu/cascader/conftest.py Outdated

mbaret added 7 commits January 6, 2022 10:03

Add test guards
9e547ac
Change-Id: I34e48e4ce4fd14e6c92601412f5029aa08efe8fc

Fix conv2d matcher test
9370675
Change-Id: Idcb92c26863837b06fa68f9122a1a79df79ee242

Fix dma linting
276aaf7
Change-Id: I0aad319b7722a56b5c7f116adcc791951223baa3

Add further test guards
2ada439
Change-Id: Ib1633341ed84e83c29c4220dd9bc4a7b6d17a5d0

Remove unnecessary test fixtures
2e6e512
Change-Id: I5223da462a8dadfda1798e1ea1c9b30b4e804217

Used alternative skip mechanism
a6f2857
Change-Id: Ib614c21e97723571aa002a2a4a3edda3afafb9d4

mbaret force-pushed the ethosu-cascader-3 branch from 7a0cfb9 to a6f2857Compare January 6, 2022 11:42

jacobbohlin approved these changes Jan 6, 2022
View reviewed changes

Mousius requested changes Jan 7, 2022
View reviewed changes

Mousius approved these changes Jan 7, 2022
View reviewed changes

Mousius merged commit afc29e6 into apache:mainJan 7, 2022

driazati mentioned this pull request Jul 14, 2022
TVM v0.9.0.rc0 Release Candidate Notes #12102
Closed