stream: keep the ONNX registry off the operator import path by atassis · Pull Request #210 · amd/IRON

atassis · 2026-09-18T16:01:22Z

iron/common/stream/__init__.py states the rule -- its submodules need onnx/pyyaml, "so importing
an operator must not pull them in" -- and requirements_stream.txt promises the stream operator's
test skips itself when stream-dse is absent. Neither holds.

iron/operators/__init__.py re-exports SwiGLUPrefillStream; swiglu_prefill_stream/op.py imports
ELTWISE_MUL, GEMM and SILU from iron.common.stream.ops; and that module imports onnx and
onnxscript at module scope. So a core install cannot import iron.operators at all -- a
ModuleNotFoundError at collection, which takes out every operator's tests, not just the one operator
the optional dependency belongs to.

Added

iron/common/stream/kernels.py, holding the StreamKernel definitions and operand layouts.
iron/tests/core_install.py, pinning the contract: a child interpreter with onnx/onnxscript hidden from sys.meta_path must still import iron.operators and iron.common.stream.kernels.

Changed

ops.py keeps only the registry it is named for -- the private-domain schemas, the translation table, and TORCH_OPS -- and imports the kernels it references.
swiglu_prefill_stream/op.py and iron/tests/stream/kernel_layouts.py take their names from the new module.

Removed

Splitting rather than deferring the import into the functions that use it is deliberate: a module that
does not import onnx cannot regain the dependency by accident.

Verified on 27cf75d: pytest iron/tests/core_install.py -> 20 passed; kernel_layouts.py skips as
designed without stream-dse; TORCH_OPS still resolves 3 ops and translation_table() 3 entries;
black --check clean.

Note on rebasing: #203 changed the moved block (the bfp16_mmul MAC-row split and _gemm_artifacts'
base_dir -> kernels_dir rename). kernels.py carries devel's current version of that block moved
verbatim, not the older copy this branch was first written against.

PR Merge Checklist

The PR is rebased on the latest devel commit and pointing to devel.
Your PR has been reviewed and approved.
All checks are passing.

iron/common/stream/__init__.py already states the rule -- its submodules need onnx/pyyaml, "so importing an operator must not pull them in" -- and requirements_stream.txt promises the stream operator's test skips itself when stream-dse is absent. Neither holds. iron/operators/__init__.py re-exports SwiGLUPrefillStream, swiglu_prefill_stream/op.py imports ELTWISE_MUL, GEMM and SILU from iron.common.stream.ops, and that module imports onnx and onnxscript at module scope for the exporter registry. So a core install cannot import iron.operators at all. The failure is ModuleNotFoundError at collection, which takes out every operator's test, not just the one operator the optional dependency belongs to. The three names op.py wants are StreamKernel instances -- a stream-dse key, a kernel source and its operand layouts. Nothing about them is ONNX. Move that half to iron/common/stream/kernels.py and leave ops.py the registry it is named for: the private-domain schemas, the translation table, and TORCH_OPS binding a torch ATen op to one of those kernels. ops.py imports the kernels it references, so the one-op-per-entry structure is unchanged; kernel_layouts.py takes its layouts from the new module. A module that does not import onnx cannot regain the dependency by accident, which is the point of splitting rather than deferring the import into the functions that use it. iron/tests/core_install.py pins the contract: a child interpreter with onnx and onnxscript hidden from sys.meta_path must still import iron.operators and iron.common.stream.kernels. It fails before this change and passes after. The ONNX path is unaffected -- TORCH_OPS still resolves three ops and translation_table three entries.

andrej · 2026-09-18T17:32:32Z

@asyms As a sanity check, does this make sense?

atassis · 2026-09-18T17:46:32Z

This work was done before #150 landed, which removed re-export. Some redesign also happened at that time, so right now this is mostly no-op.
The only thing left is a preference rather than a defect: ops.py still requires onnx from anyone who wants a StreamKernel, which is not an ONNX concept.

github-actions · 2026-09-18T18:03:55Z

CI Test Results

4c7bcd1 (2026_09_18_18_17_32)

IRON - CI Summary

Examples

iron/applications/llama_3.2_1b

Test	Krackan Status	Krackan	Phoenix Status	Phoenix
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1]	✅	-	-	-
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40]	✅	-	-	-
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1]	✅	-	-	-
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40]	✅	-	-	-

Small

iron/operators/axpy

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]	✅	192.82	✅	300.66
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]	✅	162.08	✅	345.24
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]	✅	194.72	✅	344.26
test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0]	✅	200.86	-	-

iron/operators/dequant

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]	✅	178.88	✅	415.42
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]	✅	177.54	✅	493.42
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]	✅	170.84	✅	423.76
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]	✅	174.50	✅	434.14
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]	✅	211.20	✅	484.90
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]	✅	184.46	✅	478.86
test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32]	✅	187.96	-	-
test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32]	✅	234.38	-	-

iron/operators/elementwise_add

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]	✅	178.70	✅	396.62
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]	✅	171.94	✅	436.94
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]	✅	182.56	✅	449.36
test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256]	✅	203.86	-	-

iron/operators/elementwise_mul

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]	✅	148.06	✅	445.44
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]	✅	172.00	✅	294.58
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]	✅	170.60	✅	366.80
test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256]	✅	188.68	-	-

iron/operators/flm/gemm

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_artifact_stem_differs_from_generic_gemm[M_256-K_512-N_1024]	✅	-	✅	-
test_artifact_stem_differs_from_generic_gemm[M_512-K_1024-N_2048]	✅	-	✅	-
test_gemm[M_256-K_512-N_1024-epilogue_gelu-clamp_None-rounding_conv_even]	✅	450.72	-	-
test_gemm[M_256-K_512-N_1024-epilogue_none-clamp_(-2.0, 2.0)-rounding_conv_even]	✅	362.00	-	-
test_gemm[M_256-K_512-N_1024-epilogue_none-clamp_None-rounding_conv_even]	✅	352.88	-	-
test_gemm[M_256-K_512-N_1024-epilogue_none-clamp_None-rounding_floor]	✅	311.58	-	-
test_gemm[M_256-K_512-N_1024-epilogue_silu-clamp_None-rounding_conv_even]	✅	429.44	-	-
test_gemm[M_256-K_512-N_128-epilogue_none-clamp_None-rounding_conv_even]	✅	364.58	✅	682.00
test_gemm[M_256-K_512-N_1536-epilogue_none-clamp_None-rounding_conv_even]	✅	377.58	-	-
test_gemm[M_256-K_512-N_256-epilogue_none-clamp_None-rounding_conv_even]	-	-	✅	783.08
test_gemm[M_256-K_512-N_320-epilogue_none-clamp_None-rounding_conv_even]	-	-	✅	642.36
test_gemm[M_256-K_512-N_512-epilogue_gelu-clamp_None-rounding_conv_even]	-	-	✅	1066.96
test_gemm[M_256-K_512-N_512-epilogue_none-clamp_(-2.0, 2.0)-rounding_conv_even]	-	-	✅	1219.80
test_gemm[M_256-K_512-N_512-epilogue_none-clamp_None-rounding_floor]	-	-	✅	721.78
test_gemm[M_256-K_512-N_512-epilogue_silu-clamp_None-rounding_conv_even]	-	-	✅	944.56
test_gemm[M_256-K_512-N_64-epilogue_none-clamp_None-rounding_conv_even]	-	-	✅	626.84
test_gemm[M_512-K_1024-N_2048-epilogue_none-clamp_None-rounding_conv_even]	✅	471.32	-	-
test_gemm[M_512-K_1024-N_512-epilogue_none-clamp_None-rounding_conv_even]	-	-	✅	1615.68
test_gemm_split_leg_bounds[iter0]	✅	-	✅	-
test_gemm_split_leg_bounds[iter1]	✅	-	✅	-
test_gemm_split_leg_bounds[iter2]	✅	-	✅	-
test_gemm_split_leg_bounds[iter3]	✅	-	✅	-
test_gemm_split_leg_bounds[iter4]	✅	-	✅	-
test_gemm_split_leg_bounds_runs[iter0]	✅	-	✅	-
test_gemm_split_leg_bounds_runs[iter1]	✅	-	✅	-
test_gemm_split_leg_bounds_runs[iter2]	✅	-	✅	-
test_gemm_split_leg_bounds_runs[iter3]	✅	-	✅	-
test_gemm_split_leg_bounds_runs[iter4]	✅	-	✅	-
test_gemm_tile_options[tn128-ma64-default]	✅	-	✅	-
test_gemm_tile_options[tn16-ma64-default]	✅	-	✅	-
test_gemm_tile_options[tn32-ma64-default]	✅	-	✅	-
test_gemm_tile_options[tn64-ma16-default]	-	-	✅	-
test_gemm_tile_options[tn64-ma32-default]	✅	-	-	-
test_one_xclbin_serves_every_clamp_bound[iter0]	✅	-	✅	-
test_one_xclbin_serves_every_clamp_bound[iter1]	✅	-	✅	-
test_one_xclbin_serves_every_clamp_bound[iter2]	✅	-	✅	-
test_one_xclbin_serves_every_clamp_bound[iter3]	✅	-	✅	-
test_one_xclbin_serves_every_clamp_bound[iter4]	✅	-	✅	-
test_one_xclbin_serves_every_shape[iter0]	✅	-	✅	-
test_one_xclbin_serves_every_shape[iter1]	✅	-	✅	-
test_one_xclbin_serves_every_shape[iter2]	✅	-	✅	-
test_one_xclbin_serves_every_shape[iter3]	✅	-	✅	-
test_one_xclbin_serves_every_shape[iter4]	✅	-	✅	-

iron/operators/gelu

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅	175.02	✅	418.82
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅	178.08	✅	359.94
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅	203.24	✅	426.46
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅	209.42	✅	466.06
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅	183.00	✅	427.42
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅	191.06	✅	429.16
test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅	223.38	-	-
test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]	✅	230.14	-	-

iron/operators/gemm

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1]	✅	2457.26	-	-
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]	✅	231.44	✅	494.10
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]	✅	229.20	✅	764.58
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]	✅	47811.12	✅	82161.08
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]	✅	27746.82	✅	22376.98
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1]	✅	7864.38	-	-
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]	✅	2284.08	✅	3854.68
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]	✅	3586.54	✅	5709.42
test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1]	✅	1625.82	-	-

iron/operators/gemv

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]	✅	0.22	✅	0.10
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]	✅	13.09	✅	3.57
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]	✅	24.59	✅	6.09
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]	✅	40.36	✅	9.09
test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256]	✅	41.87	-	-
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]	✅	12.75	✅	3.59
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]	✅	24.77	✅	6.44
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]	✅	39.94	✅	9.09
test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024]	✅	44.04	-	-
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]	✅	8.61	✅	2.32
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]	✅	0.88	✅	0.27
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]	✅	1.03	✅	0.45
test_gemv_batched[M_256-K_128-num_aie_columns_8-tile_size_input_1-tile_size_output_32-num_batches_100]	✅	16.38	-	-
test_gemv_batched[M_448-K_64-num_aie_columns_8-tile_size_input_1-tile_size_output_56-num_batches_192]	✅	12.46	-	-
test_gemv_batched[M_512-K_64-num_aie_columns_8-tile_size_input_4-tile_size_output_64-num_batches_32]	✅	7.19	-	-
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]	✅	5.16	✅	1.51
test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]	✅	0.21	❌	-
test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]	✅	12.63	❌	-
test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]	✅	12.54	❌	-

iron/operators/layer_norm

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅	169.54	✅	447.46
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅	173.72	✅	353.04
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅	179.86	✅	334.32
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅	189.00	✅	269.56
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅	173.32	✅	412.38
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅	207.38	✅	507.90
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅	170.10	-	-
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]	✅	210.64	-	-

iron/operators/leaky_relu

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]	✅	173.34	✅	370.92
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]	✅	162.36	✅	452.86
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]	✅	140.60	✅	351.72
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]	✅	180.06	✅	291.98
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]	✅	190.80	✅	307.12
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]	✅	194.58	✅	338.54
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]	✅	158.04	✅	375.16
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]	✅	181.02	✅	484.16
test_leaky_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-alpha_0.01]	✅	206.42	-	-
test_leaky_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-alpha_0.01]	✅	191.82	-	-

iron/operators/mem_copy

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]	✅	173.10	✅	374.02
test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128]	✅	223.76	-	-
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]	✅	156.34	✅	454.46
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]	✅	167.74	✅	374.64
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]	✅	190.34	✅	466.88
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]	✅	171.48	✅	381.56
test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256]	✅	205.18	-	-
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]	✅	193.62	✅	463.58

iron/operators/mha

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_arg_spec_matches_design_shapes[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]	✅	-	✅	-
test_arg_spec_matches_design_shapes[seq_len_16384-dim_64-num_heads_8-num_pipelines_8-num_kv_heads_2]	✅	-	✅	-
test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]	✅	52238.86	-	-

iron/operators/relu

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅	131.32	✅	334.88
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅	148.82	✅	528.74
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅	147.80	✅	331.14
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅	136.78	✅	719.52
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅	146.22	✅	426.68
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅	185.48	✅	431.64
test_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅	170.24	-	-
test_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]	✅	210.04	-	-

iron/operators/repeat

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_cols_without_a_legal_split_is_rejected[cols_1031-why_prime > 1023: the only divisors are 1 and cols, neither legal]	✅	-	✅	-
test_cols_without_a_legal_split_is_rejected[cols_2062-why_2 x 1031: the only word-aligned chunk leaves a 1031-wide chunk count]	✅	-	✅	-
test_cols_without_a_legal_split_is_rejected[cols_513-why_odd: every divisor is odd, so no chunk is a whole 32-bit word]	✅	-	✅	-
test_repeat[rows_4-cols_1024-repeat_2-transfer_size_None]	✅	172.18	✅	367.16
test_repeat[rows_8-cols_512-repeat_4-transfer_size_64]	✅	215.38	✅	364.88
test_repeat[rows_8-cols_64-repeat_4-transfer_size_None]	✅	201.26	✅	352.98

iron/operators/rms_norm

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]	✅	172.86	✅	396.34
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]	✅	189.18	✅	440.10
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]	✅	159.78	✅	644.84
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]	✅	172.70	✅	604.76
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]	✅	167.82	✅	533.08
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]	✅	207.32	✅	457.98
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]	✅	178.28	✅	770.68
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]	✅	166.36	✅	466.16
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]	✅	159.54	✅	488.88
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]	✅	171.58	✅	487.70
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]	✅	180.36	✅	433.12
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_True]	✅	204.62	-	-
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_False]	✅	177.96	-	-
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_True]	✅	224.90	-	-
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-weighted_False]	✅	216.24	-	-

iron/operators/rope

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]	✅	166.98	✅	375.82
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]	✅	149.54	✅	471.38
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]	✅	177.72	✅	382.72
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_8-method_type_0]	✅	202.92	-	-
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]	✅	195.08	✅	402.04
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]	✅	192.24	✅	427.44
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]	✅	159.98	✅	515.58
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_8-method_type_0]	✅	178.38	-	-

iron/operators/sigmoid

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅	183.12	✅	402.04
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅	174.86	✅	433.64
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅	200.88	✅	457.62
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅	160.84	✅	492.04
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅	159.68	✅	513.16
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅	173.60	✅	464.96
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅	191.18	-	-
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]	✅	221.00	-	-

iron/operators/silu

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅	146.44	✅	411.24
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅	136.60	✅	402.84
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅	148.74	✅	395.92
test_silu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅	156.20	-	-

iron/operators/softmax

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]	✅	177.18	✅	335.60
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]	✅	162.12	✅	444.30
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]	✅	161.34	✅	389.04

iron/operators/strided_copy

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_strided_copy[chunked_transfer]	✅	155.64	✅	328.26
test_strided_copy[contiguous]	✅	133.00	✅	371.48
test_strided_copy[four_channels]	✅	154.18	✅	333.38
test_strided_copy[kv_slot0]	✅	134.66	✅	478.70
test_strided_copy[kv_slot5]	✅	154.48	✅	361.74
test_strided_copy[kv_slot5_four_channels]	✅	150.54	✅	446.40
test_strided_copy[kv_slot5_two_channels]	✅	167.20	✅	337.34
test_strided_copy[kv_slot_last]	✅	140.42	✅	468.52
test_strided_copy[two_channels]	✅	162.52	✅	486.86
test_strided_copy[two_channels_chunked]	✅	144.48	✅	345.34
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter0]	✅	-	✅	-
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter1]	✅	-	✅	-
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter2]	✅	-	✅	-
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter3]	✅	-	✅	-
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter4]	✅	-	✅	-

iron/operators/swiglu_decode

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]	✅	953.52	✅	15338.84
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]	✅	1010.99	✅	13252.73

iron/operators/swiglu_prefill

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]	✅	2137.63	✅	24324.07

iron/operators/tanh

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅	178.28	✅	431.46
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅	177.64	✅	482.64
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅	181.58	✅	402.34
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅	183.24	✅	584.92
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅	208.54	✅	401.84
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅	207.48	✅	500.40
test_tanh[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅	226.78	-	-
test_tanh[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]	✅	219.92	-	-

iron/operators/transpose

Test	Krackan Status	Krackan Latency (mean)	Phoenix Status	Phoenix Latency (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]	✅	218.00	✅	568.34
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]	✅	253.52	✅	1051.96
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]	✅	224.32	✅	1005.52

Krackan - Small

IRON

Tested on 2026_09_18_18_17_32 at commit 4c7bcd1.

iron/operators/axpy

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]	✅ 5/5	192.82	0.06	n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]	✅ 5/5	162.08	0.08	n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]	✅ 5/5	194.72	0.06	n/a
test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0]	✅ 5/5	200.86	0.06	n/a

iron/operators/dequant

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]	✅ 5/5	178.88	0.03	n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]	✅ 5/5	177.54	0.03	n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]	✅ 5/5	170.84	0.03	n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]	✅ 5/5	174.50	0.03	n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]	✅ 5/5	211.20	0.03	n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]	✅ 5/5	184.46	0.03	n/a
test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32]	✅ 5/5	187.96	0.03	n/a
test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32]	✅ 5/5	234.38	0.02	n/a

iron/operators/elementwise_add

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]	✅ 5/5	178.70	0.07	n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]	✅ 5/5	171.94	0.07	n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]	✅ 5/5	182.56	0.07	n/a
test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256]	✅ 5/5	203.86	0.06	n/a

iron/operators/elementwise_mul

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]	✅ 5/5	148.06	0.08	n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]	✅ 5/5	172.00	0.08	n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]	✅ 5/5	170.60	0.08	n/a
test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256]	✅ 5/5	188.68	0.07	n/a

iron/operators/flm/gemm

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_artifact_stem_differs_from_generic_gemm[M_256-K_512-N_1024]	✅ 5/5	n/a	n/a	n/a
test_artifact_stem_differs_from_generic_gemm[M_512-K_1024-N_2048]	✅ 5/5	n/a	n/a	n/a
test_gemm[M_256-K_512-N_1024-epilogue_gelu-clamp_None-rounding_conv_even]	✅ 5/5	450.72	3.08	600.91
test_gemm[M_256-K_512-N_1024-epilogue_none-clamp_(-2.0, 2.0)-rounding_conv_even]	✅ 5/5	362.00	3.87	754.65
test_gemm[M_256-K_512-N_1024-epilogue_none-clamp_None-rounding_conv_even]	✅ 5/5	352.88	4.37	852.81
test_gemm[M_256-K_512-N_1024-epilogue_none-clamp_None-rounding_floor]	✅ 5/5	311.58	4.71	919.33
test_gemm[M_256-K_512-N_1024-epilogue_silu-clamp_None-rounding_conv_even]	✅ 5/5	429.44	3.22	628.08
test_gemm[M_256-K_512-N_128-epilogue_none-clamp_None-rounding_conv_even]	✅ 5/5	364.58	1.13	94.26
test_gemm[M_256-K_512-N_1536-epilogue_none-clamp_None-rounding_conv_even]	✅ 5/5	377.58	5.22	1087.45
test_gemm[M_512-K_1024-N_2048-epilogue_none-clamp_None-rounding_conv_even]	✅ 5/5	471.32	11.99	4675.36
test_gemm_split_leg_bounds[iter0]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds[iter1]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds[iter2]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds[iter3]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds[iter4]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds_runs[iter0]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds_runs[iter1]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds_runs[iter2]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds_runs[iter3]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds_runs[iter4]	✅ 1/1	n/a	n/a	n/a
test_gemm_tile_options[tn128-ma64-default]	✅ 5/5	n/a	n/a	n/a
test_gemm_tile_options[tn16-ma64-default]	✅ 5/5	n/a	n/a	n/a
test_gemm_tile_options[tn32-ma64-default]	✅ 5/5	n/a	n/a	n/a
test_gemm_tile_options[tn64-ma32-default]	✅ 5/5	n/a	n/a	n/a
test_one_xclbin_serves_every_clamp_bound[iter0]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_clamp_bound[iter1]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_clamp_bound[iter2]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_clamp_bound[iter3]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_clamp_bound[iter4]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_shape[iter0]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_shape[iter1]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_shape[iter2]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_shape[iter3]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_shape[iter4]	✅ 1/1	n/a	n/a	n/a

iron/operators/gelu

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	175.02	0.05	n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅ 5/5	178.08	0.05	n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	203.24	0.05	n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	209.42	0.04	n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	183.00	0.05	n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅ 5/5	191.06	0.04	n/a
test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅ 5/5	223.38	0.04	n/a
test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]	✅ 5/5	230.14	0.04	n/a

iron/operators/gemm

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1]	✅ 5/5	2457.26	3.85	1515.34
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]	✅ 5/5	231.44	1.00	42.57
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]	✅ 5/5	229.20	1.05	44.63
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]	✅ 5/5	47811.12	0.53	359.33
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]	✅ 5/5	27746.82	0.91	619.17
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1]	✅ 5/5	7864.38	3.20	2185.63
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]	✅ 5/5	2284.08	3.57	935.63
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]	✅ 5/5	3586.54	0.35	18.75
test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1]	✅ 5/5	1625.82	4.18	1290.81

iron/operators/gemv

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]	✅ 5/5	n/a	0.22	0.22
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]	✅ 5/5	n/a	13.09	13.08
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]	✅ 5/5	n/a	24.59	24.58
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]	✅ 5/5	n/a	40.36	40.33
test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256]	✅ 5/5	n/a	41.87	41.85
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]	✅ 5/5	n/a	12.75	12.74
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]	✅ 5/5	n/a	24.77	24.76
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]	✅ 5/5	n/a	39.94	39.92
test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024]	✅ 5/5	n/a	44.04	44.01
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]	✅ 5/5	n/a	8.61	8.59
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]	✅ 5/5	n/a	0.88	0.87
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]	✅ 5/5	n/a	1.03	1.02
test_gemv_batched[M_256-K_128-num_aie_columns_8-tile_size_input_1-tile_size_output_32-num_batches_100]	✅ 5/5	n/a	16.38	16.19
test_gemv_batched[M_448-K_64-num_aie_columns_8-tile_size_input_1-tile_size_output_56-num_batches_192]	✅ 5/5	n/a	12.46	12.24
test_gemv_batched[M_512-K_64-num_aie_columns_8-tile_size_input_4-tile_size_output_64-num_batches_32]	✅ 5/5	n/a	7.19	7.06
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]	✅ 5/5	n/a	5.16	5.07
test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]	✅ 5/5	n/a	0.21	0.20
test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]	✅ 5/5	n/a	12.63	12.62
test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]	✅ 5/5	n/a	12.54	12.53

iron/operators/layer_norm

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	169.54	0.05	n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅ 5/5	173.72	0.05	n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	179.86	0.05	n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	189.00	0.05	n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	173.32	0.05	n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅ 5/5	207.38	0.04	n/a
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅ 5/5	170.10	0.05	n/a
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]	✅ 5/5	210.64	0.04	n/a

iron/operators/leaky_relu

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]	✅ 5/5	173.34	0.05	n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]	✅ 5/5	162.36	0.05	n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]	✅ 5/5	140.60	0.06	n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]	✅ 5/5	180.06	0.05	n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]	✅ 5/5	190.80	0.04	n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]	✅ 5/5	194.58	0.04	n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]	✅ 5/5	158.04	0.05	n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]	✅ 5/5	181.02	0.05	n/a
test_leaky_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-alpha_0.01]	✅ 5/5	206.42	0.04	n/a
test_leaky_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-alpha_0.01]	✅ 5/5	191.82	0.04	n/a

iron/operators/mem_copy

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]	✅ 5/5	173.10	0.05	n/a
test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128]	✅ 5/5	223.76	0.04	n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]	✅ 5/5	156.34	0.05	n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]	✅ 5/5	167.74	0.05	n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]	✅ 5/5	190.34	0.05	n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]	✅ 5/5	171.48	0.05	n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256]	✅ 5/5	205.18	0.04	n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]	✅ 5/5	193.62	0.05	n/a

iron/operators/mha

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_arg_spec_matches_design_shapes[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]	✅ 5/5	n/a	n/a	n/a
test_arg_spec_matches_design_shapes[seq_len_16384-dim_64-num_heads_8-num_pipelines_8-num_kv_heads_2]	✅ 5/5	n/a	n/a	n/a
test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]	✅ 5/5	52238.86	0.16	n/a

iron/operators/relu

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	131.32	0.06	n/a
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅ 5/5	148.82	0.06	n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	147.80	0.06	n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	136.78	0.06	n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	146.22	0.06	n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅ 5/5	185.48	0.05	n/a
test_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅ 5/5	170.24	0.05	n/a
test_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]	✅ 5/5	210.04	0.04	n/a

iron/operators/repeat

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_cols_without_a_legal_split_is_rejected[cols_1031-why_prime > 1023: the only divisors are 1 and cols, neither legal]	✅ 5/5	n/a	n/a	n/a
test_cols_without_a_legal_split_is_rejected[cols_2062-why_2 x 1031: the only word-aligned chunk leaves a 1031-wide chunk count]	✅ 5/5	n/a	n/a	n/a
test_cols_without_a_legal_split_is_rejected[cols_513-why_odd: every divisor is odd, so no chunk is a whole 32-bit word]	✅ 5/5	n/a	n/a	n/a
test_repeat[rows_4-cols_1024-repeat_2-transfer_size_None]	✅ 5/5	172.18	0.15	n/a
test_repeat[rows_8-cols_512-repeat_4-transfer_size_64]	✅ 5/5	215.38	0.20	n/a
test_repeat[rows_8-cols_64-repeat_4-transfer_size_None]	✅ 5/5	201.26	0.03	n/a

iron/operators/rms_norm

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]	✅ 5/5	172.86	0.05	n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]	✅ 5/5	189.18	0.07	n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]	✅ 5/5	159.78	0.06	n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]	✅ 5/5	172.70	0.06	n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]	✅ 5/5	167.82	0.05	n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]	✅ 5/5	207.32	0.06	n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]	✅ 5/5	178.28	0.05	n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]	✅ 5/5	166.36	0.06	n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]	✅ 5/5	159.54	0.05	n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]	✅ 5/5	171.58	0.06	n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]	✅ 5/5	180.36	0.05	n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_True]	✅ 5/5	204.62	0.04	n/a
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_False]	✅ 5/5	177.96	0.05	n/a
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_True]	✅ 5/5	224.90	0.04	n/a
test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-weighted_False]	✅ 5/5	216.24	0.04	n/a

iron/operators/rope

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]	✅ 5/5	166.98	0.61	n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]	✅ 5/5	149.54	0.68	n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]	✅ 5/5	177.72	0.57	n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_8-method_type_0]	✅ 5/5	202.92	0.50	n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]	✅ 5/5	195.08	0.44	n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]	✅ 5/5	192.24	0.39	n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]	✅ 5/5	159.98	0.47	n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_8-method_type_0]	✅ 5/5	178.38	0.43	n/a

iron/operators/sigmoid

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	183.12	0.05	n/a
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅ 5/5	174.86	0.05	n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	200.88	0.04	n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	160.84	0.05	n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	159.68	0.05	n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅ 5/5	173.60	0.05	n/a
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅ 5/5	191.18	0.04	n/a
test_sigmoid[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]	✅ 5/5	221.00	0.04	n/a

iron/operators/silu

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	146.44	0.06	n/a
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	136.60	0.06	n/a
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	148.74	0.06	n/a
test_silu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅ 5/5	156.20	0.05	n/a

iron/operators/softmax

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]	✅ 5/5	177.18	0.74	n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]	✅ 5/5	162.12	0.85	n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	161.34	0.85	n/a

iron/operators/strided_copy

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_strided_copy[chunked_transfer]	✅ 5/5	155.64	0.03	n/a
test_strided_copy[contiguous]	✅ 5/5	133.00	0.03	n/a
test_strided_copy[four_channels]	✅ 5/5	154.18	0.03	n/a
test_strided_copy[kv_slot0]	✅ 5/5	134.66	0.99	n/a
test_strided_copy[kv_slot5]	✅ 5/5	154.48	0.88	n/a
test_strided_copy[kv_slot5_four_channels]	✅ 5/5	150.54	0.91	n/a
test_strided_copy[kv_slot5_two_channels]	✅ 5/5	167.20	0.80	n/a
test_strided_copy[kv_slot_last]	✅ 5/5	140.42	0.96	n/a
test_strided_copy[two_channels]	✅ 5/5	162.52	0.03	n/a
test_strided_copy[two_channels_chunked]	✅ 5/5	144.48	0.03	n/a
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter0]	✅ 1/1	n/a	n/a	n/a
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter1]	✅ 1/1	n/a	n/a	n/a
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter2]	✅ 1/1	n/a	n/a	n/a
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter3]	✅ 1/1	n/a	n/a	n/a
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter4]	✅ 1/1	n/a	n/a	n/a

iron/operators/swiglu_decode

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]	✅ 5/5	953.52	0.00	n/a
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]	✅ 5/5	1010.99	0.01	n/a

iron/operators/swiglu_prefill

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]	✅ 5/5	2137.63	0.98	n/a

iron/operators/tanh

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	178.28	0.05	n/a
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅ 5/5	177.64	0.05	n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	181.58	0.05	n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	183.24	0.05	n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	208.54	0.04	n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅ 5/5	207.48	0.04	n/a
test_tanh[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]	✅ 5/5	226.78	0.04	n/a
test_tanh[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]	✅ 5/5	219.92	0.04	n/a

iron/operators/transpose

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]	✅ 5/5	218.00	2.52	n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]	✅ 5/5	253.52	4.23	n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]	✅ 5/5	224.32	2.40	n/a

Krackan - Examples

IRON

Tested on 2026_09_18_18_39_15 at commit 4c7bcd1.

iron/applications/llama_3.2_1b

Test	Checks	TTFT (mean)	TPS (mean)
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1]	✅ 5/5	2.06	n/a
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40]	✅ 5/5	2.08	7.85
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1]	✅ 5/5	2.02	n/a
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40]	✅ 5/5	2.02	7.64

Phoenix - Small

IRON

Tested on 2026_09_18_18_11_27 at commit 4c7bcd1.

iron/operators/axpy

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]	✅ 5/5	300.66	0.04	n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]	✅ 5/5	345.24	0.04	n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]	✅ 5/5	344.26	0.04	n/a

iron/operators/dequant

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]	✅ 5/5	415.42	0.01	n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]	✅ 5/5	493.42	0.01	n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]	✅ 5/5	423.76	0.01	n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]	✅ 5/5	434.14	0.01	n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]	✅ 5/5	484.90	0.01	n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]	✅ 5/5	478.86	0.01	n/a

iron/operators/elementwise_add

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]	✅ 5/5	396.62	0.03	n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]	✅ 5/5	436.94	0.03	n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]	✅ 5/5	449.36	0.03	n/a

iron/operators/elementwise_mul

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]	✅ 5/5	445.44	0.03	n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]	✅ 5/5	294.58	0.04	n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]	✅ 5/5	366.80	0.03	n/a

iron/operators/flm/gemm

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_artifact_stem_differs_from_generic_gemm[M_256-K_512-N_1024]	✅ 5/5	n/a	n/a	n/a
test_artifact_stem_differs_from_generic_gemm[M_512-K_1024-N_2048]	✅ 5/5	n/a	n/a	n/a
test_gemm[M_256-K_512-N_128-epilogue_none-clamp_None-rounding_conv_even]	✅ 5/5	682.00	0.73	53.55
test_gemm[M_256-K_512-N_256-epilogue_none-clamp_None-rounding_conv_even]	✅ 5/5	783.08	0.85	86.65
test_gemm[M_256-K_512-N_320-epilogue_none-clamp_None-rounding_conv_even]	✅ 5/5	642.36	1.22	136.09
test_gemm[M_256-K_512-N_512-epilogue_gelu-clamp_None-rounding_conv_even]	✅ 5/5	1066.96	1.21	155.34
test_gemm[M_256-K_512-N_512-epilogue_none-clamp_(-2.0, 2.0)-rounding_conv_even]	✅ 5/5	1219.80	1.38	176.06
test_gemm[M_256-K_512-N_512-epilogue_none-clamp_None-rounding_floor]	✅ 5/5	721.78	1.53	195.27
test_gemm[M_256-K_512-N_512-epilogue_silu-clamp_None-rounding_conv_even]	✅ 5/5	944.56	1.29	164.96
test_gemm[M_256-K_512-N_64-epilogue_none-clamp_None-rounding_conv_even]	✅ 5/5	626.84	0.59	27.66
test_gemm[M_512-K_1024-N_512-epilogue_none-clamp_None-rounding_conv_even]	✅ 5/5	1615.68	2.12	433.99
test_gemm_split_leg_bounds[iter0]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds[iter1]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds[iter2]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds[iter3]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds[iter4]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds_runs[iter0]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds_runs[iter1]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds_runs[iter2]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds_runs[iter3]	✅ 1/1	n/a	n/a	n/a
test_gemm_split_leg_bounds_runs[iter4]	✅ 1/1	n/a	n/a	n/a
test_gemm_tile_options[tn128-ma64-default]	✅ 5/5	n/a	n/a	n/a
test_gemm_tile_options[tn16-ma64-default]	✅ 5/5	n/a	n/a	n/a
test_gemm_tile_options[tn32-ma64-default]	✅ 5/5	n/a	n/a	n/a
test_gemm_tile_options[tn64-ma16-default]	✅ 5/5	n/a	n/a	n/a
test_one_xclbin_serves_every_clamp_bound[iter0]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_clamp_bound[iter1]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_clamp_bound[iter2]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_clamp_bound[iter3]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_clamp_bound[iter4]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_shape[iter0]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_shape[iter1]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_shape[iter2]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_shape[iter3]	✅ 1/1	n/a	n/a	n/a
test_one_xclbin_serves_every_shape[iter4]	✅ 1/1	n/a	n/a	n/a

iron/operators/gelu

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	418.82	0.02	n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅ 5/5	359.94	0.02	n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	426.46	0.02	n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	466.06	0.02	n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	427.42	0.02	n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅ 5/5	429.16	0.02	n/a

iron/operators/gemm

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]	✅ 5/5	494.10	0.47	19.84
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]	✅ 5/5	764.58	0.40	16.97
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]	✅ 5/5	82161.08	0.31	209.12
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]	✅ 5/5	22376.98	1.13	768.71
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]	✅ 5/5	3854.68	2.77	726.46
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]	✅ 5/5	5709.42	0.23	12.20

iron/operators/gemv

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]	✅ 5/5	n/a	0.10	0.10
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]	✅ 5/5	n/a	3.57	3.57
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]	✅ 5/5	n/a	6.09	6.08
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]	✅ 5/5	n/a	9.09	9.09
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]	✅ 5/5	n/a	3.59	3.59
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]	✅ 5/5	n/a	6.44	6.44
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]	✅ 5/5	n/a	9.09	9.09
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]	✅ 5/5	n/a	2.32	2.31
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]	✅ 5/5	n/a	0.27	0.27
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]	✅ 5/5	n/a	0.45	0.45
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]	✅ 5/5	n/a	1.51	1.48
test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]	❌ 0/5	n/a	n/a	n/a
test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]	❌ 0/5	n/a	n/a	n/a
test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]	❌ 0/5	n/a	n/a	n/a

iron/operators/layer_norm

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	447.46	0.02	n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅ 5/5	353.04	0.03	n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	334.32	0.03	n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	269.56	0.03	n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	412.38	0.02	n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅ 5/5	507.90	0.02	n/a

iron/operators/leaky_relu

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]	✅ 5/5	370.92	0.02	n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]	✅ 5/5	452.86	0.02	n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]	✅ 5/5	351.72	0.03	n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]	✅ 5/5	291.98	0.03	n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]	✅ 5/5	307.12	0.03	n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]	✅ 5/5	338.54	0.03	n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]	✅ 5/5	375.16	0.02	n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]	✅ 5/5	484.16	0.02	n/a

iron/operators/mem_copy

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]	✅ 5/5	374.02	0.02	n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]	✅ 5/5	454.46	0.02	n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]	✅ 5/5	374.64	0.03	n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]	✅ 5/5	466.88	0.02	n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]	✅ 5/5	381.56	0.02	n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]	✅ 5/5	463.58	0.02	n/a

iron/operators/mha

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_arg_spec_matches_design_shapes[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]	✅ 5/5	n/a	n/a	n/a
test_arg_spec_matches_design_shapes[seq_len_16384-dim_64-num_heads_8-num_pipelines_8-num_kv_heads_2]	✅ 5/5	n/a	n/a	n/a

iron/operators/relu

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	334.88	0.03	n/a
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅ 5/5	528.74	0.02	n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	331.14	0.03	n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	719.52	0.02	n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	426.68	0.02	n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅ 5/5	431.64	0.02	n/a

iron/operators/repeat

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_cols_without_a_legal_split_is_rejected[cols_1031-why_prime > 1023: the only divisors are 1 and cols, neither legal]	✅ 5/5	n/a	n/a	n/a
test_cols_without_a_legal_split_is_rejected[cols_2062-why_2 x 1031: the only word-aligned chunk leaves a 1031-wide chunk count]	✅ 5/5	n/a	n/a	n/a
test_cols_without_a_legal_split_is_rejected[cols_513-why_odd: every divisor is odd, so no chunk is a whole 32-bit word]	✅ 5/5	n/a	n/a	n/a
test_repeat[rows_4-cols_1024-repeat_2-transfer_size_None]	✅ 5/5	367.16	0.07	n/a
test_repeat[rows_8-cols_512-repeat_4-transfer_size_64]	✅ 5/5	364.88	0.12	n/a
test_repeat[rows_8-cols_64-repeat_4-transfer_size_None]	✅ 5/5	352.98	0.02	n/a

iron/operators/rms_norm

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]	✅ 5/5	396.34	0.02	n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]	✅ 5/5	440.10	0.03	n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]	✅ 5/5	644.84	0.02	n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]	✅ 5/5	604.76	0.02	n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]	✅ 5/5	533.08	0.02	n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]	✅ 5/5	457.98	0.03	n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]	✅ 5/5	770.68	0.02	n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]	✅ 5/5	466.16	0.02	n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]	✅ 5/5	488.88	0.02	n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]	✅ 5/5	487.70	0.02	n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]	✅ 5/5	433.12	0.02	n/a

iron/operators/rope

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]	✅ 5/5	375.82	0.30	n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]	✅ 5/5	471.38	0.22	n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]	✅ 5/5	382.72	0.28	n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]	✅ 5/5	402.04	0.20	n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]	✅ 5/5	427.44	0.21	n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]	✅ 5/5	515.58	0.15	n/a

iron/operators/sigmoid

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	402.04	0.02	n/a
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅ 5/5	433.64	0.02	n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	457.62	0.02	n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	492.04	0.02	n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	513.16	0.02	n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅ 5/5	464.96	0.02	n/a

iron/operators/silu

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	411.24	0.02	n/a
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	402.84	0.02	n/a
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	395.92	0.02	n/a

iron/operators/softmax

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]	✅ 5/5	335.60	0.41	n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]	✅ 5/5	444.30	0.35	n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	389.04	0.35	n/a

iron/operators/strided_copy

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_strided_copy[chunked_transfer]	✅ 5/5	328.26	0.01	n/a
test_strided_copy[contiguous]	✅ 5/5	371.48	0.01	n/a
test_strided_copy[four_channels]	✅ 5/5	333.38	0.01	n/a
test_strided_copy[kv_slot0]	✅ 5/5	478.70	0.30	n/a
test_strided_copy[kv_slot5]	✅ 5/5	361.74	0.38	n/a
test_strided_copy[kv_slot5_four_channels]	✅ 5/5	446.40	0.32	n/a
test_strided_copy[kv_slot5_two_channels]	✅ 5/5	337.34	0.41	n/a
test_strided_copy[kv_slot_last]	✅ 5/5	468.52	0.30	n/a
test_strided_copy[two_channels]	✅ 5/5	486.86	0.01	n/a
test_strided_copy[two_channels_chunked]	✅ 5/5	345.34	0.01	n/a
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter0]	✅ 1/1	n/a	n/a	n/a
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter1]	✅ 1/1	n/a	n/a	n/a
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter2]	✅ 1/1	n/a	n/a	n/a
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter3]	✅ 1/1	n/a	n/a	n/a
test_transfer_size_not_dividing_per_channel_share_is_rejected[iter4]	✅ 1/1	n/a	n/a	n/a

iron/operators/swiglu_decode

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]	✅ 5/5	15338.84	0.00	n/a
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]	✅ 5/5	13252.73	0.00	n/a

iron/operators/swiglu_prefill

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]	✅ 5/5	24324.07	0.09	n/a

iron/operators/tanh

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]	✅ 5/5	431.46	0.02	n/a
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]	✅ 5/5	482.64	0.02	n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]	✅ 5/5	402.34	0.02	n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]	✅ 5/5	584.92	0.02	n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]	✅ 5/5	401.84	0.02	n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]	✅ 5/5	500.40	0.02	n/a

iron/operators/transpose

Test	Checks	Latency (mean)	Bandwidth (mean)	Throughput (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]	✅ 5/5	568.34	1.05	n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]	✅ 5/5	1051.96	1.92	n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]	✅ 5/5	1005.52	1.08	n/a

Phoenix - Examples

IRON

Tested on 2026_09_18_18_02_23 at commit 4c7bcd1.

Trend tables omitted, the comment hit GitHub's size limit. Full report in the workflow run.

atassis requested review from andrej, hunhoffe and jgmelber as code owners September 18, 2026 16:01

Navigation Menu

stream: keep the ONNX registry off the operator import path - #210

stream: keep the ONNX registry off the operator import path#210
atassis wants to merge 1 commit into
amd:develfrom
atassis:pr/stream-ops-import-without-onnx

atassis commented Sep 18, 2026 •

edited

Loading

Uh oh!

andrej commented Sep 18, 2026

Uh oh!

atassis commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026 •

edited

Loading

IRON

IRON

IRON

IRON

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

atassis commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Added

Changed

Removed

PR Merge Checklist

Uh oh!

andrej commented Sep 18, 2026

Uh oh!

atassis commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI Test Results

IRON - CI Summary

Examples

Small

IRON

IRON

IRON

IRON

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

atassis commented Sep 18, 2026 •

edited

Loading

github-actions Bot commented Sep 18, 2026 •

edited

Loading