XLS backend

[vasdommes]

Description

This PR adds XLS backend. It is based on PR #1343, with most of the code rewritten and new features added.

Google XLS is an open-source (Apache 2) High Level Syntesis toolchain that produces an RTL (Verilog or SystemVerilog) design from a high-level description (DSLX or C++).

Adding XLS as a new hls4ml backend allows to generate RTL without vendor-specific dependencies and benefit from the developments that XLS brings to HLS field.

XLS workflow

XLS backend performs the following transformations:

• write(): hls4ml representation -> DSLX project
• compile(): DSLX -> XLS IR -> Optimized XLS IR
• build(): Optimized XLS IR -> (System)Verilog-> IP (UPD: Vivado call removed, now we produce only Verilog.)

DSLX -> IR -> (System)Verilog conversion is done by XLS.
IP can generated from Verilog file manually e.g. via Vivado or another vendor.

XLS features

XLS backend supports the following layers: Input, ApplyAlpha, BatchNormalization, Dense, Conv1D, DepthwiseConv1D, Conv2D, DepthwiseConv2D, Pooling1D, Pooling2D, GlobalPooling1D, GlobalPooling2D, Merge, Concatenate, Dot, Activation, HardActivation, ParametrizedActivation, PReLU, Reshape, Softmax, Transpose, TernaryTanh.

You can override default codegen options as follows:

config=hls4ml.utils.config_from_keras_model(model)
# This sets hls_model.config['XLSCodegenFlags']hls_model=hls4ml.converters.convert_from_keras_model(
model, hls_config=config, backend='XLS',
xls_codegen_flags={'delay_model': 'asap7', 'generator': 'pipeline', 'use_system_verilog': False}
)

DSLX standard library has only signed FixedPoint type (similar to ap_fixed). Thus, unsigned types are not supported.

Currently, XLS backend implements only IOType: io_parallel. Strategy is ignored.
All operations are fully unrolled.

io_stream could be implemented via DSLX procs. @calad0i and I are going to work on that after finishing this PR.

Other changes

I made some minor changes in non-XLS code:

• Fixedtest_softmax.py does not test argmax and latency implementations; latency fails #1443, since it was needed to test all softmax implementations in XLS.
• Added hook to ModelGraph to call custom backend.get_top_function(). This is needed for XLS because it uses optimized XLS IR file instead of .so library generated by other backends.
• Updateddocs/ir/attributes.rst. Aside from adding XLS, this commit some other missing layers and attributes.

Dependencies

XLS backend uses xls-python to access XLS API. It is enabled by dependency group xls:

pip install hls4ml[xls]

xls-python comes with batteries (libxls.so and DSLX standard library) included, no separate XLS installation is required.
The code has been tested for the version xls-python=0.1.9875.

Known issues

XLS doesn't work with Dense layer imported from PyTorch Linear layer because of shape mismatch: PyTorch stores Linear weights as (out_features, in_features), while hls4ml Dense layers use the Keras-style layout (in_features, out_features).

Repro: add XLS backend to test_pytorch_api.py/test_squeeze and run the test.

Note that the weights in this test are constant, and other backends flatten them without checking shape.
So, it is unclear whether they handle this situation correctly or not.

Type of change

• Documentation update
• New feature (non-breaking change which adds functionality)

Tests

📝 Please describe the tests that you ran to verify your changes.

• Provide instructions so we can reproduce.
• Please also list any relevant details for your test configuration.

XLS has been added to the following tests:
test_activations.py, test_auto_precision.py, test_binary_cnn.py, test_causalpadding.py, test_depthconv1d.py, test_depthconv2d.py, test_keras_api.py, test_keras_v3_api.py, test_merge.py, test_multi_dense.py, test_pointwiseconv.py, test_pooling.py, test_pytorch_api.py, test_reshape.py, test_sepconv1d.py, test_sepconv2d.py, test_softmax.py.

Test Configuration

Add xls dependency, e.g.

pip install .[da,testing,testing-keras2,sr,optimization,xls]"
# or: pip install .[da,testing,testing-keras3,sr,xls]"

and run tests, e.g.:

pytest test/pytest --randomly-dont-reset-seed -k XLS

Notes on performance

Some test cases are very slow for XLS (e.g. ~30 minutes vs ~10 seconds on other backends).
This happens because XLS generates (in model.compile()) and uses (in model.predict()) an optimized XLS IR code, where all loops are fully unrolled. The resulting file can be huge and thus slow for the likes of Conv2D.

During development, I made test faster by reducing dimensions in some tests.
For example, in test_keras_api.py/test_conv2d I replaced

input_shape = (28, 28, 3)
filters=32

with

input_shape = (14, 14, 3)
filters=8

I haven't pushed such changes, but that could be one of the ways of speeding things up.

UPD: I reduced XLS dimensions in many tests, see 8578a62 and 75a04fb.

Checklist

• I have read the guidelines for contributing.
• I have commented my code, particularly in hard-to-understand areas.
• I have made corresponding changes to the documentation.
• My changes generate no new warnings.
• I have installed and run pre-commit on the files I edited or added.
• I have added tests that prove my fix is effective or that my feature works.

[calad0i]

Could we do something like f'{node.class_name}{node.index}' to save the whole case-match?

[vasdommes]

No, here we need qualified names for library functions, e.g. activations::softmax_stable or pooling::global_pooling_2d.

We could put all layer functions into one giant DSLX module (note that DSLX does not allow re-exports), but that's ugly. Also simply returning node.class_name and removing ValueError would lead to more obscure DSLX errors if layer is not supported.

[calad0i]

Asserting in variable definition is probably a bit misplaced, and we likely don't really need to assert these outside the corresponding layer specific handlers. Maybe remove?

[vasdommes]

Ideally yes, everything should be handled earlier. But if something goes wrong either with XLS or with layer handling, as it does in PyTorch Linear layer case:

hls4ml/test/pytest/test_pytorch_api.py

Lines 659 to 662 in 9fda567

	# TODO: this test fails for XLS due to PyTorch weights shape mismatch.
	@pytest.mark.parametrize('backend', ['Vivado', 'Vitis', 'Quartus', 'oneAPI'])
	@pytest.mark.parametrize('io_type', ['io_parallel', 'io_stream'])
	deftest_squeeze(test_case_id, backend, io_type):

then assertion error

AssertionError: Weights shape mismatch: expected (1, 5, 3), got (1, 3, 5)

is more readable than XLS error

layer_6_linear.x:50:5-50:27 TypeInferenceError: type mismatch: FixedPoint<16, -10>[5] vs. FixedPoint<16, -10>[LAYER6_OUT_DIM_1]. The body of function transform_2d does not actually return the function's declared return type, which is FixedPoint<16, -10>[LAYER6_OUT_DIM_1][LAYER6_OUT_DIM_0]

[calad0i]

If warp predict() directly, this part could be removed.

[vasdommes]

The problem is that _get_top_function is called inside _predict.

hls4ml/hls4ml/model/graph.py

Lines 881 to 882 in 2ec9ef3

	def_predict(self, x):
	top_function, ctype=self._get_top_function(x)

I see two options here:

1. Keep get_top_function hook.
2. Remove get_top_function hook, lift self._get_top_function(x) call from _predict() to predict(). Then every predict() implementation would need to make an extra self._get_top_function(x) call:

defpredict(self, x)
# <...>top_function, ctype=self._get_top_function(x)
returnself._predict(x, top_function, ctype)

Option 1 seems lees intrusive to me, although it adds three extra lines of code.

[vasdommes]

test_softmax failed for Quartus with implementation=latency:

FAILED test_softmax.py::test_softmax[16,6-input_shape4-9,6-io_parallel-False-latency-Quartus-standalone] - assert 0.5308 >= 0.98
FAILED test_softmax.py::test_softmax[16,6-input_shape5-9,6-io_stream-False-latency-Quartus-standalone] - assert 0.5216 >= 0.98
FAILED test_softmax.py::test_softmax[9,6-input_shape6-18,8-io_parallel-False-latency-Quartus-standalone] - assert 0.5636 >= 0.98
FAILED test_softmax.py::test_softmax[9,6-input_shape7-18,8-io_stream-False-latency-Quartus-standalone] - assert 0.5598 >= 0.98

This became visible because of my fixes for #1443 (enable latency and argmax in test_softmax), which are not directly related to XLS.
I can revert these changes to let CI tests pass for now. @jmitrevs made a similar fix, included in PRs #1476 and #1494.

P.S. Other CI failures seem to be due to timeout (probably unrelated to XLS, same tests work on my machine).