AMD Backend Support

[amd-asalykov]

adding support for HIP

An example:

python scripts/run_and_check.py ref_origin=local ref_arch_src_path=src/prompts/model_ex_add.py kernel_src_path=src/prompts/model_new_ex_add_hip.py eval_mode=local

[salykova]

Hi @simonguozirui, could you please take a look at the PR? Currently, we use os.environ["CXX"] = "hipcc" to inline-compile HIP kernels and rely on LLM to add this line of code (see model_new_ex_add_hip.py). Alternatively, we can prefix this LOC to LLM's output during the post-processing to guarantee hipcc is chosen as the default compiler for HIP backend

[simonguozirui]

Thanks, @amd-asalykov, for adding the AMD HIP support. Let me look through it with team @kesavanramakrishnan. We need to consider how to best add support for multiple hardware backends (in addition to the ongoing NVIDIA DSL backend effort). We will probably need to validate on an actual AMD machine as well.

[salykova]

Hi @simonguozirui, @kesavanramakrishnan, did you have a chance to look into it? Please let me know if you need access to an AMD machine to validate this, we'd be happy to provide one