NGA FY22: extend std::blas interface to KokkosKernels

[fnrizzi]

• Use the tpl-customization point interface to implement possible mappings to KokkosKernels
• Add missing kernels to KokkosKernels for BLAS 1/2/3
• (If time permits) Add tpl-customization point implementation for standard BLAS

[fnrizzi]

List below is used to track progress:

Rules for making PRs

• we first make PRs for the algorithms impl ONLY
• after we have all blas1 algorithms merged, we post a single PR to update the linalg_kokkoskernels include header and the Cmakelist under tests/kokkos-based
• we do the same for blas2 and blas3

This allows all the implementations to be merged independently without having conflicts popping up all the time

BLAS 1

• dot
  • has three overloads
  • customization point
  • KK impl
  • example code link
  • kokkos-test
• dotc
  • has three overloads
  • customization point
  • impl forwards to dot after conjugating
  • example code link
  • kokkos-test
• add
  • has three overloads
  • customization point
  • KK impl
  • example code link
  • kokkos-test
• scale (started by CTrott)
  • has three overloads
  • customization point
  • KK impl
  • example code link
  • kokkos-test
• idx_abs_max: (keep an eye on idx_abs_max: behavior differs from what k-kernels does #114)
  • has three overloads
  • customization point
  • KK impl
  • example code link
  • kokkos-test
• vector_abs_sum
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• vector_sum_of_squares
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• vector_norm2: init is inside sqrt as per spec, this is different from what the default impl does this issue.
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• givens_rotation_apply (need to sync with Luc about this since he started something similar alrady for KK)
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test

The following operate on matrices but are in BLAS 1, see related #107

• matrix_frob_norm
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• matrix_one_norm
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• matrix_inf_norm
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test

Defined in BLAS 1, buy accepting also rank-2

These functions are here because while they are defined in BLAS-1, they accept both rank-1 and rank-2 operands.
See related #106

• copy
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• swap_elements
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test

BLAS 2

product

• matrix_vector_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• symmetric_matrix_vector_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• hermitian_matrix_vector_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• triangular_matrix_vector_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test

solve and update

• triangular_matrix_vector_solve (in review: [ready] kokkos impl triangular_matrix_vector_solve #194)
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• matrix_rank_1_update
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• matrix_rank_1_update_c
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• symmetric_matrix_rank_1_update
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• hermitian_matrix_rank_1_update
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• symmetric_matrix_rank_2_update
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• hermitian_matrix_rank_2_update
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test

BLAS 3

• matrix_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
  • complete the updating overloads following the overwriting ones
• triangular_matrix_left_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• triangular_matrix_right_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• symmetric_matrix_left_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• symmetric_matrix_right_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• hermitian_matrix_left_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• hermitian_matrix_right_product
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• triangular_matrix_matrix_solve (only dispatches between left and right - has no own implementation)
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• triangular_matrix_matrix_left_solve
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• triangular_matrix_matrix_right_solve
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• symmetric_matrix_rank_2k_update
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• hermitian_matrix_rank_2k_update
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• symmetric_matrix_rank_k_update
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test
• hermitian_matrix_rank_k_update
  • has three overloads
  • customization point
  • KK impl
  • example code
  • kokkos-test

Notes

By "three overloads", we mean:

void fnc(/* no exec policy, other args */)
void fnc(/* generic ex pol, other args */)
void fnc(/* inline_exec_t, other args */)