[DNNL] Add support of QNN primitives for DNNL runtime

[apeskov]

The main value of that change is enable qnn.conv2d and qnn.dense primitive for DNNL base json runtime.
Some of these changes is useful for all type of workloads, not only int8 specific.

Together with that there was performed some refactoring of internal infrastructure of DNNL plugin. The main int8 unrelated changes are:

• Improved thread safety. Now DNNL runtime can be used in multi instance mode.
• Zero copy input/output handling
• Scratchpad specification
• Use DNNL query api to define proper layouts (additional data copy is possible)
• Indirect addressing of memory objects. Internal tensor registry allow to clone temp/const tensor depending particular thread id.
• Relative positioning of input arguments. Index of each optional arguments specified via attributes
• Introduced ability of calculating constant subgraphs in DNNL code generator stage

[masahi]

Please update this PR.

[apeskov]

@masahi PR is updated. You may continue review.

[apeskov]

@tmoreau89@gfursin Just FYI.

[tmoreau89]

Thanks @apeskov - I'm also looping in @mbs-octoml on the review of this PR

[masahi]

Together with that there was performed some refactoring of internal infrastructure of DNNL plugin

If that's the case, I want this PR to be split into smaller ones. It's hard to review 3K PR with lots of moving pieces. Indeed 3K is too big for just introducing int8 support.

[masahi]

At least changes in dnnl_json_runtime.cc and dnnl_node_helper.h are relatively unrelated to int8. Please send them first.

Also we use CamelCase throughout the codebase, so please follow the convention.

[areusch]

@apeskov@masahi what's the status on this one? should we close?

[masahi]

Closing, feel free to reopen.