GPU Support

[kaushikcfd]

Draft because:

• Clean commit history

[kaushikcfd]

cc @inducer

[miklos1]

Maybe IntFlag instead of IntEnum? (just something to perhaps consider...)

[wence-]

Didn't manage everything, but a bunch of comments/queries.

[wence-]

Completely impenetrable name.

[kaushikcfd]

Renamed it to split_n_across_workgroups, better?

[wence-]

Again, comments about vec state management apply here, and mutatis mutandis, the rest of the PR.

[wence-]

This cached data is generated by a computer. Therefore there should be no need to parse and evaluate it, because the on-disk representation can just be the in-memory representation.

[kaushikcfd]

This cached data is generated by a computer

I preferred a human-readable format to keep the representation valid across pyop2-versions which might not be the case with pickling. (Happy to serialize in a different format if there's a better alternative)

no need to parse and evaluate it,

I don't think we can avoid the evaluates because the grid sizes are typically symbolic expressions of the form: (end - start) // 32 + 1. Therefore the grid sizes are evaluated at runtime during a GlobalKernel.__call__ invocation.

[wence-]

Pickle version incompatibilities are fine (they would appear when you upgrade python and we don't guarantee very much about the portability of on disk cached data between firedrake-update calls, let alone python version upgrades).

What you're attempting to do is cache a function that computes these things. So in the code_to_compile case, just write that function, and pickle it.

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

[kaushikcfd]

Pickle version incompatibilities are fin

Cool, will change. (Thanks!)

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

Yes, the compiled code stores only the plain CUDA/OpenCL device kernel. Loopy's cache on disk mechanism works by storing both the device kernel and the host code that would call end up calling this device kernel. But adapting PyOP2 to call loopy kernels would be a bit disruptive and I think loopy's cache dir being $XDG_CACHE_DIR has run into some problems in the past.

[kaushikcfd]

just write that function,

Planning to simply generate a python function corresponding to the expression: inducer/pymbolic#107

[inducer]

For my education, how come you chose to not go with the generated invokers?

[kaushikcfd]

For PyOpenCL we could borrow the invoker, but for PyCUDA/C-target we don't have an invoker in loopy (yet).

[kaushikcfd]

• Also, for the pyopencl invoker, a minor concern is that the disk-caching semantics don't play well with PyOP2's disk-caching semantics.
• I feel adding an invoker for the PyCUDA target shouldn't be too difficult. Taking a look at it.

[inducer]

There is invoker generation for the C target. I'd expect that putting together CUDA invokers wouldn't be a massive effort based on a copy-paste job of the CL ones. Do you think that might be the right approach?

[kaushikcfd]

I am working on integrating the PyCUDA/PyOpenCL invokers. For the C-target, reorganizing pyop2/compilation.py is a bigger undertaking and also out of the scope for this PR.

[wence-]

This is data to be consumed by a computer, so don't make it human-readable and require parsing.

[kaushikcfd]

Do you recommend pickling?

[wence-]

This is tremendously fragile. Please think of a better way of saving this data in a way that doesn't rely on dubiously-stable properties of numpy.savez/load.

[kaushikcfd]

Agreed. While np.savez-ing the arrays also stored a string-array that tells us the keys that were used for storing these arrays. See https://github.com/OP2/PyOP2/compare/9b7eb97878f2b6609d64affe896c5d4d901a2934..0e58f08d3ea1487e01c45df2270fa9e5790a845d.

[connorjward]

@kaushikcfd just to make you aware of a couple things:

• We are currently in the process of ingesting PyOP2 (and TSFC) into the Firedrake repository, so this repository will shortly be archived.
• We are hoping to soon drop PyOP2 entirely in favour of pyop3. We then, assuming funding, want to add GPU support there. We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

[kaushikcfd]

Hi Connor!
Sounds good.

We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

Let me know if anything in this patch isn't clear.

[connorjward]

@kaushikcfd I just read the GPU chapter of your thesis (and liked it!). Looking through the code here I can’t find the second transform that you describe nor the various heuristics. Are those available anywhere?

[kaushikcfd]

Are those available anywhere?

You can find them here: https://github.com/OP2/PyOP2/tree/auto_tiling.

[connorjward]

Ah great. Thanks!

[kaushikcfd]

Draft because:

• Clean commit history

[kaushikcfd]

cc @inducer

[miklos1]

Maybe IntFlag instead of IntEnum? (just something to perhaps consider...)

[wence-]

Didn't manage everything, but a bunch of comments/queries.

[wence-]

Completely impenetrable name.

[kaushikcfd]

Renamed it to split_n_across_workgroups, better?

[wence-]

Again, comments about vec state management apply here, and mutatis mutandis, the rest of the PR.

[wence-]

This cached data is generated by a computer. Therefore there should be no need to parse and evaluate it, because the on-disk representation can just be the in-memory representation.

[kaushikcfd]

This cached data is generated by a computer

I preferred a human-readable format to keep the representation valid across pyop2-versions which might not be the case with pickling. (Happy to serialize in a different format if there's a better alternative)

no need to parse and evaluate it,

I don't think we can avoid the evaluates because the grid sizes are typically symbolic expressions of the form: (end - start) // 32 + 1. Therefore the grid sizes are evaluated at runtime during a GlobalKernel.__call__ invocation.

[wence-]

Pickle version incompatibilities are fine (they would appear when you upgrade python and we don't guarantee very much about the portability of on disk cached data between firedrake-update calls, let alone python version upgrades).

What you're attempting to do is cache a function that computes these things. So in the code_to_compile case, just write that function, and pickle it.

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

[kaushikcfd]

Pickle version incompatibilities are fin

Cool, will change. (Thanks!)

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

Yes, the compiled code stores only the plain CUDA/OpenCL device kernel. Loopy's cache on disk mechanism works by storing both the device kernel and the host code that would call end up calling this device kernel. But adapting PyOP2 to call loopy kernels would be a bit disruptive and I think loopy's cache dir being $XDG_CACHE_DIR has run into some problems in the past.

[kaushikcfd]

just write that function,

Planning to simply generate a python function corresponding to the expression: inducer/pymbolic#107

[inducer]

For my education, how come you chose to not go with the generated invokers?

[kaushikcfd]

For PyOpenCL we could borrow the invoker, but for PyCUDA/C-target we don't have an invoker in loopy (yet).

[kaushikcfd]

• Also, for the pyopencl invoker, a minor concern is that the disk-caching semantics don't play well with PyOP2's disk-caching semantics.
• I feel adding an invoker for the PyCUDA target shouldn't be too difficult. Taking a look at it.

[inducer]

There is invoker generation for the C target. I'd expect that putting together CUDA invokers wouldn't be a massive effort based on a copy-paste job of the CL ones. Do you think that might be the right approach?

[kaushikcfd]

I am working on integrating the PyCUDA/PyOpenCL invokers. For the C-target, reorganizing pyop2/compilation.py is a bigger undertaking and also out of the scope for this PR.

[wence-]

This is data to be consumed by a computer, so don't make it human-readable and require parsing.

[kaushikcfd]

Do you recommend pickling?

[wence-]

This is tremendously fragile. Please think of a better way of saving this data in a way that doesn't rely on dubiously-stable properties of numpy.savez/load.

[kaushikcfd]

Agreed. While np.savez-ing the arrays also stored a string-array that tells us the keys that were used for storing these arrays. See https://github.com/OP2/PyOP2/compare/9b7eb97878f2b6609d64affe896c5d4d901a2934..0e58f08d3ea1487e01c45df2270fa9e5790a845d.

[connorjward]

@kaushikcfd just to make you aware of a couple things:

• We are currently in the process of ingesting PyOP2 (and TSFC) into the Firedrake repository, so this repository will shortly be archived.
• We are hoping to soon drop PyOP2 entirely in favour of pyop3. We then, assuming funding, want to add GPU support there. We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

[kaushikcfd]

Hi Connor!
Sounds good.

We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

Let me know if anything in this patch isn't clear.

[connorjward]

@kaushikcfd I just read the GPU chapter of your thesis (and liked it!). Looking through the code here I can’t find the second transform that you describe nor the various heuristics. Are those available anywhere?

[kaushikcfd]

Are those available anywhere?

You can find them here: https://github.com/OP2/PyOP2/tree/auto_tiling.

[connorjward]

Ah great. Thanks!

[kaushikcfd]

Draft because:

• Clean commit history

[kaushikcfd]

cc @inducer

[miklos1]

Maybe IntFlag instead of IntEnum? (just something to perhaps consider...)

[wence-]

Didn't manage everything, but a bunch of comments/queries.

[wence-]

Completely impenetrable name.

[kaushikcfd]

Renamed it to split_n_across_workgroups, better?

[wence-]

Again, comments about vec state management apply here, and mutatis mutandis, the rest of the PR.

[wence-]

This cached data is generated by a computer. Therefore there should be no need to parse and evaluate it, because the on-disk representation can just be the in-memory representation.

[kaushikcfd]

This cached data is generated by a computer

I preferred a human-readable format to keep the representation valid across pyop2-versions which might not be the case with pickling. (Happy to serialize in a different format if there's a better alternative)

no need to parse and evaluate it,

I don't think we can avoid the evaluates because the grid sizes are typically symbolic expressions of the form: (end - start) // 32 + 1. Therefore the grid sizes are evaluated at runtime during a GlobalKernel.__call__ invocation.

[wence-]

Pickle version incompatibilities are fine (they would appear when you upgrade python and we don't guarantee very much about the portability of on disk cached data between firedrake-update calls, let alone python version upgrades).

What you're attempting to do is cache a function that computes these things. So in the code_to_compile case, just write that function, and pickle it.

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

[kaushikcfd]

Pickle version incompatibilities are fin

Cool, will change. (Thanks!)

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

Yes, the compiled code stores only the plain CUDA/OpenCL device kernel. Loopy's cache on disk mechanism works by storing both the device kernel and the host code that would call end up calling this device kernel. But adapting PyOP2 to call loopy kernels would be a bit disruptive and I think loopy's cache dir being $XDG_CACHE_DIR has run into some problems in the past.

[kaushikcfd]

just write that function,

Planning to simply generate a python function corresponding to the expression: inducer/pymbolic#107

[inducer]

For my education, how come you chose to not go with the generated invokers?

[kaushikcfd]

For PyOpenCL we could borrow the invoker, but for PyCUDA/C-target we don't have an invoker in loopy (yet).

[kaushikcfd]

• Also, for the pyopencl invoker, a minor concern is that the disk-caching semantics don't play well with PyOP2's disk-caching semantics.
• I feel adding an invoker for the PyCUDA target shouldn't be too difficult. Taking a look at it.

[inducer]

There is invoker generation for the C target. I'd expect that putting together CUDA invokers wouldn't be a massive effort based on a copy-paste job of the CL ones. Do you think that might be the right approach?

[kaushikcfd]

I am working on integrating the PyCUDA/PyOpenCL invokers. For the C-target, reorganizing pyop2/compilation.py is a bigger undertaking and also out of the scope for this PR.

[wence-]

This is data to be consumed by a computer, so don't make it human-readable and require parsing.

[kaushikcfd]

Do you recommend pickling?

[wence-]

This is tremendously fragile. Please think of a better way of saving this data in a way that doesn't rely on dubiously-stable properties of numpy.savez/load.

[kaushikcfd]

Agreed. While np.savez-ing the arrays also stored a string-array that tells us the keys that were used for storing these arrays. See https://github.com/OP2/PyOP2/compare/9b7eb97878f2b6609d64affe896c5d4d901a2934..0e58f08d3ea1487e01c45df2270fa9e5790a845d.

[connorjward]

@kaushikcfd just to make you aware of a couple things:

• We are currently in the process of ingesting PyOP2 (and TSFC) into the Firedrake repository, so this repository will shortly be archived.
• We are hoping to soon drop PyOP2 entirely in favour of pyop3. We then, assuming funding, want to add GPU support there. We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

[kaushikcfd]

Hi Connor!
Sounds good.

We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

Let me know if anything in this patch isn't clear.

[connorjward]

@kaushikcfd I just read the GPU chapter of your thesis (and liked it!). Looking through the code here I can’t find the second transform that you describe nor the various heuristics. Are those available anywhere?

[kaushikcfd]

Are those available anywhere?

You can find them here: https://github.com/OP2/PyOP2/tree/auto_tiling.

[connorjward]

Ah great. Thanks!

[kaushikcfd]

Draft because:

• Clean commit history

[kaushikcfd]

cc @inducer

[miklos1]

Maybe IntFlag instead of IntEnum? (just something to perhaps consider...)

[wence-]

Didn't manage everything, but a bunch of comments/queries.

[wence-]

Completely impenetrable name.

[kaushikcfd]

Renamed it to split_n_across_workgroups, better?

[wence-]

Again, comments about vec state management apply here, and mutatis mutandis, the rest of the PR.

[wence-]

This cached data is generated by a computer. Therefore there should be no need to parse and evaluate it, because the on-disk representation can just be the in-memory representation.

[kaushikcfd]

This cached data is generated by a computer

I preferred a human-readable format to keep the representation valid across pyop2-versions which might not be the case with pickling. (Happy to serialize in a different format if there's a better alternative)

no need to parse and evaluate it,

I don't think we can avoid the evaluates because the grid sizes are typically symbolic expressions of the form: (end - start) // 32 + 1. Therefore the grid sizes are evaluated at runtime during a GlobalKernel.__call__ invocation.

[wence-]

Pickle version incompatibilities are fine (they would appear when you upgrade python and we don't guarantee very much about the portability of on disk cached data between firedrake-update calls, let alone python version upgrades).

What you're attempting to do is cache a function that computes these things. So in the code_to_compile case, just write that function, and pickle it.

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

[kaushikcfd]

Pickle version incompatibilities are fin

Cool, will change. (Thanks!)

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

Yes, the compiled code stores only the plain CUDA/OpenCL device kernel. Loopy's cache on disk mechanism works by storing both the device kernel and the host code that would call end up calling this device kernel. But adapting PyOP2 to call loopy kernels would be a bit disruptive and I think loopy's cache dir being $XDG_CACHE_DIR has run into some problems in the past.

[kaushikcfd]

just write that function,

Planning to simply generate a python function corresponding to the expression: inducer/pymbolic#107

[inducer]

For my education, how come you chose to not go with the generated invokers?

[kaushikcfd]

For PyOpenCL we could borrow the invoker, but for PyCUDA/C-target we don't have an invoker in loopy (yet).

[kaushikcfd]

• Also, for the pyopencl invoker, a minor concern is that the disk-caching semantics don't play well with PyOP2's disk-caching semantics.
• I feel adding an invoker for the PyCUDA target shouldn't be too difficult. Taking a look at it.

[inducer]

There is invoker generation for the C target. I'd expect that putting together CUDA invokers wouldn't be a massive effort based on a copy-paste job of the CL ones. Do you think that might be the right approach?

[kaushikcfd]

I am working on integrating the PyCUDA/PyOpenCL invokers. For the C-target, reorganizing pyop2/compilation.py is a bigger undertaking and also out of the scope for this PR.

[wence-]

This is data to be consumed by a computer, so don't make it human-readable and require parsing.

[kaushikcfd]

Do you recommend pickling?

[wence-]

This is tremendously fragile. Please think of a better way of saving this data in a way that doesn't rely on dubiously-stable properties of numpy.savez/load.

[kaushikcfd]

Agreed. While np.savez-ing the arrays also stored a string-array that tells us the keys that were used for storing these arrays. See https://github.com/OP2/PyOP2/compare/9b7eb97878f2b6609d64affe896c5d4d901a2934..0e58f08d3ea1487e01c45df2270fa9e5790a845d.

[connorjward]

@kaushikcfd just to make you aware of a couple things:

• We are currently in the process of ingesting PyOP2 (and TSFC) into the Firedrake repository, so this repository will shortly be archived.
• We are hoping to soon drop PyOP2 entirely in favour of pyop3. We then, assuming funding, want to add GPU support there. We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

[kaushikcfd]

Hi Connor!
Sounds good.

We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

Let me know if anything in this patch isn't clear.

[connorjward]

@kaushikcfd I just read the GPU chapter of your thesis (and liked it!). Looking through the code here I can’t find the second transform that you describe nor the various heuristics. Are those available anywhere?

[kaushikcfd]

Are those available anywhere?

You can find them here: https://github.com/OP2/PyOP2/tree/auto_tiling.

[connorjward]

Ah great. Thanks!

[kaushikcfd]

Draft because:

• Clean commit history

[kaushikcfd]

cc @inducer

[miklos1]

Maybe IntFlag instead of IntEnum? (just something to perhaps consider...)

[wence-]

Didn't manage everything, but a bunch of comments/queries.

[wence-]

Completely impenetrable name.

[kaushikcfd]

Renamed it to split_n_across_workgroups, better?

[wence-]

Again, comments about vec state management apply here, and mutatis mutandis, the rest of the PR.

[wence-]

This cached data is generated by a computer. Therefore there should be no need to parse and evaluate it, because the on-disk representation can just be the in-memory representation.

[kaushikcfd]

This cached data is generated by a computer

I preferred a human-readable format to keep the representation valid across pyop2-versions which might not be the case with pickling. (Happy to serialize in a different format if there's a better alternative)

no need to parse and evaluate it,

I don't think we can avoid the evaluates because the grid sizes are typically symbolic expressions of the form: (end - start) // 32 + 1. Therefore the grid sizes are evaluated at runtime during a GlobalKernel.__call__ invocation.

[wence-]

Pickle version incompatibilities are fine (they would appear when you upgrade python and we don't guarantee very much about the portability of on disk cached data between firedrake-update calls, let alone python version upgrades).

What you're attempting to do is cache a function that computes these things. So in the code_to_compile case, just write that function, and pickle it.

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

[kaushikcfd]

Pickle version incompatibilities are fin

Cool, will change. (Thanks!)

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

Yes, the compiled code stores only the plain CUDA/OpenCL device kernel. Loopy's cache on disk mechanism works by storing both the device kernel and the host code that would call end up calling this device kernel. But adapting PyOP2 to call loopy kernels would be a bit disruptive and I think loopy's cache dir being $XDG_CACHE_DIR has run into some problems in the past.

[kaushikcfd]

just write that function,

Planning to simply generate a python function corresponding to the expression: inducer/pymbolic#107

[inducer]

For my education, how come you chose to not go with the generated invokers?

[kaushikcfd]

For PyOpenCL we could borrow the invoker, but for PyCUDA/C-target we don't have an invoker in loopy (yet).

[kaushikcfd]

• Also, for the pyopencl invoker, a minor concern is that the disk-caching semantics don't play well with PyOP2's disk-caching semantics.
• I feel adding an invoker for the PyCUDA target shouldn't be too difficult. Taking a look at it.

[inducer]

There is invoker generation for the C target. I'd expect that putting together CUDA invokers wouldn't be a massive effort based on a copy-paste job of the CL ones. Do you think that might be the right approach?

[kaushikcfd]

I am working on integrating the PyCUDA/PyOpenCL invokers. For the C-target, reorganizing pyop2/compilation.py is a bigger undertaking and also out of the scope for this PR.

[wence-]

This is data to be consumed by a computer, so don't make it human-readable and require parsing.

[kaushikcfd]

Do you recommend pickling?

[wence-]

This is tremendously fragile. Please think of a better way of saving this data in a way that doesn't rely on dubiously-stable properties of numpy.savez/load.

[kaushikcfd]

Agreed. While np.savez-ing the arrays also stored a string-array that tells us the keys that were used for storing these arrays. See https://github.com/OP2/PyOP2/compare/9b7eb97878f2b6609d64affe896c5d4d901a2934..0e58f08d3ea1487e01c45df2270fa9e5790a845d.

[connorjward]

@kaushikcfd just to make you aware of a couple things:

• We are currently in the process of ingesting PyOP2 (and TSFC) into the Firedrake repository, so this repository will shortly be archived.
• We are hoping to soon drop PyOP2 entirely in favour of pyop3. We then, assuming funding, want to add GPU support there. We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

[kaushikcfd]

Hi Connor!
Sounds good.

We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

Let me know if anything in this patch isn't clear.

[connorjward]

@kaushikcfd I just read the GPU chapter of your thesis (and liked it!). Looking through the code here I can’t find the second transform that you describe nor the various heuristics. Are those available anywhere?

[kaushikcfd]

Are those available anywhere?

You can find them here: https://github.com/OP2/PyOP2/tree/auto_tiling.

[connorjward]

Ah great. Thanks!

[kaushikcfd]

Draft because:

• Clean commit history

[kaushikcfd]

cc @inducer

[miklos1]

Maybe IntFlag instead of IntEnum? (just something to perhaps consider...)

[wence-]

Didn't manage everything, but a bunch of comments/queries.

[wence-]

Completely impenetrable name.

[kaushikcfd]

Renamed it to split_n_across_workgroups, better?

[wence-]

Again, comments about vec state management apply here, and mutatis mutandis, the rest of the PR.

[wence-]

This cached data is generated by a computer. Therefore there should be no need to parse and evaluate it, because the on-disk representation can just be the in-memory representation.

[kaushikcfd]

This cached data is generated by a computer

I preferred a human-readable format to keep the representation valid across pyop2-versions which might not be the case with pickling. (Happy to serialize in a different format if there's a better alternative)

no need to parse and evaluate it,

I don't think we can avoid the evaluates because the grid sizes are typically symbolic expressions of the form: (end - start) // 32 + 1. Therefore the grid sizes are evaluated at runtime during a GlobalKernel.__call__ invocation.

[wence-]

Pickle version incompatibilities are fine (they would appear when you upgrade python and we don't guarantee very much about the portability of on disk cached data between firedrake-update calls, let alone python version upgrades).

What you're attempting to do is cache a function that computes these things. So in the code_to_compile case, just write that function, and pickle it.

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

[kaushikcfd]

Pickle version incompatibilities are fin

Cool, will change. (Thanks!)

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

Yes, the compiled code stores only the plain CUDA/OpenCL device kernel. Loopy's cache on disk mechanism works by storing both the device kernel and the host code that would call end up calling this device kernel. But adapting PyOP2 to call loopy kernels would be a bit disruptive and I think loopy's cache dir being $XDG_CACHE_DIR has run into some problems in the past.

[kaushikcfd]

just write that function,

Planning to simply generate a python function corresponding to the expression: inducer/pymbolic#107

[inducer]

For my education, how come you chose to not go with the generated invokers?

[kaushikcfd]

For PyOpenCL we could borrow the invoker, but for PyCUDA/C-target we don't have an invoker in loopy (yet).

[kaushikcfd]

• Also, for the pyopencl invoker, a minor concern is that the disk-caching semantics don't play well with PyOP2's disk-caching semantics.
• I feel adding an invoker for the PyCUDA target shouldn't be too difficult. Taking a look at it.

[inducer]

There is invoker generation for the C target. I'd expect that putting together CUDA invokers wouldn't be a massive effort based on a copy-paste job of the CL ones. Do you think that might be the right approach?

[kaushikcfd]

I am working on integrating the PyCUDA/PyOpenCL invokers. For the C-target, reorganizing pyop2/compilation.py is a bigger undertaking and also out of the scope for this PR.

[wence-]

This is data to be consumed by a computer, so don't make it human-readable and require parsing.

[kaushikcfd]

Do you recommend pickling?

[wence-]

This is tremendously fragile. Please think of a better way of saving this data in a way that doesn't rely on dubiously-stable properties of numpy.savez/load.

[kaushikcfd]

Agreed. While np.savez-ing the arrays also stored a string-array that tells us the keys that were used for storing these arrays. See https://github.com/OP2/PyOP2/compare/9b7eb97878f2b6609d64affe896c5d4d901a2934..0e58f08d3ea1487e01c45df2270fa9e5790a845d.

[connorjward]

@kaushikcfd just to make you aware of a couple things:

• We are currently in the process of ingesting PyOP2 (and TSFC) into the Firedrake repository, so this repository will shortly be archived.
• We are hoping to soon drop PyOP2 entirely in favour of pyop3. We then, assuming funding, want to add GPU support there. We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

[kaushikcfd]

Hi Connor!
Sounds good.

We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

Let me know if anything in this patch isn't clear.

[connorjward]

@kaushikcfd I just read the GPU chapter of your thesis (and liked it!). Looking through the code here I can’t find the second transform that you describe nor the various heuristics. Are those available anywhere?

[kaushikcfd]

Are those available anywhere?

You can find them here: https://github.com/OP2/PyOP2/tree/auto_tiling.

[connorjward]

Ah great. Thanks!

[kaushikcfd]

Draft because:

• Clean commit history

[kaushikcfd]

cc @inducer

[miklos1]

Maybe IntFlag instead of IntEnum? (just something to perhaps consider...)

[wence-]

Didn't manage everything, but a bunch of comments/queries.

[wence-]

Completely impenetrable name.

[kaushikcfd]

Renamed it to split_n_across_workgroups, better?

[wence-]

Again, comments about vec state management apply here, and mutatis mutandis, the rest of the PR.

[wence-]

This cached data is generated by a computer. Therefore there should be no need to parse and evaluate it, because the on-disk representation can just be the in-memory representation.

[kaushikcfd]

This cached data is generated by a computer

I preferred a human-readable format to keep the representation valid across pyop2-versions which might not be the case with pickling. (Happy to serialize in a different format if there's a better alternative)

no need to parse and evaluate it,

I don't think we can avoid the evaluates because the grid sizes are typically symbolic expressions of the form: (end - start) // 32 + 1. Therefore the grid sizes are evaluated at runtime during a GlobalKernel.__call__ invocation.

[wence-]

Pickle version incompatibilities are fine (they would appear when you upgrade python and we don't guarantee very much about the portability of on disk cached data between firedrake-update calls, let alone python version upgrades).

What you're attempting to do is cache a function that computes these things. So in the code_to_compile case, just write that function, and pickle it.

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

[kaushikcfd]

Pickle version incompatibilities are fin

Cool, will change. (Thanks!)

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

Yes, the compiled code stores only the plain CUDA/OpenCL device kernel. Loopy's cache on disk mechanism works by storing both the device kernel and the host code that would call end up calling this device kernel. But adapting PyOP2 to call loopy kernels would be a bit disruptive and I think loopy's cache dir being $XDG_CACHE_DIR has run into some problems in the past.

[kaushikcfd]

just write that function,

Planning to simply generate a python function corresponding to the expression: inducer/pymbolic#107

[inducer]

For my education, how come you chose to not go with the generated invokers?

[kaushikcfd]

For PyOpenCL we could borrow the invoker, but for PyCUDA/C-target we don't have an invoker in loopy (yet).

[kaushikcfd]

• Also, for the pyopencl invoker, a minor concern is that the disk-caching semantics don't play well with PyOP2's disk-caching semantics.
• I feel adding an invoker for the PyCUDA target shouldn't be too difficult. Taking a look at it.

[inducer]

There is invoker generation for the C target. I'd expect that putting together CUDA invokers wouldn't be a massive effort based on a copy-paste job of the CL ones. Do you think that might be the right approach?

[kaushikcfd]

I am working on integrating the PyCUDA/PyOpenCL invokers. For the C-target, reorganizing pyop2/compilation.py is a bigger undertaking and also out of the scope for this PR.

[wence-]

This is data to be consumed by a computer, so don't make it human-readable and require parsing.

[kaushikcfd]

Do you recommend pickling?

[wence-]

This is tremendously fragile. Please think of a better way of saving this data in a way that doesn't rely on dubiously-stable properties of numpy.savez/load.

[kaushikcfd]

Agreed. While np.savez-ing the arrays also stored a string-array that tells us the keys that were used for storing these arrays. See https://github.com/OP2/PyOP2/compare/9b7eb97878f2b6609d64affe896c5d4d901a2934..0e58f08d3ea1487e01c45df2270fa9e5790a845d.

[connorjward]

@kaushikcfd just to make you aware of a couple things:

• We are currently in the process of ingesting PyOP2 (and TSFC) into the Firedrake repository, so this repository will shortly be archived.
• We are hoping to soon drop PyOP2 entirely in favour of pyop3. We then, assuming funding, want to add GPU support there. We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

[kaushikcfd]

Hi Connor!
Sounds good.

We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

Let me know if anything in this patch isn't clear.

[connorjward]

@kaushikcfd I just read the GPU chapter of your thesis (and liked it!). Looking through the code here I can’t find the second transform that you describe nor the various heuristics. Are those available anywhere?

[kaushikcfd]

Are those available anywhere?

You can find them here: https://github.com/OP2/PyOP2/tree/auto_tiling.

[connorjward]

Ah great. Thanks!

[kaushikcfd]

Draft because:

• Clean commit history

[kaushikcfd]

cc @inducer

[miklos1]

Maybe IntFlag instead of IntEnum? (just something to perhaps consider...)

[wence-]

Didn't manage everything, but a bunch of comments/queries.

[wence-]

Completely impenetrable name.

[kaushikcfd]

Renamed it to split_n_across_workgroups, better?

[wence-]

Again, comments about vec state management apply here, and mutatis mutandis, the rest of the PR.

[wence-]

This cached data is generated by a computer. Therefore there should be no need to parse and evaluate it, because the on-disk representation can just be the in-memory representation.

[kaushikcfd]

This cached data is generated by a computer

I preferred a human-readable format to keep the representation valid across pyop2-versions which might not be the case with pickling. (Happy to serialize in a different format if there's a better alternative)

no need to parse and evaluate it,

I don't think we can avoid the evaluates because the grid sizes are typically symbolic expressions of the form: (end - start) // 32 + 1. Therefore the grid sizes are evaluated at runtime during a GlobalKernel.__call__ invocation.

[wence-]

Pickle version incompatibilities are fine (they would appear when you upgrade python and we don't guarantee very much about the portability of on disk cached data between firedrake-update calls, let alone python version upgrades).

What you're attempting to do is cache a function that computes these things. So in the code_to_compile case, just write that function, and pickle it.

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

[kaushikcfd]

Pickle version incompatibilities are fin

Cool, will change. (Thanks!)

Although is this a consequence of the on-disk kernel/compiled code not remembering the loopy representation?

Yes, the compiled code stores only the plain CUDA/OpenCL device kernel. Loopy's cache on disk mechanism works by storing both the device kernel and the host code that would call end up calling this device kernel. But adapting PyOP2 to call loopy kernels would be a bit disruptive and I think loopy's cache dir being $XDG_CACHE_DIR has run into some problems in the past.

[kaushikcfd]

just write that function,

Planning to simply generate a python function corresponding to the expression: inducer/pymbolic#107

[inducer]

For my education, how come you chose to not go with the generated invokers?

[kaushikcfd]

For PyOpenCL we could borrow the invoker, but for PyCUDA/C-target we don't have an invoker in loopy (yet).

[kaushikcfd]

• Also, for the pyopencl invoker, a minor concern is that the disk-caching semantics don't play well with PyOP2's disk-caching semantics.
• I feel adding an invoker for the PyCUDA target shouldn't be too difficult. Taking a look at it.

[inducer]

There is invoker generation for the C target. I'd expect that putting together CUDA invokers wouldn't be a massive effort based on a copy-paste job of the CL ones. Do you think that might be the right approach?

[kaushikcfd]

I am working on integrating the PyCUDA/PyOpenCL invokers. For the C-target, reorganizing pyop2/compilation.py is a bigger undertaking and also out of the scope for this PR.

[wence-]

This is data to be consumed by a computer, so don't make it human-readable and require parsing.

[kaushikcfd]

Do you recommend pickling?

[wence-]

This is tremendously fragile. Please think of a better way of saving this data in a way that doesn't rely on dubiously-stable properties of numpy.savez/load.

[kaushikcfd]

Agreed. While np.savez-ing the arrays also stored a string-array that tells us the keys that were used for storing these arrays. See https://github.com/OP2/PyOP2/compare/9b7eb97878f2b6609d64affe896c5d4d901a2934..0e58f08d3ea1487e01c45df2270fa9e5790a845d.

[connorjward]

@kaushikcfd just to make you aware of a couple things:

• We are currently in the process of ingesting PyOP2 (and TSFC) into the Firedrake repository, so this repository will shortly be archived.
• We are hoping to soon drop PyOP2 entirely in favour of pyop3. We then, assuming funding, want to add GPU support there. We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

[kaushikcfd]

Hi Connor!
Sounds good.

We fully intend to use a lot of what you have done here but will have to adapt it a fair amount.

Let me know if anything in this patch isn't clear.

[connorjward]

@kaushikcfd I just read the GPU chapter of your thesis (and liked it!). Looking through the code here I can’t find the second transform that you describe nor the various heuristics. Are those available anywhere?

[kaushikcfd]

Are those available anywhere?

You can find them here: https://github.com/OP2/PyOP2/tree/auto_tiling.

[connorjward]

Ah great. Thanks!