Migrate FFC interface to Firedrake

[kynan]

Needs to go in with firedrakeproject/firedrake#160.

[kynan]

Note that this currently fails since many demos and all of our regressions tests actually depend on the FFC interface. Options include:

• Remove all regression tests
• Hard code the FFC generated kernels in those demos

[kynan]

Losing all regression tests isn't great, therefore go with option 2 and hard code the generated kernels, but leave the door open to run with Firedrake and optionally update the generated kernels.

Note that this doesn't work for CUDA due to #308.

[wence-]

Did we get anywhere with this?

[kynan]

We still need to think about a solution for CUDA and OpenCL which ideally doesn't involved replicating canned regression test kernels for these backends. Otherwise I think this is fine.

I could already push everything up to 01e79c9 to master since it's not really part of this pull request. Any objection to that?

[kynan]

This has been ready for a while, any thoughts?

The canned regression test kernels now work for all backends since I'm pickling the AST instead of dumping generated sequential code. The downside is that the pickle files are binaries.

[wence-]

I buy this. This needs to go in at the same time as firedrakeproject/firedrake#194 though, right?

[wence-]

Needs update of pyop2-clean script (which now fails)

[kynan]

I've removed pyop2-clean since it has become obsolete. firedrakeproject/firedrake#194 depends on this, and they should be merged at the same time since Firedrake would break otherwise.

[wence-]

Can you add a firedrake-clean script on the other side please?

[kynan]

Yes, I'll do that.

[wence-]

Now that we don't use instant for compilation, I think we should still have a pyop2-clean script that deletes the cached compiled libraries. That would look like this:

    Modified   pyop2/compilation.py
diff --git a/pyop2/compilation.py b/pyop2/compilation.py
index 000aee2..0d27c23 100644
--- a/pyop2/compilation.py
+++ b/pyop2/compilation.py
@@ -203,3 +203,37 @@ def load(src, fn_name, cppargs=[], ldargs=[], argtypes=None, restype=None):
     fn.argtypes = argtypes
     fn.restype = restype
     return fn
+
+
+def clear_cache(prompt=False):
+    """Clear the PyOP2 compiler cache.
+
+    :arg prompt: if ``True`` prompt before removing any files
+    """
+    cachedir = configuration['cache_dir']
+
+    files = [os.path.join(cachedir, f) for f in os.listdir(cachedir)
+             if os.path.isfile(os.path.join(cachedir, f))]
+    nfiles = len(files)
+
+    if nfiles == 0:
+        print "No cached libraries to remove"
+        return
+
+    remove = True
+    if prompt:
+
+        user = raw_input("Remove %d cached libraries from %s? [Y/n]: " % (nfiles, cachedir))
+
+        while user.lower() not in ['', 'y', 'n']:
+            print "Please answer y or n."
+            user = raw_input("Remove %d cached libraries from %s? [Y/n]: " % (nfiles, cachedir))
+
+        if user.lower() == 'n':
+            remove = False
+
+    if remove:
+        print "Removing %d cached libraries from %s" % (nfiles, cachedir)
+        [os.remove(f) for f in files]
+    else:
+        print "Not removing cached libraries"
    New        scripts/pyop2-clean
diff --git a/scripts/pyop2-clean b/scripts/pyop2-clean
new file mode 100755
index 0000000..23ac63a
--- /dev/null
+++ b/scripts/pyop2-clean
@@ -0,0 +1,6 @@
+#!/usr/bin/env python
+
+from pyop2.compilation import clear_cache
+
+if __name__ == '__main__':
+    clear_cache(prompt=True)

[kynan]

Yes, that makes sense.

[wence-]

I've repurposed pyop2-clean appropriately, barring buildbot/travis happiness, I think this is good. Note that the most recent build (http://buildbot-ocean.ese.ic.ac.uk:8080/builders/pyop2-testing/builds/1210) failed device backend regression tests. Some kind of failure in the canned kernel stuff?

[kynan]

I'm not entirely sure what's going on, will investigate.

[wence-]

Can these two changes finally land?

[kynan]

Yes as far as I'm concerned. We can have another debate whether to actually scrap the PyOP2 regression tests (and move them all to Firedrake) rather than conserving them.

[wence-]

Please go ahead on both sides then.

[kynan]

So we had another discussion and decided that there's no point keeping the regression tests and conserve pickled FFC kernels for them. The corresponding demos are also removed because they're more or less re-implementations of bits of Firedrake.

Any objections to this?

[wence-]

Assuming firedrake has gained the installation requirements that pyop2 lost, I think yes, go ahead.

[kynan]

It has, that's on the requirements branch which I'll propose for merging now.

There is one caveat with this: test coverage of the OpenCL and CUDA backends is significantly reduced, since the unit tests don't nearly have full code coverage. We should probably identify some test cases that solve pre-assembled systems s.t. we can test those with CUDA/OpenCL at the Firedrake level.