Use ST instead of polymorphic m in Vector/MVector type classes

[Shimuuar]

This is implementation of idea from #315. Instead of using polymorphic m in methods of MVector/Vector type class use ST and lift it to PrimMonad at use sites. It's probably easier to illustrate idea with few diffs:

- basicUnsafeNew :: PrimMonad m => Int -> m (v (PrimState m) a)
+ basicUnsafeNew :: Int -> ST s (v s a)

 unsafeRead v i = UNSAFE_CHECK(checkIndex) "unsafeRead" i (length v)
+ $ stToPrim
$ basicUnsafeRead v i

Main motivation for change is to allow GND for those type classes. It's partial success. In following code snippet:

newtypeinstanceMVectors (Suma) =MV_Sum (MVectorsa)
newtypeinstanceVector (Suma) =V_Sum (Vectora)
derivinginstanceUnboxa=>G.VectorVector (Suma)
derivinginstanceUnboxa=>M.MVectorMVector (Suma)

MVector instance derived successfully. Vector fails with:

 • Couldn't match representation of type ‘m1 a’
with that of ‘m1 (Sum a)’

because of basicUnsafeIndexM :: Monad m => v a -> Int -> m a. GHC cannot coerce between m a and m (Sum a). I'm a bit at loss here.

[Shimuuar]

I said that I don't know what to do and then I used same trick: use box monad in order to express strictness in indexing and keep indexed value unevaluated. Everything works except GHC8.0 can't derive instances for some reason. While this Box trick seems to work I didn't test whether it really works.

I also want to test whether this approach works with DerivingVia. GND is surely nice but I'm not convinced that it alone make this change valuable enough.

[Shimuuar]

In the end idea of using concrete monads in the definition of vector's type classes worked out nicely. basicWhatever methods which used PrimMonad m now are defined in terms of ST, and basicIndexM is defined in terms of Box. Rest of API is not changed at all.

1. GND works. It's now possible to define unbox instances for all newtypes using it. (Except GHC8.0 fails with weird error, and we'll have to wait until we drop support for it)

2. DerivingVia works too. As an example:

newtypeinstanceMVectorsInt=MV_Int (P.MVectorsInt)
newtypeinstanceVectorInt=V_Int (P.VectorInt)
instanceUnboxIntderivingvia (UnboxViaPrimInt) instanceM.MVectorMVectorIntderivingvia (UnboxViaPrimInt) instanceG.VectorVectorInt

3, Box tricked for indexM works too:

*Data.Vector> _ <- indexM (Data.Vector.fromList [undefined]) 0
*Data.Vector> _ <- indexM undefined 0
*** Exception: Prelude.undefined
CallStack (from HasCallStack):
error, called at libraries/base/GHC/Err.hs:80:14 in base:GHC.Err
undefined, called at <interactive>:5:13 in interactive:Ghci1

4. As small test for breakage I built statistics with this change. It and its dependencies has both TH and handwritten instances. All builds without any problems

Overall I think this is big improvement for vector usability.

[lehins]

@Shimuuar This is great news! There is only one concern I have about GND and unboxing is that deriving does not add INLINE pragmas and I have seen this happen before where GHD causes derived class functions from not being inlined. (Here is a real life example where I saw this being a problem: commercialhaskell/rio#166) If the same happens with deriving unboxed vector it can have some serious performance implications. I think it would be great to have some benchmarks comparison of at least few functions of derived vs underived approach. Have you tried checking performance or differences in compiled core?

Other than the above concern I think this is a great improvement. I'll do a proper PR review either today or the latest tomorrow, just in case if there is anything I can spot.

[Shimuuar]

Good point! I would've expect that GHC inlines coerced methods eagerly. I'll look into it on the weeekends

[Shimuuar]

I've converted all existing benchmarks which use Double to CDouble. There's no measurable difference:

Double:

quickhull mean 85.75 ms ( +- 994.5 μs )
spectral mean 3.183 ms ( +- 35.50 μs )
tridiag mean 99.24 ms ( +- 1.458 ms )
findIndexR mean 189.7 μs ( +- 2.424 μs )
findIndexR_naïve mean 109.1 ms ( +- 3.609 ms )
findIndexR_manual mean 352.2 μs ( +- 10.20 μs )

CDouble:

quickhull mean 85.88 ms ( +- 967.8 μs )
spectral mean 3.180 ms ( +- 92.69 μs )
tridiag mean 99.59 ms ( +- 2.563 ms )
findIndexR mean 194.0 μs ( +- 7.610 μs )
findIndexR_naïve mean 109.8 ms ( +- 4.609 ms )
findIndexR_manual mean 349.3 μs ( +- 5.304 μs )

P.S. We don't have instances for types from Foreight.C.Types. Looks like oversight for me

[lehins]

@Shimuuar Besides few minor suggestions this PR looks great. Do you mind adding an entry to change log that mentions this breaking change

[lehins]

I think it would be great to also include GND approach here as well as the third alternative that doesn't rely on via. This PR makes it possible, right? I was under impression that it would be possible to derive Unbox with this approach for things like newtype RGB = RGB (Double, Double, Double). If deriving using an already existing Unbox instance is still not possible, then feel free ignore this comment.

[Shimuuar]

Adding GND example is very good idea but it's probably better to add it in documentation o Unboxed module. Add section about deriving of unboxed vector.

[lehins]

With regards to failing CI. I've seen doctests consistently fail on other projects when executed with ghc-8.10. I have not investigated the issue what is the cause of it, since doctests are mainly intended to check that examples are still correct. So we might wanna turn off doctests for ghc-8.10 until this issue resolves itself with future ghc+doctests.

[Bodigrim]

So we might wanna turn off doctests for ghc-8.10 until this issue resolves itself with future ghc+doctests.

Let's disable them for now, we've seen the same issue with random some time ago.

[Shimuuar]

I've fixed everything except documentation on using GND for unboxed vectors.Will write it tomorrow

[Shimuuar]

I've added note about GND. I didn't change existing documentation.But it seems to be somewhat misleading:

Implementing unboxed vectors for new data types can be very easy.

In particular this line seems to be out of place. BTW approach outlined there (and it's one used by vector-th-unbox) could be turned into deriving via with approach a la https://hackage.haskell.org/package/iso-deriving It's probably should be done in separate PR

[lehins]

@Shimuuar Do you mind rebasing on master and squashing all the commits? Other than that I think we can merge it in

[Shimuuar]

@lehins done

@Bodigrim thanks! I've updated changelog

[Bodigrim]

Awesome stuff!