Is there any way that Closure/Static could be factored into a separate package? about distributed-process HOT 10 OPEN

Yes, the old implementation support for Static/Closure was very tightly interwoven with the rest. This is no longer the case, although you are right that the Template Haskell stuff does recognize Process for convenience. It should be possible to factor most of this out. I think that's a good idea, even just for clarifying the current implementation. I will have a look at this (right after I give you your performance benchmarks :).

from distributed-process.

I have made a lot of progress on this today but haven't quite finished yet. Hopefully I'll be able to push something tomorrow.

from distributed-process.

I have pushed (and released to Hackage) a new version of CH where static is a separate package (Control.Distributed.Static -- http://hackage.haskell.org/package/distributed-static-0.1.0.0). This package does not provide any Template Haskell support but the TH layer in Cloud Haskell is now so thin, and so little of it is not Cloud Haskell specific, that I'm not sure it's worth abstracting it further (in fact, C.D.Static is quite usable without TH at all).
Please have a look at let me know what part of the TH would be useful in metapar (or elsewhere) and would be worthwhile moving from CH proper to C.D.Static.

from distributed-process.

Awesome, thanks. We will switch over to this as soon as we get a chance
and let you know how it goes.

On Fri, Aug 10, 2012 at 12:12 PM, Edsko de Vries
[email protected]:

I have pushed (and released to Hackage) a new version of CH where static
is a separate package (Control.Distributed.Static --
http://hackage.haskell.org/package/distributed-static-0.1.0.0). This
package does not provide any Template Haskell support but the TH layer in
Cloud Haskell is now so thin, and so little of it is not Cloud Haskell
specific, that I'm not sure it's worth abstracting it further (in fact,
C.D.Static is quite usable without TH at all).
Please have a look at let me know what part of the TH would be useful in
metapar (or elsewhere) and would be worthwhile moving from CH proper to
C.D.Static.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/21#issuecomment-7649171.

from distributed-process.

Ryan, did you have a chance to look at this?

from distributed-process.

Moving to Misc/Admin as

(a) the move has already taken place and
(b) we've not heard anything back for >= 3months

Ryan, are you happy for me to close or is there more to us here?

from distributed-process.

To address an earlier question raised by @rrnewton on this issue. The closure/static representation in HdpH was also cut away from the rest of HdpH back in July 2012. At the time this was done to improve the chances of a unified representation for meta-par, hdph and cloud-haskell. @edsko has made similar de-coupling efforts from the CH side. The motivated design of the closure representation is described in Section 3.2 of the first hdph paper, in particular pay attention to the dual closure representation paragraph. The key implementation facts about static in HdpH is in the documentation throughout HdpH.Closure.Static.hs , and the template haskell code for mkClosure and so on is in HdpH.Closure.Internal.hs . I realise that this comes months after the CloudHaskell 2.0 closure implementation. I also, however, see many benefits to having one one representation across all distributed haskells. This would for example enable CloudHaskell's task layer to benefit from load balancing in HdpH's network-based work-stealing scheduler.

It is worth adding that a considerable HdpH refactor will soon be pushed onto github, which will see adoption of the network-transport-tcp backend, and an upload of the library to hackage.

from distributed-process.

Hi @robstewart57 - thanks for explaining a bit of the history behind this.

a unified representation for meta-par, hdph and cloud-haskell

We should definitely aim for that if possible.

I also, however, see many benefits to having one one representation across all distributed haskells.

Me too - lots and lots of them. I'd really like to be able to mix and match.

It is worth adding that a considerable HdpH refactor will soon be pushed onto github, which will see adoption of the network-transport-tcp backend, and an upload of the library to hackage.

Great. After that happens, we should look at properly unifying these layers then. Ryan (or someone on his behalf) will need to let one of us know what's left to be done before all three libraries can share the same implementation. From @edsko's last comment, it sounds like the main thing that's left to be done is to possibly share some of the TH code, though I'm not clear on which bits would be useful - are we talking about all of mkClosure available across all three libraries?

I'm happy to look into this, but it's a bit further down the priority stack than some of our pressing bug fixes right now and I know little about TH so it might not get done until I've had time to chat with @edsko about it. Unless someone wants to either contribute a pull request or give me some pointed ideas about how to achieve it, I'd estimate we're looking at post 0.2.0. Sorry I realise that's not the answer everyone was hoping for, but I want to give a realistic picture of what I'm likely to get done in my spare time over the next couple of months. I will probably drop a note out onto the parallel-haskell mailing list asking if anyone wants to step into the gap and assist with this specific task.

from distributed-process.

Note that the dual closure representation cannot be implemented without using unsafePerformIO (or proper implementation of 'static'). That's the main reason I didn't adopt it yet.

from distributed-process.

Note that the dual closure representation cannot be implemented without using unsafePerformIO (or proper implementation of 'static'). That's the main reason I didn't adopt it yet.

Hmn, I'm inclined to hold off on adopting it for the same reason.

from distributed-process.