On Thu, Oct 30, 2008 at 02:32:51PM -0400, Oren Laadan wrote:
quoted text
>=20
>=20
> Louis Rilling wrote:
> > On Thu, Oct 30, 2008 at 01:45:25PM -0400, Oren Laadan wrote:
> >>
> >> Louis Rilling wrote:
> >>> In Kerrighed this is kernel-based, and will remain kernel-based becau=
se we
quoted text
> >>> checkpoint a distributed task tree, and want to restart it as mush as=
 possible
quoted text
> >>> with the same distribution. The distributed protocol used for restart=
 is
quoted text
> >>> currently too fragile and complex to rely on customized user-space
> >>> implementations. That said, if someone brings very good arguments in =
favor of
quoted text
> >>> userspace implementations, we might consider changing this.
> >> Zap also has distributed checkpoint which does not require strict
> >> kernel-side ordering. Do you need that because you do SSI ?
> >=20
> > Yes. Tasks from different nodes have parent-children, session leader, e=
tc.
quoted text
> > relationships, and the distributed management of struct pid lifecycle i=
s a bit
quoted text
> > touchy too. By the way, splitting the checkpoint image in one file for =
each task
quoted text
> > helps us a lot to make restart parallel, because it is more efficient f=
or the file
quoted text
> > system to handle parallel reads of different files from different nodes=
 than
quoted text
> > parallel reads on a single file descriptor from different nodes.
>=20
> You can also make parallel restart work with the single stream, without
> much effort. Particularly if you store everything on the file system.

Sure we can use a single stream, since we already share file descriptors ac=
cross
nodes. But the distributed synchronization of the file pointer is costly
compared to having each node access different files. This way we push the
parallelization bottelneck down to the file system rather than in the
distributed VFS layer.

quoted text
>=20
> In both cases, the limiting factor is shared resources - where one task
> cannot proceed with checkpoint because it waits for another task to first
> (re)create that resource.

We just try to avoid other bottlenecks :) And besides file descriptors, sha=
red
resources are as common as multi-threaded programs, which are not the major=
ity
of the workloads we can address.

Louis

--=20
Dr Louis Rilling			Kerlabs
Skype: louis.rilling			Batiment Germanium
Phone: (+33|0) 6 80 89 08 23		80 avenue des Buttes de Coesmes
http://www.kerlabs.com/			35700 Rennes