[Pegasus-users] pegasus-monitord took 30G memory

[Yu Huang]

On Thu, May 3, 2012 at 3:02 PM, Mats Rynge <rynge at isi.edu> wrote:

> On 05/03/2012 01:58 PM, Yu Huang wrote:
> > The error message looks like this:
> >
> > Held     00:01   ??????PEAlignmentByBWA.sh_ID0009937
> >
> >  ??????Error from slot1 at n6231.S1335855707.27@n6231: Failed to open
> >
> '/u/home/eeskin2/polyacti/NetworkData/vervet/vervetPipeline/work/ShortRead2Al..
> >
> > The folder above is a symbolic link to a local folder on central manager
> > n6223. slot1 at n6231 would just hit a dead link.
>
> Is your condor pool configured with the same filesystemdomain across all
> the nodes? You can find out by running:
>
>  condor_status -l | grep -i ^filesystemdomain
>
> If so, what you are seeing is Condor thinking that the submit directory
> is on shared filesystem (I know, which it used to be).
>
> you totally nailed it.


> I suggest you leave the existing workflow on the shared filesystem and
> let it complete the. For future ones, let Condor transfer the
> stdout/stderr for you. First configure your pool to have
> filesystemdomain set based on the hostname. In your Condor config:
>
>    FILESYSTEM_DOMAIN = $(FULL_HOSTNAME)
>
ok i will do this from now on.

>
> Then for the jobs, set the following profiles in your site catalog:
>
>    <profile namespace="condor"
> key="should_transfer_files">YES</profile> <profile>
>    <profile namespace="condor"
> key="when_to_transfer_output">ON_EXIT_OR_EVICT</profile>
>
> ok. where do the output files reside then? scratch/local or scratch/shared
or storage/local or storage/shared defined in my site catalog?

Thanks
yu


> --
> Mats Rynge
> USC/ISI - Pegasus Team <http://pegasus.isi.edu>
>



-- 
Yu Huang
Postdoc in Nelson Freimer Lab,
Center for Neurobehavioral Genetics, UCLA
Office Phone:* +1.310-794-9598*
Skype ID: crocea
http://www-scf.usc.edu/~yuhuang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.isi.edu/pipermail/pegasus-users/attachments/20120503/fab4559f/attachment.html