From m.khademi at gmail.com Thu Feb 21 01:09:16 2008 From: m.khademi at gmail.com (Maryam Khademi) Date: Thu, 21 Feb 2008 17:09:16 +0800 Subject: [Pegasus-discuss] condor pool Message-ID: Hello, >From the config file "sites.xml", it sounds to me that Pegasus is able to map jobs and partitions into several condor pools. right? So here is the question: how Pegasus decides which pool should be selected for which partition!? Do you use information provided by MDS to find resources which adapt jobs' requirement? Thanks, Maryam -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.isi.edu/pipermail/pegasus-discuss/attachments/20080221/1ee66469/attachment.html From m.khademi at gmail.com Thu Feb 21 01:33:06 2008 From: m.khademi at gmail.com (Maryam Khademi) Date: Thu, 21 Feb 2008 17:33:06 +0800 Subject: [Pegasus-discuss] Using multiple condor pool Message-ID: Hi, Is it possible to force condor_dagman to use a special condor pool? Or in an other way, is DAGMan able to use different condor pools? As I checked the available arguments for condor_dagman, I couldn't find such thing! Any suggestion? Thanks and Regards, Maryam -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.isi.edu/pipermail/pegasus-discuss/attachments/20080221/4ca7be97/attachment.html From gmehta at ISI.EDU Thu Feb 21 11:23:50 2008 From: gmehta at ISI.EDU (Gaurang Mehta) Date: Thu, 21 Feb 2008 11:23:50 -0800 Subject: [Pegasus-discuss] Using multiple condor pool In-Reply-To: References: Message-ID: <242BED26-5CB7-4B09-ADDA-2B51C947FC9B@isi.edu> Maryam, Dagman is just a dependency manager. If you want to use different condor pools the names of the pool or the classads need to be specified in the Job submit files. For more information please read the Condor manual. _Gaurang On Feb 21, 2008, at 1:33 AM, Maryam Khademi wrote: > Hi, > > Is it possible to force condor_dagman to use a special condor pool? > Or in an other way, is DAGMan able to use different condor pools? > As I checked the available arguments for condor_dagman, I couldn't > find such thing! > Any suggestion? > > Thanks and Regards, > > Maryam > _______________________________________________ > Pegasus-discuss mailing list > Pegasus-discuss at isi.edu > http://mailman.isi.edu/mailman/listinfo/pegasus-discuss -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2869 bytes Desc: not available Url : http://mailman.isi.edu/pipermail/pegasus-discuss/attachments/20080221/d3ad6fc9/smime.bin From gmehta at ISI.EDU Thu Feb 21 11:28:32 2008 From: gmehta at ISI.EDU (Gaurang Mehta) Date: Thu, 21 Feb 2008 11:28:32 -0800 Subject: [Pegasus-discuss] condor pool In-Reply-To: References: Message-ID: <3A832724-0FA1-43FD-A0DD-2CDF21EE7799@isi.edu> Pegasus decides which pool the job to schedule on using the site selector module. Currently we have implement Round Robin and Random site selectors. There are some experimental site selectors also. We currently dont use MDS to find resources but are looking at other ways to get more dynamic information. You can easily implement your own site selector and plug it into pegasus. You can use any language (perl, java, c, bash, csh) to write your selector. Please look at the site selector java api on the pegasus website. For non java http://pegasus.isi.edu/docs/javadoc-2.0.1/org/griphyn/cPlanner/selector/site/NonJavaCallout.html -Gaurang On Feb 21, 2008, at 1:09 AM, Maryam Khademi wrote: > Hello, > > From the config file "sites.xml", it sounds to me that Pegasus is > able to map jobs and partitions into several condor pools. right? > So here is the question: how Pegasus decides which pool should be > selected for which partition!? > Do you use information provided by MDS to find resources which adapt > jobs' requirement? > > Thanks, > > Maryam > > _______________________________________________ > Pegasus-discuss mailing list > Pegasus-discuss at isi.edu > http://mailman.isi.edu/mailman/listinfo/pegasus-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.isi.edu/pipermail/pegasus-discuss/attachments/20080221/d15efa47/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2869 bytes Desc: not available Url : http://mailman.isi.edu/pipermail/pegasus-discuss/attachments/20080221/d15efa47/smime.bin From embahsi at gmail.com Thu Feb 21 12:24:31 2008 From: embahsi at gmail.com (emir mahmut bahsi) Date: Thu, 21 Feb 2008 14:24:31 -0600 Subject: [Pegasus-discuss] Siteselector that performs load balancing Message-ID: Hi all, I am planning to implement a siteselector that does selection based on queue statuses of each site. I wonder if anybody has implemented any siteselector so far. It may be helpful to me to have an example of previously-implemented siteselector code. Thanks, -- Emir Mahmut Bahsi Research assistant Dept of Computer Science at LSU Room 231, Johnston Hall Louisiana State University Baton Rouge, LA 70803 embahsi at cct.lsu.edu http://www.cct.lsu.edu/~embahsi -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.isi.edu/pipermail/pegasus-discuss/attachments/20080221/8ab528db/attachment.html From m.khademi at gmail.com Mon Feb 25 19:29:25 2008 From: m.khademi at gmail.com (Maryam Khademi) Date: Tue, 26 Feb 2008 11:29:25 +0800 Subject: [Pegasus-discuss] Using multiple condor pool In-Reply-To: <242BED26-5CB7-4B09-ADDA-2B51C947FC9B@isi.edu> References: <242BED26-5CB7-4B09-ADDA-2B51C947FC9B@isi.edu> Message-ID: Okay, right! But as long as I know, there is no direct parameter in submit job files to specify the name of pool! We can just name the machine or UidDomain or FileSystemDomain. I've noticed in Pegasus you add several classads containing "vds_site". But it seems your wrapper over condor is in charge of reading those not condor itself. right? So how do you translate it for condor to use the specified pool? Thanx, Maryam On Fri, Feb 22, 2008 at 3:23 AM, Gaurang Mehta wrote: > Maryam, Dagman is just a dependency manager. If you want to use > different condor pools the names of the pool or the classads need to > be specified in the Job submit files. For more information please read > the Condor manual. > _Gaurang > > On Feb 21, 2008, at 1:33 AM, Maryam Khademi wrote: > > > Hi, > > > > Is it possible to force condor_dagman to use a special condor pool? > > Or in an other way, is DAGMan able to use different condor pools? > > As I checked the available arguments for condor_dagman, I couldn't > > find such thing! > > Any suggestion? > > > > Thanks and Regards, > > > > Maryam > > _______________________________________________ > > Pegasus-discuss mailing list > > Pegasus-discuss at isi.edu > > http://mailman.isi.edu/mailman/listinfo/pegasus-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.isi.edu/pipermail/pegasus-discuss/attachments/20080226/6e178b95/attachment.html From gmehta at ISI.EDU Mon Feb 25 20:13:24 2008 From: gmehta at ISI.EDU (Gaurang Mehta) Date: Mon, 25 Feb 2008 20:13:24 -0800 Subject: [Pegasus-discuss] Using multiple condor pool In-Reply-To: References: <242BED26-5CB7-4B09-ADDA-2B51C947FC9B@isi.edu> Message-ID: <66521971-5F76-43E2-B373-011C0746F979@isi.edu> Hi Maryam, The classads we add are not for condor's use. We add that for our own mining purpose. For deciding which site we add universe=grid and grid_scheduler= string in the submit file if the jobs are to be run on a grid. If they are to run on a regular condor pool we generally add A filesystemdomain classad to decide which pool it runs on. -Gaurang On Feb 25, 2008, at 7:29 PM, Maryam Khademi wrote: > Okay, right! > > But as long as I know, there is no direct parameter in submit job > files to specify the name of pool! > We can just name the machine or UidDomain or FileSystemDomain. > > I've noticed in Pegasus you add several classads containing > "vds_site". But it seems your wrapper > over condor is in charge of reading those not condor itself. right? > So how do you translate it for condor to use the specified > pool? > > Thanx, > Maryam > > On Fri, Feb 22, 2008 at 3:23 AM, Gaurang Mehta wrote: > Maryam, Dagman is just a dependency manager. If you want to use > different condor pools the names of the pool or the classads need to > be specified in the Job submit files. For more information please read > the Condor manual. > _Gaurang > > On Feb 21, 2008, at 1:33 AM, Maryam Khademi wrote: > > > Hi, > > > > Is it possible to force condor_dagman to use a special condor pool? > > Or in an other way, is DAGMan able to use different condor pools? > > As I checked the available arguments for condor_dagman, I couldn't > > find such thing! > > Any suggestion? > > > > Thanks and Regards, > > > > Maryam > > _______________________________________________ > > Pegasus-discuss mailing list > > Pegasus-discuss at isi.edu > > http://mailman.isi.edu/mailman/listinfo/pegasus-discuss > > > _______________________________________________ > Pegasus-discuss mailing list > Pegasus-discuss at isi.edu > http://mailman.isi.edu/mailman/listinfo/pegasus-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.isi.edu/pipermail/pegasus-discuss/attachments/20080225/c815de7a/attachment.html