parameterize (implicit description)

AComparisonofParameterStudyCreationandJobSubmissionToolsAdrianDeVivo,MauriceYarrow,ComputerSciencesCorporation,MailStopT27A-1ASAAmesResearchCenter,MoffettField,CA9405devivo,yarrow,mccann@ASTechnicalumber:AS-01-002AbstractWeciderthedifferencesbetweentheavailablegeneralpurposeparameterstudyandjobsubmissiontools.Thesetoolsnecessarilysharemanyfeatures,butfrequentlywithdifferencesinthewaytheyaredesignedandimple-sclassoffeatures,wewillonlybrieflr,wewillfocusontheuniquefeatureswhichdistinguishtheILabparameterstudyandjobsubmissiontoolfromotherpackages,andwhichmaketheILabtooleasierandmoresuitableforuseinourresearchandengineeringenvironment.MotivationandBackgroundThegenerationandsubmissionofphysicalparametricstudiesinvolvingsimulatiwithengineeringprogramsisualcreationofsuchstudiestypicallyfirstinvolveseditingalargenumberofASCIIfiles,theresultantindividualsimulationjobrunsmustbeorganizedten,therequiredcomputatiwillactuallyoccuronasupercomputerdistinctfromthezationalrequirementswillusuallyrequiretheassemblyofaspecialdirectorystructureonthetargetcomputersystems,andtheimportationtothisdirectorystructureofthemanyfilestion,mostsupercomputercentersrequirethatsubmittedjobsgothroughajobscheduler(queuingsystem),andthusparameterstudytoolsmustcoordinatewiththescheduler.Supportingthesetasksarethemostimportantofthefunctionalitiesorgeneralrequirementstobeexpectedfromation,specificuserneedsrequireincertaincasesspecificfeatures,withoutwhichalltheotherfeaturesavailablemaynotsufficetomakeaparameterstudytoolusable.WehavedevelopedtheILabtooltoaddresstheneedsofnotonlygeneralandsimpleparameterstudyusers,butconcentrateonadescriptionoftheseimportantfeaturesinthisreport.ToolDescriptiILab“ILab:TheInformationPowerGridVirtualLaboratory”isaPerlprogramwhichprovidesaPerlTkuserinterfaceandgeneratesinputfilesandshellscriptsforparameterstudiesbeingrunontheGrid.(otethattheexpression“Grid”isnowbeingusedtodescribetheworld-widenetworkofsupercomputersthatscientistsareusingtorunscientificproblemsthatinvolvedatasetstoolargetofitondesktopsystems.)Thisprogramhasanextensiveuserinterface;wealreadyhaveabout2,000linesofPerl,andweanticipateabout50,haveadoptedanobject-orientedapproach:alldataandoperatiareorganizedintopackagesinordertospeedup’s“container”or“document”objectiscalledan“Experiment”:itcontainsallthe1

information-dataandfilepaths-thatisnecessarytodefineacomplexseriesofrepeatedandorsequencedandorjectisserializedtoandfromdisk,foruser’ime,ILab’scurrent“Experiment”isusedtogeneraterequiredinputfilesandshellscripts,createdirectories,copydatafiles,andthenbothinitiateandmonitortheexecutionofallprocessesintheExperiment.[1]imrodClustorandimrodGimrod,imrodG,andClustor[2][4]arehistoricallyrelatedtoolswhichcreateparameterstudiesandhandlejobingtheresourcesoftheGlobusGridtoolkit,imrodGperformstaskssuchas:schedulingofajob,resourcediscovery,remotestorage,andasortofsystemspassport(userauthentication).imrodGemploysthefunctionalityof“computationaleconomy”[4]:thisisaprocessthatattemptstonegotiatefortimeslotsofadequatesizercanalsonegotiateresourcesandmakeadecisionastowhenthejobcanbecompletedeatingaparameterstudytheusermayneedtoprogramaspecialinternalscriptrdoesnotsupportvisualparameterization:manualeditingofdatafilesisnecessarytopreparethemforparameterization;subsequently,parameterizedinputfilescanbegeneratedautomatically.CondorCondor[5][8],adistributedjoblauncher,utilizesidlesystemcyclesystemsinthe“Condorpool”.TheCondorpoolcistsofsystemsrunningCondorsoftwareonaLA,WA,orovertheinternet(notbehindafirewall).Condormonitorssystemsforspecificuserdefioseparalarintervalswhilethejobisrunning,ventthatthejobisstoppedbythehostsystem’suserCondorwilmarydrawbackofCondoristhatitcrealsosomesecurityissuestobeaddressed,namely“...asufficientlymaliciousandcleverusercouldcauseproblemsbydoinglocalsystemcallsontheexecutingmachine.”[5]Condorisappropriateforsimpleprocesecution,whereprocessesdoesnotcreateparameterstudies.AppLeSAPSTAppLeS(Application-Levelscheduler)[6][7]isajobsubmissiontoolwhichevaluateseverythingaboutthesystembaseduponpredictiofthestateofthesystemwhentheapplicationkstatusinformationisreceivedfrometworkWeatherService(WS),whichcan“forecasttheperformancevariousnetworkandcomputationalresourcescandeliveroveragiventimeinterval”[6].AppLeSalsodoesnotdoparameterization;itisa“performancepredictor”,“AppLeSParameterSweepTemplate”,launchespreexistingparameterstudies.ChartDescriptionThefollowingchartoutlinesthefunctionalityofILabandseveralotherparameterstudyandschedulerutilities.Clustor,imrod,andimrodGaretheclosestcomparisontoILab’scapabilities,sincetheytoocancreateparameterstudiesinadditiontosubmittingjobs.ChartKey(,whatthetermssignifyinthecontextofthesecomparis)ComplexProcesses:asetofprocesseswhereeachprocessdependsontheinputfromthepreviousprocess;mayormaynotinvolvebranchingandloops.MultipleInputFiles:morethanonefileisrequiredforrunningaprocess.ProcessLooping:therepetitionofaprocessuntilsomecondition(typically,aprogramreturnvalue,programinternalvalue,ortheexistenceofanoutputfile)issatisfied.GUIParameterization:(Visualparameterization)agraphicalscreeninwhichusercanchoosefieldstoparameterizebyhighlightingthefieldswithamouse,andthenenteringalistofvaluesforthatfield.2

imrodPerson.Yes*MultipleInputFilesYesYes*GUIParameterizationoYesGUIYesoBuilt-inMetalanguageSimpleYesCADInterfaceoYesPluginsoYesSingleJobcapacityYesYesGraphicsVideoscreensoYes*Problem-SolvingEnvoYesMetacomputingEnvoYesGlobusEnabledoYes*ComparisonTableGooYesYesYesoYesooYes?oAA?AAAYesooYesoAAAA??Yeso?YesAo