speciedfilezillaclient

IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksWebInterfacetoRforHigh-PerformanceComputingJunjiNAKANOEi-jiNAKAMATheInstituteofStatisticalMathematics,JapanCOM-ONELtd.
,JapanTheRUserConference2009July8-10,Agrocampus-Ouest,Rennes,FranceIntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarks1Introduction2Rdwebsystem3Examplesofexecution4InstallingRdweb5ConcludingremarksIntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksRandrequirementforhugecalculationR:afreesoftwareenvironmentforstatisticalcomputingandgraphicsforstatisticianstoimplementnewstatisticalmethodspractitionerstoanalyzerealdatasetsinvariouseldsRecently,bothusersrequirehugeamountofcalculationfortheirownpurposesParallelcomputingisapracticalmethodforrealizinghugecalculationbyexecutingcalculationsonseveralcomputersand/ormanyCPUcoresatthesametimeIntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksParallelcomputingtechniquesonRParallelBLAS(BasicLinearAlgebraSubprograms)usingthreadsATLASFreeparallelandoptimizedBLASGotoBLASFastestparallelandoptimizedBLASIntelMKL,AMDACMLParallelandoptimizedBLASprovidedbyvendersMPItypelibrariesforRusingclusteredcomputersRpvmanRinterfacetoPVM(ParallelVirtualMachine)RmpianRinterfacetoMPI(MessagePassingInterface)snow(SimpleNetworkofWorkstations)ApackageforrealizingparallelcomputingbyparallelapplyfunctionsUsinglowerlevelparallellibrariessuchasSocket,MPI,PVM,nwsfortransferringdataamongprocessesAsitconcealsdierenceoflowerlevellibraries,itiseasytouseforparallelcomputing.
multicoreRunningparallelcomputationsinRonmachineswithmultiplecoresorCPUs.
.
.
.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksExistingWebenvironmentsforRRwebAWebbasedinterfacetoRforsubmittingthecodeRpadAworkbook-styleuserinterfacetoRthroughaWebbrowserrapacheEmbeddingRintheApacheWebserverRserveTCP/IPserverthatallowsotherprogramstousefacilitiesofRRWebServicesExposingRfunctionsasWebservicesthroughJava/Axis/Apache.
.
.
Parallelcomputingisnotthemainconcernoftheseprograms.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksSupercomputersinISMWehavethreesupercomputersystemsintheInstituteofStatisticalMathematics(ISM),Japan.
(Wewillreplacethemnextyear.
)Presentsupercomputersprovideparallelcomputingfacilities.
WeuseRonoursupercomputers.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksOurproblemsTroublesEachsupercomputerusesdierent(Unix-like)environment.
Unix-likeenvironmentsarenoteasytousefornovices.
Severalparametersforparallelcomputingneedtobespecieddierentlyforeachsupercomputer.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksOursolutionApproach:WebinterfaceWehavemade"Rdweb",aWebinterfacetoRforusingparallelcomputingfunctionsinRIntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksStructureofRdwebRdweb(RdaemonforWeb)systemconsistsofthreecomponents:Webinterface(viaWebbrowseronuser'scomputer)ItisrathersimpleandprogrammedbyHTMLandJavaScript.
JavaScriptisusedtoassistusers'inputslightly.
Webserver(onRdwebgatewaycomputer)ItisaCGIprogramforauthentication,letransfer,jobcontrol(start,stopandcheck),creationofJCL(JobControlLanguage)scriptandscatteringtheprogramtoremotecomputersasaclientofRdaemonRdaemon(onthefront-endcomputerofclustersystem)Itchecksauthentication,transfersrequiredles,startsandendsjobs,andshowsthestatus.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksCharacteristicsofRdwebRdwebisdesignedforsupercomputersandpersonalPCclustersystems.
AbovestatedthreecomponentsofRdwebandRslavescanresideondierentorsamecomputers.
Text-basedWebbrowserscanbeused(withalittlelimitation).
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksRdwebonsupercomputersinISMShared-MemoryDistributed-MemoryIntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksDierencesbetweenRwebandRdwebFromtheuserside,RdwebissimilartoRweb.
Rdwebcancontrolsystemresourcessuchasuser,CPU,memoryandqueue.
AlthoughRwebdoesnotallowtheuseof"system"commandfromthesecurityreason,RdwebdoesnothavesuchlimitationbecauseRdwebhasrigidauthenticationmechanism.
RwebandRdwebRwebRdwebAuthenticationnonePAM,NISorUnixpasswardFileuploadoneleAlotoflesControlofparallelBLASimpossibleEachsessionControlofsnowimpossibleEachsessionIntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksAuthenticationofRdweb(1)-WebserverRdwebadoptstwoauthenticationstages.
FirststageutilizesWebserverauthenticationmechanismwhentheuserisconnectedtotheWebserveronthegatewaycomputer.
ThemechanismisrealizedbymodauthpamofApache.
sites-enabledOptions.
.
.
.
AllowOverrideNoneOrderallow,denyAllowfromallAuthPAM_EnabledonAuthTypeBasicAuthName"RdwebUserLogin"Requirevalid-userIntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksAuthenticationofRdweb(2)-RdaemonAssecondstageofRdwebauthentication,RdaemonutilizesauthenticationmethodssuchasPAM(recommended),NISandUnixpassword.
WecanselectoneofthemwhenwecompileRdwebsystem.
CookiemustbeenabledintheWebbrowserforWebinterfaceofRdweb.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksPAMauthenticationPAM(PluggableAuthenticationModules)istheAPIforauthenticationusedinLinux,Solaris,MacOSXandAIX(5.
3orlater).
PAMusesNISorLDAPorUnixpassword.
IfPAMisnotavailable,NISorUnixpasswordcanbedirectlyusedforauthenticationinRdaemon.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksLocationofles"Rdweb"directoryiscreatedinthehomedirectoryonthefront-end.
Directoryforexecutionis/Rdweb/Uploadedlesarealsostoredin/Rdweb/Logsandscriptsarestoredin/Rdweb/YYYYMMDDhhmmss/whereYYYYMMDDhhmmssshowsyear,month,day,hour,minuteandsecond,accordingtotheISO-8601dateformat.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksUploadinglesTouploaddataand/orprogramles,weclick"Choose"button,selectale,andclick"upload"button.
Theseoperationscanberepeatedwithoutaectingeditedscriptandotherfunctions.
SCPorSFTPclientssuchasFilezillaclientarerecommendedforuploadinglargelesbecauseHTTPuploadsometimescausestimeoutandstops.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksPreparingdataandprogramByusingatexteditor,wepreparethefollowingdatale.
HW.
csvheight,weight1.
70,651.
85,801.
75,86Savethisleas"HW.
csv".
WealsoprepareRprogramBMI.
RBMI<-function(H,W){W/H^2}andsaveitas"BMI.
R".
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksInputUploadtwoles"HW.
csv"and"BMI.
R".
TheninputthefollowingRprograminputtextareaHW<-read.
csv("HW.
csv")source("BMI.
R")HWB<-cbind(HW,BMI=BMI(HW$height,HW$weight))HWBplot(HWB)intheeditorareaofWebinterfacewhichisconnectedtoRdwebgateway.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksExecutionJobisstartedbyclicking"Execute"button.
Jobstatusisshownin"JOBInformation".
Jobinformationisrefreshedbyclicking"Refresh"buttonortoptitle.
Resultsofcalculationarestoredasleswithextensions.
Rout(textformat)and.
pdf(pdfGraphics).
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksUseofsnowUsuallyinR,wehavetospecifythenumberofprocessesdierentlyaccordingtotheclustertype.
makeClusternormal#SOCKclustercl<-makeCluster(c("hostname1","hostname2"))#MPIclusterwith2slaveprocessescl<-makeCluster(2)Weaddnewfunction"setDefaultClusterOptions"touseparametersgivenintheWebinterfaceinthesamewayforallclustertypes.
makeClusterRdwebcl<-makeCluster(getClusterOption("spec"))IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksSelectionofparametersforparallelcomputingWeneedtoselectqueue,numberofslaveprocesses,numberofthreadsofparallelBLAS,andclustertypebyusingpull-downmenusinthisorder.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksExecutionJobisstartedbyclicking"Execute"button.
Creationofnewresultlesisshownbyclicking"Refresh"button.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksBatchsystemRdwebrequiresabatchsystem.
Severalbatchsystemsareavailable.
at,batchStandardbatchsystemofUnixspeciedinXPG4(X/OpenportabilityguideVer.
4).
Ithassimplequeuemechanism.
OpenPBS(NASAetc.
)Queuingandschedulingcontrolsystemforclustersystems.
Developmentstoppedin1998.
Torque(ClusterResourceInc.
)FreesystembasedonOpenPBSLoadLeveler(IBM)BatchsystembyavenderLSF(PlatformComputingInc.
)CommercialjobcontrollingtoolSLURMFreeresourcecontrolutilityIntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksPlatformsRdwebshouldworkonalmostallUnix-likeOSs.
WehavecheckedthefollowingsystemsinISMandCOM-ONE.
MPIOSBATCHSYSTEMHP-MPILinuxLSF+slurmLAM-MPILinuxTorqueOpenMPILinuxTorqueLAM-MPILinuxOpenPBSLAM-MPILinuxatLAM-MPISolarisatLAM-MPIAIXLoadLevelerLAM-MPIMacOSXatNote:Installationofthesebatchsystemsissometimescomplicated.
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksInstallationWekeepsourcecodesofRdwebathttp://prs.
ism.
ac.
jp/~nakama/rdweb/RequiredinstallationprocedurePreparetheskeletonoftheshellletoafront-endDenethesysteminformationonWebserverTheydependheavilyontheclustersystem.
Detailsofthesettinginformationcanbeseenin"README"leinRdwebarchive.
WeputrequiredpackagesforDebianGNU/Linux(Lenny)athttp://prs.
ism.
ac.
jp/~nakama/debian/lenny-ism/.
TheyincludehelperpackagesforGotoBLAS,Torque,andpackagesoflam-mpiandopenmpiforTorque.
(Unfortunately,thesearestillbuggy.
).
IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksExamplesinISM(1)IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksExamplesinISM(2)IntroductionRdwebsystemExamplesofexecutionInstallingRdwebConcludingremarksConcludingremarksAdvantagesofRdwebNovicescanuseparallelexecutionfunctionsofwithlesseorts.
NumberofparallelexecutioncanbespeciedeasilyforparallelBLASandsnow.
SecureauthenticationisavailablebyPAMwhichcanuseLDAPorNIS.
DisadvantagesofpresentRdwebSysteminstallationiscomplicatedandcompletelyplatformdependentFutureworkEncryptingcommunicationbetweenWebserverandRdaemonPortingtovariousRRwithmanyBLASsRcompiledbyseveralcompilersRonmanyOSs