integratingubuntu9.04

IEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING1AnOpenSourceSolutionforVirtualInfrastructureManagementinPrivateandHybridCloudsBorjaSotomayor,RubenS.
Montero,IgnacioM.
Llorente,andIanFosterAbstractMoreoftenthannot,a"cloud"referstoan"Infrastructure-as-a-Service"(IaaS)cloud,suchasAmazonEC2,whereITinfrastructureisdeployedinacloudprovider'sdatacenterintheformofvirtualmachines.
WiththegrowingpopularityofIaaSclouds,anecosystemoftoolsandtechnologiesisemergingthatcantransformanorganization'sexistinginfrastructureintoaprivatecloudorahybridcloud.
Inthisarticle,wepresentOpenNebula,anopensourcevirtualinfrastructuremanagerthatcanbeusedtodeployvirtualizedservicesonbothalocalpoolofresourcesandonexternalIaaSclouds,andHaizea,aresourceleasemanagerthatcanactasaschedulingbackendforOpenNebulaprovidingfeaturesnotfoundinothercloudsoftwareorvirtualization-baseddatacentermanagementsoftware,suchasadvancereservationsandresourcepreemption,whichwearguetobespeciallyrelevantforprivateandhybridclouds.
IndexTermsC.
0.
aEmergingtechnologies,D.
4.
7.
bDistributedsystemsCloudComputingis,touseyetanothercloud-inspiredpun,anebulouslydenedterm.
However,itwasarguablyrstpopularizedin2006byAmazon'sElasticComputeCloud(orEC2,seehttp://www.
amazon.
com/ec2/),whichstartedofferingvirtualmachines(VMs)for$0.
10/hourusingbothasimplewebinterfaceandaprogrammer-friendlyAPI.
Althoughnotthersttoproposeautilitycomputingmodel,AmazonEC2contributedtopopularizingthe"InfrastructureasaService"(IaaS)paradigm,whichbecamecloselytiedtothenotionofCloudComputing.
AnIaaScloudenableson-demandprovisioningofcomputationalresources,intheformofVMsdeployedinacloudprovider'sdatacenter(suchasAmazon's),minimizingoreveneliminatingassociatedcapitalcostsforcloudconsumers,allowingcapacitytobeaddedorremovedfromtheirITinfrastructureinordertomeetpeakoructuatingservicedemands,whileonlypayingfortheactualcapacityused.
Overtime,anecosystemofproviders,users,andtechnologieshascoalescedaroundthisIaaScloudmodel.
MoreIaaScloudproviders,suchasGoGrid,FlexiScale,andElasticHostshaveemerged.
AgrowingnumberofBorjaSotomayoriswiththeUniversityofChicago(Chicago,IL,USA)RubenS.
MonteroandIgnacioM.
LlorentearewiththeFacultaddeInformaticaoftheUniversidadComplutensedeMadrid(Madrid,Spain)IanFosteriswiththeUniversityofChicago(Chicago,IL,USA)andArgonneNationalLaboratory(Argonne,IL,USA)July7,2009DRAFTIEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING2companiesbasetheirITstrategyoncloud-basedresources,spendinglittleornocapitaltomanagetheirownITinfrastructure(seehttp://aws.
amazon.
com/solutions/case-studies/forseveralexamples).
Someproviders,suchasElastraandRightscale,focusondeployingandmanagingservicesontopofIaaSclouds,includingwebanddatabaseserversthatbenetfromtheelasticcapacityofIaaSclouds,allowingtheirclientstoprovisionservicesdirectly,insteadofhavingtoprovisionandsetuptheinfrastructurethemselves.
OtherprovidersofferproductsthatfacilitateworkingwithIaaSclouds,suchasrPath'srBuilder(http://www.
rpath.
org/),whichallowsdynamiccreationofsoftwareenvironmentstorunonacloud.
Althoughthisecosystemhasevolvedaroundpublicclouds—commercialcloudprovidersthatofferapublicly-accessibleremoteinterfacetocreateandmanagevirtualmachineinstanceswithintheirproprietaryinfrastructure—thereisalsoagrowinginterestinopen-sourceCloudComputingtoolsthatalloworganizationstobuildtheirownIaaScloudsusingtheirinternalinfrastructure.
TheprimaryaimoftheseprivateclouddeploymentsisnottosellcapacityovertheInternetthroughpublicly-accessibleinterfaces,buttoprovidelocaluserswithaexibleandagileprivateinfrastructuretorunserviceworkloadswithintheiradministrativedomain.
Privatecloudscanalsosupportahybridcloudmodelbysupplementinglocalinfrastructurewithcomputingcapacityfromanexternalpubliccloud.
Privateandhybridcloudsarenotexclusivewithbeingapubliccloud;aprivate/hybridcloudcanallowremoteaccesstoitsresourcesovertheInternetusingremoteinterfaces,suchasthewebservicesinterfacesusedinAmazonEC2.
However,toprovidethesamefeaturesfoundincommercialclouds,private/hybridcloudsoftwaremustmeetavarietyofrequirements:provideauniformandhomogeneousviewofvirtualizedresources,regardlessoftheunderlyingvirtualizationplatform(e.
g.
,Xen,KVM,VMWare,etc.
);managethefulllifecycleofavirtualmachine,includingsettingupnetworksdynamicallyforgroupsofVMsandmanagingthestoragerequirementsofVMs,suchasdeploymentofVMdiskimagesoron-the-ycreationofsoftwareenvironments;supportforcongurableresourceallocationpoliciestomeetthespecicgoalsoftheorganization(e.
g.
,highavailability,serverconsolidationtominimizepowerusage,etc.
);andadaptabilitytoanorganization'schangingresourceneeds,includingpeakswherelocalresourcesareinsufcient,andchangingresources,includingadditionorfailureofphysicalresources.
Thus,akeycomponentinprivate/hybridcloudswillbevirtualinfrastructure(VI)management,thedynamicorchestrationofvirtualmachinesonapoolofphysicalresources,meetingtherequirementsoutlinedabove.
Inthisarticle,wediscusstherelevanceofVImanagementnotjustforthecreationofprivate/hybridclouds,butalsowithintheemergingcloudecosystem.
WepresentOpenNebula[1](http://www.
opennebula.
org/)andHaizea[2](http://haizea.
cs.
uchicago.
edu/),twocomplementaryopensourceprojectsthatcanbeusedtomanagevirtualinfrastructuresinprivate/hybridclouds.
OpenNebulaisavirtualinfrastructuremanagerthatcanbeusedtodeployandmanagevirtualmachines,eitherindividuallyoringroupsthatmustbeco-scheduled,onlocalresourcesoronexternalpublicclouds,automatingthesetupofthevirtualmachines(preparingdiskimages,settingupnetworking,etc.
)regardlessoftheunderlyingvirtualizationlayer(Xen,KVM,orVMWarearecurrentlysupported)orexternalcloud(EC2orElasticHostsarecurrentlysupported).
HaizeaisaresourceleasemanagerthatcanactasaschedulingbackendforOpenNebula,providingleasingcapabilitiesnotfoundinothercloudsystems,suchasadvancereservationsandJuly7,2009DRAFTIEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING3Fig.
1.
TheCloudecosystemforbuildingprivatecloudsresourcepreemption,andwhichwearguetobespeciallyrelevantforprivateclouds.
I.
THECLOUDECOSYSTEMVirtualinfrastructuremanagementtoolsfordatacentershavebeenaroundsincebeforeCloudComputingbe-cametheindustry'snewbuzzword.
Severalofthese,suchasPlatformVMOrchestrator(http://www.
platform.
com/Products/platform-vm-orchestrator),andVMwarevSphere(www.
vmware.
com/products/vsphere/),andOvirt(http://ovirt.
org/),meetmanyoftherequirementsforVImanagementoutlinedearlier,providingfeaturessuchasdynamicplacementandmanagementofvirtualmachinesonapoolofphysicalresources,automaticloadbalancing,serverconsolidation,anddynamicresizingandpartitioningofinfrastructure.
Thus,althoughcreatingwhatisnowcalleda"privatecloud"wasalreadypossiblewithexistingtools,thesetoolslackotherfeaturesthatarerelevantforbuildingIaaSclouds,suchaspubliccloud-likeinterfaces,mechanismstoaddsuchinterfaceseasily,ortheabilitytodeployVMsonexternalclouds.
Ontheotherhand,projectslikeGlobusNimbus[3](http://workspace.
globus.
org/)andEucalyptus[4](http://www.
eucalyptus.
com/),whichwetermcloudtoolkits,canbeusedtotransformexistinginfrastructureintoanIaaScloudwithcloud-likeinterfaces.
EucalyptusiscompatiblewithAmazon'sEC2interfaceandisdesignedtosupportadditionalclient-sideinterfaces.
GlobusNimbusexposesEC2andWSRFinterfacesandoffersself-conguringvirtualclustersupport.
However,althoughthesetoolsarefullyfunctionalwithrespecttoprovidingcloud-likeinterfacesandhigher-levelfunctionalityforsecurity,contextualizationandVMdiskimagemanagement,theirVImanagementcapabilitiesarelimitedandlackthefeaturesofsolutionsthatspecializeinVImanagement.
Thus,anecosystemofcloudtoolsisstartingtoform(seeFigure1)wherecloudtoolkitsattempttospanbothcloudmanagementandVImanagementbut,byfocusingontheformer,donotdeliverthesamefunctionalityofJuly7,2009DRAFTIEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING4softwarewrittenspecicallyforVImanagement.
AlthoughintegratingcloudmanagementsolutionswithexistingVImanagerswouldseemliketheobvioussolution,thisiscomplicatedbythelackofopenandstandardinterfacesbetweenthetwolayers,andthelackofcertainkeyfeaturesinexistingVImanagers(enumeratedbelow).
Thefocusofourworkis,therefore,toproduceaVImanagementsolutionwithaexibleandopenarchitecturethatcanbeusedtobuildprivate/hybridclouds.
Withthisgoalinmind,westarteddevelopingOpenNebulaandcontinuetoenhanceitaspartoftheEuropeanUnion'sRESERVOIRProject(http://www.
reservoir-fp7.
eu/),whichaimstodevelopopensourcetechnologiestoenabledeploymentandmanagementofcomplexITservicesacrossdifferentadministrativedomains.
OpenNebulaprovidesmuchofthefunctionalityfoundinexistingVImanagers,butalsoaimstoovercometheshortcomingsinotherVIsolutions.
Namely,(i)theinabilitytoscaletoexternalclouds,(ii)monolithicandclosedarchitecturesthatarehardtoextendorinterfacewithothersoftware,notallowingitsseamlessintegrationwithexistingstorageandnetworkmanagementsolutionsdeployedindatacenters,(iii)alimitedchoiceofpreconguredplacementpolicies(rstt,roundrobin,etc.
),and(iv)lackofsupportforscheduling,deploying,andconguringgroupsofVMs(e.
g.
,agroupofVMsrepresentingacluster,whichmustallbedeployed,ornotatall,andwherethecongurationofsomeVMsdependsonthecongurationofothers,suchasthehead-workerrelationshipincomputeclusters).
TableIprovidesamoredetailedcomparisonbetweenOpenNebulaandseveralwell-knownVImanagers,includingcloudtoolkitsthatperformVImanagement.
AkeyfeatureofOpenNebula'sarchitecture,describedinmoredetailinthenextsection,isitshighlymodu-lardesign,whichfacilitatesintegrationwithanyvirtualizationplatformandthird-partycomponentinthecloudecosystem,suchascloudtoolkits,virtualimagemanagers,servicemanagers,andVMschedulers.
Forexample,alltheactionspertainingtosettingupaVMdiskimage(transferringthediskimage,installingsoftwareonit,etc.
)arespeciedintermsofwell-denedhooks;althoughOpenNebulaincludesadefault"transfermanager"thatusesthesehooks,itispossibletoleverageexistingtransfermanagersorVMimagecontextualizersjustbywritingcodethatinterfacesbetweenthehooksandthethird-partysoftware.
TheHaizeaproject,describedindetailinSectionIIIanddevelopedindependentlyfromOpenNebula,wastherstprojecttoleveragesuchanarchitectureinawaythatwasbenecialtobothprojects.
Haizea,anopen-sourceVM-basedleasemanagerwhichoriginallycouldonlysimulateVMschedulingforresearchpurposes,wasmodiedtoactasadrop-inreplacementforOpenNebula'sdefaultscheduler,withfewmodicationsrequiredintheHaizeacodeandnoneintheOpenNebulacode.
Byworkingtogether,OpenNebulawasabletoofferresourceleases,suchasadvancereservationleases,asafundamentalprovisioningabstraction,andHaizeawasabletooperatewithrealhardwarethroughOpenNebula.
Infact,theintegrationofOpenNebulaandHaizeaprovidestheonlyVImanagementsolutionofferingadvancereservationofcapacity.
AsshowninTableI,otherVImanagersuseimmediateprovisioning,whereVMsmustbeallocatedrightawayornotatall,orbest-effortprovisioning,whereVMsarequeueduntilresourcescanbeallocated.
However,privateclouds,speciallythosewithlimitedresourceswherenotallrequestsmaybesatisableimmediatelyforlackofresources,standtobenetfrommoresophisticatedVMplacementstrategiessupportingqueues,priorities,July7,2009DRAFTIEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING5andadvancereservations.
Additionally,Serviceprovisioningclouds,suchastheonebeingdevelopedintheRESERVOIRproject,haverequirementsthatcannotbesupportedonlywithanimmediateprovisioningmodel,includingtheneedforcapacityreservationsatspecictimestomeetservice-levelagreementsorpeakcapacityrequirements.
OtherintegrationeffortswithOpenNebulaarecurrentlyunderway(seehttp://opennebula.
org/doku.
phpid=ecosystem),includinganimplementationofthelibvirtinterface(http://libvirt.
org/)andaVMconsolidationschedulerdesignedtominimizeenergyconsumption.
TheRESERVOIRprojectisalsodevelopingothertoolsaroundOpenNebulaforserviceelasticitymanagement,VMplacementtomeetSLAcommitments,supportforpubliccloudinterfaces,andaVMscheduler(termed"policyengine"withintheproject)thataddssupportforpolicy-drivenprobabilisticadmissioncontrolanddynamicplacementoptimizationtosatisfysite-levelmanagementpolicies.
TherehasalsobeenanexperimentalintegrationofOpenNebulawithGlobusNimbus.
II.
THEOPENNEBULAARCHITECTURETheOpenNebulaarchitecture,depictedingure2,encompassesseveralcomponentsspecializedindifferentaspectsofvirtualinfrastructuremanagement.
Fig.
2.
OpenNebulavirtualinfrastructureenginecomponentsanditsintegrationwithanexternalcloudprovider.
InordertocontrolthelifecycleofaVM,theOpenNebulaCoreorchestratesthreedifferentmanagementareas,namely:(i)imageandstoragetechnologies(e.
g.
virtualappliancetoolsordistributedlesystems)topreparediskimagesforVMs;(ii)thenetworkfabric(e.
g.
DHCPservers,rewallsorswitches)toprovidetheVMswithaJuly7,2009DRAFTIEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING6TABLEICOMPARISONOFTOOLSPROVIDINGVIRTUALINFRASTRUCTUREMANAGEMENTCAPABILITIESToolProvisioningModelDefaultPlacementPoliciesCongurablePlacementPoliciesSupportforHy-bridCloudRemoteinterfacesAmazonEC2Best-effortProprietaryProprietaryNoEC2WSAPIVMWarevSphereImmediateInitialplacementonCPUload,anddynamicplace-menttobalanceaverageCPUormemoryloadandtoconsolidateserversNoOnlywhenboththelocalandex-ternalcloudusevSpherevCloudAPIPlatformOrchestratorImmediateInitialplacementonCPUloadandmigrationpoliciesbasedonpolicythresholdsonCPUutilizationlevelNoNoNoNimbusImmediateStaticgreedyandroundrobinresourceselectionNoIncludesa"EC2backend"thatcanforwardrequeststoEC2,butlo-calandremoteresourceshavetobemanagedsep-arately.
EC2WSAPIandNimbusWSRFEucalyptusImmediateStaticgreedyandroundrobinresourceselectionNoNoEC2WSAPIoVirtImmediateManualmodeNoNoNoOpenNebula1.
2Best-effortInitialplacementbasedonarequirement/rankpoliciestoprioritizethoseresourcesmoresuitablefortheVMus-ingdynamicinformation,anddynamicplacementtocon-solidateserversSupportforanystatic/dynamicplacementpolicyDriver-basedarchitectureallowsinterfacingwithmultipleexternalclouds.
EC2-compatiblecloudsandElasticHostsarecurrentlysupported.
NoOpenNebula1.
2/HaizeaImmediate,Best-effortandAdvancereservationDynamicplacementtoim-plementadvancereserva-tionleasesVMplacementstrategiessupportingqueuesandprioritiesOpenNebula1.
2/RESERVOIRImmediateandBest-effortLoadbalancingandpowersavingpoliciesSupportforpolicy-drivenprobabilisticadmissioncontrolanddynamicplacementoptimizationtosatisfysite-levelmanagementpoliciesRESERVOIRVMIJuly7,2009DRAFTIEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING7virtualnetworkenvironment;and(iii)theunderlyinghypervisorstocreateandcontrolVMs.
TheCoreperformsspecicstorage,networkorvirtualizationoperationthroughpluggableDrivers.
Thus,OpenNebulaisnottiedtoanyspecicenvironment,providingauniformmanagementlayerregardlessoftheunderlyinginfrastructure.
BesidesmanagingthelifecycleofindividualVMs,theCoreisalsodesignedtosupportdeploymentofservices,whichtypicallyincludeofasetofinterrelatedcomponents(e.
g.
webserver,DBbackend,etc.
)requiringseveralVMs.
Thus,agroupofrelatedVMscanbetreatedasarst-classentityinOpenNebula.
BesidesmanagingtheVMsasaunit,theCorealsohandlesdeliveryofcontextinformation(e.
g.
theIPaddressofthewebserver,digitalcerticates,softwarelicenses,etc.
)totheVMs.
VirtualMachineplacementdecisionsaremadebyaseparateSchedulercomponent.
Morespecically,theschedulerhasaccesstoinformationonallrequestsreceivedbyOpenNebulaand,basedonthem,isresponsibleforkeepingtrackofcurrentandfutureallocations,creatingandupdatingaresourceschedule,andsendingtheappropriatedeploymentcommandstotheOpenNebulaCore.
TheOpenNebuladefaultschedulerprovidesarankschedulingpolicythatplacesVMsonphysicalresourcesaccordingtoarankingalgorithmthatishighlycongurablebytheadministrator,andreliesonreal-timedatafromboththerunningVMsandavailablephysicalresources.
TheHaizealeasemanager,describedinthenextsection,canbeusedinplaceofthedefaultscheduler.
OpenNebulaoffersManagementInterfacestointegratetheCorefunctionalitywithinotherdatacentermanagementtools,suchasaccountingormonitoringframeworks.
Tothisend,OpenNebulaimplementsthelibvirtAPI,anopeninterfaceforVMmanagement,andacommandlineinterface(CLI).
AlsoasubsetofthisfunctionalitycanbeexposedtoexternalusersthroughaCloudInterface.
Finally,OpenNebulacansupportahybridcloudmodelbyusingCloudDriverstointerfacewithexternalclouds.
Inthisway,thelocalinfrastructurecanbesupplementedwithcomputingcapacityfromapubliccloudtomeetpeakdemands,tobetterserveuseraccessrequests(e.
g.
bymovingtheserviceclosertotheuser),ortoimplementhighavailabilitystrategies.
OpenNebulacurrentlyincludesanEC2driver,whichcansubmitrequeststoAmazonEC2andEucalyptus,andanElasticHostsdriver.
III.
THEHAIZEALEASEMANAGERHaizeaisanopen-sourceresourceleasemanager,andcanactasaVMschedulerforOpenNebulaorusedonitsownasasimulatortoevaluatetheperformanceofdifferentschedulingstrategiesovertime.
ThefundamentalresourceprovisioningabstractioninHaizeaisthelease.
Intuitively,aleaseissomeformofcontractwhereonepartyagreestoprovideasetofresources(anapartment,acar,etc.
)toanotherparty.
WhenauserwantstorequestcomputationalresourcesfromHaizea,itdoessointheformofalease,whicharethenimplementedasVMsmanagedbyOpenNebula.
TheleasetermssupportedbyHaizeaincludethehardwareresources,softwareenvironments,andtheavailabilityperiodduringwhichthehardwareandsoftwareresourcesmustbeavailable.
Currently,Haizeasupportsadvancereservationleases,wheretheresourcesmustbeavailableataspecictime;best-effortleases,whereresourcesareprovisionedassoonaspossible,andrequestsareplacedonaqueueifnecessary;andimmediateleases,whereresourcesareprovisionedwhenrequested,ornotatall.
July7,2009DRAFTIEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING8Advancereservationofcomputationalresources,oneofthesalientfeaturesinOpenNebulaandHaizea,hasbeenpreviouslystudiedinthecontextofparallelcomputing[5],[6],[7]and,intheabsenceofsuspension/resumptioncapabilities,isknowntoproduceresourceunderutilizationduetotheneedtovacateresourcesbeforeanadvancereservation(AR)starts.
Byusingvirtualmachinestoimplementleases,ARscanbesupportedmoreefciently[8],[2]throughresourcepreemption,suspendingtheVMsoflower-priorityleasesbeforeareservationstarts,resumingthemafterthereservationends,andpotentiallymigratingthemtootheravailablenodesorevenotherclouds.
Althoughresourcepreemptionhasalsobeenstudiedinthecontextofparallelcomputing,VMshavetheattractivequalityofallowingcomputationtobesuspendedwithouthavingtomaketheapplicationsinsidetheVMawarethattheyaregoingtobesuspended,resumed,orevenmigrated.
However,usingVMsintroducesoverheadsthatposeadditionalschedulingchallenges.
Morespecically,thepreparationoverheadofdeployingtheVMdiskimagesneededbytheleasecanhaveanoticeableimpactonperformanceifnotadequatelymanaged[9].
Haizea'sapproachistoseparatelyschedulethisoverhead,insteadofassumingitshouldjustbedeductedfromauser'sallocation.
However,thisiscomplicatedbyhavingtosupportmultipletypesofleaseswithconictingrequirementsthatmustbereconciled;thetransfersforaleasestartingat2pmcouldrequiredelayingtransfersforbest-effortleases,resultinginlongerwaitingtimes.
Haizeausesanumberofoptimizations,suchasreusingdiskimagesacrossleases,tominimizetheimpactofpreparationoverhead.
Similarly,Haizeaalsoschedulesruntimeoverhead,suchasVMsuspensions,resumptionsandmigrations,whichmayalsoberequiredtonishbeforeaspecictime.
HaizeabasesitsschedulingonaresourceslottablethatrepresentsallthephysicalnodesmanagedbyHaizeaovertime.
Best-effortleasesarescheduledusingaFirst-Come-First-Servequeuewithbacklling(acommonoptimizationinqueue-basedsystems),whileARleasesuseagreedyalgorithmtoselectphysicalresourcesthatminimizethenumberofpreemptions.
Althoughtheresourceselectionalgorithmiscurrentlyhardcoded,futureversionswillincludeapolicydecisionmoduletoallowdeveloperstospecifytheirownresourceselectionpolicies(e.
g.
,policiestoprioritizeleasesbasedonuser,group,project,etc.
).
Thispolicydecisionmodulewillalsobeusedtospecifytheconditionsunderwhichaleaseshouldbeacceptedorrejected.
IV.
EXPERIENCESWITHOPENNEBULAANDHAIZEAAlthoughOpenNebulaandHaizeabothoriginatedinresearchprojects,oneofourgoalsistoproduceproduction-qualityreleasesthatmeettheneedsofothercommunities.
Infact,wefeelstronglyaboutusingadevelopmentmodelthat,rstandforemost,producesstablesoftware,suitableforproductionenvironments,whichwecanalsouseforourownresearch,incorporatingtheresultsofourresearchintothenextstableversion.
Thisallowsustosupporttherequirementsofvirtualinfrastructureusers,whileincorporatingnoveltechniquesandsolutionsintoourreleases.
OpenNebulahasalreadyseenseveralstablereleases,andhasagrowinguserbasethroughitsinclusioninthepopularUbuntuGNU/Linuxdistribution(http://www.
ubuntu.
com/)startingwithUbuntu9.
04("JauntyJackalope").
Ourrst-handexperienceswithOpenNebulahavemostlytakenplaceintheEURESERVOIRproject,whereOpenNebulaisbeingenhancedtomeettherequirementsofseveralbusinessusecases[10].
Inrecentwork,wehaveshownOpenNebulatobeaneffectivetoolformanagingclusteredservices,usingittodeployandmanagethebackendJuly7,2009DRAFTIEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING9nodesofaSunGridEnginecomputecluster[1]andofaNGINXwebserver[11]onbothlocalresourcesandanexternalcloud.
Haizeaisstillina"technologypreview"stage,althougharststablereleaseisplannedlaterin2009.
Inpreviousjointwork[2]withK.
Keahey(ArgonneNationalLaboratory),weusedHaizeatosimulate7230-dayworkloadsinsixdifferentcongurations,or36yearsofleasescheduling,producingexperimentalresultsshowingthat,whenusingworkloadsthatcombinebest-effortandadvancereservationrequests,aVM-basedapproachwithsuspend/resumecanovercometheutilizationproblemstypicallyassociatedwiththeuseofadvancereservations.
Morespecically,whenmeasuringthetotaltimerequiredtoprocessalltherequestsintheworkload,wefoundthataVM-basedapproachperformedconsistentlybetter(upto32.
97%),despitetheoverheadofusingVMs.
OurresultsalsoshowedthatthepreparationoverheadofVMs,intheformoftransferringVMdiskimagesfromarepository,canbeminimizedthroughtheuseofimagetransferschedulingandcachingstrategies.
Inmorerecentwork[12],[13],wehaveusedOpenNebulaandHaizeatogethertoperformexperimentsonaphysicaltestbedanddeveloparesourcemodelforpredictingtheruntimeoverheadofsuspending/resumingVMsunderavarietyofcongurations.
Wefoundthat,similarlytoschedulingpreparationoverhead,explicitlyandseparatelyschedulingsuspensionsandresumptionsisnecessarytoavoidunnecessarydelaysinleases(e.
g.
,ifaleasemustbesuspendedtomakewayforahigher-prioritylease,likeanadvancereservation).
Furthermore,wefoundthataccuratelyestimatingthetimetosuspendandresumeleasesdependsonavarietyoffactorsthatmustbetakenintoaccountwhenschedulingleases.
V.
CONCLUSIONAsinterestinprivateandhybridIaaScloudsgrows,sowilltheneedforadiverseecosystemoftoolsandtechnologiesthatcanbeusedasbuildingblockstocreateandmanagetheseclouds.
Althoughsomesolutionshaveemergedacrossthreebroadcategories–cloudmanagement,virtualinfrastructuremanagement,andvirtualmachinemanagement–,thechallengeaheadwillbeintegratingmultiplecomponentstocreatecompleteIaaScloud-buildingsolutions.
Privateandhybridcloudswillalsofacethechallengeofefcientlymanaginganiteamountofresources.
However,existingvirtualinfrastructuremanagersrelyonanimmediateresourceprovisioningthatimplicitlyassumesthatcapacityispracticallyinnite.
Whilethisisafairassumptionforlargecloudproviders,suchasAmazonEC2whichfrequentlyreferstoacloudashaving"innitecapacity",itisnotapplicabletosmallerproviderswherethelikelihoodofbeingoverloadedisgreater.
Inordertosatisfyservice-levelagreements,requestsforresourceswillinevitablyhavetobeprioritized,queued,pre-reserved,deployedonexternalclouds,orevenrejected,andVImanagementsolutionswiththesecapabilitieswillberequired.
HerewehavepresentedOpenNebulaandHaizea,twoopensourceprojectsthataddressthesetwochallenges.
Byrelyingonaexible,open,andlooselycoupledarchitecture,OpenNebulaisdesignedfromtheoutsettobeeasytointegratewithothercomponents,suchastheHaizealeasemanager.
Whenusedtogether,OpenNebulaandHaizeaaretheonlyvirtualinfrastructuremanagementsolutionthatprovideleasingcapabilitiesbeyondimmediateprovisioning,includingbest-effortleasesandadvancereservationofcapacity.
July7,2009DRAFTIEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING10ACKNOWLEDGMENTSWegratefullyacknowledgethehardworkoftheOpenNebuladevelopers:JavierFontanandTinoVazquez.
Wealsothanksouranonymousreviewersfortheirinsightfulanddetailedcomments.
DevelopmentofOpenNebulaissupportedbyConsejeradeEducaciondelaComunidaddeMadrid,FondoEuropeodeDesarrolloRegional(FEDER)andFondoSocialEuropeo(FSE),throughBIOGRIDNETResearchProgramS-0505/TIC/000101,byMinisteriodeEducacionyCiencia,andthroughtheresearchgrantTIN2006-02806,andbytheEuropeanUnionthroughtheresearchgrantRESERVOIRGrantNumber215605.
DevelopmentofHaizeaissupportedbyRESERVOIR,theUniversityofChicago,andtheU.
S.
DepartmentofEnergyunderContractDE-AC02-06CH11357.
EarlyworkonHaizeawasdoneincollaborationwithK.
Keahey(ArgonneNationalLaboratory)andfundedbyNSFgrant#509408"VirtualPlaygrounds".
REFERENCES[1]I.
Llorente,R.
Moreno-Vozmediano,andR.
Montero,"Cloudcomputingforon-demandgridresourceprovisioning,"ToappearinAdvancesinParallelComputing,2009.
[2]B.
Sotomayor,K.
Keahey,andI.
Foster,"Combiningbatchexecutionandleasingusingvirtualmachines,"inHPDC'08:Proceedingsofthe17thInternationalSymposiumonHighPerformanceDistributedComputing.
ACM,2008,pp.
87–96.
[3]K.
Keahey,I.
Foster,T.
Freeman,andX.
Zhang,"Virtualworkspaces:Achievingqualityofserviceandqualityoflifeonthegrid,"ScienticProgramming,vol.
13,no.
4,pp.
265–276,2005.
[4]D.
Nurmi,R.
Wolski,C.
Grzegorczyk,G.
Obertelli,S.
Soman,L.
Youseff,andD.
Zagorodnov,"Theeucalyptusopen-sourcecloud-computingsystem,"inCloudComputingandApplications2008(CCA08),2008.
[5]I.
Foster,C.
Kesselman,C.
Lee,R.
Lindell,K.
Nahrstedt,andA.
Roy,"Adistributedresourcemanagementarchitecturethatsupportsadvancereservationsandco-allocation,"inProceedingsoftheInternationalWorkshoponQualityofService,1999.
[6]W.
Smith,I.
Foster,andV.
Taylor,"Schedulingwithadvancedreservations,"inIPDPS'00:Proceedingsofthe14thInternationalSymposiumonParallelandDistributedProcessing.
IEEEComputerSociety,2000,p.
127.
[7]Q.
Snell,M.
J.
Clement,D.
B.
Jackson,andC.
Gregory,"Theperformanceimpactofadvancereservationmeta-scheduling,"inIPDPS'00/JSSPP'00:ProceedingsoftheWorkshoponJobSchedulingStrategiesforParallelProcessing.
London,UK:Springer-Verlag,2000,pp.
137–153.
[8]B.
Sotomayor,K.
Keahey,I.
Foster,andT.
Freeman,"Enablingcost-effectiveresourceleaseswithvirtualmachines,"inHotTopicssessioninACM/IEEEInternationalSymposiumonHighPerformanceDistributedComputing2007(HPDC2007),2007.
[9]B.
Sotomayor,K.
Keahey,andI.
Foster,"Overheadmatters:Amodelforvirtualresourcemanagement,"inVTDC'06:Proceedingsofthe1stInternationalWorkshoponVirtualizationTechnologyinDistributedComputing.
IEEEComputerSociety,2006,p.
5.
[10]B.
Rochwerger,D.
Breitgand,E.
Levy,A.
Galis,K.
Nagin,I.
Llorente,R.
Montero,Y.
Wolfsthal,E.
Elmroth,J.
Caceres,M.
Ben-Yehuda,W.
Emmerich,andF.
Galan,"Thereservoirmodelandarchitectureforopenfederatedcloudcomputing,"IBMSystemsJournal,October2008.
[11]R.
Moreno,R.
Montero,andI.
Llorente,"Elasticmanagementofcluster-basedservicesinthecloud,"ToappearinProceedingsoftheFirstWorkshoponAutomatedControlforDatacentersandClouds(ACDC2009),June2009.
[12]B.
Sotomayor,R.
S.
Montero,I.
M.
Llorente,andI.
Foster,"Capacityleasingincloudsystemsusingtheopennebulaengine,"inCloudComputingandApplications2008(CCA08),2008.
[13]B.
Sotomayor,R.
Montero,I.
Llorente,andI.
Foster,"Resourceleasingandtheartofsuspendingvirtualmachines,"ToappearinProceedingsoftheThe11thIEEEInternationalConferenceonHighPerformanceComputingandCommunications(HPCC-09),June2009.
July7,2009DRAFTIEEEINTERNETCOMPUTING,SPECIALISSUEONCLOUDCOMPUTING11BIOGRAPHIESBorjaSotomayorisaPhDcandidateintheDepartmentofComputerScienceattheUniversityofChicago.
Hisresearchinterestsincluderesourceprovisioningandscheduling,distributedsystems,andvirtualization.
SotomayorhasaM.
Sc.
degreefromtheUniversityofChicago,andaComputerEngineeringdegreefromtheUniversityofDeusto(Bilbao,Spain).
Contacthimatborja@cs.
uchicago.
edu.
RubenSantiagoMonteroisanAssociateProfessorintheDepartmentofComputerArchitectureattheCom-plutenseUniversityofMadrid.
Hisresearchinterestsliemainlyinresourceprovisioningmodelsfordistributedsystems,inparticular:Gridresourcemanagementandscheduling,distributedmanagementofvirtualmachinesandcloudcomputing.
RubenhasaPhDinphysics(computerscienceprogram)fromComplutenseUniversity.
Contacthimatrubensm@dacya.
ucm.
esIgnacioMartnLlorenteisaFullProfessorandtheHeadoftheDistributedSystemsArchitectureResearchgroupattheComplutenseUniversityofMadrid.
Hisresearchinterestsincludeadvanceddistributedcomputingandvirtualizationtechnologies,architectureoflarge-scaledistributedinfrastructuresandresourceprovisioningplatforms.
IgnaciohasaPhDinComputerScienceandaExecutiveMasterinBusinessAdministration.
Contacthimatllorente@dacya.
ucm.
es.
IanFosterisdirectoroftheComputationInstituteattheUniversityofChicagoandArgonneNationalLaboratoryandtheArthurHollyComptonDistinguishedServiceProfessorofcomputerscienceattheUniversityofChicago.
Hisresearchinterestsincludedistributedcomputing,parallelcomputing,andcomputationalscience.
FosterhasaPhDincomputersciencefromImperialCollege,London.
Contacthimatfoster@anl.
gov.
ThesubmittedmanuscripthasbeencreatedinpartbyUChicagoArgonne,LLC,OperatorofArgonneNationalLaboratory("Argonne").
Argonne,aU.
S.
DepartmentofEnergyOfceofSciencelaboratory,isoperatedunderContractNo.
DE-AC02-06CH11357.
TheU.
S.
Governmentretainsforitself,andothersactingonitsbehalf,apaid-upnonexclusive,irrevocableworldwidelicenseinsaidarticletoreproduce,preparederivativeworks,distributecopiestothepublic,andperformpubliclyanddisplaypublicly,byoronbehalfoftheGovernment.
July7,2009DRAFT