serverindexnslookup

CopyrightIBMCorporation2006TrademarksMakeaquickerandeasierrecoveryfromanunplanneddeploymentmanagerfailoverPage1of7MakeaquickerandeasierrecoveryfromanunplanneddeploymentmanagerfailoverRohithAshokNovember22,2006Fewthingsaremorestressfulandfrustratingthananunplannedoutage.
ThisisespeciallytruewhentheunplannedoutageisthedeploymentmanagerofalargeNetworkDeploymentcell.
ThisarticlewalksthroughexamplesofsomefeaturesavailableinIBMWebSphereApplicationServerNetworkDeploymenttohelpyouprepareforaneasyrecoveryfromsuchanoutage.
IntroductionByitsverynature,WebSphereApplicationServerNetworkDeploymentisadistributedsystemrangingacrossmanymachines.
Whilefewthingsaremorestressfulandfrustratingthananunplannedoutage,therearewaysyoucanlessentheimpact.
Thegoalofthisarticleistoshowhowyoucanharnessthissystemandmakerecoveryaquickandsimpletask.
ThisarticleassumesyouhaveagoodunderstandingofWebSphereApplicationServerNetworkDeploymentandsomeknowledgeofnetworkconfiguration.
SamplecellForthepurposesofthisarticle,let'sassumewehaveacellwithfivenodes(Figure1).
Thenodecountisn'tsignificant,butthemorenodesyouhave,thegreatertheimpactwhenthedeploymentmanagerislost.
Figure1.
SamplecelldeveloperWorksibm.
com/developerWorks/MakeaquickerandeasierrecoveryfromanunplanneddeploymentmanagerfailoverPage2of7Thiscellisspreadacrossfivedifferentsystems:System1:Node1(1NodeAgent/20applicationservers)Host:node1.
samplecompany.
com(10.
10.
0.
1)System2:Node2(1NodeAgent/20applicationservers)Host:node2.
samplecompany.
com(10.
10.
0.
2)System3:Node3(1NodeAgent/20applicationservers)Host:node3.
samplecompany.
com(10.
10.
0.
3)System4:Node4(1NodeAgent/20applicationservers)Host:node4.
samplecompany.
com(10.
10.
0.
4)System5:DeploymentManagernodeHost:dmgr.
samplecompany.
com(10.
10.
0.
5)Oneachcluster,asingleapplicationisinstalledforatotalof20applications.
Ifthiswasaproductioncell,andthedeploymentmanager'smachinesimplydiedandwascompletelyunrecoverable,itwouldbeverydifficulttoreconstitutethiscellquickly.
Yourbestbetwouldbetotryandrestorethedeploymentmanagerfromsomebackup,assumingonewastaken.
Thedeploymentmanagerholdstheconfiguration--themasterrepository--fortheentirecell.
Ifyoulookatthefilesystemonthedeploymentmanager,youwillseeallfournodesdefined,aswellasthedeploymentmanagernode.
Thisconfigurationmakesuptheentirecell.
Listing1.
System5/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/dmgrNode/.
.
.
/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/node1/.
.
.
/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/node2/.
.
.
/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/node3/.
.
.
/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/node4/.
.
.
Theapplicationservernodes,however,donotneedtheentirecellconfiguration;rather,theyneedonlythesubsetoftheconfigurationthattheythemselvesrequire.
Eachindividualnodehastheentireconfigurationpertainingtoitsownnodeandjusttheserverindex.
xmloftheothernodes.
Listing2.
System1/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/node1/.
.
.
/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/node2/serverindex.
xml/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/node3/serverindex.
xml/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/node4/serverindex.
xml/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/dmgrNode/serverindex.
xmlMethod1:SetupthebackupnodeTheplanhereistouseanapplicationservernodeasabackup.
Byreplicatingtheentireconfigurationtreefromtheactualdeploymentmanagernodetoanotherapplicationservernode,ibm.
com/developerWorks/developerWorksMakeaquickerandeasierrecoveryfromanunplanneddeploymentmanagerfailoverPage3of7thedeploymentmanagercanbequicklyrestartedonthebackupnode.
Tobegintheprocess,youfirstneedanapplicationservernode.
Inaproductionenvironment,itisbestnottouseanodethatservesapplications,butrathertochooseanewnodededicatedasabackup.
A.
Createbackupnode1.
WewillfirstinstallWebSphereApplicationServeronanewsystem:System6.
Wheninstallingtheproduct,installitintothesamelocationonthefilesystemasthedeploymentmanagersystem.
IfyouinstalledWebSphereApplicationServerinthe/WebSphere/AppServerdirectoryonthedeploymentmanagersystem,makesureitisinstalledonthebackupsysteminthesameplace.
2.
CreateanewCustomorStandaloneprofile.
ThenodenamewewillgiveisbackupNode.
Then,federatethenewnodeintotheexistingcell.
Listing3.
System6/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/backupNode/.
.
.
System6:BackupnodeHost:backup.
samplecompany.
com(10.
10.
0.
6)Thisnode'spurposeisstrictlyforbackup/failover,notforhostinganyapplicationservers.
Whenthebackupdeploymentmanagerisrunning,itwillcreateitslogsintheexactsameplaceasdoesthedeploymentmanagersystem.
Eventhough/WebSphere/AppServer/profiles/Dmgr01/logshasnotbeencreatedbytheprofilemanagementtools,itwillbecreatedbytherunningserver.
B.
ConfigurebackupnodeRightnow,thebackupNodeisjustlikeanyothernode;itonlymanagesitsownconfigurationandasmallsubset(serverindex.
xml)fortheothernodes.
Tosetupthisnodeasabackup,youneedtoautomaticallyreplicatethedeploymentmanager'scell-wideconfiguration.
Todothis,youneedtosetupacustompropertyonthebackupNode'snodeagent,eitherusingtheadministrativeconsoleorbyusingasimplejythonscript.
Tousetheadministrativeconsole:3.
Thecustompropertycanbesetontheadministrativeconsole(Figure2)bynavigatingtoSystemAdministration=>NodeAgents=>nodeagent=>FileSynchronizationService=>CustomProperties=>New.
developerWorksibm.
com/developerWorks/MakeaquickerandeasierrecoveryfromanunplanneddeploymentmanagerfailoverPage4of7Figure2.
AddcustompropertyusingadministrativeconsoleThenewpropertytocreateis:Name:recoveryNodeValue:trueDescription:(optional)4.
Oncethispropertyisset,thenodeagentmustberestarted.
Thenexttimethenodeagentissynchronized,itwillextracttheentirecellconfiguration,notjustitsown.
Anytimeaconfigurationchangeismadeonthedeploymentmanagerandsynchronized,thisbackupNodewillalsodownloadthenewchange,regardlessofthenodethechangewasactuallydirectedto.
5.
YoudonotneedtocreateanyapplicationserversonthebackupNode.
Youonlyneedthenodeagentupandrunningtokeeptheconfigurationcurrent.
Alternatively,youcanuseajythonscripttosetthecustomproperty.
BelowisasimplescriptthatillustrateshowtosetthecustompropertyonthebackupNode'snodeagent.
(Beforerunning,changealloccurrencesofinthescripttotheactualnodename.
)Listing4.
Jythonscripttosetacustomproperty#ScripttocreatetherecoveryNodecustomproperty#Step1:Getahandletothenodeagentnodeagent=AdminConfig.
getid('/Node:/Server:nodeagent/')printnodeagent#Step2:GetahandletotheConfigSynchronizationServicesyncservice=AdminConfig.
list('ConfigSynchronizationService',nodeagent)printsyncservice#Step3:CreatethecustompropertyAdminConfig.
create("Property",syncservice,[["name","recoveryNode"],["value","true"]])#Step4:SaveAdminConfig.
saveibm.
com/developerWorks/developerWorksMakeaquickerandeasierrecoveryfromanunplanneddeploymentmanagerfailoverPage5of7C.
CreateascripttostartthedeploymentmanagerNowthatyouhavethedeploymentmanagerconfigurationautomaticallyreplicatedonthebackupNode,youneedawaytostartit.
YoucannotusethestandardStartServer.
shscript,asthescriptmakessomeassumptionsaboutrunningonthelocalnode.
StartServer[.
sh|.
bat]/startManager[.
sh|.
bat]worksbyreadingtheconfigurationofthelocalnodeandstartingaserveronthatnode.
IfyoutrytorunstartManager[.
sh|.
bat]onthebackupNode,itwillnotletyoutostartthedeploymentmanager;thedeploymentmanagerdoesnothavebackupNodeasitsnodename,butratherdmgrNode.
SinceyoucannotusetheexistingstartManagerscript,youmustcreateanotherscripttostarttheserver:6.
Run.
/startNode.
[sh|.
bat]-script.
Thiscommandwillgenerate:start_nodeagent.
sh(or.
batifWindows).
Youmusttheneditthescripttopointtothedeploymentmanagerconfigurationratherthanthenodeagent.
7.
Renamethescripttostart_backup_dmgr.
sh.
8.
Changethecontentsofthefile;thekeychangeisattheendofthefileintheexeccall:Listing5.
Modifystart_backup_dmgr.
shexec"/build/websphere/WASX/u0622.
06.
wasx/AppServer/java/bin/java".
.
.
"/build/websphere/WASX/u0622.
06.
wasx/AppServer/profiles/AppSrv01/config""productionCell""backupNode""nodeagent"""9.
Changethe"backupNode"todmgrNodeand"nodeagent"todmgrtoread:Listing6.
Modifystart_backup_dmgr.
shexec"/build/websphere/WASX/u0622.
06.
wasx/AppServer/java/bin/java".
.
.
"/build/websphere/WASX/u0622.
06.
wasx/AppServer/profiles/AppSrv01/config""productionCell""dmgrNode""dmgr"""Theoutputofyournewstart_backup_dmgr.
shisverydifferentthantheexpectedstartManager[.
sh|.
bat]output.
Thebackupdeploymentmanagershouldonlybeusedforoperationalcontrol,sinceanychangesmadetothesystemarenotreflectedbacktotheoriginaldeploymentmanager.
Method2:SharedfilesystemAsecondoptionisuseafilesystemthatissharedbetweentheprimarydeploymentmanagersystemandabackupsystem.
Youcansetupasecondsystemtosharethedeploymentmanagerprofileitself,andstartthedeploymentmanageronthatsecondsystem.
IfyouexamineaV6.
0.
XorV6.
1installation,allthedatapertainingtoadeploymentmanager(oranyprofileforthatmatter)isstoredundertheWAS_HOME/profiles/directory.
ConsidertheearlierexamplewherethedeploymentmanagerisinstalledonSystem5undertheHFS:/WebSphere/AppServer/profiles/Dmgr01/config/cells/productionCell/nodes/dmgrNode/.
.
.
Again,youneedtoinstallWebSphereApplicationServeronSystem6.
However,thistimeyoudonotneedtocreateanyprofiles,youcaninsteadsharethedeploymentmanager'sprofiledirectory.
developerWorksibm.
com/developerWorks/MakeaquickerandeasierrecoveryfromanunplanneddeploymentmanagerfailoverPage6of7BymountingtheshareddirectoryinexactlythesameplaceonSystem6,youhaveessentiallyduplicatedthesystem.
Theduplicationoftheprofileenablesthesamedeploymentmanagertobestartedontheothersystem.
Oncethisdirectoryisshared,youcanrunstartManager[.
sh|.
bat]fromthebackupsystem.
Thedownsidetothismethodisthesharedfilesystemmustbemaintainedbyyou.
Therecanalsobesomeconsiderableperformancecoststousingasharedfilesystem,bothonthemastersystemandonthebackupsystem.
Onemajoradvantageofthissystemisthatanychangesmadetothedeploymentmanager'smasterrepositorywhileusingthebackupsystemarecommitted.
Oncetheprimarydeploymentmanagermachineisrebuilt,thechangeswillnotbelostwhenthefilesystemisremounted.
ConfiguringDNSformulti-homedhostsOnceyouhavefoundawaytostartadeploymentmanageronaremotesystem,youstillmayhavesomeissuestoresolve.
Whenyouoriginallyconfiguredthatdeploymentmanager,aspecifichostthatresolvedtoaspecificIPwasgiven.
Inourexample,thehostnamedmgr.
samplecompany.
comresolvedto10.
0.
0.
5.
Toenablethedeploymentmanagertobelocatedonthebackupsystem(10.
0.
0.
6)weneedtosetupwhat'scalledamulti-homeDNSentry.
Inourexample,weareusingmultipleinterfacesacrossmultiplemachines,eachwithasingleIPaddresspermachine.
HereisanexampleofourDNSentry:Listing7.
DNSentryADMIN:SYSTEM6:/u/ADMIN>nslookupdmgr.
samplecompany.
comDefaultingtonslookupversion4Startingnslookupversion4Server:dnsserver.
samplecompany.
comAddress:10.
0.
0.
100Name:dmgr.
samplecompany.
comAddresses:10.
0.
0.
5,10.
0.
0.
6ByconfiguringtheDNSentryfortwohosts,itenablesaneasytransitionbetweenthemsincenore-configurationisneeded.
Aslongasonlyonesystemoneitherhostisrunningatanygiventime,onlyonedeploymentmanagerwillbeused.
Thenodeagentsandclientswillautomaticallytrythefirst(primary)addressandthenfailovertotrythesecond(backup)address.
Onceacellhasbeensetupinthisway,thedeploymentmanagerandallnodeagentsmustberestartedtore-readtheconfiguration.
Bydefault,theJVMcachesDNSentriesandtherestartisneededtorefreshthem.
ConclusionThisarticlediscussedonewayofbuildingamorefaulttolerantdeploymentmanagerusingfeaturesthatexistinWebSphereApplicationServerNetworkDeploymentV6.
x,anddescribedhowtouseeitheraWebSphereApplicationServerconfigurationoptionorasharedfilesystemtoautomaticallyreplicatetheentirecell-wideconfigurationtoabackupsystem.
ThisbackupsystemthenusesapreconfiguredDNSentrytostartupandreplacethefaileddeploymentmanager.
Thelossofadeploymentmanagerisagreatblowtoaproductioncell,andsoitiscriticalthatthisserverremainrunningandfunctionaltoensureahighlyavailabletopology.
ibm.
com/developerWorks/developerWorksMakeaquickerandeasierrecoveryfromanunplanneddeploymentmanagerfailoverPage7of7RelatedtopicsWebSphereApplicationServerInformationCenterSettingtherecoveryNodepropertyCopyrightIBMCorporation2006(www.
ibm.
com/legal/copytrade.
shtml)Trademarks(www.
ibm.
com/developerworks/ibm/trademarks/)