wwwsources.list

UsingOpenSourceSoftwaretoTeachMathematicalStatisticsDouglasM.
Batesbates@R-project.
orgUniversityofWisconsin–MadisonUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
1/29OutlineOpenSourceSoftwareforStatisticsWhatisRObtainingandinstallingRAnExample-MLEConclusionsfromtheexampleLinksUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
2/29StatisticalsoftwareinMathStatItiscommontousesomestatisticalsoftwareinappliedcoursestoworkwithrealisticsizeddatasetstcomplicatedmodelsgaininsightfromsimulationSoftwareinMathStatcourseshassimilaruses(especiallysimulation)butwemustprovideasimpleinterfaceforittobeuseful.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
3/29LinkingthecomputingtothetextItisimportantthattheuseofthecomputingsystembeintegratedwiththelecturesandtext.
Twowaysthiscanbeaccomplishedare:Writeatextthatistiedtospecicsoftwareandillustratestheuseofthatsoftware,e.
g.
NolanandSpeed(2000)Adoptaconventionaltextbutprovidetheexamplesfromthetextinthecomputingsystem.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
4/29AdvantagesofOpenSourcesoftwareNolicensefeesormanagementoflicenses.
Wecaninstallthesoftwareonallourdepartmentalcomputers.
Wedonotneedtowaitforacommericalsoftwareprovidertoporttoaparticularoperatingsystemorworryabouttheirdiscontinuingaport.
Itismucheasiertoconvinceothergroups(Computer-AidedEngineeringandtheComputingCenter,inmycase)toinstallthesoftware.
Studentscaninstallthesoftwareontheirowncomputerswithoutcharge(andwithoutviolatinglicenses).
OpenSourceprojectsencouragecontributionsfromuserssoextensionsareeasier.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
5/29TheSlanguageItisalanguageandsystemdevelopedbyJohnChambersandhisco-workersatBellLaboratories(formerlypartofAT&T,nowpartofLucentTechnologies).
TheAssociationforComputingMachinerypresentedits1999SoftwareSystemAwardtoJohnstating,"Shasforeveralteredthewaypeopleanalyze,visualize,andmanipulatedata"ItisthedefactostandardforcomputingintheStatisticsresearchcommunity.
Itisdocumentedinmanybooks,thebestknownofwhicharebyVenablesandRipley(ModernAppliedStatisticswithS-PLUSandSProgramming)andbyChambers(ProgrammingwithData).
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
6/29DesignGoalsforSToprovideanenvironmentforinteractivecomputingwithdata.
Toallowuserstotransitioneasilyintoprogrammersastheneedarises.
Toprovidebothexploratoryandpresentationgraphics.
Toavoidreinventingexistingtools.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
7/29CharacteristicsofSItisaninteractivelanguagebasedonfunctionsandfunctioncalls.
Itprovidesanenvironmentwithpersistent,self-describingobjects.
Inparticular,functionsarerst-classobjects.
Ithasawidevarietyofgraphicsdevicedriversandcontrols.
Itprovidesinterfacestoexistingcodeandsystems.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
8/29ImplementationsofSS-PLUS,soldbyInsightfulCorp.
(formerlyMathSoft,Inc.
)andbasedontheirexclusivelicenseoftheoriginalSsourcefromLucentTechnologies.
R,anOpenSourceprojectconformingforthemostparttothepublisheddescriptionsofS.
InitiallydevelopedbyRossIhakaandRobertGentlemanattheUniversityofAucklandNowdevelopedandmaintainedbyanwidely-dispersed,internationalgroupofvolunteersfromacademiaandindustry.
Operatesthroughwebsites(www.
r-project.
org),archives(cran.
r-project.
org)e-maillists,CVSsites,rsyncsites.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
9/29WhatisRUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29WhatisRAnOpenSourceimplementationofJohnChambers'award-winningSlanguageUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29WhatisRAnOpenSourceimplementationofJohnChambers'award-winningSlanguageAlanguageandenvironmentfordataanalysisandgraphicsUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29WhatisRAnOpenSourceimplementationofJohnChambers'award-winningSlanguageAlanguageandenvironmentfordataanalysisandgraphicsAmeansoftechnologytransferthroughpackagesUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29WhatisRAnOpenSourceimplementationofJohnChambers'award-winningSlanguageAlanguageandenvironmentfordataanalysisandgraphicsAmeansoftechnologytransferthroughpackagesAexibledataexchangemechanismaccessing:UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29WhatisRAnOpenSourceimplementationofJohnChambers'award-winningSlanguageAlanguageandenvironmentfordataanalysisandgraphicsAmeansoftechnologytransferthroughpackagesAexibledataexchangemechanismaccessing:textlesandsavedRworkspacesUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29WhatisRAnOpenSourceimplementationofJohnChambers'award-winningSlanguageAlanguageandenvironmentfordataanalysisandgraphicsAmeansoftechnologytransferthroughpackagesAexibledataexchangemechanismaccessing:textlesandsavedRworkspacesS-PLUSdataobjects,SASXPORTdatasets,SPSSsaveddatasets,Minitabworksheets,UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29WhatisRAnOpenSourceimplementationofJohnChambers'award-winningSlanguageAlanguageandenvironmentfordataanalysisandgraphicsAmeansoftechnologytransferthroughpackagesAexibledataexchangemechanismaccessing:textlesandsavedRworkspacesS-PLUSdataobjects,SASXPORTdatasets,SPSSsaveddatasets,Minitabworksheets,relationaldatabases–ODBC,PostgreSQL,MySQLUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29WhatisRAnOpenSourceimplementationofJohnChambers'award-winningSlanguageAlanguageandenvironmentfordataanalysisandgraphicsAmeansoftechnologytransferthroughpackagesAexibledataexchangemechanismaccessing:textlesandsavedRworkspacesS-PLUSdataobjects,SASXPORTdatasets,SPSSsaveddatasets,Minitabworksheets,relationaldatabases–ODBC,PostgreSQL,MySQLAnembeddableextensionlanguageUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29WhatisRAnOpenSourceimplementationofJohnChambers'award-winningSlanguageAlanguageandenvironmentfordataanalysisandgraphicsAmeansoftechnologytransferthroughpackagesAexibledataexchangemechanismaccessing:textlesandsavedRworkspacesS-PLUSdataobjects,SASXPORTdatasets,SPSSsaveddatasets,Minitabworksheets,relationaldatabases–ODBC,PostgreSQL,MySQLAnembeddableextensionlanguagePartoftheGNU(GNU'sNotUnix)softwaresystem.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29WhatisRAnOpenSourceimplementationofJohnChambers'award-winningSlanguageAlanguageandenvironmentfordataanalysisandgraphicsAmeansoftechnologytransferthroughpackagesAexibledataexchangemechanismaccessing:textlesandsavedRworkspacesS-PLUSdataobjects,SASXPORTdatasets,SPSSsaveddatasets,Minitabworksheets,relationaldatabases–ODBC,PostgreSQL,MySQLAnembeddableextensionlanguagePartoftheGNU(GNU'sNotUnix)softwaresystem.
Justinstallitandtryit.
Comeswithamoney-backguarantee.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
10/29HowdoIgetRUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
11/29HowdoIgetRTheinformationalwebsitehttp://www.
r-project.
org/UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
11/29HowdoIgetRTheinformationalwebsitehttp://www.
r-project.
org/CRAN-theComprehensiveRArchiveNetworkUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
11/29HowdoIgetRTheinformationalwebsitehttp://www.
r-project.
org/CRAN-theComprehensiveRArchiveNetworkTheprimarysiteishttp://cran.
r-project.
org/UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
11/29HowdoIgetRTheinformationalwebsitehttp://www.
r-project.
org/CRAN-theComprehensiveRArchiveNetworkTheprimarysiteishttp://cran.
r-project.
org/Mirrorsitesareavailableformanycountries,e.
g.
http://cran.
us.
r-project.
org/.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
11/29HowdoIgetRTheinformationalwebsitehttp://www.
r-project.
org/CRAN-theComprehensiveRArchiveNetworkTheprimarysiteishttp://cran.
r-project.
org/Mirrorsitesareavailableformanycountries,e.
g.
http://cran.
us.
r-project.
org/.
CRANsiteshavebinarydistributionsforWindows95,98,ME,NT4and2000onIntel,fortheMacintosh(System8.
6to9.
1andMacOSX),andforseveralLinuxdistributions.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
11/29HowdoIgetRTheinformationalwebsitehttp://www.
r-project.
org/CRAN-theComprehensiveRArchiveNetworkTheprimarysiteishttp://cran.
r-project.
org/Mirrorsitesareavailableformanycountries,e.
g.
http://cran.
us.
r-project.
org/.
CRANsiteshavebinarydistributionsforWindows95,98,ME,NT4and2000onIntel,fortheMacintosh(System8.
6to9.
1andMacOSX),andforseveralLinuxdistributions.
Newreleasesoccurfrequently-aboutevery3months.
Bepreparedtore-installfrequently.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
11/29InstallingRUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
12/29InstallingRWindowsDownloadandruntheinstaller,SetupR.
exe.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
12/29InstallingRWindowsDownloadandruntheinstaller,SetupR.
exe.
WindowsAlternatively,seetheminiRdirectoryforoppy-sizedimagesandtheirinstaller.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
12/29InstallingRWindowsDownloadandruntheinstaller,SetupR.
exe.
WindowsAlternatively,seetheminiRdirectoryforoppy-sizedimagesandtheirinstaller.
MacintoshThedistributionconsistsofonebinhexed(.
hxq)lethatyouexpandusingstandardtools.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
12/29InstallingRWindowsDownloadandruntheinstaller,SetupR.
exe.
WindowsAlternatively,seetheminiRdirectoryforoppy-sizedimagesandtheirinstaller.
MacintoshThedistributionconsistsofonebinhexed(.
hxq)lethatyouexpandusingstandardtools.
LinuxRPMlesareavailableforRedHat,SuSE,andMandrake.
DeblesareavailableforDebian.
UnderDebianyoucanlistaCRANarchivein/etc/apt/sources.
listforautomaticupdates.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
12/29InstallingRWindowsDownloadandruntheinstaller,SetupR.
exe.
WindowsAlternatively,seetheminiRdirectoryforoppy-sizedimagesandtheirinstaller.
MacintoshThedistributionconsistsofonebinhexed(.
hxq)lethatyouexpandusingstandardtools.
LinuxRPMlesareavailableforRedHat,SuSE,andMandrake.
DeblesareavailableforDebian.
UnderDebianyoucanlistaCRANarchivein/etc/apt/sources.
listforautomaticupdates.
UnixDownloadandexpandthecompressedtarleofthesources.
Run.
/congurethenmake;makecheck;makeinstallUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
12/29AnExample-maximumlikelihoodExampleisfromaMathematicalStatisticscourseforIndustrialEngineeringstudentsIusethetextProbabilityandStatisticsforEngineeringandtheSciences(5thed)byJayDevore(Duxbury,2000).
OurI.
E.
Departmentrequiresthatstudentslearnmaximumlikelihood.
Studentsarenotatallenthusiastic.
Asiscommon,thetextconvertstheproblemofmaximizingthelikelihoodorlog-likelihoodtotheproblemofsolvingthelikelihoodequations.
Derivingthelikelihoodequationssometimesinvolvestakingpartialderivativesofcomplicatedexpressions.
Thisiserror-prone.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
13/29WeibullexamplefromDevore(2000)LetbearandomsamplefromaWeibullpdf¤§¨¤otherwiseWritingthelikelihoodandln(likelihood),thensettingbothandyieldstheequations¨Thesetwoequationscannotbesolvedexplicitlytogivegeneralformulasforthemle'sand.
Instead,foreachsample,theequationsmustbesolvedusinganiterativenumericalprocedure.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
14/29WeibullexampleinRRegardtheproblemofcalculatingMLEsasanoptimizationproblem.
Usuallyitiseasiertooptimizethelog-likelihoodratherthanthelikelihooditself.
Theparametervaluesthatoptimizethelog-likelihoodarealsotheMLEs.
Probabilitydensityfunctions(probabilityfunctionsfordiscretedistributions)havenamesstartingwithd—dnorm,dbinom,dunif,dweibull.
Alldensityfunctionstakeanoptionalargumentlogwhich,whenTRUEcausesevaluationofthelog-density.
Therecipebecomes:Regardtheparametersasthevariablesandsumthelogdensityforyourdistributionusingyourdata.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
15/29AnRsessionontheWeibullexample>library(Devore5)>data(xmp04.
30)#importthedata>str(xmp04.
30)#examinethestructure'data.
frame':10obs.
of1variable:$lifetime:num2825017418511072.
.
.
>#Reasonablestartingestimatesareshape=1,scale=1000>#Doasimpleevaluationatthissetofparameters>sum(dweibull(xmp04.
30$lifetime,shape=1,scale=1000,log=TRUE))[1]-80.
47655>#Optimizationfunctionsminimizesousenegativelog-likelihood>llfuncmlestr(mle)#structureofthereturnedvalueListof6$minimum:num77.
1$estimate:num[1:2]2.
151289.
34$gradient:num[1:2]-1.
50e-052.
98e-08$hessian:num[1:2,1:2]3.
75e+00-3.
20e-03-3.
20e-032.
78e-05$code:int1$iterations:int20>solve(mle$hessian)#approximatevariance-covariancematrix[,1][,2][1,]0.
295964234.
06422[2,]34.
064220039835.
81998Weseethatthemaximumofthelog-likelihoodis,achievedatand.
Theapproximatestandarderrorsoftheestimatesareand.
Wecanusethestandarderrorstodetermineagridofvaluesforcontouringthelog-likelihoodfunction.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
17/29Plottingthedensityattheestimates>plot(function(x)dweibull(x,shape=2.
15,scale=1289.
34),0,3000,+col="red",xlab="lifetime(hr)",ylab="density",+main="WeibulldensityusingMLEsfromthelifetimedata")>rug(xmp04.
30$lifetime,col="blue")0500100015002000250030000e+003e046e04WeibulldensityusingMLEsfromthelifetimedatalifetime(hr)densityUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
18/29Contouringthelog-likelihoodfunction>gridscvalsshvalsfor(iinseq(along=scvals))+for(jinseq(along=shvals))+grid[i,j]contour(scvals,shvals,grid,levels=77:85)>points(mle$estimate[1],mle$estimate[2],pch="+",cex=1.
5)>title(xlab=expression(alpha),ylab=expression(beta))>#Oruselevelscalculatedfromthechi-squaredistribution>contour(scvals,shvals,grid,+levels=mle$min+qchisq(c(0.
5,0.
8,0.
9,0.
95,0.
99),2),+labels=paste(c(50,80,90,95,99),"%",sep=""))UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
19/29Log-likelihoodcontours-Weibull0.
51.
01.
52.
02.
53.
03.
55001000150020002500+αβ0.
51.
01.
52.
02.
53.
03.
55001000150020002500+αβUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
20/29LessonsfromtheWeibullexampleThelikelihoodfunctionisthesameastheprobabilitydensitybutwiththeparametersvaryingandthedataxed.
Forarandomsample,thelog-likelihoodissum(d(,par1,par2,.
.
.
,log=TRUE))Weminimizethenegativeofthelog-likelihoodllfunc(,par1=x[1],.
.
.
,log=TRUE))mle,hessian=TRUE)Theinverseofthehessianprovidesanestimateofthevariance-covariancematrix.
Fortwo-parametermodelswecanevaluateagridoflog-likelihoodvaluesandgetcontours.
Standarderrorsfromtheinversehessianarenotalwaysrealisticindicationsofthevariabilityintheparameterestimates.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
21/29AMethodofMomentsexampleExample6.
12inDevore(2000)discussedmethodofmomentsestimatesfortheparametersinadistributionusingsomesurvivaldata.
Rprovidesfacilitiesforapackageauthortodocumenteitherfunctionsordatasets.
TheexamplesectionofthedocumentationcanberuninRusingtheexamplefunction.
>example(xmp06.
12)x06.
12>data(xmp06.
12)x06.
12>gamma.
MoMprint(surv.
MoMllsurvmle2mle2$estimate[1]8.
79920312.
893209>solve(mle2$hessian)[,1][,2][1,]7.
513112-11.
01259[2,]-11.
01258517.
08703>shvalsscvalsfor(iinseq(along=shvals))+for(jinseq(along=scvals))+grid[i,j]contour(shvals,scvals,grid,levels=seq(100,110,2))UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
23/29Contoursforthegammalog-likelihood5101520251020304050+αβ5101520251020304050+αUsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
24/29ConclusionsfromthegammaexampleGeneralizingtheWeibullexampletootherdistributions(especiallytwo-parameterdistributions)isstraightforward.
Thelog-likelihoodcontoursforthisexamplearebadlynon-elliptical.
Condenceintervalsforandbasedonthestandarderrorwouldhavepoorcoverage.
Ifyouwanttoapproximatethevariabilityintheparameterestimateswithsymmetricintervals,youarebetteroffusingand.
Ifyouwanttoseeareallybadexample,lookatthelog-likelihoodcontoursfortheparametersforthenegativebinomialusingthedatainExample6.
12ofDevore(2000).
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
25/29ProlelikelihoodInhisdescriptionoftheWeibullMLEs,Jayprovidestheexpression¨showingthattheMLEof,conditionalon,canbecalculateddirectly.
Thiscanbeusedtointroduceprolelikelihood>profilellprofilell(2.
15)#negativeprofilelog-likelihoodatestimate[1]77.
0951>mle3unlist(mle3[-3])minimumestimatehessiancodeiterations77.
0950882.
1520013.
3784371.
0000007.
000000UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
26/29ProlelikelihoodingeneralWecannowdiscusstheuseofprolelikelihoodtoobtaincondenceintervalswithbettercoverageproperties.
ForcaseswherethereisnoexplicitexpressionfortheconditionalMLE,nestedoptimizationscanbeused.
ThisworksinRbutisverytricky,ifnotimpossible,todoinS-PLUS.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
27/29WhatdoessoftwareaddtoMathStatArichsetofdistributionsinourstatisticalsoftwareallowsustoillustratedifferentdistributions.
Plottingandexploringdensities,probabilityfunctions,cumulativedistributionfunctions,empiricalcdf's,etc.
canhelptointernalizetheconceptofthesefunctionsaswellasconceptslikeshapeandscaleparameters.
(RecallhoweasyitwastoplottheWeibulldensity.
)WecantreatMLEsasoptimizationproblemsandavoidthepotentiallyconfusingpartialderivatives.
Wecanillustrateexamplesforwhichtherearenoanalyticsolutions.
Wecandiscussmoreadvancedconcepts(condenceregionsbasedoncontoursofthelog-likelihood,prolelikelihood,likelihood-basedcondenceintervalsthatdonothavetobesymmetric)inelementarycourses.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
28/29SourcesofinformationaboutRThewebsitehttp://www.
r-project.
org/andCRANThefrequentlyaskedquestions(FAQ)listathttp://www.
ci.
tuwien.
ac.
at/hornik/R/,mirroredathttp://cran.
r-project.
org/doc/FAQ/,isagreatsourceofinformation,especiallyforusersswitchingfromS-PLUS.
Themanualsinthedocumentationdirectoryhttp://cran.
r-project.
org/doc/manuals.
SeeespeciallyR-intro.
pdfandR-data.
pdf.
UsingOpenSourceSoftwaretoTeachMathematicalStatistics–p.
29/29