pwrite卡巴斯基授权文件
卡巴斯基授权文件 时间:2021-02-28 阅读:()
AllFileSystemsAreNotCreatedEqual:OntheComplexityofCraftingCrash-ConsistentApplicationsThanumalayanSankaranarayanaPillai,VijayChidambaram,RamnatthanAlagappan,SamerAl-Kiswany,AndreaC.
Arpaci-Dusseau,RemziH.
Arpaci-DusseauAllFileSystemsAreNotCreatedEqual:OntheComplexityofCraftingCrash-ConsistentApplicationsThanumalayanSankaranarayanaPillai,VijayChidambaram,RamnatthanAlagappan,SamerAl-Kiswany,AndreaC.
Arpaci-Dusseau,RemziH.
Arpaci-DusseauCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicImportantinsystems‐FileSystems‐RelationalDatabases‐Key-ValueStoresCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicImportantinsystems‐FileSystems‐RelationalDatabases‐Key-ValueStoresHardtogetright:ARIESinventedonlyin1992‐ProvingARIEStook5moreyears(1997)Lotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
File-SystemCrashConsistencyLotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
FSconsistencyfocusesoninternalmetadata‐DodirectoriesonlycontainvaliddirectoryentriesFile-SystemCrashConsistencyLotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
FSconsistencyfocusesoninternalmetadata‐DodirectoriesonlycontainvaliddirectoryentriesWhataboutuser-leveldataFile-SystemCrashConsistencyWhatguaranteesdofilesystemsgiveapplications‐Thatcanbeusedforconsistencyofuser-leveldataThisworkstudies.
.
.
Whatguaranteesdofilesystemsgiveapplications‐Thatcanbeusedforconsistencyofuser-leveldataDoapplicationsmaintainconsistencycorrectly‐Importantapplicationsrequireuser-levelconsistency‐Databases,key-valuestores,distributedsystems.
.
.
Thisworkstudies.
.
.
Filesystemguaranteesvarywidely‐Studied16configsofext2,ext3,ext4,btrfs,xfs,reiserfs‐Guaranteesvaryamongconfigsofsamefilesystem‐Guaranteesoftenside-effectsofFSimplementation‐POSIXstandardsofguarantees,ifany,aredebatedWefind.
.
.
Filesystemguaranteesvarywidely‐Studied16configsofext2,ext3,ext4,btrfs,xfs,reiserfs‐Guaranteesvaryamongconfigsofsamefilesystem‐Guaranteesoftenside-effectsofFSimplementation‐POSIXstandardsofguarantees,ifany,aredebatedApplicationsdependonguaranteesinsubtleways‐Studied11applications:Databases,Distributedsystems,Virtualizationsoftware,Key-valuestores,VCS‐60vulnerabilitiesunderaweakfilesystemmodel‐Morethan30vulnerabilitiesunderext3,ext4,btrfsWefind.
.
.
IntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineAfileinitiallycontainsthestring"afoo"‐Assumeeachcharacterin"afoo"isablockofdataTask:Atomicallychangethecontentsto"abar"‐Onapowerloss,wemustretrieveeither"afoo"or"abar"ToyExample:OverviewToyExample:SimpleOverwriteInitialstate/x/f1"afoo"Finalstate/x/f1"abar"Modificationpwrite(/x/f1,2,"bar")IntermediatestatespossibleoncrashToyExample:SimpleOverwriteInitialstate/x/f1"afoo"Modificationpwrite(/x/f1,2,"bar")Finalstate/x/f1"abar"Intermediatestate1/x/f1"aboo"Intermediatestate2/x/f1"afar"Intermediatestates3,4,5.
.
.
.
WhatifcrashatomicityisneededUseapplication-levellogging(a.
k.
a.
undologging/rollbackjournaling)a.
Makeacopyofolddatain"log"fileb.
Modifyactualfilec.
Deletelogfiled.
Onacrash,datacanberecoveredfromthelogToyExample:MaintainingConsistencyWhatifcrashatomicityisneededUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);WritetologActualmodificationDeletelogToyExample:Protocol#1Worksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ToyExample:Protocol#1Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
Worksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
ToyExample:Protocol#1SimplydeletelogfileduringrecoveryWorksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
ToyExample:Protocol#1RecoverfromlogfileduringrecoveryWorksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ToyExample:Protocol#1Worksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ext3(ordered)canre-orderthesetworequests,sendingpwrite(f1)todiskfirst,beforewrite(log1)ToyExample:Protocol#1Worksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Apossibleintermediatestate/x/f1"aboo"/x/log1""Recoverynotpossible!
ToyExample:Protocol#1Worksinext3(data-journal),(data-ordered)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#2Worksinext3(data-journal),(data-ordered)!
Doesn'tworkinext3(writeback)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);Apossibleintermediatestates/x/f1"afoo"/x/log1"2,3,#!
@"Filesizealoneincreasesforlog1,andgarbageoccurs.
Recoverycannotdifferentiatebetweengarbageanddata!
CrashhereToyExample:Protocol#2Worksinext3(data-journal),(data-ordered),(writeback)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#3Worksinext3(data-journal),(data-ordered),(writeback)Notenough,accordingtoLinuxmanpagesUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);Apossibleintermediatestates/x/f1"aboo"Thelogfile'sdirectoryentrymightneverbecreatedToyExample:Protocol#3WorksinallfilesystemsUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);fsync(/x);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#4Worksinallfilesystems(Additionalfsync()requiredfordurabilityinallFS)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);fsync(/x);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);fsync(/x);ToyExample:Protocol#5Filesystemsvaryincrash-relatedbehavior‐ext3(ordered)re-orders,whileext3(journaled)doesnotApplicationsusuallydependonsomebehavior‐Dependonordering:Somefsync()callscanbeomittedExample:SummaryIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineTwoclassesofproperties:atomicityandordering‐Atomicityexample:Isawrite()callatomicintheFS‐Orderingexample:Arewrite()callssenttodiskin-orderStudiedext2,ext3,ext4,btrfs,xfs,reiserfs‐Westudied16configurationsofthesixfilesystemsFSBehavior:PersistenceProperties1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()callsMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkloadMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkload3.
Reconstructdisk-statespossibleonapower-loss‐Allstatespossibleifdisk-cachedoesnotre-order‐Afewstateswheredisk-cachere-ordersMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkload3.
Reconstructdisk-statespossibleonapower-loss‐Allstatespossibleifdisk-cachedoesnotre-order‐Afewstateswheredisk-cachere-orders4.
RunFSrecovery,verifypropertyoneachdisk-state‐Example:Isall8KBwrittenatomicallyMethodology:TheBlock-OrderBreaker(BOB)FileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncFileSystemDifferentConfigurationsofFileSystemFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncPersistencePropertiesconsideredFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncIsadirectoryoperation,likerename(),atomiconasystemcrashFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncPropertycertainlynotobeyedWedidnotseeaviolationFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncMainresult:FilesystemsvaryintheirpersistencepropertiesApplicationsshouldnotrelyonpersistencepropertiesTestingapplicationsonaspecificFSisnotenough‐ext3(data-journal):Re-orderingvulnerabilitiesarehiddenFileSystemStudy:ConclusionIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineALICE:Goal"Application-LevelIntelligentCrashExplorer"Goal:Tooltofindcrashvulnerabilitiesofanapplication‐Findvulnerabilitiesacrossallfilesystems‐Relatevulnerabilitiestospecificsourcelines‐RelatevulnerabilitiestofilesystembehaviorALICE:TechniqueUser-suppliedApplicationWorkloadALICEUsersuppliesALICEwithanapplicationworkload–Example:AdatabasetransactionALICE:TechniqueSystem-callTraceALICErunsworkloadandrecordssystem-calltraceALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceAPM:AbstractPersistenceModelAPMmodelsallcrashstatesthatcanoccuronanFS–Default,weakAPMallowsmanypossiblestates–CustomAPMscanbeconfiguredbyuserforaspecificfilesystem–Eg:ext3(ordered)APMallowsstateswithoverwritesre-ordered;ext3(data-journal)APMdoesnotALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceExplorerAPM:AbstractPersistenceModelExplorerreconstructssomestatesusingtheAPMALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceExplorer.
.
.
ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)Explorertargetsspecificstates–Relatingtoatomicityandre-orderingofeachsyscallUser-suppliedApplicationWorkloadTheapplicationisrunoneachreconstructedstate–Usersuppliesapplicationchecker–Example:WasACIDpreservedALICE:TechniqueSystem-callTraceExplorer.
.
.
User-suppliedApplicationCheckerALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)User-suppliedApplicationWorkloadCheckershowswhichstatesarerecoveredfromcorrectlyALICE:TechniqueSystem-callTraceExplorer.
.
.
ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)CorrectIncorrectUser-suppliedApplicationWorkloadUser-suppliedApplicationCheckerFromcheckeroutputs,wedeterminevulnerabilitiesALICE:TechniqueSystem-callTraceExplorer.
.
.
Crashvulnerability:Re-orderingsyscall-1and2ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)CorrectIncorrectUser-suppliedApplicationWorkloadUser-suppliedApplicationCheckerHDFSZooKeeperVMWarePlayerLMDBGDBMLevelDBPostgresHSQLDBSQLiteMercurialGitVulnerabilityStudy:ApplicationsNon-relationalDatabasesRelationalDatabasesVersionControlSystemsVirtualizationSoftwareDistributedServicesExample:Gitcreat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Readthefullpapertocorrectlyinterpretresults!
creat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Example:GitAtomicityvulnerabilityWhichsystemcallsneedtobeatomicReadthefullpapertocorrectlyinterpretresults!
creat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Example:GitAtomicityvulnerabilityOrderingvulnerabilityWhichsystem-callre-orderingscauseincorrectnessReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:Default(Weak)APMReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:Default(Weak)APM60VulnerabilitiesManyresultinsilentdataloss,inaccessibleapplicationsReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:BtrfsAPMReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:BtrfsAPM31VulnerabilitiesReadthefullpapertocorrectlyinterpretresults!
Garbageduringfile-appends‐Affects3applications‐But,partialappendswithactualdata:0vulnerabilitiesFSsafetyheuristicsseeminglydon'thelpmuch‐Only2foundvulnerabilitiesby"Flushdatabeforerename"‐HeuristicsmighthelpothertypesofapplicationsNon-synchronousdirectoryoperations‐Affectsdurabilityof6applicationsWhatFSbehavioraffectsapplicationsIn-depth:WhatFSbehaviorsaffectapplicationsVulnerabilitiesunderotherAPMsInteractionswithapplicationdevelopersHownottointerpretourresultsAnefficientFSdesignwithsafetyvalidatedbyALICEInthepaper.
.
.
FSvaryinbehavioraffectingapplicationconsistency‐ext2,ext3,ext4,btrfs,xfs,reiserfsvaryevenamongtheirdifferentconfigurations‐SubtleimplementationdetailsaffectbehaviorApplicationprotocolsarecomplex,vulnerable‐60vulnerabilitiesunderweakAPM‐Morethanhalfexposedunderext3,ext4,btrfs‐Depend(bydesignorunwittingly)onFSimplementationSummaryApartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoApartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoExperiencedAcademic:Realfilesystemsdon'tdothatButdoesjustthatAcademic:Mystudentswouldflunkclassiftheybuiltafilesystemthatway.
.
.
Apartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoExperiencedAcademic:Realfilesystemsdon'tdothatButdoesjustthatAcademic:Mystudentswouldflunkclassiftheybuiltafilesystemthatway.
.
.
Thankyou!
Downloadtools:http://research.
cs.
wisc.
edu/adsl/Software/alice
Arpaci-Dusseau,RemziH.
Arpaci-DusseauAllFileSystemsAreNotCreatedEqual:OntheComplexityofCraftingCrash-ConsistentApplicationsThanumalayanSankaranarayanaPillai,VijayChidambaram,RamnatthanAlagappan,SamerAl-Kiswany,AndreaC.
Arpaci-Dusseau,RemziH.
Arpaci-DusseauCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicImportantinsystems‐FileSystems‐RelationalDatabases‐Key-ValueStoresCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicImportantinsystems‐FileSystems‐RelationalDatabases‐Key-ValueStoresHardtogetright:ARIESinventedonlyin1992‐ProvingARIEStook5moreyears(1997)Lotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
File-SystemCrashConsistencyLotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
FSconsistencyfocusesoninternalmetadata‐DodirectoriesonlycontainvaliddirectoryentriesFile-SystemCrashConsistencyLotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
FSconsistencyfocusesoninternalmetadata‐DodirectoriesonlycontainvaliddirectoryentriesWhataboutuser-leveldataFile-SystemCrashConsistencyWhatguaranteesdofilesystemsgiveapplications‐Thatcanbeusedforconsistencyofuser-leveldataThisworkstudies.
.
.
Whatguaranteesdofilesystemsgiveapplications‐Thatcanbeusedforconsistencyofuser-leveldataDoapplicationsmaintainconsistencycorrectly‐Importantapplicationsrequireuser-levelconsistency‐Databases,key-valuestores,distributedsystems.
.
.
Thisworkstudies.
.
.
Filesystemguaranteesvarywidely‐Studied16configsofext2,ext3,ext4,btrfs,xfs,reiserfs‐Guaranteesvaryamongconfigsofsamefilesystem‐Guaranteesoftenside-effectsofFSimplementation‐POSIXstandardsofguarantees,ifany,aredebatedWefind.
.
.
Filesystemguaranteesvarywidely‐Studied16configsofext2,ext3,ext4,btrfs,xfs,reiserfs‐Guaranteesvaryamongconfigsofsamefilesystem‐Guaranteesoftenside-effectsofFSimplementation‐POSIXstandardsofguarantees,ifany,aredebatedApplicationsdependonguaranteesinsubtleways‐Studied11applications:Databases,Distributedsystems,Virtualizationsoftware,Key-valuestores,VCS‐60vulnerabilitiesunderaweakfilesystemmodel‐Morethan30vulnerabilitiesunderext3,ext4,btrfsWefind.
.
.
IntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineAfileinitiallycontainsthestring"afoo"‐Assumeeachcharacterin"afoo"isablockofdataTask:Atomicallychangethecontentsto"abar"‐Onapowerloss,wemustretrieveeither"afoo"or"abar"ToyExample:OverviewToyExample:SimpleOverwriteInitialstate/x/f1"afoo"Finalstate/x/f1"abar"Modificationpwrite(/x/f1,2,"bar")IntermediatestatespossibleoncrashToyExample:SimpleOverwriteInitialstate/x/f1"afoo"Modificationpwrite(/x/f1,2,"bar")Finalstate/x/f1"abar"Intermediatestate1/x/f1"aboo"Intermediatestate2/x/f1"afar"Intermediatestates3,4,5.
.
.
.
WhatifcrashatomicityisneededUseapplication-levellogging(a.
k.
a.
undologging/rollbackjournaling)a.
Makeacopyofolddatain"log"fileb.
Modifyactualfilec.
Deletelogfiled.
Onacrash,datacanberecoveredfromthelogToyExample:MaintainingConsistencyWhatifcrashatomicityisneededUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);WritetologActualmodificationDeletelogToyExample:Protocol#1Worksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ToyExample:Protocol#1Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
Worksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
ToyExample:Protocol#1SimplydeletelogfileduringrecoveryWorksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
ToyExample:Protocol#1RecoverfromlogfileduringrecoveryWorksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ToyExample:Protocol#1Worksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ext3(ordered)canre-orderthesetworequests,sendingpwrite(f1)todiskfirst,beforewrite(log1)ToyExample:Protocol#1Worksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Apossibleintermediatestate/x/f1"aboo"/x/log1""Recoverynotpossible!
ToyExample:Protocol#1Worksinext3(data-journal),(data-ordered)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#2Worksinext3(data-journal),(data-ordered)!
Doesn'tworkinext3(writeback)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);Apossibleintermediatestates/x/f1"afoo"/x/log1"2,3,#!
@"Filesizealoneincreasesforlog1,andgarbageoccurs.
Recoverycannotdifferentiatebetweengarbageanddata!
CrashhereToyExample:Protocol#2Worksinext3(data-journal),(data-ordered),(writeback)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#3Worksinext3(data-journal),(data-ordered),(writeback)Notenough,accordingtoLinuxmanpagesUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);Apossibleintermediatestates/x/f1"aboo"Thelogfile'sdirectoryentrymightneverbecreatedToyExample:Protocol#3WorksinallfilesystemsUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);fsync(/x);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#4Worksinallfilesystems(Additionalfsync()requiredfordurabilityinallFS)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);fsync(/x);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);fsync(/x);ToyExample:Protocol#5Filesystemsvaryincrash-relatedbehavior‐ext3(ordered)re-orders,whileext3(journaled)doesnotApplicationsusuallydependonsomebehavior‐Dependonordering:Somefsync()callscanbeomittedExample:SummaryIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineTwoclassesofproperties:atomicityandordering‐Atomicityexample:Isawrite()callatomicintheFS‐Orderingexample:Arewrite()callssenttodiskin-orderStudiedext2,ext3,ext4,btrfs,xfs,reiserfs‐Westudied16configurationsofthesixfilesystemsFSBehavior:PersistenceProperties1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()callsMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkloadMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkload3.
Reconstructdisk-statespossibleonapower-loss‐Allstatespossibleifdisk-cachedoesnotre-order‐Afewstateswheredisk-cachere-ordersMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkload3.
Reconstructdisk-statespossibleonapower-loss‐Allstatespossibleifdisk-cachedoesnotre-order‐Afewstateswheredisk-cachere-orders4.
RunFSrecovery,verifypropertyoneachdisk-state‐Example:Isall8KBwrittenatomicallyMethodology:TheBlock-OrderBreaker(BOB)FileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncFileSystemDifferentConfigurationsofFileSystemFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncPersistencePropertiesconsideredFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncIsadirectoryoperation,likerename(),atomiconasystemcrashFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncPropertycertainlynotobeyedWedidnotseeaviolationFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncMainresult:FilesystemsvaryintheirpersistencepropertiesApplicationsshouldnotrelyonpersistencepropertiesTestingapplicationsonaspecificFSisnotenough‐ext3(data-journal):Re-orderingvulnerabilitiesarehiddenFileSystemStudy:ConclusionIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineALICE:Goal"Application-LevelIntelligentCrashExplorer"Goal:Tooltofindcrashvulnerabilitiesofanapplication‐Findvulnerabilitiesacrossallfilesystems‐Relatevulnerabilitiestospecificsourcelines‐RelatevulnerabilitiestofilesystembehaviorALICE:TechniqueUser-suppliedApplicationWorkloadALICEUsersuppliesALICEwithanapplicationworkload–Example:AdatabasetransactionALICE:TechniqueSystem-callTraceALICErunsworkloadandrecordssystem-calltraceALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceAPM:AbstractPersistenceModelAPMmodelsallcrashstatesthatcanoccuronanFS–Default,weakAPMallowsmanypossiblestates–CustomAPMscanbeconfiguredbyuserforaspecificfilesystem–Eg:ext3(ordered)APMallowsstateswithoverwritesre-ordered;ext3(data-journal)APMdoesnotALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceExplorerAPM:AbstractPersistenceModelExplorerreconstructssomestatesusingtheAPMALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceExplorer.
.
.
ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)Explorertargetsspecificstates–Relatingtoatomicityandre-orderingofeachsyscallUser-suppliedApplicationWorkloadTheapplicationisrunoneachreconstructedstate–Usersuppliesapplicationchecker–Example:WasACIDpreservedALICE:TechniqueSystem-callTraceExplorer.
.
.
User-suppliedApplicationCheckerALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)User-suppliedApplicationWorkloadCheckershowswhichstatesarerecoveredfromcorrectlyALICE:TechniqueSystem-callTraceExplorer.
.
.
ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)CorrectIncorrectUser-suppliedApplicationWorkloadUser-suppliedApplicationCheckerFromcheckeroutputs,wedeterminevulnerabilitiesALICE:TechniqueSystem-callTraceExplorer.
.
.
Crashvulnerability:Re-orderingsyscall-1and2ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)CorrectIncorrectUser-suppliedApplicationWorkloadUser-suppliedApplicationCheckerHDFSZooKeeperVMWarePlayerLMDBGDBMLevelDBPostgresHSQLDBSQLiteMercurialGitVulnerabilityStudy:ApplicationsNon-relationalDatabasesRelationalDatabasesVersionControlSystemsVirtualizationSoftwareDistributedServicesExample:Gitcreat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Readthefullpapertocorrectlyinterpretresults!
creat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Example:GitAtomicityvulnerabilityWhichsystemcallsneedtobeatomicReadthefullpapertocorrectlyinterpretresults!
creat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Example:GitAtomicityvulnerabilityOrderingvulnerabilityWhichsystem-callre-orderingscauseincorrectnessReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:Default(Weak)APMReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:Default(Weak)APM60VulnerabilitiesManyresultinsilentdataloss,inaccessibleapplicationsReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:BtrfsAPMReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:BtrfsAPM31VulnerabilitiesReadthefullpapertocorrectlyinterpretresults!
Garbageduringfile-appends‐Affects3applications‐But,partialappendswithactualdata:0vulnerabilitiesFSsafetyheuristicsseeminglydon'thelpmuch‐Only2foundvulnerabilitiesby"Flushdatabeforerename"‐HeuristicsmighthelpothertypesofapplicationsNon-synchronousdirectoryoperations‐Affectsdurabilityof6applicationsWhatFSbehavioraffectsapplicationsIn-depth:WhatFSbehaviorsaffectapplicationsVulnerabilitiesunderotherAPMsInteractionswithapplicationdevelopersHownottointerpretourresultsAnefficientFSdesignwithsafetyvalidatedbyALICEInthepaper.
.
.
FSvaryinbehavioraffectingapplicationconsistency‐ext2,ext3,ext4,btrfs,xfs,reiserfsvaryevenamongtheirdifferentconfigurations‐SubtleimplementationdetailsaffectbehaviorApplicationprotocolsarecomplex,vulnerable‐60vulnerabilitiesunderweakAPM‐Morethanhalfexposedunderext3,ext4,btrfs‐Depend(bydesignorunwittingly)onFSimplementationSummaryApartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoApartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoExperiencedAcademic:Realfilesystemsdon'tdothatButdoesjustthatAcademic:Mystudentswouldflunkclassiftheybuiltafilesystemthatway.
.
.
Apartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoExperiencedAcademic:Realfilesystemsdon'tdothatButdoesjustthatAcademic:Mystudentswouldflunkclassiftheybuiltafilesystemthatway.
.
.
Thankyou!
Downloadtools:http://research.
cs.
wisc.
edu/adsl/Software/alice
- pwrite卡巴斯基授权文件相关文档
- 标准卡巴斯基授权文件
- 信息卡巴斯基授权文件
- 信任卡巴斯基授权文件
- 数据卡巴斯基授权文件
- 居中卡巴斯基授权文件
- 节点卡巴斯基授权文件
DiyVM：2G内存/50G硬盘/元起线路香港vps带宽CN2线路,香港VPS五折月付50元起
DiyVM是一家低调国人VPS主机商，成立于2009年，提供的产品包括VPS主机和独立服务器租用等，数据中心包括香港沙田、美国洛杉矶、日本大阪等，VPS主机基于XEN架构，均为国内直连线路，主机支持异地备份与自定义镜像，可提供内网IP。最近，商家对香港机房VPS提供5折优惠码，最低2GB内存起优惠后仅需50元/月。下面就以香港机房为例，分享几款VPS主机配置信息。CPU：2cores内存：2GB硬...
木木云35元/月，美国vps服务器优惠,1核1G/500M带宽/1T硬盘/4T流量
木木云怎么样?木木云品牌成立于18年，此为贵州木木云科技有限公司旗下新运营高端的服务器的平台，目前已上线美国中部大盘鸡，母鸡采用E5-267X系列，硬盘全部组成阵列。目前，木木云美国vps进行了优惠促销，1核1G/500M带宽/1T硬盘/4T流量,仅35元/月。点击进入：木木云官方网站地址木木云优惠码：提供了一个您专用的优惠码： yuntue目前我们有如下产品套餐：DV型 1H 1G 500M带宽...
Ftech：越南vps,2核/2G/20G SSD/1Gbps不限流量/可安装Windows系统,$12.5月
ftech怎么样?ftech是一家越南本土的主机商，成立于2011年，比较低调，国内知道的人比较少。FTECH.VN以极低的成本提供高质量服务的领先提供商之一。主营虚拟主机、VPS、独立服务器、域名等传统的IDC业务，数据中心分布在河内和胡志明市。其中，VPS提供1G的共享带宽，且不限流量，还可以安装Windows server2003/2008的系统。Ftech支持信用卡、Paypal等付款，但...
卡巴斯基授权文件为你推荐
-
郭吉军什么叫做广告联盟网络明星网络明星是什么，出现这一现象的原因是什么外网和内网内网和外网是什么意思啊？？优酷路由宝怎么赚钱优酷路由宝是如何赚钱的？暴风影音怎么截图怎么截取暴风影音图片pwlosera,pw是什么，是不认识的人发的短信。请解释::安卓应用平台现在android平台的手机都有哪些？童之磊网文大学很强吗？iphone越狱后怎么恢复iPhone越狱后怎么恢复？开机滚动条开机滚动条太多怎么办？