pwrite卡巴斯基授权文件
卡巴斯基授权文件 时间:2021-02-28 阅读:()
AllFileSystemsAreNotCreatedEqual:OntheComplexityofCraftingCrash-ConsistentApplicationsThanumalayanSankaranarayanaPillai,VijayChidambaram,RamnatthanAlagappan,SamerAl-Kiswany,AndreaC.
Arpaci-Dusseau,RemziH.
Arpaci-DusseauAllFileSystemsAreNotCreatedEqual:OntheComplexityofCraftingCrash-ConsistentApplicationsThanumalayanSankaranarayanaPillai,VijayChidambaram,RamnatthanAlagappan,SamerAl-Kiswany,AndreaC.
Arpaci-Dusseau,RemziH.
Arpaci-DusseauCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicImportantinsystems‐FileSystems‐RelationalDatabases‐Key-ValueStoresCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicImportantinsystems‐FileSystems‐RelationalDatabases‐Key-ValueStoresHardtogetright:ARIESinventedonlyin1992‐ProvingARIEStook5moreyears(1997)Lotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
File-SystemCrashConsistencyLotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
FSconsistencyfocusesoninternalmetadata‐DodirectoriesonlycontainvaliddirectoryentriesFile-SystemCrashConsistencyLotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
FSconsistencyfocusesoninternalmetadata‐DodirectoriesonlycontainvaliddirectoryentriesWhataboutuser-leveldataFile-SystemCrashConsistencyWhatguaranteesdofilesystemsgiveapplications‐Thatcanbeusedforconsistencyofuser-leveldataThisworkstudies.
.
.
Whatguaranteesdofilesystemsgiveapplications‐Thatcanbeusedforconsistencyofuser-leveldataDoapplicationsmaintainconsistencycorrectly‐Importantapplicationsrequireuser-levelconsistency‐Databases,key-valuestores,distributedsystems.
.
.
Thisworkstudies.
.
.
Filesystemguaranteesvarywidely‐Studied16configsofext2,ext3,ext4,btrfs,xfs,reiserfs‐Guaranteesvaryamongconfigsofsamefilesystem‐Guaranteesoftenside-effectsofFSimplementation‐POSIXstandardsofguarantees,ifany,aredebatedWefind.
.
.
Filesystemguaranteesvarywidely‐Studied16configsofext2,ext3,ext4,btrfs,xfs,reiserfs‐Guaranteesvaryamongconfigsofsamefilesystem‐Guaranteesoftenside-effectsofFSimplementation‐POSIXstandardsofguarantees,ifany,aredebatedApplicationsdependonguaranteesinsubtleways‐Studied11applications:Databases,Distributedsystems,Virtualizationsoftware,Key-valuestores,VCS‐60vulnerabilitiesunderaweakfilesystemmodel‐Morethan30vulnerabilitiesunderext3,ext4,btrfsWefind.
.
.
IntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineAfileinitiallycontainsthestring"afoo"‐Assumeeachcharacterin"afoo"isablockofdataTask:Atomicallychangethecontentsto"abar"‐Onapowerloss,wemustretrieveeither"afoo"or"abar"ToyExample:OverviewToyExample:SimpleOverwriteInitialstate/x/f1"afoo"Finalstate/x/f1"abar"Modificationpwrite(/x/f1,2,"bar")IntermediatestatespossibleoncrashToyExample:SimpleOverwriteInitialstate/x/f1"afoo"Modificationpwrite(/x/f1,2,"bar")Finalstate/x/f1"abar"Intermediatestate1/x/f1"aboo"Intermediatestate2/x/f1"afar"Intermediatestates3,4,5.
.
.
.
WhatifcrashatomicityisneededUseapplication-levellogging(a.
k.
a.
undologging/rollbackjournaling)a.
Makeacopyofolddatain"log"fileb.
Modifyactualfilec.
Deletelogfiled.
Onacrash,datacanberecoveredfromthelogToyExample:MaintainingConsistencyWhatifcrashatomicityisneededUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);WritetologActualmodificationDeletelogToyExample:Protocol#1Worksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ToyExample:Protocol#1Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
Worksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
ToyExample:Protocol#1SimplydeletelogfileduringrecoveryWorksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
ToyExample:Protocol#1RecoverfromlogfileduringrecoveryWorksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ToyExample:Protocol#1Worksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ext3(ordered)canre-orderthesetworequests,sendingpwrite(f1)todiskfirst,beforewrite(log1)ToyExample:Protocol#1Worksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Apossibleintermediatestate/x/f1"aboo"/x/log1""Recoverynotpossible!
ToyExample:Protocol#1Worksinext3(data-journal),(data-ordered)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#2Worksinext3(data-journal),(data-ordered)!
Doesn'tworkinext3(writeback)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);Apossibleintermediatestates/x/f1"afoo"/x/log1"2,3,#!
@"Filesizealoneincreasesforlog1,andgarbageoccurs.
Recoverycannotdifferentiatebetweengarbageanddata!
CrashhereToyExample:Protocol#2Worksinext3(data-journal),(data-ordered),(writeback)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#3Worksinext3(data-journal),(data-ordered),(writeback)Notenough,accordingtoLinuxmanpagesUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);Apossibleintermediatestates/x/f1"aboo"Thelogfile'sdirectoryentrymightneverbecreatedToyExample:Protocol#3WorksinallfilesystemsUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);fsync(/x);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#4Worksinallfilesystems(Additionalfsync()requiredfordurabilityinallFS)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);fsync(/x);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);fsync(/x);ToyExample:Protocol#5Filesystemsvaryincrash-relatedbehavior‐ext3(ordered)re-orders,whileext3(journaled)doesnotApplicationsusuallydependonsomebehavior‐Dependonordering:Somefsync()callscanbeomittedExample:SummaryIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineTwoclassesofproperties:atomicityandordering‐Atomicityexample:Isawrite()callatomicintheFS‐Orderingexample:Arewrite()callssenttodiskin-orderStudiedext2,ext3,ext4,btrfs,xfs,reiserfs‐Westudied16configurationsofthesixfilesystemsFSBehavior:PersistenceProperties1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()callsMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkloadMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkload3.
Reconstructdisk-statespossibleonapower-loss‐Allstatespossibleifdisk-cachedoesnotre-order‐Afewstateswheredisk-cachere-ordersMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkload3.
Reconstructdisk-statespossibleonapower-loss‐Allstatespossibleifdisk-cachedoesnotre-order‐Afewstateswheredisk-cachere-orders4.
RunFSrecovery,verifypropertyoneachdisk-state‐Example:Isall8KBwrittenatomicallyMethodology:TheBlock-OrderBreaker(BOB)FileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncFileSystemDifferentConfigurationsofFileSystemFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncPersistencePropertiesconsideredFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncIsadirectoryoperation,likerename(),atomiconasystemcrashFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncPropertycertainlynotobeyedWedidnotseeaviolationFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncMainresult:FilesystemsvaryintheirpersistencepropertiesApplicationsshouldnotrelyonpersistencepropertiesTestingapplicationsonaspecificFSisnotenough‐ext3(data-journal):Re-orderingvulnerabilitiesarehiddenFileSystemStudy:ConclusionIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineALICE:Goal"Application-LevelIntelligentCrashExplorer"Goal:Tooltofindcrashvulnerabilitiesofanapplication‐Findvulnerabilitiesacrossallfilesystems‐Relatevulnerabilitiestospecificsourcelines‐RelatevulnerabilitiestofilesystembehaviorALICE:TechniqueUser-suppliedApplicationWorkloadALICEUsersuppliesALICEwithanapplicationworkload–Example:AdatabasetransactionALICE:TechniqueSystem-callTraceALICErunsworkloadandrecordssystem-calltraceALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceAPM:AbstractPersistenceModelAPMmodelsallcrashstatesthatcanoccuronanFS–Default,weakAPMallowsmanypossiblestates–CustomAPMscanbeconfiguredbyuserforaspecificfilesystem–Eg:ext3(ordered)APMallowsstateswithoverwritesre-ordered;ext3(data-journal)APMdoesnotALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceExplorerAPM:AbstractPersistenceModelExplorerreconstructssomestatesusingtheAPMALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceExplorer.
.
.
ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)Explorertargetsspecificstates–Relatingtoatomicityandre-orderingofeachsyscallUser-suppliedApplicationWorkloadTheapplicationisrunoneachreconstructedstate–Usersuppliesapplicationchecker–Example:WasACIDpreservedALICE:TechniqueSystem-callTraceExplorer.
.
.
User-suppliedApplicationCheckerALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)User-suppliedApplicationWorkloadCheckershowswhichstatesarerecoveredfromcorrectlyALICE:TechniqueSystem-callTraceExplorer.
.
.
ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)CorrectIncorrectUser-suppliedApplicationWorkloadUser-suppliedApplicationCheckerFromcheckeroutputs,wedeterminevulnerabilitiesALICE:TechniqueSystem-callTraceExplorer.
.
.
Crashvulnerability:Re-orderingsyscall-1and2ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)CorrectIncorrectUser-suppliedApplicationWorkloadUser-suppliedApplicationCheckerHDFSZooKeeperVMWarePlayerLMDBGDBMLevelDBPostgresHSQLDBSQLiteMercurialGitVulnerabilityStudy:ApplicationsNon-relationalDatabasesRelationalDatabasesVersionControlSystemsVirtualizationSoftwareDistributedServicesExample:Gitcreat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Readthefullpapertocorrectlyinterpretresults!
creat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Example:GitAtomicityvulnerabilityWhichsystemcallsneedtobeatomicReadthefullpapertocorrectlyinterpretresults!
creat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Example:GitAtomicityvulnerabilityOrderingvulnerabilityWhichsystem-callre-orderingscauseincorrectnessReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:Default(Weak)APMReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:Default(Weak)APM60VulnerabilitiesManyresultinsilentdataloss,inaccessibleapplicationsReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:BtrfsAPMReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:BtrfsAPM31VulnerabilitiesReadthefullpapertocorrectlyinterpretresults!
Garbageduringfile-appends‐Affects3applications‐But,partialappendswithactualdata:0vulnerabilitiesFSsafetyheuristicsseeminglydon'thelpmuch‐Only2foundvulnerabilitiesby"Flushdatabeforerename"‐HeuristicsmighthelpothertypesofapplicationsNon-synchronousdirectoryoperations‐Affectsdurabilityof6applicationsWhatFSbehavioraffectsapplicationsIn-depth:WhatFSbehaviorsaffectapplicationsVulnerabilitiesunderotherAPMsInteractionswithapplicationdevelopersHownottointerpretourresultsAnefficientFSdesignwithsafetyvalidatedbyALICEInthepaper.
.
.
FSvaryinbehavioraffectingapplicationconsistency‐ext2,ext3,ext4,btrfs,xfs,reiserfsvaryevenamongtheirdifferentconfigurations‐SubtleimplementationdetailsaffectbehaviorApplicationprotocolsarecomplex,vulnerable‐60vulnerabilitiesunderweakAPM‐Morethanhalfexposedunderext3,ext4,btrfs‐Depend(bydesignorunwittingly)onFSimplementationSummaryApartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoApartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoExperiencedAcademic:Realfilesystemsdon'tdothatButdoesjustthatAcademic:Mystudentswouldflunkclassiftheybuiltafilesystemthatway.
.
.
Apartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoExperiencedAcademic:Realfilesystemsdon'tdothatButdoesjustthatAcademic:Mystudentswouldflunkclassiftheybuiltafilesystemthatway.
.
.
Thankyou!
Downloadtools:http://research.
cs.
wisc.
edu/adsl/Software/alice
Arpaci-Dusseau,RemziH.
Arpaci-DusseauAllFileSystemsAreNotCreatedEqual:OntheComplexityofCraftingCrash-ConsistentApplicationsThanumalayanSankaranarayanaPillai,VijayChidambaram,RamnatthanAlagappan,SamerAl-Kiswany,AndreaC.
Arpaci-Dusseau,RemziH.
Arpaci-DusseauCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicImportantinsystems‐FileSystems‐RelationalDatabases‐Key-ValueStoresCrashConsistencyMaintainingdatainvariantsacrossasystemcrash‐Example:DatabasetransactionsshouldbeatomicImportantinsystems‐FileSystems‐RelationalDatabases‐Key-ValueStoresHardtogetright:ARIESinventedonlyin1992‐ProvingARIEStook5moreyears(1997)Lotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
File-SystemCrashConsistencyLotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
FSconsistencyfocusesoninternalmetadata‐DodirectoriesonlycontainvaliddirectoryentriesFile-SystemCrashConsistencyLotsofworkinfilesystemcrashconsistency‐Journaling,copy-on-write,softupdates.
.
.
FSconsistencyfocusesoninternalmetadata‐DodirectoriesonlycontainvaliddirectoryentriesWhataboutuser-leveldataFile-SystemCrashConsistencyWhatguaranteesdofilesystemsgiveapplications‐Thatcanbeusedforconsistencyofuser-leveldataThisworkstudies.
.
.
Whatguaranteesdofilesystemsgiveapplications‐Thatcanbeusedforconsistencyofuser-leveldataDoapplicationsmaintainconsistencycorrectly‐Importantapplicationsrequireuser-levelconsistency‐Databases,key-valuestores,distributedsystems.
.
.
Thisworkstudies.
.
.
Filesystemguaranteesvarywidely‐Studied16configsofext2,ext3,ext4,btrfs,xfs,reiserfs‐Guaranteesvaryamongconfigsofsamefilesystem‐Guaranteesoftenside-effectsofFSimplementation‐POSIXstandardsofguarantees,ifany,aredebatedWefind.
.
.
Filesystemguaranteesvarywidely‐Studied16configsofext2,ext3,ext4,btrfs,xfs,reiserfs‐Guaranteesvaryamongconfigsofsamefilesystem‐Guaranteesoftenside-effectsofFSimplementation‐POSIXstandardsofguarantees,ifany,aredebatedApplicationsdependonguaranteesinsubtleways‐Studied11applications:Databases,Distributedsystems,Virtualizationsoftware,Key-valuestores,VCS‐60vulnerabilitiesunderaweakfilesystemmodel‐Morethan30vulnerabilitiesunderext3,ext4,btrfsWefind.
.
.
IntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineAfileinitiallycontainsthestring"afoo"‐Assumeeachcharacterin"afoo"isablockofdataTask:Atomicallychangethecontentsto"abar"‐Onapowerloss,wemustretrieveeither"afoo"or"abar"ToyExample:OverviewToyExample:SimpleOverwriteInitialstate/x/f1"afoo"Finalstate/x/f1"abar"Modificationpwrite(/x/f1,2,"bar")IntermediatestatespossibleoncrashToyExample:SimpleOverwriteInitialstate/x/f1"afoo"Modificationpwrite(/x/f1,2,"bar")Finalstate/x/f1"abar"Intermediatestate1/x/f1"aboo"Intermediatestate2/x/f1"afar"Intermediatestates3,4,5.
.
.
.
WhatifcrashatomicityisneededUseapplication-levellogging(a.
k.
a.
undologging/rollbackjournaling)a.
Makeacopyofolddatain"log"fileb.
Modifyactualfilec.
Deletelogfiled.
Onacrash,datacanberecoveredfromthelogToyExample:MaintainingConsistencyWhatifcrashatomicityisneededUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);WritetologActualmodificationDeletelogToyExample:Protocol#1Worksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ToyExample:Protocol#1Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
Worksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
ToyExample:Protocol#1SimplydeletelogfileduringrecoveryWorksinext3(data-journal)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Somepossibleintermediatestates/x/f1"afoo"/x/log1""/x/f1"afoo"/x/log1"2,3,f"/x/f1"aboo"/x/log1"2,3,foo"1.
2.
3.
ToyExample:Protocol#1RecoverfromlogfileduringrecoveryWorksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ToyExample:Protocol#1Worksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);ext3(ordered)canre-orderthesetworequests,sendingpwrite(f1)todiskfirst,beforewrite(log1)ToyExample:Protocol#1Worksinext3(data-journal)!
Doesn'tworkinext3(data-ordered)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");pwrite(/x/f1,2,"bar");unlink(/x/log1);Apossibleintermediatestate/x/f1"aboo"/x/log1""Recoverynotpossible!
ToyExample:Protocol#1Worksinext3(data-journal),(data-ordered)!
UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#2Worksinext3(data-journal),(data-ordered)!
Doesn'tworkinext3(writeback)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);Apossibleintermediatestates/x/f1"afoo"/x/log1"2,3,#!
@"Filesizealoneincreasesforlog1,andgarbageoccurs.
Recoverycannotdifferentiatebetweengarbageanddata!
CrashhereToyExample:Protocol#2Worksinext3(data-journal),(data-ordered),(writeback)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#3Worksinext3(data-journal),(data-ordered),(writeback)Notenough,accordingtoLinuxmanpagesUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);Apossibleintermediatestates/x/f1"aboo"Thelogfile'sdirectoryentrymightneverbecreatedToyExample:Protocol#3WorksinallfilesystemsUpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);fsync(/x);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);ToyExample:Protocol#4Worksinallfilesystems(Additionalfsync()requiredfordurabilityinallFS)UpdateProtocolcreat(/x/log1);write(/x/log1,"2,3,checksum,foo");fsync(/x/log1);fsync(/x);pwrite(/x/f1,2,"bar");fsync(/x/f1);unlink(/x/log1);fsync(/x);ToyExample:Protocol#5Filesystemsvaryincrash-relatedbehavior‐ext3(ordered)re-orders,whileext3(journaled)doesnotApplicationsusuallydependonsomebehavior‐Dependonordering:Somefsync()callscanbeomittedExample:SummaryIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineTwoclassesofproperties:atomicityandordering‐Atomicityexample:Isawrite()callatomicintheFS‐Orderingexample:Arewrite()callssenttodiskin-orderStudiedext2,ext3,ext4,btrfs,xfs,reiserfs‐Westudied16configurationsofthesixfilesystemsFSBehavior:PersistenceProperties1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()callsMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkloadMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkload3.
Reconstructdisk-statespossibleonapower-loss‐Allstatespossibleifdisk-cachedoesnotre-order‐Afewstateswheredisk-cachere-ordersMethodology:TheBlock-OrderBreaker(BOB)1.
Runuser-levelworkloadsstressingtheproperty‐Example:write(8KB)fortestingatomicityofwrite()calls2.
Recordblock-leveltraceoftheworkload3.
Reconstructdisk-statespossibleonapower-loss‐Allstatespossibleifdisk-cachedoesnotre-order‐Afewstateswheredisk-cachere-orders4.
RunFSrecovery,verifypropertyoneachdisk-state‐Example:Isall8KBwrittenatomicallyMethodology:TheBlock-OrderBreaker(BOB)FileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncFileSystemDifferentConfigurationsofFileSystemFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncPersistencePropertiesconsideredFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncIsadirectoryoperation,likerename(),atomiconasystemcrashFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncPropertycertainlynotobeyedWedidnotseeaviolationFileSystemStudy:ResultsFilesystemconfigurationAtomicityOrderingOnesectoroverwriteAppendcontentManysectoroverwriteDirectoryoperationOverwrite→AnyopAppend→AnyopDir-op→AnyopAppend→Renameext2asyncsyncext3writebackordereddata-journalext4writebackorderedno-delallocdata-journalbtrfsxfsdefaultwsyncMainresult:FilesystemsvaryintheirpersistencepropertiesApplicationsshouldnotrelyonpersistencepropertiesTestingapplicationsonaspecificFSisnotenough‐ext3(data-journal):Re-orderingvulnerabilitiesarehiddenFileSystemStudy:ConclusionIntroductionAnExampleBOB:ExaminingFileSystemBehaviorALICE:ExaminingApplicationsConclusionOutlineALICE:Goal"Application-LevelIntelligentCrashExplorer"Goal:Tooltofindcrashvulnerabilitiesofanapplication‐Findvulnerabilitiesacrossallfilesystems‐Relatevulnerabilitiestospecificsourcelines‐RelatevulnerabilitiestofilesystembehaviorALICE:TechniqueUser-suppliedApplicationWorkloadALICEUsersuppliesALICEwithanapplicationworkload–Example:AdatabasetransactionALICE:TechniqueSystem-callTraceALICErunsworkloadandrecordssystem-calltraceALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceAPM:AbstractPersistenceModelAPMmodelsallcrashstatesthatcanoccuronanFS–Default,weakAPMallowsmanypossiblestates–CustomAPMscanbeconfiguredbyuserforaspecificfilesystem–Eg:ext3(ordered)APMallowsstateswithoverwritesre-ordered;ext3(data-journal)APMdoesnotALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceExplorerAPM:AbstractPersistenceModelExplorerreconstructssomestatesusingtheAPMALICEUser-suppliedApplicationWorkloadALICE:TechniqueSystem-callTraceExplorer.
.
.
ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)Explorertargetsspecificstates–Relatingtoatomicityandre-orderingofeachsyscallUser-suppliedApplicationWorkloadTheapplicationisrunoneachreconstructedstate–Usersuppliesapplicationchecker–Example:WasACIDpreservedALICE:TechniqueSystem-callTraceExplorer.
.
.
User-suppliedApplicationCheckerALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)User-suppliedApplicationWorkloadCheckershowswhichstatesarerecoveredfromcorrectlyALICE:TechniqueSystem-callTraceExplorer.
.
.
ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)CorrectIncorrectUser-suppliedApplicationWorkloadUser-suppliedApplicationCheckerFromcheckeroutputs,wedeterminevulnerabilitiesALICE:TechniqueSystem-callTraceExplorer.
.
.
Crashvulnerability:Re-orderingsyscall-1and2ALICEAPM:AbstractPersistenceModelCrashstate#1(Violatesatomicityofsyscall-1)Crashstate#2(Violatesorderingofsyscall-1and2)CorrectIncorrectUser-suppliedApplicationWorkloadUser-suppliedApplicationCheckerHDFSZooKeeperVMWarePlayerLMDBGDBMLevelDBPostgresHSQLDBSQLiteMercurialGitVulnerabilityStudy:ApplicationsNon-relationalDatabasesRelationalDatabasesVersionControlSystemsVirtualizationSoftwareDistributedServicesExample:Gitcreat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Readthefullpapertocorrectlyinterpretresults!
creat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Example:GitAtomicityvulnerabilityWhichsystemcallsneedtobeatomicReadthefullpapertocorrectlyinterpretresults!
creat(index.
lock)mkdir(o/x)creat(o/x/tmp_y)append(o/x/tmp_y)fsync(o/x/tmp_y)link(o/x/tmp_y,o/x/y)unlink(o/x/tmp_y)append(index.
lock)rename(index.
lock,index)stdout(finishedadd)Example:GitAtomicityvulnerabilityOrderingvulnerabilityWhichsystem-callre-orderingscauseincorrectnessReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:Default(Weak)APMReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:Default(Weak)APM60VulnerabilitiesManyresultinsilentdataloss,inaccessibleapplicationsReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:BtrfsAPMReadthefullpapertocorrectlyinterpretresults!
VulnerabilityStudy:BtrfsAPM31VulnerabilitiesReadthefullpapertocorrectlyinterpretresults!
Garbageduringfile-appends‐Affects3applications‐But,partialappendswithactualdata:0vulnerabilitiesFSsafetyheuristicsseeminglydon'thelpmuch‐Only2foundvulnerabilitiesby"Flushdatabeforerename"‐HeuristicsmighthelpothertypesofapplicationsNon-synchronousdirectoryoperations‐Affectsdurabilityof6applicationsWhatFSbehavioraffectsapplicationsIn-depth:WhatFSbehaviorsaffectapplicationsVulnerabilitiesunderotherAPMsInteractionswithapplicationdevelopersHownottointerpretourresultsAnefficientFSdesignwithsafetyvalidatedbyALICEInthepaper.
.
.
FSvaryinbehavioraffectingapplicationconsistency‐ext2,ext3,ext4,btrfs,xfs,reiserfsvaryevenamongtheirdifferentconfigurations‐SubtleimplementationdetailsaffectbehaviorApplicationprotocolsarecomplex,vulnerable‐60vulnerabilitiesunderweakAPM‐Morethanhalfexposedunderext3,ext4,btrfs‐Depend(bydesignorunwittingly)onFSimplementationSummaryApartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoApartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoExperiencedAcademic:Realfilesystemsdon'tdothatButdoesjustthatAcademic:Mystudentswouldflunkclassiftheybuiltafilesystemthatway.
.
.
Apartingnote.
.
.
.
ExperiencedApp-Developer:POSIXdoesn'tletFSesdothatCanyoupointustotheexactPOSIXdocumentationDeveloper:Ican'tfindit,butIremembersomeonesayingsoExperiencedAcademic:Realfilesystemsdon'tdothatButdoesjustthatAcademic:Mystudentswouldflunkclassiftheybuiltafilesystemthatway.
.
.
Thankyou!
Downloadtools:http://research.
cs.
wisc.
edu/adsl/Software/alice
- pwrite卡巴斯基授权文件相关文档
- 标准卡巴斯基授权文件
- 信息卡巴斯基授权文件
- 信任卡巴斯基授权文件
- 数据卡巴斯基授权文件
- 居中卡巴斯基授权文件
- 节点卡巴斯基授权文件
Megalayer 香港CN2优化线路VPS主机速度和性能综合评测
对于Megalayer云服务器提供商在之前也有对于他们家的美国服务器和香港服务器进行过评测和介绍，但是对于大部分网友来说需要独立服务器和站群服务器并不是特别的普及，我们很多网友使用较多的还是云服务器或者VPS主机比较多。在前面也有在"Megalayer新增香港VPS主机 1GB内存 50GB SSD 2M带宽 月59元"文章中有介绍到Megalayer商家有新增香港CN2优化VPS主机。那时候看这...
王小玉网-美国洛杉矶2核4G 20元/月,香港日本CN2 2核2G/119元/季,美国300G高防/80元/月!
活动方案：美国洛杉矶 E5 2696V2 2核4G20M带宽100G流量20元/月美国洛杉矶E5 2696V2 2核4G100M带宽1000G流量99元/季香港CN2 E5 2660V2 2核2G30M CN2500G流量119元/季日本CN2E5 2660 2核2G30M CN2 500G流量119元/季美国300G高防 真实防御E5 2696V2 2核2G30M...
亚洲云Asiayu,成都云服务器 4核4G 30M 120元一月
点击进入亚云官方网站（www.asiayun.com）公司名：上海玥悠悠云计算有限公司成都铂金宿主机IO测试图亚洲云Asiayun怎么样？亚洲云Asiayun好不好？亚云由亚云团队运营，拥有ICP/ISP/IDC/CDN等资质，亚云团队成立于2018年，经过多次品牌升级。主要销售主VPS服务器，提供云服务器和物理服务器，机房有成都、美国CERA、中国香港安畅和电信，香港提供CN2 GIA线路，CE...
卡巴斯基授权文件为你推荐
-
淘宝收费淘宝卖东西收多少手续费支付宝查询余额怎样查支付宝余额伪静态什么是伪静态网站？伪静态网站有什么优势如何建立一个网站怎样能创建一个网站蘑菇街美丽说蘑菇街、美丽说这类网站前期是怎么推广的？ios7固件下载iOS的固件有正版盗版之分吗？我看到了蜂威网有iOS7的固件想下载试用一下，那里是测试版是正版吗迅雷云点播账号求个迅雷VIP 是VIP就可以 只用来看云点播 改密码是孙子。 谢了 ！ 362135668@qq.comxp系统停止服务xp系统停止服务怎么办安装迅雷看看播放器怎样安装迅雷看看播放器商标注册查询官网商标注册查询官方网站?