handle169pp

ComputerModelinginEngineering&SciencesCMES,vol.
122,no.
3,pp.
1039-1053,2020CMES.
doi:10.
32604/cmes.
2020.
08268www.
techscience.
com/journal/CMESALaneDetectionMethodBasedonSemanticSegmentationLingDing1,2,HuyinZhang1,*,JinshengXiao3,*,ChengShu3andShejieLu2Abstract:Thispaperproposesanovelmethodoflanedetection,whichadoptsVGG16asthebasisofconvolutionalneuralnetworktoextractlanelinefeaturesbycavityconvolution,whereinthelanelinesaredividedintodottedlinesandsolidlines.
Expandingthefieldofexperiencethroughhollowconvolution,thefullconnectionlayerofthenetworkisdiscarded,thelastlargestpoolinglayeroftheVGG16networkisremoved,andtheprocessingofthelastthreeconvolutionlayersisreplacedbyholeconvolution.
Atthesametime,CNNadoptstheencoderanddecoderstructuremode,andusestheindexfunctionofthemaximumpoolinglayerinthedecoderparttoupsampletheencoderinacounter-poolingmanner,realizingsemanticsegmentation.
Andcombinedwiththeinstancesegmentation,andfinallythroughthefittingtoachievethedetectionofthelaneline.
Inaddition,thecurrentlydisclosedlanelinedatasetsarerelativelysmall,andthereisnodistinctionbetweenlanesolidlinesanddashedlines.
Tothisend,ourworkmadealanelinedatasetforthelanevirtualandrealidentification,andbasedontheproposedalgorithmeffectiveverificationofthedatasetachievedbytheincreasedsegmentation.
Thefinaltestshowsthattheproposedmethodhasagoodbalancebetweenlanedetectionspeedandaccuracy,whichhasgoodrobustness.
Keywords:CNN,VGG16,semanticsegmentation,instancesegmentation,lanedetection.
1IntroductionTheinstallationoftheadvanceddriverassistancesystem[He,ShanandSong(2018)](ADAS)withlanedetectionfunctiononthevehiclenotonlypromptsthedrivertoshiftthelaneintime,butalsoalertsthenearbyvehicletoavoidintime.
Toacertainextent,accidentscanavoid.
Theadvanceddriverassistancesystemisdesignedtohelpdriversworksafely.
Ithasmanyfunctionssuchasadaptivecruisecontrol,blindspotdetection,collisionavoidance,andtrafficsigndetection.
Inaddition,thesystemalsoincludesthefunctionsoflanelinedetectionandlanedeparturemonitoring.
ModerncarscombinedwithADASarebecomingmorestandard.
Withtherapiddevelopmentofdriverlesscars,1SchoolofComputerScience,WuhanUniversity,Wuhan,430072,China.
2CollegeofComputerScienceandTechnology,HubeiUniversityofScienceandTechnology,Xianning,437100,China.
3SchoolofElectronicInformation,WuhanUniversity,Wuhan,430072,China.
*CorrespondingAuthor:HuyinZhang.
Email:zhy2536@whu.
edu.
cn;JinshengXiao.
Email:xiaojs@whu.
edu.
cn.
Received:11August2019;Accepted:12January2020.
1040CMES,vol.
122,no.
3,pp.
1039-1053,2020theautomaticlanekeepingfunctioninADASplaysanincreasinglyimportantrole,andthecarcorrectlypositionedtotravel.
Inthelane,itprovidesanimportantbasisforsubsequentlanedepartureandtrajectoryplanningdecisions.
Thispaperproposesarobustlanedetectionmethodbasedonthecurrentmainstreamdeeplearningalgorithm.
Thismethodmainlyusesconvolutionalneuralnetwork,inwhichtheencoderanddecoderstructureareadded.
Basedontheexistingalgorithms,thispaperimprovestheEncoderandDecoderpartsrespectively.
TheEncoderdiscardsthefullyconnectedlayeroftheVGG16networkandthelast2*2maximumpoolinglayer,andtheEncoderendthree-volumelayerissettoholeconvolution.
Decoderhastwobranches,oneistheupsamplingofEncoder,whichimplementssemanticsegmentation.
Inthispaper,theindexingfunctionofthepoolinglayerisusedtoperformupsamplinginanunpoolingmanner.
Eachupsamplingfollowedbymultipleconvolutionallayers,andthestandardcrossentropylossfunctionisusedtotrainthesegmentationnetwork.
Theotherbranchistheinstancesegmentationbranch.
Thenetworkgeneratesthepixelvectorisinthehigh-dimensionalfeaturespace.
Thediscriminantlossfunctionhascombinedwiththesemanticsegmentationresulttorealizetheinstancesegmentation.
Finally,theinstancedetectionofthelanelinehasrealizedbyfitting.
Inaddition,thispaperhasmadearesearchonthedatasetofthelanevirtuallinetodistinguishthelanevirtualrealitydetection.
Byincreasingthedecoder'ssemanticsegmentationofthebranchnetworkoutput,thetransitionlanelineandbackgroundaretransformedintothesolidlane.
Theaccuracyandrobustnessofthedashedverifiedbyactualtests,whichisthesamewiththesolidlineidentificationoftheproposedalgorithm.
2RelatedworkRegardingthedetectionoflanelines,thetraditionalmethodofdetecting[TalibandRamli(2015);Li,Long,Mingetal.
(2014);Wu,LinandLee(2012);Yoo,Yang,Sohnetal.
(2013)]mainlyusesimageprocessingtechnologytoperformedgedetection,thresholdingprocessingandcurvefittingonroadimages.
Themainstepsaretopre-processtheimage,selecttheRegionofInterest(ROI),andperformedgedetection.
AfterHoughtransform,thresholdingisperformed,andthentheresultisprocessedbythestraightlineorcurvefit.
Commonfittingmethodsmainlyincludeleastsquaresmethod,polynomialfitting,andrandomsampleConsensus(RANSAC)algorithmfitting.
Manyscholarsathomeandabroadhavealotofresearchonthis.
In2014,theliterature[KimandLee(2014)]proposedaConvolutionalNeuralNetworks(CNN)combinedwithRANSAClanedetectionmethod.
Firstly,theoriginalimageisedge-detectedandthelaneinformationisenhanced.
Then,inthesimpleroadscene,theauthorthinksthatthedetectioncanhavecompletedbyusingtheRANSACmethod.
Forcomplexroadconditionssuchasshadowsandfences,itisprocessedbyCNN,theusedtheRANSACmethod.
TheCNNnetworkstructureconsistsofthreeconvolutionallayers,twodownsamplinglayers,amulti-layerperceptron,andthreefullyconnectedlayers.
TheedgeimageofROIisinput,andtheCNNnetworkoutputonlycontainswhitelanelinesandblackbackground.
Thecomplexityjudgmentofthescenedependsonthesettingoftheconditionalthreshold.
Therequirementsofthedifferentscenariosaredifferent.
Atthesametime,theCNNnetworkstructureisverysimple,sotherobustnessofthewholeALaneDetectionMethodBasedonSemanticSegmentation1041algorithmisnothigh.
Thelanelinedetection[Xiao,Luo,Yaoetal.
(2018)]isbasedinpartonvisualsensors.
First,theroadinformationcapturedbythemonocularcameraonthewindshieldofthevehicle,andthentheedgepointinformationisextracted.
Then,thelanelineselectedaccordingtothecustomparameterspacevotingmethod,andcombinedwiththeExtendedKalmanFilter(EKF)pair.
Thecoordinateparametersoftheedgepointsofthelanelinearetrackedandestimated.
Finally,thecurrentvehiclepositioninformationisobtainedaccordingtotheGPSpositioningsystem,andthelateraloffsetbetweenthevehicleandthecurrentlanelineiscalculatedtorealizetheearlywarningfunction.
Parameterspacevotingisaclassicmethod,butitissusceptibletointerferencepoints,whichrequirehighextractionofedgepointsanddoesnothandlecorneringwell.
Traditionallanedetectionmethods[Narote,Bhujbal,Naroteetal.
(2018)]relyonhighlyspecialized,handcraftedfeaturesandheuristicconstraints,oftenrequiringvariouspost-processingtechniquesforoptimization,whichareextremelyunstableduetochangesinroadscenes.
In2016,theliterature[He,Ai,Yanetal.
(2016)]obtainedthecorrespondingtopviewfromthefrontviewoftheimagethroughtheinverseperspectivetransformationmethod,andthecandidateregionandthecandidatelanelinethroughthehat-weightingfilter.
ThedoubleviewnetworkarchitectureofDoubleViewConvolutionalNeuralNetworkdesigned,whichinputtheoriginalforward-lookingimage.
ThetopviewcorrespondstothecandidateregionintotheDVCNNnetworkatthesametime,andthelastoptimallaneobtainedbyusingtheglobaloptimizationfunctionconsideringinformationincludingthelength,number,probability,direction,andwidthofthelaneline.
Thecombineddifferentviewsimprovetheaccuracyofdetection,butitalsoincreasesthespeedofthealgorithm.
Inconsideringspeed,in2017,Kimetal.
[Kim,Kim,Jangetal.
(2017)]proposedafastlearningalgorithmbasedonExtremeLearningMachine(ELM)convolutionalneuralnetworkandappliedittolanelinedetection.
Theinputimagebeforelanedetectionisenhancedbyeliminatingnoiseandobstacles,whicharenotrelatedtotheedgedetectionresult.
ELMisafastlearningmethodforcalculatingthenetworkweightbetweentheoutputlayerandthehiddenlayerinoneiteration.
Thisapproachreducesthelearningtimeofthenetwork,buttheroleofthenetworkisfocusedonimageenhancement.
In2018,Liuetal.
[LiuandDeng(2018)]proposedanRPPmodelbasedonsingleconvolutionvisualroaddetection.
Specifically,RPPisadeepfullconvolutionresidualpartitionnetworkwithpyramidpool.
InordertoimprovethepredictionaccuracyoftheKITTI-ROADdetectiontask,Liuproposesanewstrategybyaddingroadedgetagsandintroducingappropriatedataenhancements.
Itisaneffectiveideatousethesemanticsegmentationindeeplearningtocompletethedetectionofroadsorlanes.
3ProposedmethodThealgorithmisbasedontheroadsegmentationandlanedetectionalgorithm[Oliveira,Burgard,Broxetal.
(2016)],anddrawsontheuseofthediscriminantlossfunction[De,NevenandVan(2017)]inLaneNet[Neven,De,Georgoulisetal.
(2018)]algorithmandthecavityconvolution[YuandKoltun(2016)]appliedinthealgorithmofDeepLabv1[Chen,Papandreou,Kokkinosetal.
(2014)]andLMD[Chen,Lo,Hangetal.
(2018)].
1042CMES,vol.
122,no.
3,pp.
1039-1053,2020Voidconvolutionisthereplacementofthecommonconvolutionallayeroftheextractedfeatureparttoexpandthereceptivefield.
Thedistinguishingfeatureofthediscriminantlossfunctionisthatitiseasytointegrateintodifferentnetworkstructures[Ren,He,Girshicketal.
(2015)],andtheinstancesegmentationisrealizedbypost-processing.
Inaddition,therearesomeworksonimagedenoisingpreprocessingpresentedinpreviousarticles[Xiao,Tian,Zhangetal.
(2018);Ding,Zhang,Xiaoetal.
(2018)].
3.
1NetworkstructureThealgorithmadoptstheEncoder-Decoderstructuremode[Redmon,Divvala,Girshicketal.
(2016)].
TheEncoderpartusestheVGG16networkasthebasemodeltoextractthelanelinefeatures,discardsthefullyconnectedlayeroftheVGG16network,retainsonlythefirstfour2*2maximumpoolinglayersinVGG16,andusestheholes.
Convolutioncanexpandthecharacteristicsofthereceptivefield[Liu,Anguelov,Erhanetal.
(2016)].
Inthispaper,the11th,12th,and13thlayersaresetasacavityconvolutionwithavoidratioof2.
Decoderhastwobranches;oneistheupsamplingofEncoder,whichimplementssemanticsegmentation[Chen,Papandreou,Kokkinosetal.
(2016)].
Itmainlyusestheindexfunctionofthelargestpoolinglayer.
ItconsistsoffourUpsamplinglayersandtenconvolutionlayers.
TheUpsamplinglayerisusedtobetterhandlethegradientdisappearanceproblemusingtheactivationfunctionReLU,andthesegmentationnetworkistrainedusingthestandardcrossentropylossfunction.
Theotherbranchistheinstancesegmentationbranch.
Thesegmentationlossfunctionbasedonthedistancemetriclearningisusedtoimplementtheinstancesegmentationonthegeneratedpixelvectorfeaturemap.
Finally,theinstancedetectionofthelanelineiscompletedbyclusterfitting.
ThespecificnetworkstructureandworkflowchartareshowninFig.
1:Figure1:Networkstructureandworkflowofthispaper3.
2EncoderEncoderisthepartofthealgorithmnetworkstructuretoextractimagefeatures.
Basedonvgg16,itismainlycomposedofconvolutionlayerandpoolinglayer.
ThespecificparametersareshowninTab.
1.
DilatedconvolutionMax-poolingUpsampling+ReluSoftmaxConvolutionSemanticsegmentationInstancesegmentationALaneDetectionMethodBasedonSemanticSegmentation1043Table1:EncodernetworkstructureNameKernelStrideOutputconv1_13*31512*256*64conv1_23*31512*256*64Pool1,max_indices2*22256*128*64conv2_13*31256*128*128conv2_13*31256*128*128Pool2,max_indices2*22128*64*128conv3_13*31128*64*256conv3_23*31128*64*256conv3_33*31128*64*256Pool3,max_indices2*2264*32*256conv4_13*3164*32*512conv4_23*3164*32*512conv4_33*3164*32*512Pool4,max_indices2*2232*16*512Dilatedconv5_13*3132*16*512Dilatedconv5_23*3132*16*512Dilatedconv5_33*3132*16*512Theinputtrainingsampleresolutionisadjustedto512*256,convolutedwiththefilterbanktogenerateasetoffeaturemaps,andthentheyaremassnormalized,followedbytheactivationfunctionlinearrectificationfunction.
Then,themaximumpoolingisusedfor2-xdownsampling,andthepositionofthelargesteigenvalueineachpooledwindowisstoredforeachfeaturemapbeforedownsampling.
Inthelastthreeconvolutionallayers,themaximumpoolingoperationisnotcontinued,butaholeconvolution[Chen,Papandreou,Schroffetal.
(2017)]withaholeratioof2isusedinsteadoftheordinaryconvolutionoperation.
Therefore,theresolutionofthefeaturemapattheendoftheencodernetworkwitha2-xincrease,thevoidconvolutioncanexpandthereceptivefieldwithoutanyadditionalparametersandcomputationalcosts.
3.
3RemovethefullyconnectedlayerInmanycommonalgorithms,FullyConnectedLayersgenerallyfollowtheconvolutionalandpoolinglayers,whichactasa"classifier".
Theconvolutionallayerandthepoolinglayermaptheinputrawdatatothehiddenlayerfeaturespaceforfeatureextraction,andthefullyconnectedlayermapsthelearnedfeaturestothesamplemarkspace.
AsshowninFig.
2,thenodesa,b,andcofthefullyconnectedlayerarerespectivelyconnectedtothenodesX,Y,andZoftheupperlayer,andfunctiontosynthesizethepreviouslyextractedfeatures.
Nevertheless,atthesametime,inviewofthefactthatallthenodesareconnected,theparametersofthefullconnectionlayergenerallyaccountforthe1044CMES,vol.
122,no.
3,pp.
1039-1053,2020largestproportioninthenetworkstructure.
Forexample,inthefamiliarVGG16,fortheinputof224*224*3,thefirstfullyconnectedlayerFC6has4096nodes,andtheupperlayerofFC6isthefifthlargestpoolinglayerwith7*7*512,whichhasatotalof25088nodes,thenitmeansthat4096*25088weightsareneeded,whichconsumesahugeamountofmemory.
abcXYZFigure2:SchematicdiagramofthefullconnectionlayerInthesemanticsegmentationFCNnetworkandtheliterature[Oliveira,Burgard,Broxetal.
(2016)]network,thefullconnectionlayerisused,whichreducesthefiltersnumberofthefullyconnectedlayerfrom4096to1024.
However,eventheparametersofthefullyconnectedlayerarestillredundant.
Inordertomaintaintheresolutionofthefeaturemapandconsidertherunningspeedofthealgorithm,thispaperchoosestodiscardtheoperationofthefullyconnectedlayerdirectly,therebygreatlyreducingthenumberofparameters,andthespeedcomparisonisconfirmedbythefinalcomparisonexperiment.
3.
4IncreasethecavityconvolutionTheexistenceofdownsamplingmakestherunningfilterhavealargerreceptivefield,whichisbeneficialtocollectmorecontextinformationandimprovetheaccuracyofsegmentation.
However,theresultofsemanticsegmentationrequiresthesameresolutionastheinput,whichmeansthatpowerfuldownsamplingwillrequirethesamepowerfulupsampling.
Ontheotherhand,downsamplingwillalsoloseedgeresolutionifthefeatureresolutionisreduced.
Theimportantspatialinformation,andtheoperabilityoftheoriginalinformationforthelostinformationisgreatlyreduced.
Inthisregard,theproposedandwell-conceivedcavityconvolution[YuandKoltun(2016)]avoidstheseproblems.
Thecavityconvolutionprovidesaneffectivemechanismtocontrolthefieldofview.
ItcanexpandthereceptivefieldofthefilterwithoutdownsamplingtoincludelargerContextualinformation.
3.
5SemanticsegmentationThesemanticsegmentation[Chen,Papandreou,Schroffetal.
(2017);Chen,Zhu,Papandreouetal.
(2018);Peng,Zhang,Yuetal.
(2017)]partmainlyrealizesthesegmentationofthelanelineandthebackground.
Byupsamplingtheencoder,theoutputhasthesameresolutionastheinputdata.
Upsamplingincomputervisiongenerallyincludesthreemethods,whicharenamelybilinearinterpolation,inversepooling,anddeconvolution.
Themainideaofbilinearinterpolationistoperformlinearinterpolationintwodirections,respectively.
Deconvolutionistheinverseofconvolutionoperation.
Comparedwiththeformertwo,theparametersinthedeconvolutionprocessneedtotrain.
Intheory,iftheconvolutionkernelparametersaresetproperly,deconvolutioncanALaneDetectionMethodBasedonSemanticSegmentation1045achieveanti-pooling.
Anti-poolingoperationstendtobemoreefficientintermsofmemoryusagebecausetheyonlyneedtostorefewerindexes.
Thede-poolinganddeconvolutiondiagramisshowninFig.
3,whereFig.
3(a)showsthemaximumpoolingandthecorrespondingde-poolingoperation,andFig.
3(b)showstheconvolutionandthecorrespondingdeconvolutionoperation.
(a)(b)Figure3:Comparisonofanti-poolinganddeconvolutionTheroleofsemanticsegmentationismainlytoprovideamaskforinstancesegmentation.
Thecasesegmentationinvolvesthepost-processingofclustering.
Ifclusteringactsoverallimage,itwillconsumealotoftime,andthemaskprovidedbysemanticsegmentationcanignoretheproportionofthescale.
Largebackgroundinformationcanspeedupclustering.
3.
6DataimbalanceDataimbalancemeansthattheproportionofeachcategoryvariesgreatly.
Ifthedataisnotbalanced,suchascategory1accountingfor1%andcategory2accountingfor99%,thenthenetworkmodelbiasedpredictionresulttocategory2willgetthehighest.
Therateisaccuracy,buttheeffectisnotgoodinpracticalapplications.
Tosolvethedataimbalanceproblem,theclassweightaddstoweightthecrossentropy,asshowninEq.
(1).
1()classclasswlncp=+(1)Amongthem,pistheprobabilitythatthecorrespondingcategoryappearsintheoverallsample,cisahyperparameter,whichissetto1.
03.
3.
7InstancesegmentationTheinstancesegmentationbranchnetworkhasrealizedbydiscriminatingthelossfunction.
Thediscriminantlossfunctioncontainsthreeitems,whicharethevarianceterm,1324250173124053574513240002010030000040.
.
.
(a)MaxPoolingMaxUnpoolingInput4*4Output2*2Input2*2Output4*4ConvolutionDeconvolution1046CMES,vol.
122,no.
3,pp.
1039-1053,2020thedistanceterm,andtheregularterm.
Boththevariancetermandthedistancetermhaveacertainrangeofdistances,whichismanifestedinthattheembeddedpixelsarenolongersubjectedtotensionwhentheyarewithinthecentervδofthecluster,whichmeansthattheembeddedpixelsdonothavetobeaggregatedtoasinglepoint.
Similarly,theembeddedpixelsarenolongersubjecttothrustwhentheyarefartherawayfromthecenteroftheclusterthan2dδ.
Wherevδanddδarethethresholdsassociatedwiththetrainingsamples,basedonthedistancebetweentheinstances.
Thespecificcalculationofthediscriminantlossfunctionisshowninthefollowingequations:21111[||||]cNCvarcivcicLxCNδ+===∑∑(2)2111[2(1)ABABCCdistCCABCCLCCCCδ+===≠∑∑(3)11||||CregCCLC==∑(4)11CNCiixN==∑(5)Eq.
(2)representsthepullingforce,andEq.
(3)representsthethrust,Crepresentsthenumberofinstancesinthereallabel,cNindicatesthetotalnumberofpixelsinaninstance.
ixdenotestheembeddingvectorgeneratedbythepixelimappingintheexample,andcisthecenteroftheembeddingvectorofthemappingcorrespondingtoallthepixelsoftheinstanceinthereallabel,thatis,theembeddedaveragevalue,whichiscalculatedbytheEq.
(5).
Eq.
(4)isaregulartermtoensurethatthedistancebetweeneachclustercentermappedtothefeaturespaceandtheorigindoesnotbecomeveryfar.
Finally,thelossfunctionofthewholealgorithmistheweightedsumoftherespectivelossfunctionsofthetwobranchnetworks,andtheweightsareall0.
5.
ALaneDetectionMethodBasedonSemanticSegmentation1047Figure4:ClusteringfitresultgraphAsshowninFig.
4,Figs.
4(a)and4(b)areeffectdiagramsafterclustering,Figs.
4(c)and4(d)aretheresultsaftercorrespondingfitting.
Itcanbeclearlyseenfromthefigurethattheclusteringhascompletedthedetectionofthelaneline[He,Gkioxari,Dollaretal.
(2017);KimandPark(2017)],butcontainsmorebackgroundinformation,whichcanmoreaccuratelyrepresentthespecificpositionofthelanelineafterfitting.
4IdentificationoflanelinesIfonlythelanelineisidentified,thefinaloutputofthesegmentationnetworkactuallyhastwotypes,thelaneline,andtheimagebackground.
Afterthevirtualrealitydetectionfunctionisadded,theactualoutputofthesegmentationnetworkbecomesthreecategories,whicharenamelythelanesolidline,thelanedottedline,andtheimagebackground.
4.
1BuildingadatasetThedatasetusedinthispaperisthelabelmetool,whichusesthemonocularcameraonthewindshieldofthevehicletorecordvideointhesurroundingareaincludingthehigh-speedsectiontoobtainthedatamaterial.
50videoclipsaresampledevery30framesforeachvideo.
Theimagewithnolanelineisculled,theimagesizeis1280*720,and1800imagesareselectedasthetrainingset,andthedatasetisincreasedbyrandomleftandrightflip.
4.
2IdentificationprocessThealgorithminthispaperusesthesemanticsegmentationmethod.
Itonlyneedstochangethelastoutputchannelofthesemanticsegmentationfromtheoriginal2channelsto3channelstoincreasetherecognitionofacategory.
Thespecificoperationsareasfollows:(1)Changeofdataset:AsshowninFig.
5,intheFig.
5(a),allthelanelinepixelvaluesare(a)(b)(c)(d)1048CMES,vol.
122,no.
3,pp.
1039-1053,2020255,andthepixelvalueis0forthebackgroundandintheFig.
5(b),thepixelvalueis255.
Thedottedlineindicatesthatthepixelvalueis100forthesolidline;thepixelvaluefor0indicatesthebackground.
ThetrainingdataFig.
5(d)iscorrespondingtotheexamplesegmentationandtheoriginalFig.
5(c)donotneedtobechanged.
Inactualtraining,0,1,and2correspondtothecategorybackground,solidline,anddottedlinerespectively.
Figure5:Virtualandrealidentificationdataset(2)Algorithmchange:Thelastconvolutionallayerofthesemanticsegmentationbranchnetworkaddsanoutputchannel;thefeaturevectorofthelanedottedlineandthelanesolidlinearerespectivelyobtainedontheinstancesegmentationmapbythesemanticsegmentationmaskmap.
ThefinallanevirtualrecognitioneffectisshowninFig.
6.
Figure6:LanevirtualrealityrecognitioneffectmapFig.
6(a)showstheoriginalimagetodetect,andFig.
6(b)showstheeffectofthelanevirtualrealityrecognition.
Thesolidlinedrawninthefigureindicatestheactuallanesolidline,andthedottedlineindicatestheactuallanedottedline.
AsshowninFig.
6,theeffectivenessoftheproposedalgorithminthelanevirtualidentificationfunctionisverified.
(a)(b)(c)(d)(a)(b)ALaneDetectionMethodBasedonSemanticSegmentation10495Experimentalresults5.
1ExperimentalenvironmentandnetworkparametersTheexperimentalsoftwareenvironmentofthisarticleincludesubuntu16.
04(x64),python3.
5,cuda-9.
0,cudnn-7.
0,TensorFlow1.
10.
Hardwareplatform:GTX-1060GPU6gmemory.
Theprocessorisintel(R)core(TM)i5-6500MCPU@3.
20GHZ.
Theparametersinvolvedinthisalgorithmmainlyhavelearningrate,thesizeissetto0.
0001,thetrainingsampleimageresolutionis512*216,andthebatchsizeissetto4.
ThesizeoftheBatchsizecannothavecontinuedbythelimitationofGPUmemory.
TheexperimentaldatasetrespectivelyisaKITTIdatasetandaself-madedataset.
TheKITTIdatasetdoesnotdistinguishbetweenvirtuallanesandreallanes.
Theself-madedatasetisusedforlanevirtualrealitydetection.
Thecorrespondingembeddedpixelmappingfeaturespatialdimensionsare3and4,respectively,dδsetto3,vδsetto0.
5.
Asuitableneuralnetworkoptimizationalgorithmhelpsthemodeltoproducebetterandfasterresults.
5.
2ExperimentalresultsandanalysisThealgorithmisusedtotestthevideosequencesinvariousenvironmentalscenarios,includingdaytime,high-speed,night,rainydays,andsoon.
Thesescenesalsoincludecorners,vehicleinterference,occlusion,strongillumination,groundstripinterference,andsoon.
Inthecomplicatedroadconditions,8setsofvideowereusedandthealgorithmrunningspeedandaccuracywereusedasevaluationcriteria.
Thetestresultsareshownineachsceneoftheself-madedataset,differentcolorsrepresentdifferentlaneinstances,solidlinesindicatesolidlanes,anddashedlinesindicatelanedashedlines.
(a)(b)(c)Figure7:Testresultsindifferentenvironments:(a)Testresultsofcurvedroadsections;(b)Testresultsofrainydays;(c)TestresultsofnightFig.
7showsthelanelinerecognitionofgoodroadconditionsindifferentenvironments.
Fig.
7(a)showsthecorrectdetectionresultsofthelanesofthemulti-vehicleroadsegment,andlanelinerecognition.
Fig.
7(b)showsthecorners.
Thelanelineisaccuratedetectionresults,whichincludestheeffectoflanelinedetectionintherainyenvironment[Xiao,Zou,Chenetal.
(2018)],andFig.
7(c)showstheresultoflanelinerecognitioninthenightenvironment.
Opensourcedatasettestresults:differentcolorsrepresentdifferentlaneinstances,solid1050CMES,vol.
122,no.
3,pp.
1039-1053,2020linesindicatesolidlanes,anddashedlinesindicatelanedashedlines.
Figure8:TestresultsofTucsondatasetFig.
8showsthematerialsoftheTucsondatasetaretakenfromthehighway,butitisnoteasytodetect.
Thedifficultyliesintheseriouswearofthelaneline,andthelanelinefeaturesarenotobvious,butthedetectionresultofthealgorithmisbetter.
Figure9:CULanedatasettestresultsFig.
9showstheresolutionofCULanedatasetisrelativelylarge,anditcontainstheinterferenceinformationofmanyvehicleheads.
Inthecaseofgoodroadconditions,thealgorithmcanstillaccuratelyidentifythelaneline.
ALaneDetectionMethodBasedonSemanticSegmentation1051Figure10:KITTIdatasetdetectionFig.
10showstheKITTIdatasetprovidestwotypesofmarkings,theroad,andthecurrentlane.
Itismainlyusedforthesegmentationofthevehicle'stravelablearea.
Thelanelinemainlyexistsinthemiddleoftheroad[BehrendtandWitt(2017)].
Themethodaccuratelyidentifiesthelaneline.
6ConclusionThispaperproposesarobustlanedetectionmethodbasedonthecurrentmainstreamdeeplearningalgorithm.
Thealgorithmtestedbydifferentdatasetsandlanesindifferentweatherenvironments,theaccuracyandrobustnessoftheproposedalgorithmareverified.
Inthealgorithmcomparisonexperiment,thedetectionspeedofthealgorithmisfasterthanthealgorithminref.
[Oliveira,Burgard,Broxetal.
(2016)],andthecorrespondingaccuracyisalmostthesame,onlyabout1.
39%.
Incontrasttothetraditionalmethodbasedonvotingofref.
[Li,Zhou,Lietal.
(2018)],thealgorithmismuchslowerindetectionspeed,butitismuchhigherindetectionaccuracy,especiallyforthedetectionoffalsereallanelinesandthedetectionofcorners.
Toachieveeffectivedetection,incontrast,thealgorithmusingdeeplearningdoesnothavetheseproblems,andcanachieveaccuratedetection.
Acknowledgement:ThisworkissupportedbytheNationalNaturalScienceFoundationofChina(61772386);Jointfundproject(nsfc-guangdongbigdatasciencecenterproject),projectnumber:U1611262,HubeiUniversityofScienceandTechnology,MasterofEngineering,specialconstruction,projectnumber:2018-19GZ01,HubeiUniversityofScienceandTechnologyTeachingReformProject,projectnumber:2018-XB-023,S201910927028.
ConflictsofInterest:Theauthorsdeclarethattheyhavenoconflictsofinteresttoreportregardingthepresentstudy.
(a)(b)(c)(d)1052CMES,vol.
122,no.
3,pp.
1039-1053,2020ReferencesBehrendt,K.
;Witt,J.
(2017):Deeplearninglanemarkersegmentationfromautomaticallygeneratedlabels.
InternationalConferenceonIntelligentRobotsandSystems,pp.
777-782.
Chen,L.
C.
;Papandreou,G.
;Kokkinos,I.
;Murphy,K.
;Yuille,A.
L.
(2014):SemanticimagesegmentationwithdeepconvolutionalnetsandfullyconnectedCRFs.
arXiv:1412.
7062.
Chen,L.
C.
;Papandreou,G.
;Kokkinos,I.
;Murphy,K.
;Yuille,A.
L.
(2016):DeepLab:semanticimagesegmentationwithdeepconvolutionalnets,atrousconvolution,andfullyconnectedCRFs.
IEEETransactionsonPatternAnalysis&MachineIntelligence,vol.
40,no.
4,pp.
834-848.
Chen,L.
C.
;Papandreou,G.
;Schroff,F.
;Adam,H.
(2019):Rethinkingatrousconvolutionforsemanticimagesegmentation.
arXiv:1706.
05587.
Chen,L.
C.
;Zhu,Y.
;Papandreou,G.
;Schroff,F.
;Adam,H.
(2018):Encoder-decoderwithatrousseparableconvolutionforsemanticimagesegmentation.
EuropeanConferenceonComputerVision,pp.
801-818.
Chen,P.
;Lo,S.
;Hang,H.
;Chan,S.
;Lin,J.
(2018):Efficientroadlanemarkingdetectionwithdeeplearning.
InternationalConferenceonDigitalSignalProcessing,pp.
1-5.
De,B.
B.
;Neven,D.
;Van,G.
L.
(2017):Semanticinstancesegmentationwithadiscriminativelossfunction.
arXiv:1708.
02551.
Ding,L.
;Zhang,H.
;Xiao,J.
;Li,B.
;Lu,S.
etal.
(2018):Animprovedimagemixednoiseremovalalgorithmbasedonsuper-resolutionalgorithmandCNN.
NeuralComputingandApplications,vol.
31,no.
1,pp.
325-336.
He,A.
;Shan,Y.
;Song,D.
B.
(2018):Theinfluenceofin-carADASalarmsoundondriverbehavior.
AutomobilePracticalTechnology,pp.
38-39.
He,B.
;Ai,R.
;Yan,Y.
;Lang,X.
(2016):Accurateandrobustlanedetectionbasedondual-viewconvolutionalneutralnetwork.
IEEEIntelligentVehiclesSymposium,pp.
1041-1046.
He,K.
;Gkioxari,G.
;Dollar,P.
;Girshick,R.
(2017):MaskR-CNN.
InternationalConferenceonComputerVision,pp.
2961-2969.
Kim,J.
;Kim,J.
;Jang,G.
J.
;Lee,M.
(2017):Fastlearningmethodforconvolutionalneuralnetworksusingextremelearningmachineanditsapplicationtolanedetection.
NeuralNetworks,vol.
87,pp.
109-121.
Kim,J.
;Lee,M.
(2014):Robustlanedetectionbasedonconvolutionalneuralnetworkandrandomsampleconsensus.
InternationalConferenceonNeuralInformationProcessing,pp.
454-461.
Kim,J.
;Park,C.
(2017):End-to-endegolaneestimationbasedonsequentialtransferlearningforself-drivingcars.
IEEEConferenceonComputerVisionandPatternRecognitionWorkshops,pp.
30-38.
Li,Q.
;Chen,L.
;Li,M.
;Shaw,S.
L.
;Nüchter,A.
(2013):Asensor-fusiondrivable-regionandlane-detectionsystemforautonomousvehiclenavigationinchallengingroadscenarios.
IEEETransactionsonVehicularTechnology,vol.
63,no.
2,pp.
540-555.
ALaneDetectionMethodBasedonSemanticSegmentation1053Li,Q.
;Zhou,J.
;Li,B.
;Guo,Y.
;Xiao,J.
(2018):Robustlane-detectionmethodforlow-speedenvironments,Sensors,vol.
18,no.
12,pp.
4274.
Liu,W.
;Anguelov,D.
;Erhan,D.
;Szegedy,C.
;Berg,A.
(2016):SSD:singleshotmultiBoxdetector.
EuropeanConferenceonComputerVision,pp.
21-37.
Liu,X.
;Deng,Z.
(2018):Segmentationofdrivableroadusingdeepfullyconvolutionalresidualnetworkwithpyramidpooling.
CognitiveComputation,vol.
10,no.
2,pp.
1-10.
Narote,S.
P.
;Bhujbal,P.
N.
;Narote,A.
S.
;Dhane,D.
M.
(2018):Areviewofrecentadvancesinlanedetectionanddeparturewarningsystem.
PatternRecognition,vol.
73,pp.
216-234.
Neven,D.
;Brabandere,B.
D.
;Georgoulis,S.
;Proesmans,M.
;Gool,L.
V.
(2018):Towardsend-to-endlanedetection:aninstancesegmentationapproach.
IEEEIntelligentVehiclesSymposium,pp.
286-291.
Oliveira,G.
L.
;Burgard,W.
;Brox,T.
(2016):Efficientdeepmodelsformonocularroadsegmentation.
InternationalConferenceonIntelligentRobotsandSystems,pp.
4885-4891.
Peng,C.
;Zhang,X.
;Yu,G.
;Luo,G.
;Sun,J.
(2017):Largekernelmatters-improvesemanticsegmentationbyglobalconvolutionalnetwork.
IEEEConferenceonComputerVisionandPatternRecognition,pp.
4353-4361.
Redmon,J.
;Divvala,S.
K.
;Girshick,R.
B.
;Farhadi,A.
(2016):Youonlylookonce:unified,real-timeobjectdetection.
ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.
779-788.
Ren,S.
;He,K.
;Ross,G.
;Sun,J.
(2015):FasterR-CNN:towardsreal-timeobjectdetectionwithregionproposalnetworks.
IEEETransactionsonPatternAnalysis&MachineIntelligence,vol.
39,no.
6,pp.
1137-1149.
Talib,M.
L.
;Ramli,S.
(2015):AreviewofmultipleedgedetectioninroadlanedetectionusingimprovedHoughtransform.
AdvancedMaterialsResearch,vol.
1125,pp.
541-545.
Wu,C.
F.
;Lin,C.
J.
;Lee,C.
Y.
(2012):Applyingafunctionalneurofuzzynetworktoreal-timelanedetectionandfront-vehicledistancemeasurement.
IEEETransactionsonSystemsMan&CyberneticsPartC,vol.
42,no.
4,pp.
577-589.
Xiao,J.
;Luo,L.
;Yao,Y.
;Zou,W.
;Reinhard,K.
(2018):LanedetectionbasedonroadmoduleandextendedKalmanfilter.
PSIVT2017,pp.
382-395.
Xiao,J.
;Tian,H.
;Zhang,Y.
;Zhou,Y.
;Lei,J.
(2018):Blindvideodenoisingviatexture-awarenoiseestimation.
ComputerVisionandImageUnderstanding,vol.
169,pp.
1-13.
Xiao,J.
;Zou,W.
;Chen,Y.
;Wang,W.
;Lei,J.
(2018):Singleimagerainremovalbasedondepthoffieldandsparsecoding.
PatternRecognitionLetters,vol.
116,pp.
212-217.
Yoo,H.
;Yang,U.
;Sohn,K.
(2013):Gradient-enhancingconversionforillumination-robustlanedetection.
IEEETransactionsonIntelligentTransportationSystems,vol.
14,no.
3,pp.
1083-1094.
Yu,F.
;Koltun,V.
(2016):Multi-scalecontextaggregationbydilatedconvolutions.
arXiv:1511.
07122.