partmathplayer

C.
StephanidisandM.
Antona(Eds.
):UAHCI/HCII2014,PartII,LNCS8514,pp.
111–120,2014.
SpringerInternationalPublishingSwitzerland2014EAR-Math:EvaluationofAudioRenderedMathematicsHernisaKacorri1,2,ParaskeviRiga1,andGeorgiosKouroupetroglou11NationalandKapodistrianUniversityofAthensDepartmentofInformaticsandTelecommunications,Panepistimiopolis,Ilisia,15784Athens,Greece2CityUniversityofNewYork(CUNY)DoctoralPrograminComputerScience,TheGraduateCenter,365FifthAve,NewYork,NY10016USA{c.
katsori,p.
riga,koupe}@di.
uoa.
gr,hkacorri@gc.
cuny.
eduAbstract.
Audiorenderingofmathematicalexpressionshasaccessibilitybene-fitsforpeoplewithvisualimpairment.
Seekingasystematicwaytomeasureparticipants'perceptionoftherenderedformulaewithaudiocues,weinvesti-gatethedesignofperformancemetricstocapturethedistancebetweenreferenceandperceivedmathexpressions.
WeproposeEAR-Math,amethodo-logicalapproachforuser-basedevaluationofmathrenderingagainstabaseline.
EAR-Mathmeasuressystems'performanceusingthreefine-grainederrorratesbasedonthestructuralelements,arithmeticoperators,numbersandidentifiersinaformula.
Theproposedmethodologyandmetricsweresuccessfullyappliedinapilotstudy,where5sightedand2blindparticipantsevaluated39stimulirenderedbyMathPlayerinGreek.
Intheobtainedresults,weobservedthatstructuralelementshadthehighestmeanandvarianceoferrors,whichim-provedfrom18%inthefirstattemptto10%and7%intwofollowingattempts.
Keywords:mathematics,audiorendering,visuallyimpaired,blind,evaluation,userstudy.
1IntroductionTheWorldHealthOrganizationestimatesatotalof285millionpeoplewithvisualimpairmentofwhich39millionareblind[1].
Oneconsiderableaccessibilitybarrieristhecomprehensionofmathematicalconceptsandformulaethroughaudioorhapticmodality,whichrequiresignificantadditionalcognitiveprocessing[2].
Braille,linearinnature,facesdifficultiesinkeepingupwithcomplexmathematicalexpressionsrepresentedintwodimensions.
Moreover,as6-dotbrailleislimitedto64characters,itresultsincomplexnotationstorepresentthevastnumberofmathematicalstructuresandsymbols.
Usuallythisisperformedthroughescapesequences,whichmapmorethanonemeaningtothesamebraillecharacterdependingonthecontext.
Audio,apopularoutputmediumforthevisuallyimpaired,isoneapproachfavoredbyresearcherstocreateanaccessibleplatformformathematics[3].
Similartobraille,audiorenderingofamathematicalformula,eitherbyatrainedreaderfollowinga112H.
Kacorri,P.
Riga,andG.
Kouroupetroglouspokenstructure(e.
g.
[4-5])orbysyntheticspeechdrivenbyarule-basedsystemsuchas[6-10],[24],communicatestheexpressionusinglexicalandprosodiccuesinalinearway.
Researchershaveinvestigatedthepossibilityofalternatingbetweenmul-tipleaudioviewssuchassummarizationanddetaileddescription[7],usercustomiza-tionoftherenderingrules[7-8][10],andtheuseofnon-speechaudiocuesorspatialaudiotoindicatestructuraldelimiterswithintheformula[11-13].
Blindstudentsmayavoidadoptingapproachesforaccessingmathematicsthatre-quiretheuseofalanguageorrepresentationofmathematicswhichdiffersfromtheoneusedbytheirteachersorpeers.
Anadaptationoflexicalcues,whichindicatethestructuralinformationinthestudent'snativelanguage,isoftenrequired.
Inthepastfewyears,theSpeechandAccessibilityLab,UniversityofAthens,hasfocusedonaccessiblemathematics[14-15]andthesupportoftheGreeklanguageforMathPlayer[8].
Toconstructtheaudiorenderingruleswetypicallyconsultblindstudentsduringtheinitialdesignandthedevelopmentphase.
Thechallengesduringthesephasesare:achievingadelicatebalancebetweenresolvingambiguityandreduc-ingverbosity,andfine-tuningothernon-lexicalparameterssuchaspauses,pitch,andvolume.
Whenselectingbetweendifferentlyparameterizedrenderingstyles,weareinterestedinquantitativeresultsobtainedfromtheuserstomeasuretheperformanceofoursystem.
Despiteadvancesintheaudiorenderingofmathematics,therearenowidelyacceptedmethodologiesorbenchmarks.
Inthispaper,weproposeEAR-Math,amethodologicalapproachalongwithasetofmetrics,designedtoevaluateandcompareaudiorenderingsystemsformathex-pressions.
Furthermore,wepresentresultsfromanapplicationoftheproposedme-thodologyinapilotstudyevaluatingtheGreekaudiorenderingrulesofMathPlayer.
2RelatedWorkRelativelyfewresearchershaveevaluatedtheperformanceofmathaudio-accessap-proaches.
Wereviewedpriorstudiesevaluatingmathaudiorenderingsystemsbasedonperceptionbyvisuallyimpairedandsightedparticipants.
WepresentinTable1therelativemethodologiesbasedonthegroupsofparticipantsrecruited,selectedstimuli,andthetypeoffeedbackobtainedfromtheparticipants.
MathTalk[9]incorporatesasetofrulestoinsertprosodiccuesintospokenexpres-sions.
Inauserstudy,participantswrotetheirrecalloftheformulaeoncetherender-ingwasoverusingeitherquestionmarksorellipsestodenoteanymissingobjects.
Responsesweregradedforcomprehensionofthestructureandretentionofcontent.
Acorrectanswerrequiredover75%ofanexpression'scontentandthemajorstructuralfeatures.
TechRead's[16]usedprosodytoindicatenestedstructure.
Sightedparticipantswereaskedtochooseamong4answersmatchingtheaudiostimuli.
Of16renderedmathexpressions,3wereintrainingwhiletheremainingwereplayedin3speechmodes.
EAR-Math:EvaluationofAudioRenderedMathematics113Gellenbecketal.
[17]conductedastudytoassesswhetherinsertionofpausesin-sidespokenmathematicalexpressionsreducesambiguitybetweensimilaralgebraicexpressions.
Participantsheardstimuliandratedside-by-sideformulae,usinga0-10Likert-scale,onhowwelltheythoughtthevisualexpressionmatchedtheaudio.
Be-forethestudy,participantsgainedfamiliaritythroughfourungradedexampletasks.
Murphyetal.
[13]usedamixtureofnon-speechauditorycues,modifiedspeech,andbinauralspatializationfordisambiguation.
Toevaluateinitialcues,theauthorsconductedafour-partaccessibleonlinesurveywithnotrainingphase.
Thefirstparttestedwordrecognitionandlexicallanguageofspokenmathematics,thesecondmeasuredunderstandingofspokenequationswithnon-speechsoundelements,thethirdassessedintuitivenessofspatialattributes,whilethefourthwaspurelyqua-litative.
Thelistenerschoseamong3alternativesrenderedbothgraphicallyandacoustically.
I-Math[18]wasevaluatedbyparticipantswithvaryingvisualability.
Audiostimuliweregradedonspeechquality,comprehensioneffort,andtranscription.
Transcriptionswerecomparedtooriginaltextualexpressionsandcorrect,missing,andincorrectwordswerecounted.
Thenumberofcorrectwordsandpositionswereevaluatedusingprecision,recall,andF-score.
Table1.
UserstudiesevaluatingapproachesforaudiorenderingofmathematicsMathTalkTechReadGellenbecketal[10]Murphyetal[14]I-Math#Participants24(sighted)20(sighted)16(sighted)35(sighted)21(blind)35(sighted)6(blind),4(v.
i.
)Participants'Ageuniversitystudentsuniversitystudentsuniversitystu-dents~22,7~38studentsandteachersKnowledge/UseofMathdaily-infrequentuse(qualication>'O'Level)goodknowledgeofmathconstructsinstimuliN/A10yearssincelastexam(mean)N/A#Stimuli2matchedsetsof12and3samples16(3samples)30min.
session13(6partial)35StimuliCategoriesfractions,parenthe-sisedsubexpres-sions,superscriptsimplefractions,radicals,sum,limits,integrals,trigono-metryselectedfromthegeneralareaofalgebraN/Afractions,vectors,superscripts,log,radicals,lim,sum.
,trigonometry,inte-gralsTrainingexplanationandsyntheticspeechfamiliarizationexplanationandexamplesdescriptionandexamples-N/ALanguageEnglishEnglishEnglishN/AThai(tonal)Participants'Answerswrittenmathexpres-sion4multiple-choicesolutionsratingof2alter-natives3multiple-choicesolutionstranscribedtextBaseline/Com-parisonlexicalvs.
prosodiccues,lexicalvs.
neitherlexicalnorprosodiccuesnaturalvs.
enhancedvs.
unenhancedspeechpausesvs.
nopausessightedvs.
blindparticipants-Quantitativeaccuracyaccuracyaverageratingaccuracyintuitiveness,confusionintelligibility,speechquality,understand-ingeffortQualitative---Alternatives,commentsusefulness,easeofuse114H.
Kacorri,P.
Riga,andG.
Kouroupetroglou3EvaluationMethodologyEvaluationofAudioRenderedMath(EAR-Math)isproposedasanexperimentalmethodologyforuser-basedperformanceevaluationofmathematicalexpressionsdescribedbyarule-basedsystemwithaudiooutput.
Theaccuracyscoresachievedoverthestimuliinauserstudydependnotonlyonthesystemoutput,butalsoonthedifficultyofthemathematicalexpressionschosenandonthegroupofparticipants.
Thus,EARincludesabaselineinthestudiestomaketheresultsmoremeaningful.
Thisbaselinemaytypicallybeatrainedreaderfollowingaspokenstructuretoreadmathematicalexpressionsaloud.
Thebaselinecouldbeapreviousversionofthesys-temforresearcherswhowishtoevaluateimprovementswithintheirsystem.
Similar-ly,EAR-Mathallowsfordirectcomparisonbetweentwoormorealternativesystems.
3.
1ExperimentDesignSinceitisnotfeasibletoexhaustivelyexaminethemathematicalexpressionslikelytoberenderedbyasystem,thedesignofasmallerrepresentativesubsetisrequired.
Stimulishouldcoveranextendednumberofmathematicalstructures(e.
g.
fractions,integrals,roots,subscripts,andarrays).
Theyshouldbeengineeredtorevealpotentialambiguityinthesystemoutputandhavearealisticlength.
Thenumberofstimulishouldalsoallowincorporationinanexperimentalsessionlastinglessthan2hours.
WeconsideredthepubliclyavailablesetofformulaeforASTeRdemonstration[19]asapotentialcandidate.
TheEAR-Mathuserstudyconsistsofthreephases.
Asapreliminaryphase,partic-ipantsareintroducedtothesystem'sruleswithsampleexamplestogainfamiliaritywiththegeneratedoutput.
Symbolsnaming,alistthatcanextendover2.
000Unicodecharacters,ismentionedonlyforasmallnumberofcharacters,whichareincludedinthestimuliandareassignedanon-familiarname.
Inphase2,participantslistentothedescriptionofthemathformula(renderedei-therbythesystembeingevaluatedorthebaseline)andarepromptedinparalleltokeepnotesoftheformula.
NotesmaybetakenonpaperforsightedparticipantsinBrailleforblindparticipants,inLaTeX,orinanyotherformatandmeansthepartici-pantisfamiliarwith.
Stimuliareplayeduponrequesttwomoretimesandthepartici-pantsareallowedtocorrecttheirinitialguess.
Inphase3ofthestudy,participantsheartwosequentialversionsofthesamema-thematicalexpressionrenderedbythesystemandbaseline(withalternatingse-quence).
Theparticipantscanhearanyoftheoutputsasmanytimesastheywish.
Theyarethenaskedtorespondona1-to-10Likert-scalequestionabouthowsuretheyarethattheexpressionsareidentical.
Foreachofthetwoversions,participantswereaskedtwo1-to-10Likert-scalequestionsabouttheunderstandabilityandnatu-ralnessoftherenderedformula.
E3.
2PerformanceMetriEAR-Mathproposesaperftendedmathexpressionansightedorblindusers.
OurSpecifically,theperformanoffine-grainederrorratesobersandidentifiersofthem/Theproposedmetricsarpressionwiththeoriginaleformedbetweenthesyntaxdemonstratingtheprocess.
ofinsertions,deletions,andreferenceoverthetotalnumthreecategoriesofelementnentsofamathformulas(OER)isfocusedonmathIdentifierandNumberErroersandnumericalvalueswtheperceivedexpressionpaFig.
1.
AnexamplEAR-Math:EvaluationofAudioRenderedMathematicsicsformancemetrictomeasurethedistancebetweenthendtheactualonedeliveredbythesystemasperceivedmetricistailoredtoaccountforbothcontentandstructunceofamathaudiorenderingsystemisdefinedasatriponthestructuralelements,arithmeticoperators,andnumathformula:#############rederivedbyfirstaligningtheperceivedmathematicalexpressiontoberenderedbythesystem.
Alignmentispxtreesofbothexpressions.
Figure1illustratesanexamOurmetricscanthenbecomputedaserrorrates;numdsubstitutionsintheperceivedexpressioncomparedtomberofelementsinthereferenceexpressionforeachofts.
StructureErrorRate(SER)involvesstructuralcomsuchasfractions,roots,andarrays.
OperatorErrorRhematicaloperatorse.
g.
plus,minus,andtimes.
AndlorRate(INER)representthenumberoferrorsforidentwithintheexpression.
Forexample,givenareferenceair,wegetthesyntaxtreesandalignmentofFig.
1.
leofalignmentandSER,OER,andINERcalculation115in-dbyure.
pletum-ex-per-mplemberthefthempo-Ratelast,tifi-and116H.
Kacorri,P.
Riga,andG.
KouroupetroglouWordErrorRate(WER),commonlyusedinthefieldofspeechrecognition,wasadoptedforthedesignoftheproposedperformancemetrics.
Togroupthecompo-nentsthatcompriseamathematicalformula,wewereinspiredbythetokenandlayoutelementsinPresentationMathML,suchasforoperators,foridentifiers,andfornumbers.
Thisgroupingsimplifiedincorporatinguser'sfeedbackto-wardstheimprovementofourGreekaudiorenderingrulesforMathMLexpressionsinMathPlayer.
4PilotUserStudyToassessfeasibility,time,andstatisticalvariabilityrequiredforevaluatingMath-Player'sGreekaudiorenderingrules,weconductedapilotuserstudy.
Thishelpedtotest,adjustandobtainvaluableinsightsinthedesignofEAR-Mathandparticularlyitsmetricsandisshownasanappliedexampleoftheproposedmethodology.
Forthepurposesofthisstudy,thesystemwasnotevaluatedagainstabaseline.
Therefore,onlythefirsttwophasesofEAR-Mathwereemployed.
StimuliwerebasedonthesetofformulaeforASTeRdemonstration[19]renderedthroughtheDimitrisofAcapelaText-to-Speech[20]drivenbyMathPlayerwithlexicalandprosodiccues.
Thevoice,volume,andspeedforallstimuliwereidentical.
Mathematicalexpressionswereprerecorded,thustherewasnonavigationandnocontroloverthespeechvelocity.
Ofthe7participantsrecruitedforthestudy:2werecongenitallyblindand5weresighted.
ParticipantattendedaGreekuniversitywhoseentrancerequiredahighlevelofmathematicsaccordingtotheGreekeducationalsystem(6weremajoringincom-puterscienceand1inlaw).
Thus,allparticipantshadbeenexposedtomorecomplexmathematicalexpressionsthanthestimuli.
Therewere5menand2womenofages20-34(averageage25.
9).
Whileusingasightedcomparisongroupwouldn'tbeei-therfairorappropriate,weincludedpeoplebothwithandwithoutvisualimpairment.
Therationalebehindthisdecisionisthataudiorenderingofmathematicshasuniver-salbenefits(e.
g.
totemporaryprintdisabledpeople).
Inaddition,recruitmentofahomogeneousgroupofpeoplewithvisualimpairmentandsimilarperformanceinmathematicassessmentsfromageographicallydispersedpopulationisdifficult.
Thesechallengesmaysetuserstudiesinvestigatingmultipleimprovementswithinthesamesystemwithadequatepowerofvisuallyimpairedparticipantsimpossible.
Duringthestudy,participantswouldlistentomathematicalexpressionsandwritedowntheperceivedformulae.
Theywereallowedtomakechangestotheirinitialguesstwomoretimes.
Wecollectedallthreeperceivedversionsforeachoftheex-pressions.
Sightedparticipantswrotedowntheiranswersusingapen-and-paperexperimentalpacketwhileblindparticipantsusedformatsandtechnologythatweremostfamiliartothem.
OneparticipantusedNemeth[21-22].
Theotherusedavariantmathnotationhehaddevelopedandusedthroughouttheyears.
Aftertheex-perimentalsession,blindparticipantswouldreadtheirnotesanddescribetheiran-swerstoasightedmemberoftheteamwhowouldthenvisualizetheexpressionintwo-dimensions.
EAR-Math:EvaluationofAudioRenderedMathematics117ShortmathematicalexpressionsdescribedinthedesignofGreekaudiorenderingrulesformath[23]servedassampleexpressionstofamiliarizetheparticipantswiththeexperimentandtheoutputofthesystem.
5ResultsInourstudy,MathPlayer'sGreekaudiorenderingofmathisevaluatedwithasetof49stimuliwith190structuralelements,113operators,and295identifiersornum-bers.
Whilenotcomparedagainstabaseline,thisevaluationdemonstratestheEAR-Mathperformancemetricswithintheproposedexperimentalsetup.
Inourcalculations,allreferenceexpressionsfromthestimuliandperceivedex-pressionsweredrawnassyntaxtrees.
AcomputersciencePhDstudent'manually'alignedtheperceivedformulaesyntaxtreetothereferencetreeandrecordedtheer-rors.
Whileidentifyingtheoperator,identifier,andnumbererrorswereastraightfor-wardprocess,structuralelementserrorsweremorechallenging,especiallywhenthecorrectstructurewasimproperlypositionedinthetree.
Animproperpositionofthestructuralelementwascalculatedasadeleteandre-insert.
Thenumberofmathematicalelementsforeachstimulivariedbycontext:0-13structural(mean5),1-10operators(mean2.
97),and3-18identifiersandnumbers(mean7.
76).
Thereforetheerrorrateswerecalculatedfortheaggregatedelementsamongallstimuli,asshowninTable2.
Table2.
OverallerrorratesinthestimulisetSEROERINER1stAttempt18%12%11%2ndAttempt10%6%4%3rdAttempt7%4%2%Figure2showsthedistributionoftheSER,OER,andINERintheperceivedma-thematicalexpressionsfromtheparticipantsasboxplotswithwhiskersatthe1.
5IQR(inter-quartilerange).
Toaidthecomparison,meanvalues,illustratedwithastar,areaddedaslabelsatthetopofeachplot.
Forthefirstattempt,weobservethatboththestructuralerrorrate(SER)andtheerrorrateforidentifiersandnumbers(INER)havehighervariancethantheoperators'errorrate(OER).
WespeculatethisisduetotheinherentdependenceoftheINERonSER.
Forexample,whenparticipantsdonotunderstandastructureandomitit,theyoftenomittheidentifiersandnumberswithinthestructure.
Wealsoobservethatparticipantstendtoimprovetheirerrorsthesecondandthethirdtimetheyhearthemathematicalexpression.
Thissuggeststhattheaudiorenderingmighthavebeenaccurate,butotherfactors(suchasaudiomemoryandfamiliaritywiththesystem)mayhaveaneffectontheresultsandshouldbetakenintoaccountwhendesigningtheexperiment.
118H.
Kacorri,P.
Riga,Fig.
2.
ErroFig.
3.
StimuliwiWeanalyzedthedataanturallevel(12and39)hadand38%forexpression(126DiscussionandFThispaperhasdescribedEAmathaudiorenderingsystemthedistancebetweenthereEAR-MathmetricsaredefRate,andIdentifierandNuandG.
KouroupetroglouorratedistributionineachofthethreeattemptsththeHighestStructureErrorRateintheUserStudyndfoundthatthetwomostmisperceivedstimuliatastrSERimprovingineachoftheattemptsfrom93%to62)andfrom14%to3%and3%forexpression(39).
FutureWorkAR-Math,amethodologicalapproachfortheevaluationmswithusersandasetofperformancemetricsthatcapteferenceandtheperceivedmathexpressions.
Inparticufinedaserrorrates(StructureErrorRate,OperatorErumberErrorRate)calculatedbycomparingthesyntaxtrruc-69%noftureular,rrorreesEAR-Math:EvaluationofAudioRenderedMathematics119ofthereferenceandperceivedmathematicalexpressions.
Ourmethodologyandme-tricsaredemonstratedinapilotuserstudyevaluatingtheGreekaudiorenderingrulesofMathPlayerwith7participantsand39stimuli.
Obtainedresults,showthatStruc-tureErrorRatehadhighermeanandvariancethantheothertwometrics,whichwasimprovedthesecondandthirdtimeparticipantsheardthestimuli.
Thisresearchhastwokeycontributions.
First,itprovidesguidanceforresearchersconductinguser-basedevaluationstudiestomeasuretheperformanceofmathaudiorenderingsystemsagainstabaseline,comparealternativesystems,oriterativelyeva-luateimprovements/styles.
Second,itprovidesresultsfromapilotstudytoassessfeasibility,time,andstatisticalvariabilityrequiredforacasestudy.
Infutureworkwewanttoexplorealternativewaystoautomatethecalculationoftheproposedmetricswithouthumanintervention.
Thiswouldallowformorerobustresultswithlesshumanerrorsduringthedataprocessingsteps.
Acknowledgments.
WewouldliketothankNeilSofferfromDesignScienceforhissupportduringthedevelopmentoftheGreekrulesinMathPlayer.
Thisresearchhasbeenco-financedbytheEuropeanUnion(EuropeanSocialFund–ESF)andGreeknationalfundsthroughtheOperationalProgram"EducationandLifelongLearning"oftheNationalStrategicReferenceFramework(NSRF)undertheResearchFundingProject:"THALIS-UniversityofMacedonia-KAIKOS:AudioandTactileAccesstoKnowledgeforIndividualswithVisualImpairments",MIS380442.
References1.
WorldHealthOrganization.
FactSheet282:VisualImpairmentandBlindness(2013),http://www.
who.
int/mediacentre/factsheets/fs282/en/index.
html2.
Dick,T.
,Kubiak,E.
:Issuesandaidsforteachingmathematicstotheblind.
TheMathemat-icsTeacher90(5),344–349(1997)3.
Freitas,D.
,Kouroupetroglou,G.
:SpeechTechnologiesforBlindandLowVisionPersons.
TechnologyandDisability20,135–156(2008)4.
Chang,L.
A.
,White,C.
M.
,Abrahamson,L.
:Handbookforspokenmathematics.
LawrenceLivermoreNationalLaboratory(1983)5.
Nemeth,A.
:TipsforReadingMathOutloudtoBlindStudents,http://www.
rit.
edu/easi/easisem/talkmath.
htm6.
Edwards,A.
D.
,McCartney,H.
,Fogarolo,F.
:Lambda:amultimodalapproachtomakingmathematicsaccessibletoblindstudents.
In:Proceedingsofthe8thInternationalACMSIGACCESSConferenceonComputersandAccessibility,pp.
48–54(2006)7.
Raman,T.
V.
:ASTER-Towardsmodality-independentelectronicdocuments.
In:ElectronicMultimediaPublishing,pp.
53-63,SpringerUS(1998)8.
Soiffer,N.
:Aflexibledesignforaccessiblespokenmath.
In:Stephanidis,C.
(ed.
)UAHCI2009,PartIII.
LNCS,vol.
5616,pp.
130–139.
Springer,Heidelberg(2009)9.
Stevens,R.
D.
:Principlesforthedesignofauditoryinterfacestopresentcomplexinforma-tiontoblindpeople.
Doctoraldissertation,UniversityofYork(1996)10.
Yamaguchi,K.
,Komada,T.
,Kawane,F.
,Suzuki,M.
:Newfeaturesinmathaccessibilitywithinftysoftware.
In:Miesenberger,K.
,Klaus,J.
,Zagler,W.
L.
,Karshmer,A.
I.
(eds.
)ICCHP2008.
LNCS,vol.
5105,pp.
892–899.
Springer,Heidelberg(2008)120H.
Kacorri,P.
Riga,andG.
Kouroupetroglou11.
Bates,E.
,Fitzpatrick,D.
:Spokenmathematicsusingprosody,earconsandspearcons.
In:Miesenberger,K.
,Klaus,J.
,Zagler,W.
,Karshmer,A.
(eds.
)ICCHP2010,PartII.
LNCS,vol.
6180,pp.
407–414.
Springer,Heidelberg(2010)12.
Brewster,S.
A.
:Usingnon-speechsoundtoovercomeinformationoverload.
Dis-plays17(3),179–189(1997)13.
Murphy,E.
,Bates,E.
,Fitzpatrick,D.
:Designingauditorycuestoenhancespokenmathe-maticsforvisuallyimpairedusers.
In:Proceedingsofthe12thInternationalACMSIGACCESSConferenceonComputersandAccessibility,pp.
75–82(2010)14.
Kouroupetroglou,G.
,Kacorri,H.
:DerivingAccessibleScienceBooksfortheBlindStu-dentsofPhysics.
ProceedingsoftheAmericanInstituteofPhysics1203,1308–1313(2010)15.
Tsonos,D.
,Kacorri,H.
,Kouroupetroglou,G.
:Adesign-for-allapproachtowardsmulti-modalaccessibilityofmathematics.
AssistiveTechnologyResearchSeries25,393–397(2009)16.
Fitzpatrick,D.
:TowardsAccessibleTechnicalDocuments:ProductionofSpeechandBrailleOutputfromFormattedDocuments.
Doctoraldissertation,DublinCityUniversity(1999)17.
Gellenbeck,E.
,Stefik,A.
:Evaluatingprosodiccuesasameanstodisambiguatealgebraicexpressions:anempiricalstudy.
In:Proceedingsofthe11thInternationalACMSIGACCESSConferenceonComputersandAccessibility,pp.
139–146(2009)18.
Wongkia,W.
,Naruedomkul,K.
,Cercone,N.
:I-Math:anIntelligentAccessibleMathemat-icssystemforPeoplewithVisualImpairment.
ComputationalApproachestoAssistiveTechnologiesforPeoplewithDisabilities253,83–108(2013)19.
Raman,T.
V.
:Mathematicsforcomputergeneratedspokendocuments-ASTeRDemon-stration,http://www.
cs.
cornell.
edu/home/raman/aster/demo.
html20.
Acapelatext-to-speech,http://www.
acapela-group.
com/21.
Kouroupetroglou,G.
,Florias,E.
:GreekBrailleScientificNotation–ApplicationinIn-formationSystemsfortheBlind.
EducationandRehabilitationCenterfortheBlind,Athens(2003)ISBN960-87918-0-422.
Nemeth,A.
:TheNemethBraillecodeformathematicsandsciencenotation:1972revi-sion.
ProducedinbraillefortheLibraryofCongress,NationalLibraryServicefortheBlindandPhysicallyHandicappedbytheAmericanPrintingHousefortheBlind(1972)23.
Kacorri,H.
:AudioRenderingRulesforMathematicalMathMLExpressionsinGreek,DiplomaThesis,UniversityofAthens(2006)24.
Ferreira,H.
,Freitas,D.
:AudiorenderingofmathematicalformulaeusingMathMLandAudioMath.
In:Stary,C.
,Stephanidis,C.
(eds.
)UI4ALL2004.
LNCS,vol.
3196,pp.
391–399.
Springer,Heidelberg(2004)