openmathplayer

DevelopmentandInitialEvaluationoftheClearSpeakStyleforAutomatedSpeakingofAlgebraLoisFrankel,1BethBrownstein,1NeilSoiffer,2&EricHansen11EducationalTestingService,Princeton,NJ2DesignScience,LongBeach,CATheworkdescribedinthisreportisthefirstphaseofaprojecttoprovideeasy-to-usetoolsforauthoringandrenderingsecondary-schoolalgebra-levelmathexpressionsinsynthesizedspeechthatisusefulforstudentswithblindnessorlowvision.
Thisreportdescribestheinitialdevelopment,softwareimplementation,andevaluationoftheClearSpeakspeechstyle—implementedforinitialtestingpurposesusingprerecordedsyntheticspeech,andimplementedlaterintheprojectasacollectionofpredefinedrulesandinsomecasesvariations(calledpreferencesinClearSpeak)—forautomaticallygeneratedsyntheticspeech.
Inadditiontowording,speechstylescanspecifypausing(orotherprosodiccues)withinthespeech.
TheClearSpeakstylefocusesonspeechforsecondaryschoolalgebra.
TheevaluationcomparesaprototypeoftheClearSpeakstyletotwopre-existingspeechstyles:MathSpeakandSimpleSpeak.
Theprimaryparametersevaluatedarestudents'successindrawingconclusionsaboutthecontentandstructureofmathexpressionsandtheirperceptionsregardingthefamiliarity,helpfulness,andunderstandabilityoftheexpressionsasspoken.
ThisWorddocumentisfullyaccessibleusingthetoolsdescribed,whicharedescribedinAppendixE,alongwithadditionalaccessibilitynotes.
Keywordsmath,accessibility;blindness;visualimpairment;text-to-speech;MathML;algebra;STEM;ClearSpeak;assistivetechnology;screenreaderCorrespondingauthor:L.
Frankel,E-mail:lfrankel@ets.
orgThereisagreatneedtoimprovemathematicsachievementinstudentswithvisualimpairments(SVIs),forwhomthereisalargeachievementgapcomparedtostudentswithoutdisabilities(Blackorby,Chorost,Garza,&Guzman,2003).
Thisgapisconsistentwiththeverylownumberwhoworkinscience,technology,engineering,andmath(STEM)careersandisrecognizedbyleadingorganizationsservingindividualswithblindnessorlowvision.
Itisonereasonmathisafocusoftheirresearchandproductdevelopmentefforts(AmericanPrintingHousefortheBlind,2007).
Removingaccessbarrierstomathinstructionandtestingisanecessarystepinnarrowingthisgap.
OurprojectseekstoremoveaccessbarriersbyimprovingSVIs'accesstomathexpressionsincomputer-basedmaterials,includingassessments,instruction,andotherinformationalmaterials—materialsthatarealreadyfullyaccessibletotheirsightedpeers.
Alltoooften,suchmaterialsarenotfullyaccessible(orarecompletelyinaccessible)toSVIs.
Thusthereisaclearaccessgap,whichcanreasonablybeexpectedtobeacontributortothedocumentedachievementgap.
ThisreportdescribesthedevelopmentandinitialevaluationofClearSpeak,aspeechstyleintendedtoimproveaudioaccesstomathexpressionsbyallowingMathML-encodedexpressionscontainedinMicrosoftWordorHTMLdocumentstoberenderedinunderstandable,mathematicallyaccurate,andeasilynavigablesyntheticspeech.
SVIstypicallybenefitfromavarietyoflearningmodalities,dependingonindividualneeds.
Forexample,studentsmightusemodalitiesthatarevisual(largeprint,mechanicalorelectronicenlargement,fontorcolormodifications),tactile(refreshableorhardcopybraille,tactilegraphics),and/orauditory(livehumanreader,prerecordedhumanspeech,orliveorprerecordedsyntheticspeech).
Textbooksandothermaterialspreparedinadvancecanbeproducedintherequiredformats,butdoingsoinatimelyfashionisoftenproblematicformathmaterials,withdifficultiesincreasingwiththelevelofmath.
Materialsgeneratedonthespot(e.
g.
,worksheetsorassignmentsdevelopedtomeetimmediateneeds)byteachersorothersareevenmoreproblematic,becausetheycannotalwaysbereadilypreparedinthesomeoftheformatsneeded.
Eventhoughalivereadercan,intheory,workwithmaterialonthespot,suchareadergenerallyneedsdetailedguidelinesoracarefullypreparedscriptinordertoreliablyrelatethemathunderstandablyandconsistently.
Thatthereisaperceivedneedforsuchguidelinesisevidencedbythedevelopmentanduseoverthepastfewdecadesofavarietyofguidelinesandrules(seediscussionunderProgress).
Assistivetechnology(AT)suchasscreenreaders,whichprovidesyntheticspeechforelectronictext,allowscompatiblematerialtoberepresentedinmultiplesimultaneousmodesofaccess.
Screenreadersenablethecontent-providertopresentthesametextualmaterialsimultaneouslyinaudio,onscreen,orviarefreshablebrailletoprovideamultimodalpresentationthatcanmakethematerialmoreaccessibletosomelearners,asarguedbyStevens,Edwards,andHarling(1997)whentheyconcludedthat"thedifficultyoftheproblemofmakingmathematicsaccessiblesuggeststhatamulti-modalapproachisnotonlyattractivebutnecessary"(p.
89).
However,althoughsomeschoolsareabletoobtainelectronicversionsofsomebooksandtestsfrompublishersforusewiththeirstudents'AT,andalthoughteacherscaneasilydevelopelectronictext-onlydocumentsforclassroomorhomeworkuse,mathematicalmaterialhadnotheretoforebeenabletobemadeaccessibleinthiswaybecausethemathematicalexpressionshadusuallybeenopaquetotheAT:Itdidnot"know"howtospeakthem,especiallyinawaythatprovidesaccesstotheexpressions'structure,whichmaybetwo-dimensional(e.
g.
,superscripts,numeratorsanddenominators).
Pasteffortstoaddresstheselimitationsarediscussedinthesection,ComputerizedSpeechRules.
OurprojectaimstohelpclosetheaccessgapbymakingmathexpressionsinmoreelectronicmaterialsaccessibletoSVIsviaaudioassistivetechnologyusingsyntheticspeech.
Thesyntheticspeechcanserveeitherasthesolemodalityorasoneofmultiplemodalities(e.
g.
,speechwithlargeprint,embossedbraille,orrefreshablebraille).
Becausethemasteryofsecondaryschool–levelmathematics(particularlyAlgebra1)hasbeenshowntobeanimportantgatewaytocollegecompletionaswellastothestudyofandcareersinmathandotherSTEMdomains(Fennelletal.
,2008),andbecauseAlgebra1introducestypesofmathematicalstructuresthatcanbenefitfromthetoolsourprojectisdeveloping,weselectedAlgebra1andsomeofAlgebra2asthecontentfocusfortheproject.
MathExpressionsinAudioMathexpressionshavebeennotoriouslydifficulttoaccessinaudio,dueinparttotheirstructure,whichisoftennonlinear,layered,and/ortwo-dimensional;theycanconsistofmultiplesubstructures,suchasfractions,exponents,androots,andcaninvolvemultipleoperationsormultiplelevelsofnesting.
Considerthequadraticformula:.
Itisafractionwithmultiplesubstructures.
Thenumeratorisasum,andthedenominatorisaproduct.
Insidethesuminthenumeratorisasquareroot,whichhassubstructuresofitsown—andsoon.
Tobeuseful,anaudiorenditionmustallowthelistenertonotethestructureandnestedsubstructuresandtodistinguishtheboundariesofeachstructure—inthecurrentcase,theboundariesofthesquareroot,theexponent,thenumerator,andthedenominator.
Thevisual/spatiallayoutallowsasightedpersontodistinguishthoseboundariesataglance.
Inaddition,anaudiorenditionshouldnotneedlesslycontributetoverbalormemoryload.
Thatconsiderationisimportantbecauseitisgenerallyheldthatshort-termmemorycanhold7±2chunksofinformationatatime(Miller,1956)andonlyforabout15–30seconds(Atkinson&Shiffrin,1971).
Anaudiorenditionthatisunambiguous,thatrevealsessentialstructuresandsubstructures,andthatavoidsexcessiveverbalmemoryloadisamuchgreaterchallengetoproducethanisthevisuallayoutusablebysightedpeople.
Toproducesucharenditionalgorithmicallyisanevengreaterchallenge.
Thesedifficultiesarecompoundedbythefactthatsolvingmathproblemstypicallyrequiresworkingthroughmultiplesteps,eachinvolvingmathematicalexpressionsofvaryingdegreesofcomplexity.
Eventhebestpossiblespeechforsomeexpressionswillbeunavoidablylong,andso,providingtheabilitytonavigateanexpressioninteractivelyisalsoessential.
Whilebeyondthescopeofthecurrentpaper,wedevelopedinteractivenavigationinanotherphaseofourproject.
SeeFrankel,Brownstein,andSoiffer(2016).
ProgressThissectiondescribesrelevantworkthatprecededthisstudy,startingwithworkinaudiomathaccessibilityingeneral,andworkthatdirectlyinfluencedthedevelopmentofthenewspeechstylethatisthesubjectofthisevaluation.
Severalmethodsforspeakingmathexpressionshavebeendevelopedoverthepastfewdecadesbasedonavarietyofgoalsandguidingprinciples.
Thissectionreviewsthemostsignificantofthese,startingwithspeechstylesthatwereinitiallydesignedforhumanreaders(asopposedtocomputerizedtext-to-speech,whichrequiresmoreformalizedrules)andcontinuingwithadiscussionofdevelopmenteffortsforcomputerizedmathspeech.
MathSpeechRulesforHumanReadersSeveralstylesofmathspeechrulesforhumanreadershavebeendeveloped.
Thebestknownsetsofsuchrulesarethosedevelopedbyablindmathematician,thelateAbrahamNemeth,bestknownasthedeveloperoftheNemethBrailleCodeforMathematicsandScienceNotation(1972),whichisthestandardforbraillemathematicsinNorthAmerica(Do-It,2015),andthosedevelopedbyablindscientist,thelateLawrenceChang.
BothNemethandChanginitiallydevelopedtheirrulesfortheuseofreadersoraideswhoreadmathexpressionstothem,andlatertheypublishedthoserules.
Theirrulesandstrategiesaresignificantlydifferentfromoneanother,aswillbedescribedinthenextsections.
MathSpeakNemethdevelopedhisrulesinthe1950s,withthegoalthatreaderswouldspeakmathexpressionsinawaythatmapsone-to-onewithsymbolsusedinNemethCode,sothathecouldconvenientlywritetheinformationinNemethCodeasitwasreadtohim.
HenamedhissystemMathSpeakandnotedthat"thespeechgeneratedbythisprotocolisnotexactlywhataprofessorinclasswoulduse,butitisabsolutelyunambiguousandresultsinaperfectNemethCodetranscription"(Nemeth,2013,p.
4).
Thatis,thissetofrulesisoptimizedforreal-timeNemethCodetranscriptionandrequirestheusertobeproficientinboththeNemethCodeanditsvocabulary.
Example:isspokenas"xsuperscriptnbaselineplus1.
"ThewordssuperscriptandbaselinecorrespondtoindicatorsinNemethCode.
Thisexpressionisdistinguishedfrom,whichisspokenas"xsuperscriptnplus1.
"Forthelatterexpression,becausetheendoftheexponentistheendoftheexpression,thebaselineindicatorisnotusedintheNemethCodeorMathSpeakrepresentationsofthelatterexpression.
Larry'sSpeakeasyChang's(1983)HandbookofSpokenMathematics:Larry'sSpeakeasy(usefullyexpandedbyResearchandDevelopmentInstitute,Inc.
,2006)tookadifferentapproachwithhisrules.
Chang'sapproachwasbasedonhisexperienceoflearningmathematicsnotfrombraillebutfromlisteningtolivereaderswhospokemathematicsusingthemorenaturallanguageofmathematicsinstruction.
Hearguedthatasymbol-by-symbolinterpretation"canbeverytediousandhardtounderstand"(Chang,1983,p.
1).
Chang'srules,someofwhichincludeoptionalvariations,asseenintheexamplebelow,whichprovidestwooptions,usecommonphrases(e.
g.
"tothenpower"fortheassociatedmathematicalnotations)ratherthanmappingtoNemethCode("superscript"and"baseline").
Example:isspokenas"xtothenpowerplus1"oras"thesumofxtothenand1"(Chang,1983,p.
20).
LearningAllyWhenadoptingguidelinesforaudiomath,organizationsincludingRecordingfortheBlindandDyslexic(RFB&D,nowLearningAlly)haveusedChang'swork,alongwithNationalBrailleAssociation's(1996)TapeRecordingManual,asreferencesforspeakingmathintheirrecordedaudioproductions.
ThisapproachisconsistentwiththegoalofprovidingaccesstoawideraudiencethanjustthoseaccustomedtoNemethCodeanditsvocabulary.
TheMathSpeakstylecanbedifficulttounderstandevenforNemethCodeusers,becauseaswefound(seetheQualitativeFindingssection),noteveryNemethCodeusercorrectlyassociatesthewords(e.
g.
,baseline)withthecorrespondingnotation.
AssessmentGuidelinesProvidingspokenmaththatisbothunambiguousandusablebyawidevarietyoflistenersisanimportantconsiderationinassessments.
AnimportantcommonthreadamongChang's(1983)andLearningAlly's(NationalBrailleAssociation,1996)setsofguidelinesistheiremphasisoncommonandcorrectmathematicalterminologyasitisusedinthecontextofthematerialbeingread,asopposedtoMathSpeak'smappingoftermstoindividualbrailleorprintsymbols.
WhentheEducationalTestingService(ETS)plannedstrategiestomakeitsassessmentandassessment-relatedaudiomaterialsusefulforavarietyoflisteners,regardlessoftheirexperiencewithNemethCodeanditsterminology,itbaseditsspeechrulesforitsrecordedaudioandscriptsforlivereadersonthoseadoptedbyLearningAlly—essentiallyadoptingtheChangstrategy.
ThisstrategyisconsistentwiththerecommendationoftheNationalBrailleAssociation(1996)to"insofaraspossible,callthingsbythesamenamesthatwillbeusedintheclassroom"(p.
14).
TheETSimplementationoftheseguidelinesincludesstrategicpausestoemphasizeandsupportgroupingwithinexpressions.
MuchmorerecentlythetwoprimaryconsortiaforCommonCoretests,PARCC(PARCC,2014)andSmarterBalanced(MeasuredProgress/ETS,2012)publishedguidelinesthatreferenceavarietyofread-aloudguidelines,includingChang,ETS,andMathSpeak,thoughtheactualguidelinesaremoreinlinewiththeChang/LearningAlly/ETSstrategythantheMathSpeakstrategy—thatis,theyfavorclassroom-likespeechoverNemethCode–likespeech.
Inintroducingitsguidelines,SmarterBalancedstates:Theseguidelinesarebasedon1)researchstudiesinvolvingthereadaloudaccommodation2)preexistingstatereadaloudguidelinesforstandardizedassessmentand3)discussionandfeedbackfromstateofficials,expertsonaccessibility,andcontentexperts.
Theguidelinesweremadetoinformdecisionsonscriptingandtaggingofmathematicsitemsforcomputer-baseddeliveryofthereadaloudaccommodation.
Oneoftheoverarchinggoalsofthisprojectistohelpfacilitatestandardizationofmathematicsreadaloudandindoingsominimizingtheinconsistenciesandcomplicationsthatexistinthecurrenthumanandtexttospeechreadalouddeliverysystems.
Theguidelinescontainedinthisdocumentarenotintendedtoberigidrules,butratheraguidetocreatingreadaloudscriptsandtagsthatbesthelpstudentsaccessthecontentwithoutviolatingtheconstructbeingmeasured.
(MeasuredProgress/ETS2012,p.
4)Similarguidelines,intendedtobeusedforcreatingscriptsandtagsforcomputer-baseddeliveryofreadaloudaccommodationsforCommonCoreStateStandards,werealsodevelopedfortheGuidelinesforAccessibleAssessmentsProject(GAAP)(MeasuredProgressInnovationLab&MDStateDepartmentofEducation,2014).
AlthoughscriptsdevelopedaccordingtothePARCC,SmarterBalanced,andGAAPguidelinescouldbeusedforliveorrecordedhumanspeechorscriptedtext-to-speech,thescriptsthemselvesarenotcomputer-generatedbutdevelopedbypeoplewhohaveanunderstandingoftheguidelines,thetestingconstruct,andthecontentofthematerial.
ComputerizedSpeechRulesAThasforsometimebeencapableofmakingtextualmaterialaccessibletopeoplewithvisualimpairments,thusprovidingfargreaterindependencethanrelianceonhumanreaders.
Formathtobesimilarlyaccessible,tomovefromastyleforhumanspeechorscript-writingtooneusablebycomputersandotherelectronicdevices,itisnecessarytocodifyspeechguidelinesintospeechrulesthatthecomputercanimplementwithlittleornohumaninteraction.
ASTEROneoftheearliestsoftwaredevelopmentsincomputerizedspokenmathwasbyT.
V.
Raman(1994)whodevelopedtheASTERsystem,whichreadsLaTeXdocuments.
Prosodiccues(i.
e.
,pausespluschangestopitch,rate,volume,andvoice)arecentralfeaturesandhelpmakethespeechmoreconcisethanispossiblethroughwordsalone.
ASTERusesahigherpitchforsuperscriptsandalowerpitchforsubscripts.
Thepitchremainsraisedorloweredforthedurationofthesuperscriptorsubscript,thusmakingthescopeunambiguousandavoidingtheneedforbegin/endlanguage.
Grouping(e.
g.
,parentheses)issimilarlyindicatedbyvariationsinthevoiceused,withpausesprovidingadditionalemphasisforgrouping.
ASTERincludesadditionalfeatures,thediscussionofwhichisbeyondthescopeofthisreport.
ExampleoftheuseofprosodiccuesinASTER:In,isspokeninahigherpitchtodistinguishtheexpressionfromwhereonlythenisspokeninahigherpitch;bothexpressionswouldbespokenwiththesamewords.
OtherProjectsInadditiontoASTER,variousothersoftwareprojectshavedevelopedsystemsforspokenmath.
SomeofthoseprojectsareMathTalk,developedbyStevens,Edwards,andHarling(1997),MathGenie(Karshmer,Bledsoe,&Stanley,2005),theLambdaProject(Edwards,McCartney,&Fogarolo,2006),WinTriangle(Gardner,2005),AudioMath(Ferreira&Freitas,2004)andReMathEx(Gaura,2002).
ThelattertwoprojectsusedmathexpressionsencodedinMathMLasthebasisforspeech,butneitherprojectisunderactivedevelopmentasofthiswriting.
SummaryRegardingComputerizedSpeechRulesAlloftheprojectsmentionedsofarwereself-containedsoftwareenvironments,asopposedtobeingabletoworkwithinstandardapplicationsorwithcommontypesofassistivetechnology,thusmakingthemnotverysuitableformakingmainstreamelectronicmaterialaccessibletoSVIs.
RequirementsfortheNewSpeechStyleInthissectionwediscusstherequirementsweestablishedforthespeechstyleandthereasonsunderlyingthoserequirements.
Therequirementsarelistedbrieflybelow,followedbyanexplicationofeach:1.
Easeofdecodingexpressions2.
Similaritytoclassroomspeech3.
Standardization4.
Customization5.
ExternalizationEaseofDecodingExpressionsAsthepurposeofspeakingmathexpressions,likethatofspeakingtext,istoallowthelistenertoreceiveandmakeuseoftheinformation,theprimarymeasureofsuccessforanyspeechstyleishoweasilyuserscandeterminewhatthemathexpressionis(or,toborrowatermfromdiscussionsofliteracy,"decode"theexpression).
Onlyonceastudenthassuccessfullydecodedtheexpression—graspedbothitsstructureanditscontent—canthestudentgoontoworkwithittosolveproblemsandfurthertheirknowledgeofmathematics.
SimilaritytoClassroomSpeechStudentsareusedtohearingmathspokenintheclassroom,andsoitisreasonabletoassumethatmathdecodingisfacilitatedbyaspeechstylethatisassimilaraspossibletotypicalclassroomspeech.
Anecdotalcommentsbyteachers,aswellascommentsstudentsofferedinourfirstfeedbackstudy,supportthisassumption.
Classroom-likespeechhastheadvantageofminimizinganyadditionallearningcurvesforstudents.
StandardizationTheexistenceofavarietyofguidesforspokenmath—whetherspokenbypeopleorbycomputers—showsthatitisoftenlessthanobvioushowagivenmathexpressionshouldbespokenandthatthereareexpectationsofvariationbasedonmathematicaloreducationalcontextfromanyproposedstandards.
Butitisdesirablethatmathspeechrulesusedininstructionshouldbeasconsistentaspossibleandthatspeechusedinassessmentsshouldbeconsistentwithspeechusedininstruction.
Justastherearestandardsforprintednotation(e.
g.
,superscriptsindicateexponents),soshouldsomewhatstandardizedspeechbegiven(e.
g.
,useofthewordpowerinexponents).
Otherwise,itcanbenecessaryforalistenertoperformamentaltranslationwhenchangingenvironments(e.
g.
,movingamongdifferenttextbooksorbetweenaninstructionalcontextandatest-takingone).
Ineachcase,themathshouldideallybespokenusingthesamebasiclanguage.
CustomizationYetalongsidetheneedforstandardizationisaneedforflexibility:Justasstandardizedprintsymbolsallowforsomedegreeofvariationandcontextualization(e.
g.
,(1,2)canrepresentthecoordinatesofapointoranopeninterval),differentcontexts,situations,oraudiencescancallforvariationsinthewaymathematicsisspoken.
Forexample,(1,2)(meaningpointcoordinates)shouldbespokendifferentlythanthesamenotationindicatinganopeninterval.
Verbosityneedsmayalsovarywiththelistener'slevelofexperienceandwiththecomplexityofthematerial.
Lesscomplexexpressionsusedbybeginningstudentsmaybeverbosebutrelativelyshortbecauseofthebrevityoftheexpression.
Morecomplexexpressionswillimposeincreasedmemoryloadifspokenverbosely,butthemoreadvancedstudentslikelytoworkwiththemorecomplexexpressionsmaypreferalessverbosepresentation.
Context,asalreadysuggested,ishighlyrelevanttothewayanexpressionshouldbespoken.
Inadditiontothefactthatthemathematicalmeaningofthesamenotationcanvarycontextually,evenwithinagivenmeaning,contextcanaffecttheoptimalspeechforanexpression.
Forexample,whileitmayseemobviousthatshouldbespoken"onehalf,"whenitbecomespartofalargerexpressionsuchasitmaybepreferabletomatchitsspeechtothatofthesurroundingfractions—"Thefractiononeovertwo[pause]plus[pause]thefractionthreeoverfive[pause]plus[pause]thefractiontwenty-threeoverfourhundredtwenty-three[pause]plus[pause]thefractionxovery,"—sothatallofthefractionswithinthelargerexpressionarespokenaccordingtothesamebasicscheme.
Concernssuchasthosedescribedinthissectionleadtotheconclusionthatinordertomakemathaccessibleviasyntheticspeech,itisimportanttodefineaspeechstylethatisrobustandflexibleandthatallowscontentcreatorsanduserstomakechoicesamongspeechstylesandamongspeechpreferenceswithintheoverallstyle.
Itisalsousefultoallowcontent-providerstoentertheexactspeech(includingpausesandotherprosodicelementsifdesired)forpartsofanexpression,forcasesinwhichtheavailablerulesandpreferencesarenotsufficientlyrobusttoprovidethedesiredspeechorforspecialsituations.
Theabilitytospecifyexactspeechisalsousefulforprototypingandtestingproposedrulesandpreferencesbeforetheyareprogrammedintoaspeechstyle.
ExternalizationAsdiscussedearlierinthispaper(seethesection,SummaryRegardingComputerizedSpeechRules),previousprojectsimplementedspokenmathinself-containedenvironments,meaningthatuserswouldhavetoobtain,learn,anduseasystemthatmightnotworkwiththeircustomaryATandthatanychangesorupdatestothespeechruleswouldbecloselytiedtotheenvironmentinwhichthesystemwasdeveloped.
Weinsteadsoughtanenvironmentthatallowedthespeechrulestobestoredexternallyandthuscapableofbeingrevisedoradoptedindependentlyofthedeliverysoftware,andwhichcouldbeusedwithawiderangeofcommonlyusedsoftware(Webbrowsers,MicrosoftWord)andAT(mostnotably,screenreadersandrefreshablebrailledisplays).
PurposeThisreportdescribesthedevelopmentandevaluationoftheClearSpeakstyle,whichisintendedtoaddresstherequirementsdetailedabove.
Theproject,ExpandingAudioAccesstoMathematicsExpressionsbyStudentsWithVisualImpairmentsviaMathML,wasfundedbyaU.
S.
DepartmentofEducation,InstituteofEducationSciencesSpecialEducationDevelopmentGrant(R324A110355),whichsupportedtheiterativedevelopmentofthespeechstyle,authoringtools,interactivenavigation,andintegrationwithMicrosoftWord.
Fourfeedbackstudies—theonedescribedinthisreport,focusingonspeechstyles;oneontheuseofcertainprosodicelementsinthespeechstyle;oneontheinteractivenavigationcapability;andoneonauthoringbyteachersandserviceproviders—guideddevelopment,culminatinginafinalpilotinthespringof2015.
MethodTheMathMLAudioProjectandItsOverallStrategyWhenthisprojectwasconceived,theMathPlayersoftware(DesignScience,2008;Soiffer,2007)wasselectedforanumberofreasons.
First,itwasanexisting,freeplug-inthatalreadyimplementedsomeaspectsofspokenmath,includingtwopre-existingspeechstyles.
Second,itsatisfiestheexternalizationrequirementdescribedabove:Itsarchitecture,whichstoresspeechrulesexternallytotheprogram,allowsformultiplesetsofintegratedspeechruleswithoptionalvariations(preferences),andiseasilyextensible,allowingfortheimplementationofClearSpeakwithoutrequiringanychangestooreliminationofitspre-existingstyles.
Moreover,MathPlayerisnotself-containedbutinsteadworkswithinwidelyusedsoftware(e.
g.
,MicrosoftWordandsomebrowsers)andwithATtosupplyspeechdata(andbrailledataforrefreshablebrailledisplays)totext-to-speechsynthesizersthatarealreadyontheuser'scomputer.
StepsintheDevelopmentoftheClearSpeakStyleThissectiondescribesthemajorstepsindevelopingClearSpeaktothepointthatenabledustoperformthepreliminaryevaluationdescribedinthispaper.
Whenourprojectbegan,thespeechstylesavailableinMathPlayerwereSimpleSpeakandanimplementationofMathSpeak,whichasnotedpreviously(seethesection,MathSpeak),isaspeechstylebasedonNemethCodeandoptimizedforreal-timeNemethCodetranscriptionbyone-to-onemappingofspeechtoNemethCodesymbols.
MathSpeakwasalsoimplementedanddocumentedbygh(2004-2006).
SimpleSpeak'sguidingprincipleistospeaksimpleexpressionssimplyandtodisambiguatecomplicatedexpressionswithbegin/endbracketing.
ItfocusesonrenderingthemeaningofthemathematicalexpressionassimplyandbrieflyaspossibleanddoesnotmaptoNemethCodeortoanyotherbraillesystem,nordoesitattempttoreplicateclassroomspeech.
WebelievedthattherewasaneedforClearSpeakbecauseofindustryandETSpractice(seethesections,LearningAllyandAssessmentGuidelines)andreportsfromblindindividualssuggestingthatMathSpeak'sfocusonmappingtoNemethCodesymbolswasnotoptimalformostnon-braillereadersorevenformostbraillereaders,andthatsomethingmoreliketheETSspeechstyle(which,likeSimpleSpeak,doesnotmaptobraillesymbols,butwhich,unlikeSimpleSpeak,attemptstoreplicateclassroomspeechandcorrectlyreflectthemathematicalmeaningoftheexpression)shouldbeavailable.
BecauseneitherMathSpeaknorSimpleSpeakmeettherequirementforclassroom-likespeech,amajorgoalofourprojectwasthecreationofanewautomatedsyntheticspeechstyle(ClearSpeak)thatwouldmeetthatandtheotherfourrequirementspresentedpreviously.
TherulesoftheClearSpeakstyleweredevelopedbycarefullyconsideringthecurrentETSspeechstyleforhumanreaders(liveorrecorded),whichisbasedonclassroom-likespeech,anddecidinghowitshouldbeadaptedforcomputer-generatedspeech—codifiedintoprogrammablealgorithms.
Webeganbycreatingacomprehensivelistofmathematicalstructuresandtypesofexpressionsusedinsecondaryschoolalgebra(masterexpressionlist)andhowtheyoughttobespokeninvariouscircumstances.
Thenwerevisedandaugmenteditasneeded.
ThislistservedasareferencefortheresultingClearSpeakrulesandpreferences(Brownstein,Soiffer,&Frankel,2015a).
AnexcerptfromthemasterexpressionlistisinAppendixD;thefulllistincludeshundredsofexpressionsandcoversawidevarietyofmathstructures,symbols,andotherconventionsfoundinsecondaryschoolmath.
SeeBrownstein,SoifferandFrankel(2015a,2015b).
Wenextfollowedaniterativeprocessofdevelopingtherulesandvariations(preferences)andtestingthemagainstthemasterexpressionlist.
Wepresentedearlydraftsoftherulesandpreferencesforthestructuresmostfundamentaltosecondaryschoolalgebra(fractions,exponents,parentheses,androots)toouradvisorycommittee,whichconsistedofexpertsinthefieldofmathematicseducationforblindstudents,includingDr.
AbrahamNemeth,SusanOsterhaus,andMayleneBird,forcomment.
Theresultingruleswereused(prototypedusingtheexactspeechcapabilitythatwasdevelopedearlyintheproject)forthespokenmathusedinthefirstfeedbackstudy,thepurposeofwhichwastoperformtheinitialevaluationoftheClearSpeakstyle.
ClearSpeakandtheSpeechStyleRequirementsAcentralcomponentofthisprojectistheconstructionofthenewautomatedspeechstyle,ClearSpeak.
TheMathPlayerplatformwithitspre-existingstylesMathSpeakandSimpleSpeakalreadyhadstandardizedexternalrules,andsosatisfiedRequirements3and5describedpreviously(standardizationandexternalization).
Thesestylesdidnot,however,supportpreferences,norwastheresultantspeechsimilartoclassroomspeech,andsotheydidnotsatisfyRequirements2and4.
ClearSpeakisdesignedtosatisfyallfiverequirements—mostimportantly,theprimaryrequirementofeaseofdecoding.
Easeofdecodingis,weargue,substantiallybasedonthesimilarityofthesyntheticspeechtothespeechstudentsarefamiliarwithfromtheirclassrooms,aswehavestatedinRequirement2.
Thatsaid,anysyntheticspeechstylewillatbestbeclassroom-like;itcannotbeidenticaltoclassroomspeech,butitcanandmustdifferincertainrespects.
Inclassroomspeech,ateachercanconsiderrelevantcharacteristicsofthestudentstheteacherisaddressing(e.
g.
,theirmathbackgroundandthespeechtheyareusedto).
Ifateacherisspeakinganexpressioninawayastudentfindsdifficulttounderstand,thestudentcanasktheteacherforclarificationwheneverneeded.
Asaresult,forhumanspeech,guidelines,asopposedtoanexhaustiverule-set,areoftenadequate.
Syntheticspeechcannotbeaskedforclarificationifitisnotimmediatelyunderstood,sothedevelopmentofrulesisafarmoreinvolvedandmeticulousprocessthanisthedevelopmentofguidelinesforhumanreaders.
Anotherimportantdifferencebetweenhumanandsyntheticspeechistheabilitytoadapttocontext,particularlytothelargercontexts(e.
g.
,thoseofaproblem,proof,orassessmentquestion)inwhichmathexpressionstypicallyoccur.
Forexample,inalessononfunctionsis"fofc,"whileinamultiplicationlessonitis"ftimesc.
"Ateacherwillnaturallyspeaktheexpressionsaccordingtotheappropriatecontext.
Sincemuchofthatcontextisoutsidetheexpressionitself,systemsforgeneratingsyntheticspeechforexpressionscannotautomaticallyadjustforthatexternalcontext.
Butwell-consideredrulesandauthoringtoolsthatallowtheselectionofrelevantvariationscanhelpbridgethisgap.
ClearSpeakprovidessuchtools,sothatcontentauthorscanmakeadjustmentstohowanexpressionisspoken(seeRequirement4).
Adjustmentsaremadeprimarilythroughtheuseofspeechpreferences,whicharealternativecommonwaysofspeakingmathexpressions.
ClearSpeakalsoprovidestheabilitytoenterexactspeechforanexpression,whenneithertherulesnoranyavailablepreferencegivethedesiredspeech.
SeeFrankel,Soiffer,&Brownstein(2013)fordetails.
Oneclassofpotentialdifficultiesinunderstandingspokenexpressionsinvolvesboundaries(forexample,whereanexponentinanexpressionbeginsandends).
Acommonpreventivestrategyforcommunicatingthelocationsofboundaries,suchaswhereafraction,exponent,root,orotherstructurebeginsandends,istoaddwordssuchas"beginroot.
.
.
endroot.
"(BothMathSpeakandSimpleSpeakusesuchstrategies.
)However,particularlyinlongerexpressions,thosewordscanbuilduptothepointthattheverballoadmakesitdifficulttokeeptheexpressioninshort-termmemory.
Accordingly,ClearSpeakendeavorstoreplaceatleastsomeofthewording(particularlybegin/endlanguage)withalternatives,suchasprosodiccues(primarilypauses,althoughchangestothespeechratehavebeenconsidered)thatdonotrequireverbalmemory,totheextentthatsuchcuescanbeeffectivereplacementsand/orhelpmitigatememory-loadproblemsthatoccurwhenthelanguagemustberetained.
Inoursecondfeedbackstudy(seeFrankel&Brownstein,inpress),weinvestigatedtheeffectivenessofvariouscombinationsofprosodiccuesandendlanguage,andusedtheresultstofurtherrefinetheClearSpeakrulesandpreferences.
TheFirstFeedbackStudyThebalanceofthisreportdescribesthefirstfeedbackstudy,conductedinMarch2012,anditsresults.
ThisstudywasthefirstofaseriesofstudiesguidingthespecificationandimplementationofClearSpeak.
Unlikethestandardization,customization,andexternalizationrequirements,whichcouldbeassessedprimafacie,determiningwhichstylesbestmettherequirementsofsimilaritytoclassroomspeechandeaseofdecodingexpressionsneededempiricalinvestigation.
Hence,theprimarypurposeofthefirststudywastodeterminewhether,forSVIswhowereinorhadcompletedAlgebra1andforAlgebra1-typemathexpressions,ClearSpeaksurpassedtheothertwostyleswithrespecttothoseremainingrequirements.
Andalthoughpositivesubjectivereactiontoaspeechstylewasnotanoverridingrequirement,webelievedthatsuchareactionwouldbehighlydesirablebecausesuchreactionswouldincreasethelikelihoodthatstudentswouldbewillingtousethespeech.
Accordingly,theprimaryresearchquestionswere:DoesClearSpeakpromoteequalorbetterdecodingofmathematicalexpressionsascomparedwiththeothertwostylesDostudentsreactatleastasormorepositivelyinavarietyofrespectstoClearSpeakastotheotherstylesForexample,howdothestylescompareonoverallstudent-preferenceandonstudents'perceptionsoffamiliarityandunderstandabilityWeattemptedtoanswerthefirstquestionbypresentingstudentswithobjectivemathandmath-recognitionquestionsinwhichthemathematicalexpressionswerepresentedpurelyinaudiointhevariousspeechstyles(seeAppendixA,questions1,9,18,26,36,46,and47).
Toanswerthesecondquestionwepresentedthemwithfeedbackquestions(theremainingquestionsinAppendixA)abouthowfamiliarandunderstandabletheyfoundthespeechineachcase.
Inend-of-sectionandend-of-studyquestions,weaskedthemtoindicateanoverallassessmentofeachstyle,whichofthethreestylestheymostpreferred,andwhichtheyleastpreferred.
WealsosoughtspecificsuggestionsforimprovingClearSpeakinlaterphases.
Furtherbreakoutsoftheseresearchquestionsandhowthestudywentaboutansweringthemaredescribedbelow(seethesection,QuantitativeResultsbyResearchQuestion).
ParticipantsSixteenstudentswithvisualdisabilities(blindnessorlowvision)whoweretakingorhadcompletedAlgebra1participatedinthestudy.
Allbutonestudent(whowasblind)completedallthreesections.
Asummaryoftheparticipants'mostrelevantcharacteristicsispresentedinTable1.
Wedidnotcollectdataonage,grade,sex,orethnicity,noneofwhichwereexpectedtoberelevant.
Itisperhapsnotablethataudiowasnotapreferredmathaccessmethodformostoftheparticipants:itwaslistedbytwooftheblindstudentsandbyonestudentwithbothlowvisionandamathlearningdisability.
Thatfactreflectsthelimitedaudiosupportthathas(priortoourproject)beenavailableformath.
Table1ParticipantCharacteristicsVisualstatusCurrentmathclassOtherdisabilitiesPreferredmathaccessmethodBlind:8Algebra1:4Autism:1Hardcopybraille:5Lowvision:8Mathmodels:a2Hearingimpairment:1Braille+reader:3Algebra2:7Orthopedic:1Reader:1Calculus:2Learningdisabilityinmath:1Largeprint/screenenlargement:6Other:1Speechimpairment:1Regularprint:1TBI:1Note.
TBI=Traumaticbraininjury.
aMathModelsisacoursegiveninthisparticularschoolbetweenAlgebra1andAlgebra2.
SamplingProceduresParticipants,allofwhomwereblindorhadlowvision,weretakingorhadcompletedAlgebra1,werefluentinEnglish,anddidnothavesignificantcognitivedisabilities,wererecruitedthroughtwoparticipatingschoolsfortheblind.
Weslightlyexceededourrecruitinggoalof15students,butasalreadynoted,oneofthestudentsbeganbutdidnotcompletethestudy.
Studentsweregiven$25ingiftcardsforcompletingthestudy.
InstitutionalReviewBoardapprovalandsignedinformedconsentformswereobtainedpriortodatacollection.
ResearchDesignEachstudentwasassignedtoreceivethreeparallelsectionsofitems.
Ineachsection,themathexpressionswerespokeninadifferentoneofthethreestyles—ClearSpeak,MathSpeak,orSimpleSpeak.
Inordertocancelpossibleordereffects,theorderofstyleswasvariedsystematically.
Afteransweringeachmathquestionthestudentwasaskedseveralfeedbackquestionsaboutthemathstatementandmathquestion.
InstrumentsThestudyusedthreeinstruments.
First,thestudentbackgroundquestionnaireaskedstudentstoself-reporttheirmath-andvision-relatedbackgroundandtheirhistorywithusingvariousformsofassistivetechnologyformath.
Second,thestudentbackgroundquestionnaireforteacherswasaparallelinstrumenttothequestionnaireforstudents.
Itpresentedteacherswiththesamequestions(onbehalfoftheirstudents)thatthestudentshadansweredforthemselves.
Centraltopicsofthetwoquestionnairesincludedthefollowing:student'scurrent"best"modeforaccessingmath(print,braille,readaloud,AT)perceivedusefulnessforthestudentofthevariousmodesforaccessingmath(print,braille,readaloud,assistivetechnology)student'sproficiencywithfractions,exponents,andparentheses(grouping)thedegreetowhichthestudentusesvisualversusnonvisualmethodsofaccessingmathThird,themathandfeedbackinstrumentcomprisedthreesections(seeAppendixAforonefullsection),eachofwhichconsistedofthefollowing:Sevenmathstatements(expressionsorequations),eachaccompaniedbyoneortwo"mathquestions.
"Inmostcases,thesewerenottypicalmath"problems,"butratherquestionstodeterminehowsuccessfulthestudentwasatdecodingthemathstatements.
Themathstatementsrepresentedkeysecondaryschoolalgebratopics(fractions,exponents,grouping).
ThepurposeofeachmathstatementisincludedasannotationstothesampleinstrumentprovidedinAppendixA.
Foreachmathquestion,therewereseveralfeedbackquestionsdesignedtogatherinformationaboutthestudent'sperceivedlevelofunderstandingofthestructureofthemathstatement(i.
e.
,howeasilytheycoulddecodethestatement)andaboutthestudent'ssubjectivereactiontohowthemathwasspokenAsectionsummaryquestionregardingthestudent'swillingnesstohearmathspokenthewayitwasinthatsectionAfterallthreesectionswerecompleted,studentswereaskedthreefinalsummaryquestionsaboutthewaymathwasspokeninallthreesections.
Notethatthethreesectionsaredescribedasparallel.
Tobemorespecific,thefeedbackquestionswereidentical,andthemathstatementsandquestionswere"clones"—differingonlyinsuperficialaspectssoastominimizeanymemorizationeffectsbetweenitemsasparticipantsprogressedfromonesectiontothenextforthethreespeechstyles.
Effortsweremadetomaintainthesamelevelofdifficultyamongtheclones.
However,tocontrolforpossibleunintendeddifferencesindifficultyamongclones,theassignmentofinstrumentvariants(i.
e.
,groupofclonedexpressions)tospeechstyleswassystematicallyvaried(seethesection,StudyManipulation).
Sampleexpressionsandtheirclones,includinginformationabouthowtheexpressionswerespokeninthedifferentspeechstyles,areincludedinAppendixB.
DeliveryMethodThemathandfeedbackinstrumentconsistedoftwoparts:atextualcomponentandaprerecordedaudiocomponent.
Allofthemathexpressionswereprovidedinaudioonly.
TheaudiocomponentwasdeliveredviaMP3players;thetextualcomponentwasdeliveredviaWordfile(whichstudentscouldusewiththeirscreenreader,includingenteringtheirownresponsesintothedocument)andhardcopyprint(forusewithalivereader/scribe).
Thisdivisionintoscreenreadable/scribereadabletextandprerecordedaudiowasemployedforthreereasons:(a)theintegrationofMathPlayeraudiointoscreenreadableWorddocumentswasnotyetreadyforuse(itisfunctionalasofthiswritingandwasusedinthefinalpilot),(b)therulesandpreferenceswereunderdevelopmentandneededtobesimulatedviaentryofexactspeechtobereadbythescreenreader,and(c)wewishedtoensurethatallstudents,regardlessofdegreeofusablevision,receivedtheexpressionsonlyinaudio,whichallowedustoisolatetheeffectsoftheaudiopresentationfromanyvisualinformationthatmightotherwisebeavailabletostudentswithsomevision.
Becausestudentswithvisionareaccustomedtousingit,however,beingrequiredtoworkwithoutvisionplausiblyincreasedthedifficultyofthetasksforsuchstudents(andaswillbeseeninthediscussionofresults,studentswithlowvisionhadlowerlevelsofsuccessthanstudentswithblindness).
Detailsofhowtheaudioportionsweredevelopedareprovidedinthesection,DevelopmentofAudio.
ThestudentexperiencecanbeseenfromtheannotatedinstrumentpresentedinAppendixA.
Thatinstrumentincludesbothmathandfeedbackquestions.
Thefeedbackquestionsweredesignedtoassessdecodingsuccessandstudents'subjectivereactionstothespeech.
DevelopmentoftheItemsTodeveloptheitems,weworkedwithtwoofourexpertconsultants,SusanOsterhausandMayleneBird(bothofwhomteachmathtoblind/lowvisionstudents).
AppendixAincludesdescriptionsofthereasonsbehindtheselectionofeachmathstatementandassociatedmathquestion.
Increatingthemathquestionstoaccompanythestatements,wesoughtquestionsthatwouldhelpidentifyerrorsinrecognitionofthemathstructuresinthestatements,asopposedtoquestionsthatwouldassessstudents'mathematicalability.
Followingthegeneralpreceptsofevidence-centereddesign(Hansen&Mislevy,2008)inordertomakesurethetestitemscouldprovideevidencetosupporttheclaimstobemadeontheirbasis,wedevelopedthefollowingcriteriaforexpressionstobeusedinthestudy:SimilaritytoexpressionstypicallyfoundinhighschoolalgebraInclusionofstructuresthatwouldbeunderstoodbyasightedstudentwhoiscurrentlytakingthesecondhalfofanAlgebra1classorasubsequentmathclassComplexitysufficienttoofferpotentialaccessibilitydifficultiesattendanttospokenpresentation;thatis,notbesosimplethatstudentswouldinstantlyunderstandthem,nomatterhowtheywerespokenSimplicitysufficienttoallowstudentstounderstandthemwithoutintensiveeffortSupportformathquestionsthatrequirelittleornocomputation(soastofocusondecoding,notcomputation)Speechthatissignificantlydifferentforatleasttwoofthethreespeechstylesbeingtested(andpreferably,differentforallthree)Supportmathquestionsthatcanhelptodiagnoseaspectsinwhichthespeechstylesucceedsorfailstomakethemathexpressionclear,asopposedtothosethatassesswhetherthestudentcandothemathematicsrequiredtoworkwiththeexpression(i.
e.
,ifstudentsfailtocorrectlyidentifythedenominatorofafraction,itispossiblethatthespeechdidnotmaketherelevantboundariessufficientlyclear)BecausethefirstClearSpeakrulestobedevelopedfocusedonfractions,exponents,andmultiplication/parentheses—allcommonandessentialinAlgebra1—theexpressionsandassociatedquestionsforthestudyfocusedonthesestructures.
The"mathquestions"werecraftedtocalluponthelowestlevelofmathknowledgeneededinordertoassessthestudents'understandingofeachexpression'sstructureandcontent(i.
e.
,whetherthestudentsuccessfullydecodedtheexpression).
Amongthefeedbackquestionstowhichstudentsrespondedwererequeststodescribehowtheyarrivedattheiranswers.
Thatinformationinmanycaseshelpedustopinpointportionsoftheexpressionthatwereorwerenotspokenunderstandablyorallowedustoinferthatastudentdid(ordidnot)correctlyidentifytheexpressionregardlessofwhetherthestudentansweredthemathquestioncorrectly.
Theresultingitemsincludedsomeconstructed-responsequestions(e.
g.
,evaluatetheexpression,reducethefraction)andsomemultiple-choicequestionsinwhichthedistractersreflectincorrectunderstandingsofthestatementsthatcouldresultfromthewaysthestatementisspokeninthevariousspeechstyles.
Forexample,astudentwouldbemorelikelytoselectaparticulardistracterifthestudenthaddrawnaparticularincorrectconclusionaboutthescopeofanexponentinthestatement.
Aftertheexpressionsandquestionswerecreated,wecreatedcloneversionstopreventmemorizationeffectsbetweenspeechstyles.
Cloneswerecarefullyconstructedtochangeonlysurfacefeaturesandtomaintainequaldifficulty.
Allitemsandcloneswerecarefullydevelopedandreviewedbytwotestdevelopers(BrownsteinandFrankel)andtwoteachers/consultants(OsterhausandBird).
SeeAppendixAforasampleinstrumentandAppendixBforasmallersamplingofexpressionsandclonesshowingthespeechusedforeachstyle.
DevelopmentofAudioThemathexpressionsintheinstrumentsweredevelopedinMicrosoftWord2007withMathType6.
8.
Foreachclonegroup,onedocumentwasproducedfortheClearSpeakimplementationandanotherfortheMathSpeakandSimpleSpeakimplementations.
FortheClearSpeakimplementation,exactspeechwasinsertedtoprototypewhatwouldbecomerulesorpreferencesasClearSpeakdevelopmentmatured.
MathSpeakandSimpleSpeakspokeaccordingtoeachstyle'salreadyestablishedrules.
Notethatatthetimeofthestudy,theMathSpeakimplementationincludedwhatwasinitiallythoughttobeanintentionalpartofthespeechstylebutlaterwasconsideredabugbecauseitwasnot,infact,intended:Inthreeofthesevenmathexpressions,onwhichfouroftheeightmathquestionswerebased,itaddedthephrase"nestedonedeep,"indicatinganestinglevel.
Althoughthisspeechpatternwasnotintended,someotherimplementationsofMathSpeakdoannouncenestinglevelsforfractionswithinfractions(gh2004-2006),andtheversionofMathSpeakagainstwhichClearSpeakwascomparedinthisstudywastheimplementationthathadbeeninthepublishedversionofMathPlayeratthetimeofthestudy.
Oncetheexpressionsandintendedspeechwerefinalized,acomputer'sdefaultvoicewassettoMicrosoftAnna,thestandardtext-to-speechvoicethatcomeswithWindows7.
(Annaisnotthehighestqualityvoice,butwedonothavedistributionlicensesforhigherqualityvoices,soitwastheonlyavailableselection.
)Thefree,open-sourceAudacitysoundeditorwasusedtocapturetheaudioproducedbyhavingthecomputerspeakeachexpressionineachspeechstyleandineachcloneversion.
Somesoundeditingwasdonetoallfilesinallspeechstylesinordertocorrectcasesofparticularlypoorpronunciation:PlacesinthesoundstreamwhereAnnapronouncedanumberorsymbol(e.
g.
,"x")correctlyreplacedinstanceswhereAnnapronouncedthesymbolincorrectly("thex"for"x"and"dwy"for"y").
Wefeltcomfortableperformingthisediting,whichwasdoneuniformlyacrossallexpressionsandstyles,becausewewantedstudentstobeabletofocusasmuchaspossibleonthevocabularyandprosody,ratherthanonpronunciationissuesthatwouldbesolvedinlaterimplementationsbyintegrationwiththeWindow-EyesandNVDAscreenreaders,eachofwhichcanuseavarietyofspeechenginesandvoices.
Atthestartofeachaudiomathstatement,weaddedaudiointroductions,suchas,"Mathstatementforquestionsonethrougheight,"spokenviaJAWS,usingitsdefaultvoiceandspeechengine(EloquenceReed),andrecorded.
Attheendofeachmathstatementweinsertedatonetosignalstudentstopausetheaudiotoanswerthequestionsbeforemovingtothenextaudioclip.
So,foreachmathexpression,astudentwouldhearthefollowing:Introduction(e.
g.
,"Mathstatementforquestionsonethrougheight")ThespokenexpressionEndtoneThefinalaudiofileswereloadedontoMP3playersandtaggedsothattheclipswouldplayinthecorrectorderanddisplayappropriatelabelsonplayers.
Theaudiofilesareavailablefordownload.
SeeAppendixEforlinks.
StudyManipulationTherewerethreeinstrumentvariants;eachusedadifferentsetofclonesoftheexpressions.
ThesectiongivenfirstwasalwaysCloneGroup1,thesecondsectionwasCloneGroup2,andthethirdwasCloneGroup3.
Eachclonegroupwasgiventodifferentgroupsofstudentsineachofthespeechstyles.
Studentswererandomlyassignedtoreceiveagivensectionorder:Sixstudents,includingtheonewhodidnotfinishthestudy,receivedtheorderandclonegroupassignmentSimpleSpeak(CloneGroup1),MathSpeak(CloneGroup2),ClearSpeak(CloneGroup3);fivereceivedtheorder/assignmentMathSpeak(CloneGroup1),ClearSpeak(CloneGroup2),SimpleSpeak(CloneGroup3);andfivestudentsreceivedtheorder/assignmentClearSpeak(CloneGroup1),SimpleSpeak(CloneGroup2),MathSpeak(CloneGroup3).
QualificationsofStudyAdministratorsStudyadministratorswereeitherprojectconsultantswhoworkedatoneofthecooperatingschoolsorcooperatingschoolpersonnel.
Forstudentswhotooknotesinbraille,thestudyadministratortranscribedthosenotesintoprintsothatwecouldconsiderthemwhenweanalyzedthedata.
SessionProcedureFirst,studentscompletedthebackgroundquestionnaireswiththehelpofthestudyadministratorwhoalsocollectedparallelinformationfromthestudent'smathandVIteachers.
Next,administratorsprovidedthestudentswiththecorrectinstrument,basedontherandomorderassignment.
Form1usedtheorderSimpleSpeak(SectionS),MathSpeak(SectionM),andClearSpeak(SectionC).
Form2wasorderedM,C,S;andForm3wasorderedC,S,M.
Thebeginningoftheinstrumentexplainedthepurposeofthestudyanddescribedtheproceduretothestudents.
Last,administratorsguidedthestudentsthroughthethreesectionsoftheform,includingthemathquestions,thefeedbackquestions,andtheend-of-sectionquestions,andthentheend-of-studyquestions.
Asneeded,administratorsassistedwithaudioplaybackandscribingstudentresponsesforthosestudentswhodidnotentertheirownresponsesintotheWorddocument.
DataAnalysisTheinstrumentfileswerereturnedtouselectronicallybythestudyadministratorsandpastedortranscribedintoaspreadsheetbyastaffmember.
Becauseonestudentcompletedonlytwoofthethreesections(SimpleSpeakandMathSpeakonly),thatstudent'sdatawerenotusedintheanalysis.
Weorganizedthespreadsheetsand,basedontheanswerkeyforthemathquestions,enteredinformationaboutwhethereachanswerwascorrectorincorrect.
Inasmallnumberofcasesastudent'sanswertothemathquestionwastechnicallyincorrectbutcorrectforthepurposesofanalysis(or"adjustedcorrect");evenmorerarely,ananswercouldbetechnicallycorrectbutincorrectforthepurposesofanalysis.
Forexample,considertheexpression,aboutwhichstudentswereaskedtotypeinananswertothequestion,"Whatisthebiggestexponentinthemathstatement"Ifastudententeredorinsteadofjust13,thatwouldbetechnicallyincorrectbutadjustedcorrect(andsowastreatedascorrectforanalysis)becauseitshowedthatthestudentcorrectlyidentifiedthebiggestexponent,eventhoughthestudentdidnotexpressitasdirected.
Forthemathstatement,studentswereaskedtogivethevalueofthemathstatementandtoexplainhowtheyarrivedatthatvalue.
Insomecases,thestudent'sexplanationclearlyindicatedthatthestudentcorrectlyidentifiedthenumbersandtheoperationofmultiplication,butthatthestudentmultipliedincorrectly.
Again,sucharesponsewastechnicallyincorrectbutadjustedandcountedascorrectbecauseitshowedthatthestudentcorrectlydecodedthemathexpression.
Forthosestudentswhowrotedown(usingbrailleorbytypingintothedocument)whattheythoughttheyheard,wedeterminedwhethertheirwrittenexpressionmatchedthespokenexpression.
Insomecasesevaluatingthistranscriptionresultedinatechnicallycorrectanswerbeingadjustedincorrectorviceversa.
Althoughanalyseswereperformedonbothtechnicalcorrectnessandadjustedcorrectness,forthepurposesofreportingweareincludingonlytheanalysesbasedonadjustedcorrectnessbecausethatisthemeasurethataddressesstudents'levelofsuccessindecodingthemathexpressions.
Toavoidrepeating"adjustedcorrect,"whenever"correct"isusedinthisdocumentitreferstothe"adjusted"correctnessmeasure.
ForClearSpeak,thereweresixadjustmentsoutof105totalresponses;forSimpleSpeak,threeadjustmentsoutof105totalresponses,andforMathSpeak,fouradjustmentsoutof105totalresponses.
ResultsThissectiondescribeshowourresultsbearoneachoftworesearchquestions:DecodingComparison:DoesusingClearSpeakresultinequalorbetterdecodingofmathematicalexpressionsrepresentativeofAlgebra1content,ascomparedwiththeothertwostylestestedThatis,asimplementedinMathPlayer,howdoestheClearSpeakprototypestylecomparewiththepre-existingstylesSimpleSpeakandMathSpeakwithregardtostudents'abilitytodecodetherepresentativemathstatementsasmeasuredbydegreeofsuccessonthemathquestions(AppendixA)FavorabilityComparison:DostudentssubjectivelyreactatleastasormorepositivelytoClearSpeak,ascomparedwiththeothertwostylesThatis,asimplementedinMathPlayer,howdoestheClearSpeakprototypestylecomparewiththepre-existingstylesSimpleSpeakandMathSpeakwithregardtostudents'(a)overallleveloffavorableattitude,asexpressedintermsoffamiliarityandeaseofunderstanding(seeQuestions4,6,andsimilarquestionsinAppendixA),(b)levelofconfidenceintheirunderstandingofthemathstatement(seeQuestion5andsimilarquestionsinAppendixA),and(c)forthesubsetofquestionsthatinvolvedsomewhatmoredifficultmathproblemsolving,levelofconfidencethattheyhadansweredthemathquestioncorrectly(mathconfidence;seeQuestions34,44,55,57,and66inAppendixA)Aswewilldescribeindetailinthissection,ourresultssupportedverypromisinganswerstobothofthesecentralquestions.
WefoundthatexpressionsspokenwiththeClearSpeakstyleperformedbetterthanexpressionsusingtheothertwostyles,bothwithregardtosuccesswiththemathdecodingquestionsandwithregardtomeasuresoffavorableattitude(confidencethattheyunderstoodtheexpressionandfoundthespeechfamiliar)andconfidenceintheirabilitytohandlethemath("mathconfidence").
Inaddition,forallthesesummarymeasures,ClearSpeakshowedtheleastvariabilityinresponsebetweenstudentswithblindnessandthosewithlowvision.
QuantitativeResultsbyResearchQuestionBecauseofthesmallsample,quantitativeanalysisisdescriptiveonly.
DecodingComparisonsTable2showsthemeanscoresforthemathitems,whichwereaproxymeasureofwhetherthestudentcomprehendedthemathstatementsufficientlytoengagewithitmathematically,thatis,whetherthestudentsuccessfullydecodedthestatement.
Eachofthe15studentsansweredeightmathquestionsforeachofthethreespeechstyles,andwerescored1(correct)or0(incorrect)foreachquestion.
Thefiguresshownrepresentthemeanscoresonallofthemathitemsforallstudents,blindstudents,andstudentswithlowvisionforeachofthethreestyles.
Notethatwhiletheblindstudents'meanscoreswerehigherthanthoseofstudentswithlowvision,thedifferencebetweenthosegroupswassmallestformathstatementsspokenaccordingtotheClearSpeakstyle.
Table2MeanMathDecodingScoresClearSpeakallClearSpeakblindClearSpeaklowvisionSimpleSpeakallSimpleSpeakblindSimpleSpeaklowvisionMathSpeakallMathSpeakblindMathSpeaklowvision0.
890.
910.
880.
700.
780.
640.
560.
670.
46Note.
0=incorrect;1=correct;N=15students(7blind,8withlowvision),8questionsperstudentperspeechstyle.
Thescoresareaveragedacrossallitemsforall(N=15),blindonly(n=7)andlowvisiononly(n=8).
FavorabilityComparisonsInadditiontocomparingcorrectresponsestothemathquestions,weanalyzedstudents'feedbackresponsesindicatingdegreeofperceivedfamiliaritywiththespeechused,whethertheyfounditeasytounderstand,andthedegreeofconfidencethattheyhadinfactunderstoodtheexpression.
Table3showsthestudents'responsesforfavorability,whichcombinesfamiliarityandunderstandability:Favorability:Percentofstudentscharacterizing"thewaythecomputerspokethemathstatement"asveryfamiliar,somewhatfamiliar,notveryfamiliar,orveryunfamiliarfortheseveninstancesperstudentperspeechstyle(onepermathstatement)ofthatquestion;andpercentofstudentscharacterizing"thewaythemathstatementwasspokenbythecomputer"asveryeasy,somewhateasy,somewhathard,orveryhardtounderstandfortheseveninstancesperstudentperspeechstyle(onepermathstatement)ofthatquestionTable4showstheirresponsesregardingconfidencethattheyunderstoodtheexpressions.
Thepercentagesshownarebasedonthefeedbackquestionsgivenaftereachmathquestionandrepresenttheresponsesforallstudentsforalloftheexpressionsineachofthethreestyles:Confidenceinunderstanding:Percentofstudentsansweringthequestion"Howsureareyouthatyouunderstoodthemathstatement"withverysure,somewhatsure,notsure,definitelydidnotunderstandfortheseveninstancesperstudentperspeechstyleofthatquestion.
Table3Students'OpinionsofFavorabilityofExpressions(BasedonFamiliarityandUnderstandability)UsingEachSpeechStyleFeedbackClearSpeakSimpleSpeakMathSpeakFamiliarityVeryfamiliar48%24%24%Somewhatfamiliar50%50%31%Notveryfamiliar2%9%23%Veryunfamiliar0%16%22%UnderstandabilityVeryeasytounderstand52%17%16%Somewhateasytounderstand33%37%38%Somewhathardtounderstand13%24%23%Veryhardtounderstand3%22%22%Note.
N=15students(7blind,8withlowvision),7questionspersection.
Table4Students'OpinionsofConfidenceinUnderstandingofExpressionsUsingEachSpeechStyleFeedbackClearSpeakSimpleSpeakMathSpeakVerysureunderstood71%42%39%Somewhatsureunderstood24%37%33%Notsureunderstood5%7%11%Definitelydidnotunderstand0%14%17%Note.
N=15students(7blind,8withlowvision),7questionspersection.
Thedatawerefurtheranalyzedtocomputescoresforfavorability(familiarityandeaseofunderstanding)andconfidenceinunderstanding(suretheyunderstoodit).
Favorabilityscoresforeachexpressionwerebasedonstudents'answerstotwoquestions:(a)"Whichofthefollowingbestdescribesthewaythemathstatementwasspokenbythecomputer"(Veryeasytounderstand,somewhateasytounderstand,somewhathardtounderstand,orveryhardtounderstand)and(b)"Thewaythecomputerspokethemathstatementwas(veryfamiliar,somewhatfamiliar,notveryfamiliar,veryunfamiliar).
"Theanswerswerescoredasfollows:3pointswereassignedforveryeasy/veryfamiliar,2forsomewhateasy/familiar,1forsomewhathard/notveryfamiliar,0forveryhard/veryunfamiliar.
Becausethisscorewasbasedonthesumoftwoquestions,therangeofavailablescoreswas0–6perstudentpermathstatement.
Confidenceinunderstandingscoresforeachexpressionwerewasbasedonthequestion:"Howsureareyouthatyouunderstoodthemathstatement"(Verysure,somewhatsure,notsure,ordefinitelydidnotunderstand).
Againverysurecounted3points,definitelydidnotunderstandcounted0points,andthetwointermediatechoicescounted2pointsor1point,respectively.
Inaddition,forthemathstatementsaboutwhichmoredifficultmathquestionswereasked,thestudentswerealsoasked,"Onceyouunderstoodwhatthecomputerwassayingforthemathstatement,themathquestionitselfwas(veryeasy,somewhateasy,somewhatdifficult,verydifficult,couldn'tfigureoutwhatthecomputerwassaying)"Forcaseswhereastudentansweredthattheycouldn'tfigureoutwhatthecomputerwassaying,apointwassubtractedfromtheconfidenceinunderstandingscorebecausewetookthisasastrongindicationthatthespeechinanapplicablecasewasnotunderstoodbythestudent.
Becausethisscorewasbasedprimarilyononequestion(withapotentialadjustmentfromanadditionalquestion),therangeofavailablescoreswas-1to3perstudentpermathstatement.
Mathconfidencescoreswerealsocalculated.
Althougheveryattemptwasmadetominimizetheamountofactualmathmanipulationorcalculationrequiredtoanswerthe"mathquestions"(thepurposeofwhichwastogaugewhetherthestudentunderstoodtheexpressionwellenoughtoworkwithit—thatis,totestdecoding—notwhetherthestudentcoulddotheactualcomputation),someofthemorecomplexexpressionsrequiredmoreactualmaththanothers.
Forthosequestions(basedonMathStatements4–7),amathconfidencescorewasgeneratedfromresponsestothequestions(a)"Onceyouunderstoodwhatthecomputerwassayingforthemathstatement,themathquestionitselfwas(veryeasy,somewhateasy,somewhatdifficult,verydifficult,couldn'tfigureoutwhatthecomputerwassaying)"(seeQuestions34,44,55,57,and66inAppendixA)and(b)"Doyouthinkyouansweredthemathquestioncorrectly(yes/no)"(seeQuestions35,45,56,58,and67inAppendixA).
Forthefirstquestion,the0–3pointscalewasusedaspreviouslydescribed;forthesecondquestion"yes"wasassigned2pointsand"no"0points.
Onemathstatementinvolvedtwomathquestionsandtwosetsoftwomathconfidencequestions.
Sothatitcouldbecomparedconvenientlytotheothers,thescoresgeneratedfromtheanswerstothetwosetsofmathconfidencequestionswereaveraged.
Therangeofavailablescoresperstudentpermathstatementformathconfidencewas0–5.
Meanscoresforallstudentsforallexpressionsforfavorability,confidenceinunderstanding,andmathconfidenceareshowninTable5,whichalsoincludesforconveniencethemeanmathdecodingscoresfromTable2.
Table5AverageFavorability,ConfidenceinUnderstanding,MathConfidence,andMeanMathDecodingScoresAveragedAcrossAllItemsGroupClearSpeakSimpleSpeakMathSpeakFavorability:Range0to6All4.
753.
252.
96Blind4.
923.
963.
65Lowvision4.
612.
632.
36Confidenceinunderstanding:Range-1to3All2.
641.
871.
64Blind2.
532.
041.
85Lowvision2.
731.
721.
46Mathconfidence:Range0to5All3.
932.
681.
88Blind3.
803.
452.
86Lowvision4.
032.
000.
95Meanmathdecodingscores:Range0to1All0.
890.
700.
56Blind0.
910.
780.
67Lowvision0.
880.
640.
46Note.
All(N=15);Blind(n=7);Lowvision(n=8).
AversionofthistableappearsinFrankel,Soiffer,andBrownstein(2013).
Table5showsthatmeanscoresforeachmeasurewerenotablyhigherforClearSpeakthanforSimpleSpeakorMathSpeakforallstudentstakentogetherandforbothgroupswhenvisualstatus(blindorlowvision)wasbrokenout.
WealsonotedthatdifferencesbetweenmeanscoresonallmeasuresforstudentswithblindnesscomparedwithmeanscoresforallmeasuresforstudentswithlowvisionwererelativelysmallforClearSpeakandmuchlargerforSimpleSpeakandMathSpeak.
WealsocomputedquantitativeresultsbyexpressionandprovidethoseresultsinAppendixC.
Thosedatashowthatdifferencesbetweenspeechstylesareminimalforsomeexpressions(seeTablesC1throughC3)andsubstantialforothers(seeTablesC4throughC7).
Students'answerstothefree-responsequestionshelpedexplainthedata(seetheQualitativeFindingssection).
Students'overallimpressionsofthespeechstyleswerecapturedthroughthesectionsummaryandfinalsummaryquestions.
Intheend-of-sectionquestions,theywereaskedregardingeachspeechstyle,"Assumingthatyouareusingacomputertospeakmath,howmuchwouldyoulikeittospeakthewayitspokeinthissection"ResponsesaresummarizedinFigure1,whichshowsthat,directlyaftertheClearSpeaksection,almostallstudentswouldmostlyorverymuchlikemathspokenaccordingtotheClearSpeakstyle,whileveryfewstudentsfeltthatwayaboutSimpleSpeakorMathSpeakattheendoftheirrespectivesections.
Figure1Sectionsummaryresponses.
Percentageofstudentsprovidingeachresponse(N=15).
Afterworkingthroughthesectionsdevotedtoallthreeofthespeechstyles,studentswereaskedtoindicatewhichstyleoveralltheythoughtmosthelpedthemtounderstandthemathstatementsandwhichstyletheythoughthelpedleast.
TheseresultsaresummarizedinFigure2,whichshowsthattwothirdsofthestudentsthoughtthatoverallClearSpeakwasthemosthelpfulandnonethoughtittheleasthelpfulofthethreestyles.
NearlyallstudentsthoughtMathSpeakwastheleasthelpful.
SimpleSpeakfellinbetween.
Figure2Stylesconsideredmostandleasthelpful(N=15).
Note.
Onestudentselectedtwoofthestylesasleasthelpful,whichwascountedasone-halfpointforeachofthetwostylessoselected.
QualitativeFindingsTakentogether,theresultsdescribedpreviouslysupportkeyaspectsofourtrajectoryfordevelopmentofClearSpeak.
AsseeninFigures1and2,studentstendedtoindicateanoverallpreferenceforClearSpeakovertheothertwostyles.
Theyalso,asseeninTables3through5,tendedtoprovidehigherfavorabilityandconfidencescoresandweremorelikelytoanswerthemathquestionscorrectlywhenthemathwasspokenusingClearSpeak.
Becauseoursisadevelopmentproject,anadditionalimportantgoalofthefirstfeedbackstudybeyondthatofvalidatingouroverallapproachtoClearSpeakwastoobtainspecificfeedbackonspeechpatternsthatwereorwerenothelpful,inordertoguidefurtherdevelopment.
Tothatendweexaminedtheresponsestothefree-responsequestions.
Providingsupportforouroverallassumptionthatclassroom-likespeechwouldpromoteeasierdecodingandmorepositivesubjectivereactions,studentsfrequentlyexpressedapreferenceforspeechtheyconsideredclassroom-likewhentheyencountereditinthespokenmathinthestudy.
Forexample:"Ilikehowitsoundsjustlikemymorningmathclass.
""[Liked]termsusedinmymathclass""Ididn'tknowwhatnestedonedeepmeans.
I'veneverheardthatinclassbefore"[inreferencetoMathSpeak].
"Allaroundjusteasiertounderstandwiththechangeoftermsandfamiliarlanguageusedineverydaymathclasses""Thenumeratoranddenominatorstuffsoundslikeelementaryschool.
Idon'tlikeit.
Justuseopenandcloseforfractions.
Ihearthatinclass.
"Overall,students'responsesshowedthattheyheldavarietyofviewsregardingmanyofthedetailsofClearSpeak'simplementation,includinguseofpauses,thespeedatwhichthemathwasspoken,theuseofparenthesesversusimplied"times"(e.
g.
,shouldtheexpressionortheexpression(2)(3)(4)bespokenwitheachopeningandclosingparenthesis,with"times"toindicatemultiplication,orwithsomecombinationofboth),andsomeoftheterminologyemployed.
Wehadnotexplicitlyformulatedresearchquestionsaroundtheseissuesanddidnotattempttocountorcodethesefree-formresponses,butwedidfindthemworthyofnoteandconsideration.
Commentscenteredonissuesofpauses,speed,terminology,approachtoimplied"times,"andvoiceselection.
Eachofthesetopicsisdiscussedbrieflyinthissection.
PausesThespeechrulesforallthreestylesincludesomepauses,butpausesaremorecarefullystructuredinClearSpeak;inaddition,speechenginessometimespauseunpredictably.
Somestudentsfeltthatthepauseswerehelpful(thoughperhapsmoreasprovidinga"breather"than,atleastconsciously,aswaystoconveymathematicalmeaning);othersthoughtsomeofthepausesbrokeuptheflowofthemath.
Theextenttowhichthestyles'pausingstrategiesmayhavemoresubtlyaffectedstudents'perceptionsordecodingperformancecannotbedetermined,butitcouldhaveaffectedthestudents'overallperceptionsofthestyles.
Overall,studentshaddifficultyunderstandinghowthe(intentional)pausesweremeanttowork,whichsuggeststhatalthoughClearSpeakshouldbeunderstandablewithlittleornodirectinstruction,sometrainingorpracticeinitsconventionsislikelytobehelpful.
Thesecondfeedbackstudy,whichisdescribedinFrankelandBrownstein(inpress),furtherexplorestheeffectivenessoftwoelementsofprosody:pausesandspeechratechanges.
SpeedManystudentsfoundthespeedtobetoofast,butafewfoundittooslowfortheirliking.
StartingwiththeimplementationofWordintegrationwithWindow-Eyes(usedinthesecondfeedbackstudy),andcarryingforwardintotheimplementationwithNVDA(NonVisualDesktopAccess)(usedinsubsequentstudies),userscannowtakecontrolofthespeechrate.
Inaddition,theimplementationofinteractivenavigationwithinexpressions(nowcompleteandthefocusofthethirdfeedbackstudy)givesusersaddedcontrolovertheirpace.
Moststudentswerewritingexpressionsdown(usingbrailleorpaper)astheyheardthem(seeQuestion2andsimilarquestionsinAppendixA),andslowerspeedand/ormorecontroloverthepacemakethateasierforthemtoaccomplish.
Therearealsopreliminaryindications(fromthethirdfeedbackstudy)thatnavigationsupportreducestheneedtowriteexpressionsdownbyallowinguserstoexploreexpressions'mathematicalstructure.
TerminologyStudentsgenerallycommentedfavorablyonClearSpeak'slanguageof"raisedto"and"powers"andunfavorablyontermslike"superscript"and"baseline"(usedinMathSpeak)or"super"(usedinSimpleSpeak).
Manystatedthattheydidnotknowwhatthelattertermsmeant.
Onesuchexamplewastheexpression,whichwasspokeninMathSpeakas"xsuperscript9baselinedotstartfractionnestedonedeepysuperscript5baselineovernestedonedeepysquaredendfractionnestedonedeepdotstartfractionnestedonedeepzsuperscriptminus7baselineovernestedonedeepzsuperscriptminus3endfractionnested[pause]onedeep.
"Onestudentcommented,"Istilldon'tunderstandwhat'nestedonedeep'and'superscript'mean.
IthinkImayunderstandthatsuperscriptmeans'tothepowerofsomething'andsubscriptistothebaseofsomething.
Likex^7,the7isthesuperscriptandx7,the7isthesubscript.
"ThecorrespondingSimpleSpeakspeechwas"xtothe9dotfractionytothefifthoverysquaredendfractiondotfractionzsupernegative7endsuperoverzsupernegative3endsuperendfraction.
"FortheSimpleSpeakversion,studentsfrequentlyobjectedtotheterm"super.
"TheClearSpeakversionwas"xraisedtothepower9[pause]times[pause]thefractionwithnumeratoryraisedtothepower5anddenominatoryraisedtothepower2[pause]times[pause]thefractionwithnumeratorzraisedtothepowernegative7anddenominatorzraisedtothepowernegative3.
"StudentcommentsontheClearSpeakversionsweregenerallypositive,particularlywithregardtotheuseof"power.
"However,somethought"raisedtothe"addedunnecessarywords.
The"raised"hassincebeenremovedfromthespeechruleforexpressionswithfairlysimpleuseofexponents,suchasthecurrentexample.
SeeBrownsteinetal.
(2015a),p.
17.
Students'expressedpreferencesfortheClearSpeakapproachtoexponentsisconsistentwiththeexpression-breakoutsseeninAppendixC:expressionswherethemostsalientdifferencesbetweenspeechstyleswereinhowtheyhandledmorecomplicatedexponents(seeTableC7)showedastrongadvantageforClearSpeakoverbothoftheothertwostyles.
Asmentionedinthesection,ClearSpeakandtheSpeechStyleRequirements,ClearSpeakfocusesonclassroom-likespeechandmathematicalmeaning(hence"power,""times,""fof")insteadofonprintsymbolssuchas"superscript"(asonestudentcalledthelatter,"astrangesymbolbeingread")oronbraillesymbols.
Overall,studentscommentedfavorablyonthesymbol-sparseterminology:"Ilikedthatitdidn'tgivemealotofstrangesigns,justtheproblemlikeyou'dhearitreadbyaperson.
"ThatMathSpeakcanusedifficultterminologywasimplicitlyrecognizedbyBouck,Meyer,Joshi,andSchleppenbach(2013)intheirexplorationsofMathSpeakasameansofaccessingalgebra.
PartoftheirresearchmethodologyinvolvedexplicitinstructionwiththeMathSpeaklanguageandareviewofalgebraicterminologysuchascoefficientandterm.
Ourfirstfeedbackstudy,bycontrast,providednoexplicitinstructioninthelanguageorspecialterminologyofeachspeechstyle(sinceClearSpeakavoidstheuseoftermsexpectedtobeunfamiliar),butitdidincludeatthebeginningofeachsectionexamplesofthesameexpressionsrenderedinthespeechstyletobeusedinthesection.
ParenthesesandImplied"Times"ClearSpeakspeaksparenthesesas"open/closeparen"or"open/closeparenthesis,"onlywhereneededformathematicalclarity,using"times"inplaceofparenthesesthatindicateonlymultiplication,asin.
SimpleSpeakspeaksallparentheses("open/close"),asdoesMathSpeak("leftparenthesis/rightparenthesis").
Students'preferencesamongtheseweremixed.
Forexample,somethoughtSimpleSpeak's"open/close"wastooterse,andstudentswonderedwhatwasbeingopenedorclosed(weretheyparentheses,brackets,braces,etc.
)Ofthosewhocommented,morestudentscommentedfavorablyonClearSpeak's"openparenthesis/closeparenthesis"ascomparedtoMathSpeak's"leftparenthesis/rightparenthesis.
"Studentsweremixedonwhether"times,"theparentheses,orbothshouldbespokenincaseswhereparenthesesdenotemultiplication.
VoiceMicrosoftAnna(seethesection,DevelopmentofAudio)wasselectedonlybecauseitwasreadilyavailableatthetimeofthisstudy,notforsuitabilityforspeakingmath.
Weeditedouttheworstpronunciationproblems;however,studentsfrequentlymentionedthevoiceassomethingtheywouldliketohavechanged(althoughsomeindicatedtheylikedit),andsomementionedapreferenceforthe"standardJAWSvoice"(EloquenceReed).
ThesecondfeedbackstudyusedtheEloquencespeechengineimplementedinWindow-Eyes.
Withtheenhancedscreenreaderintegrationnowavailable,studentscanuseanyspeechenginesupportedbysupportedscreenreaders(currentlyNVDAandWindow-Eyes)andanyvoicesavailableontheircomputersthataresupportedbythespeechenginesrecognizedbythescreenreader.
DiscussionAswedescribedintheResultssection,itisclearfromthisfirststudythattheanswerstoourprimaryresearchquestions—howdoesClearSpeakcomparetoMathSpeakandSimpleSpeakwithregardtobothstudents'abilitytodecodethemathstatementsandstudents'subjectivereactionstothespeechstyle—werefavorabletoClearSpeak.
Soiffer,whoimplementedtheClearSpeakstyleinMathPlayer,isalsoresponsibleformaintainingtheotherstylesimplementedinMathPlayer.
AsaresultofthefeedbackreceivedonSimpleSpeakandMathSpeakandfollowingthefirststudy,SoiffercorrectedtheMathSpeak"nestedonedeep"speechpatternandimplementedseveralenhancementstoSimpleSpeak(e.
g.
,replacing"super"with"tothepower")basedonsomeoftheClearSpeakrulesdevelopedinthisphaseoftheproject.
TheClearSpeakrulestestedinthisstudyhavenowbeenfinalized,implementedinMathPlayer,anddocumentedinBrownsteinetal.
(2015a).
LimitationsAspreviouslymentionedintheDevelopmentofAudiosection,anundesirablespeechpattern("nestedonedeep")laterdeterminedtobeunintendedwaspartoftheMathSpeakrenditionsofthreeofthesevenmathexpressions.
Fourmathquestions,plustheirassociatedfeedbackquestions,werebasedontheseexpressions(Questions26,36,46,and47).
Thisspeechpattern(whichwasadjustedinMathPlayer'sMathSpeakimplementationafterthestudyconcluded)likelycontributedtostudents'difficultywithMathSpeak:Severalstudentsmentionedthatparticularphrasingasbeingconfusing.
TheexpressionsinwhichthisspeechpatternoccurredshowedthegreatestdisadvantagefortheMathSpeakstyle(seeTablesC4,C5,andC6).
However,wedonotthinkthe"nestedonedeep"speechpatternaccountsfullyforMathSpeak'sdisadvantagewhencomparedtoClearSpeak,becausethedisadvantagewaspresent,thoughtoalesserextent,intheremainderoftheexpressions(seeTablesC1,C2,C3,andC7).
SubsequentWorkAspreviouslynoted,screenreaderandWordintegrationhasbeenaccomplishedandusedinasecondstudythatfocusedoncertainprosodicelements,includingpausesandchangeofspeechrate(Frankel&Brownstein,inpress).
Theinteractivenavigationfeaturehasalsobeendeveloped,integratedwiththeWindow-EyesandNVDAscreenreadersandWord,andevaluatedinathirdfeedbackstudy.
Astudyoftheauthoringtoolsforteachersandothercontentcreators,aswellasafinalpilotthatincorporatesassessmentsofbothauthoringandspokenmathtools,wasalsoconducted.
Frankeletal.
(inpress)reportedonthefinalpilotwithteachersfortheauthoringtoolsandthefinalpilotwithstudentsforthespokenmathtools.
AcknowledgmentsWearegratefultoSusanOsterhausandMayleneBirdoftheTexasSchoolfortheBlindandVisuallyImpairedfortheirassistancewiththedevelopmentofClearSpeakandoftheresearchinstrumentsandwithadministeringtheinstrumentstosomeoftheparticipatingstudents.
Theresearchreportedherewassupportedbythe,U.
S.
DepartmentofEducation,throughGrantR324A110355totheEducationalTestingService.
TheopinionsexpressedarethoseoftheauthorsanddonotrepresentviewsoftheInstituteofEducationSciencesortheU.
S.
DepartmentofEducation.
NotesReferencesAmericanPrintingHousefortheBlind.
(2007).
Researchanddevelopmentactivities,fiscal2007.
Retrievedfromhttp://www.
aph.
org/research-report-2007/ar2007.
htmAtkinson,R.
C.
,&Shiffrin,R.
M.
(1971).
Thecontrolprocessesofshort-termmemory.
Retrievedfromhttp://citeseerx.
ist.
psu.
edu/viewdoc/downloaddoi=10.
1.
1.
398.
2237&rep=rep1&type=pdf&rep=rep1&type=pdf.