Frequencyregister.com

WEAPONSOFMASSDISTRACTION:ForeignState-SponsoredDisinformationintheDigitalAgeChristinaNemrandWilliamGangwareTheauthorsaregratefultothefollowingsubjectmatterexpertswhoprovidedinputonearlydraftsofselectexcerpts:Dr.
DrewConway,Dr.
ArieKruglanski,SeanMurphy,Dr.
AlinaPolyakova,andKaterinaSedova.
TheauthorsalsoappreciatethecontributionstothispaperbyAndrewRothgaberandBrendanO'DonoghueofParkAdvisors,aswellastheeditorialassistanceprovidedbyRhondaShoreandRyanJacobs.
ThisreportwasproducedwithsupportfromtheUSDepartmentofState'sGlobalEngagementCenter.
AnyviewsexpressedinthisreportarethoseoftheauthorsanddonotnecessarilyreflecttheviewsoftheUSStateDepartment,ParkAdvisors,oritssubjectmatterexpertconsultants.
Anyerrorscontainedinthisreportaretheauthors'alone.
AcknowledgementsAuthoredbyIntroductionandcontextualanalysisHowdowedefinedisinformationWhatpsychologicalfactorsdrivevulnerabilitiestodisinformationandpropagandaAlookatforeignstate-sponsoreddisinformationandpropagandaPlatform-specificchallengesandeffortstocounterdisinformationKnowledgegapsandfuturetechnologychallenges010406142639TableofContents1IntroductionandcontextualanalysisOnJuly12,2014,viewersofRussia'smainstate-runtelevisionstation,ChannelOne,wereshownahorrificstory.
Fivemonthsprior,theRussianmilitaryhadpusheditswayintoneighboringUkraine,andChannelOnehadbeencoveringthepoliticalandmilitaryactionontheground.
TheJuly12story,however,wasparticularlydramatic.
Thatday,ChannelOnereportersinterviewedawomanatarefugeecampneartheRussianborder,whoclaimedtowitnessasquadofUkrainiansoldiersnailathree-year-oldboytoapostinhertownsquare.
Thesoldiershadtorturedtheboytodeathoveraperiodofhours,beforetyinghismothertothebackofatankanddraggingherthroughthesquare.
1ChannelOneneverquestionedthewoman'sstory.
ButatleastoneindependentRussianjournalistfoundthetalesounbelievablethathevisitedtheallegedsitetoinvestigate.
Findingnoevidencethatthisatrocityhadeveroccurred,heinterviewedoneresidentinthetownsquare,thesupposedsiteofthecrime.
"This,"theresidentsaid,"isthefirstI'mhearingofit.
"SowheredidthestorycomefromWithinafewdays,thejournalistandotherstracedthestorybacktoapoliticalscientistwithtiestotheKremlin.
DaysbeforetheshockingChannelOnereport,thismanhadpostedasimilartaleonFacebook,withnearlyidenticaldetails.
BythetimetheKremlinconnectionwasuncovered,however,thedamagewasdone:notonlyhadtheChannelOnereportrunontelevision,buttheviralstorywasnowreachingamuchbroaderaudienceonsocialmedia.
ThefalsecrucifixionstorywasbutoneexampleofKremlin-backeddisinformationdeployedduringRussia'sannexationofCrimea.
Insubsequentyears,similartacticswouldagainbeunleashedbytheKremlinonotherforeignadversaries,includingtheUnitedStatesduringthelead-uptothe2016presidentialelection.
1.
See"State-RunNewsStationAccusedofMakingUpChildCrucifixion,"TheMoscowTimes,14July2014,https://themoscowtimes.
com/news/state-run-news-station-accused-of-making-up-child-crucifixion-37289;andArkadyOstrovsky,"Putin'sUkraineUnrealityShow,"WallStreetJournal,28July2014,https://www.
wsj.
com/articles/arkady-ostrovsky-putins-ukraine-unreality-show-1406590397;andAndrewHiggins,"FakeNews,FakeUkrainians,HowaGroupofRussiansTiltedaDutchVote,"NewYorkTimes,16Feb2017,https://www.
nytimes.
com/2017/02/16/world/europe/russia-ukraine-fake-news-dutch-vote.
html.
2Yettheuseofmodern-daydisinformationdoesnotstartandendwithRussia.
Agrowingnumberofstates,inthepursuitofgeopoliticalends,areleveragingdigitaltoolsandsocialmedianetworkstospreadnarratives,distortions,andfalsehoodstoshapepublicperceptionsandunderminetrustinthetruth.
Ifthereisonewordthathascometodefinethetechnologygiantsandtheirimpactontheworld,itis"disruption.
"Themajortechnologyandsocialmediacompanieshavedisruptedindustriesrangingfrommediatoadvertisingtoretail.
However,itisnotjustthetraditionalsectorsthatthesetechnologieshaveupended.
Theyhavealsodisruptedanother,moreinsidioustrade–disinformationandpropaganda.
Theproliferationofsocialmediaplatformshasdemocratizedthedisseminationandconsumptionofinformation,therebyerodingtraditionalmediahierarchiesandundercuttingclaimsofauthority.
Theenvironment,therefore,isripeforexploitationbybadactors.
Today,statesandindividualscaneasilyspreaddisinformationatlightningspeedandwithpotentiallyseriousimpact.
Therearesignificantvulnerabilitiesintheinformationecosystemthatforeignstate-sponsoredactorscanexploit,andtheyrevolvearoundthreeprimary,interconnectedelements:1.
Themedium–theplatformsonwhichdisinformationflourishes;2.
Themessage–whatisbeingconveyedthroughdisinformation;and3.
Theaudience–theconsumersofsuchcontent.
Thefirsttwoelements,themediumandthemessage,operatehandinhand.
Socialmediaandnewsplatformsaredesignedtodeliverinformationtomassaudiencesquickly,optimizingforviralcontentthatgeneratesclicksandthusrevenue.
Asaconsequence,theyareinherentlyvulnerabletosensationalistdisinformationthatseekstocatchtheeyeandbeshared.
2Themessagesconveyedthroughdisinformationrangefrombiasedhalf-truthstoconspiracytheoriestooutrightlies.
Theintentistomanipulatepopularopiniontoswaypolicyorinhibitactionbycreatingdivisionandblurringthetruthamongthetargetpopulation.
Unfortunately,themostusefulemotionstocreatesuchconditions–uncertainty,fear,andanger–aretheverycharacteristicsthatincreasethelikelihoodamessagewillgoviral.
Evenwhendisinformationfirstappearsonfringesitesoutsideofthemainstreammedia,masscoordinatedactionthattakesadvantageofplatformbusinessmodelsreliantuponclicksandviewshelpsensuregreateraudiencepenetration.
3Botnetworksconsistingoffakeprofilesamplifythemessageandcreatetheillusionofhighactivityandpopularityacrossmultipleplatformsatonce,gamingrecommendationandratingalgorithms.
2.
InformationSocietyProjectatYaleLawSchoolandtheFloydAbramsInstituteforFreedomofExpression,"FightingFakeNews(WorkshopReport),"2017,https://law.
yale.
edu/system/files/area/center/isp/documents/fighting_fake_news_-_workshop_report.
pdf.

3.
"Connectingthebots:Researchersuncoverinvisibleinfluenceonsocialmedia,"UniversityofGeorgia,30May2017,https://www.
sciencedaily.
com/releases/2017/05/170530095910.
htm.
Agrowingnumberofstates,inthepursuitofgeopoliticalends,areleveragingdigitaltoolsandsocialmedianetworkstospreadnarratives,distortions,andfalsehoodstoshapepublicperceptionsandunderminetrustinthetruth.
3Researchshowsthatthesetechniquesforspreadingfakenewsareeffective.
Onaverage,afalsestoryreaches1,500peoplesixtimesmorequicklythanafactualstory.
4Thisistrueoffalsestoriesaboutanytopic,butstoriesaboutpoliticsarethemostlikelytogoviral.
5Forallthathaschangedaboutdisinformationandtheabilitytodisseminateit,arguablythemostimportantelementhasremainedthesame:theaudience.
Nonumberofsocialmediabotswouldbeeffectiveinspreadingdisinformationifthemessagesdidnotexploitfundamentalhumanbiasesandbehavior.
Peoplearenotrationalconsumersofinformation.
Theyseekswift,reassuringanswersandmessagesthatgivethemasenseofidentityandbelonging.
6Thetruthcanbecompromisedwhenpeoplebelieveandshareinformationthatadherestotheirworldview.
Theproblemofdisinformationisthereforenotonethatcanbesolvedthroughanysinglesolution,whetherpsychologicalortechnological.
Aneffectiveresponsetothischallengerequiresunderstandingtheconvergingfactorsoftechnology,media,andhumanbehaviors.
Thefollowinginterdisciplinaryreviewattemptstoshedlightontheseconvergingfactors,andthechallengesandopportunitiesmovingforward.
4.
RobinsonMeyer,"TheGrimConclusionsoftheLargest-EverStudyofFakeNews,"TheAtlantic,08March2018,https://www.
theatlantic.
com/technology/archive/2018/03/largest-study-ever-fake-news-mit-twitter/555104/.

5.
Meyer,"TheGrimConclusions,"TheAtlantic.
6.
DanieleAnastasion,"ThePriceofCertainty,"NewYorkTimes,01November2016,https://www.
nytimes.
com/2016/11/01/opinion/the-price-of-certainty.
html.
Onaverage,afalsestoryreaches1,500peoplesixtimesmorequicklythanafactualstory.
Thisistrueoffalsestoriesaboutanytopic,butstoriesaboutpoliticsarethemostlikelytogoviral.
4HowdowedefinedisinformationSeveraltermsandframeworkshaveemergedtodescribeinformationthatmisleads,deceives,andpolarizes.
Themostpopularofthesetermsaremisinformationanddisinformation,andwhiletheyaresometimesusedinterchangeably,researchersagreetheyareseparateanddistinct.
Misinformationisgenerallyunderstoodastheinadvertentsharingoffalseinformationthatisnotintendedtocauseharm.
7Disinformation,ontheotherhand,iswidelydefinedasthepurposefuldisseminationoffalseinformationintendedtomisleadorharm.
8Althoughastraightforwarddefinition,itcanbedifficulttoascribepreciseparameterstodisinformation.
Forexample,disinformationisnotalwayscomposedoffabrications.
Itcanconsistoftruefacts,piecedtogethertoportrayadistortedviewofreality.
9Tounderstandthedisinformationenvironment,itisusefultodissectthedifferentelementsitencompasses.
10Disinformationcanincludeauthenticmaterialusedinadeliberatelywrongcontexttomakeafalseconnection,suchasanauthenticpicturedisplayedwithafakecaption.
Itcantaketheformoffakenewssitesoronesthataredeliberatelydesignedtolooklikewell-knownsites.
Disinformationcanfurtherincludeoutrightfalseinformation,sharedthroughgraphics,images,andvideos.
Itcanalsotaketheformofmanipulatedimageandvideocontent,wherecontroversialelementsarephotoshoppedintoinnocuouscontextstoevokeangeroroutrage.
7.
HosseinDerakhshanandClairWardle,"InformationDisorder:Definitions"inUnderstandingandAddressingtheDisinformationEcosystem,AnnenbergSchoolforCommunicationsworkshop,15-16December2017,pp.
5-12,https://firstdraftnews.
org/wp-content/uploads/2018/03/The-Disinformation-Ecosystem-20180207-v2.
pdf.
8.
IssueBrief:"DistinguishingDisinformationfromPropaganda,Misinformation,and"FakeNews","NationalEndowmentforDemocracy,17October2017,https://www.
ned.
org/issue-brief-distinguishing-disinformation-from-propaganda-misinformation-and-fake-news/.

9.
Seenote82;andDonFallis,"TheVarietiesofDisinformation"inThePhilosophyofInformationQuality[Boston:NortheasternUniversity,2014],pp.
135-161,https://www.
researchgate.
net/publication/321554157_The_Philosophy_of_Information_Quality;andAliceE.
Marwick,_"WhyDoPeopleShareFakeNewsASociotechnicalModelofMediaEffects,"GeorgetownLawTechnologyReview,2018,https://www.
georgetownlawtechreview.
org/wp-content/uploads/2018/07/2.
2-Marwick-pp-474-512.
pdf.
10.
Formore,seeClaireWardle,"6TypesofMisinformationCirculatedThisElectionSeason,"ColumbiaJournalismReview,18November2016,https://www.
cjr.
org/tow_center/6_types_election_fake_news.
php;andFallis,"TheVarietiesofDisinformation,"https://www.
researchgate.
net/publication/278692847_The_Varieties_of_Disinformation.
5AnoteontermsThisinterdisciplinaryreviewismeanttoexplorethetopicofdisinformation,understandingitasatermdistinctfrommisinformation.
However,theliteratureonthetopicoverwhelminglyusesthetermsmisinformation,disinformation,andevenfakenewsandpropagandainterchangeably.
Thisreviewthereforeusesthetermsasstatedinthespecificliteraturetoadheretothespiritoftheresearch.
Furthermore,whilethisreviewisfocusedonthewaysstateactorsusedisinformationtofurthergeopoliticalgoals,theanalysisofdisinformationcontainedhereinencompassesadditionalcontextsoutsideofgeopoliticsintheinterestofpresentingathoroughreview.
6Disinformationsucceeds,inpart,becauseofpsychologicalvulnerabilitiesinthewaypeopleconsumeandprocessinformation.
Indeed,expertsona2018Newsgeistpanel–agatheringofpractitionersandthinkersfromjournalism,technology,andpublicpolicy–identifiedanumberofpsychologicalfeaturesthatmakedisinformationsoeffectivewithaudiences.
Thesefeaturesincludehowdisinformationplaystoemotionsandbiases,simplifiesdifficulttopics,allowstheaudiencetofeelasthoughtheyareexposingtruths,andoffersidentityvalidation.
11Thefollowingsectionreviewssomeofthesepsychologicalfactorsandtheirimplicationsfortheappealandpersistenceofdisinformation.
TheneedtobelongAlargebodyofresearchshowsthatpeopledesiresocialbelonging,suchasinclusionwithinacommunity,andtheresultingidentitythataccompaniessuchbelonging.
12Indeed,theresearchindicatesthatthisneedforbelongingisafundamentalhumanmotivationthatdictatesmostinterpersonalbehavior.
13Thesemotivationsplayoutinrealtimeonline,oftenwithdrasticeffect.
Forbetterorworse,theinternetandsocialmediahavefacilitatedtheabilitytoseekoutandfindacommunitythatcontributestoaperson'ssenseofbelonging.
Inparticular,researchshowsthatsocialmediacanprovidepositivepsychosocialwell-being,increasesocialcapital,andevenenableofflinesocialinteractions.
1411.
"What'sdisinformationdoing"right"–andwhatcannewsroomslearnfromit,"NiemanLab,02November2018,http://www.
niemanlab.
org/2018/11/whats-disinformation-doing-right-and-what-can-newsrooms-learn-from-it/.

12.
GregoryM.
WaltonandGeoffreyLCohen,"AQuestionofBelonging:Race,SocialFit,andAchievement,"JournalofPersonalityandSocialPsychology,Vol.
92,No.
1,(2007),pp.
82-96,http://lmcreadinglist.
pbworks.
com/f/Walton+%26+Cohen+(2007).
pdf.
13.
RoyF.
BauermeisterandMarkR.
Leary,"TheNeedtoBelong:DesireforInterpersonalAttachmentsasaFundamentalHumanMotivation,"PsychologicalBulletin,1995,http://persweb.
wabash.
edu/facstaff/hortonr/articles%20for%20class/baumeister%20and%20leary.
pdf.

14.
ElizabethA.
Vincent,"SocialMediaasanAvenuetoAchievingSenseofBelongingAmongCollegeStudents,"VistasOnline,AmericanCounselingAssociation,2016,https://www.
counseling.
org/docs/default-source/vistas/social-media-as-an-avenue.
pdfsfvrsn=f684492c_8.

Whatpsychologicalfactorsdrivevulnerabilitiestodisinformationandpropaganda7SohowdoesthisapplytotheresonanceofdisinformationandpropagandaIntheirdesireforsocialbelonging,peopleareinterestedinconsumingandsharingcontentthatconnectswiththeirownexperiencesandgivesshapetotheidentityandstatustheywanttoproject.
15Status-seekingandidentityprojectionthereforeplayaconsequentialroleinmotivatingpeopletosharestories.
16Additionally,thenatureofsocialmediaamplifiesthechoicespeoplemakeinservicetosocialidentityandbelongingbecauseitsactivitytakesplaceinapublicforum.
17Theimplicationsofsocialidentityandbelongingfurtherextendtothesourcespeopletrustwhenconsuminginformation.
Researchshowsthatindividualsdependontheirsocialnetworksastrustednewssourcesandaremorelikelytoshareapostifitoriginatesfromatrustedfriend.
18Thiscanincreasesusceptibilitytodisinformationifone'snetworkispronetosharingunverifiedorlow-qualityinformation.
DealingwiththefirehoseofinformationThesheervolumeofcontentisanotherimportantfactorenablingdisinformation.
SixthousandtweetsaresenteverysecondandFacebookhosts300millionnewphotoseveryday.
19Researchrevealsthatindividualsareill-equippedtoassessandcopewithsuchvolume,leadingthemtoquicklydiscardwhattheyperceiveasirrelevantorunwantedinformation.
20Thevolumeofinformation,combinedwithpeople'slimitedattentionspans,alsomakesitdifficulttodiscernbetweenhigh-andlow-qualityinformation,creatinganenvironmentinwhichdisinformationhasthepotentialtogoviral.
21HowdopeoplethenrespondtoallthisinformationAlthoughpeopleliketobelievetheyarerationalconsumersofinformation,decadesofpsychologicalresearchhasdemonstratedthelimitationsofthehumanbrainwhenitcomestoinformationintake,particularlyincontextsofuncertaintyandanxiety.
22Humansaregenerallyaversetouncertainandambiguoussituations,leadingthemtoseekquick,definitiveanswers.
2315.
SeeDouglasRushkoff,DavidPescovitz,andJakeDunagan,"TheBiologyofDisinformation,"InstitutefortheFuture,2018,http://www.
iftf.
org/fileadmin/user_upload/images/ourwork/digintel/IFTF_biology_of_disinformation_062718.
pdf;andAliceE.
Marwick,"WhyDoPeopleShareFakeNewsASociotechnicalModelofMediaEffects.
"16.
Rushkoffetal,"TheBiologyofDisinformation,"InstitutefortheFuture,2018.
17.
NatalieJominiStroud,EmilyThorson,andDannagalYoung,"MakingSenseofInformationandJudgingitsCredibility"inUnderstandingandAddressingtheDisinformationEcosystem,AnnenbergSchoolforCommunicationsworkshop,15-16December2017,pp.
45-50https://firstdraftnews.
org/wp-content/uploads/2018/03/The-Disinformation-Ecosystem-20180207-v2.
pdf.

18.
Marwick,"WhyDoPeopleShareFakeNewsASociotechnicalModelofMediaEffects.
"19.
JominiStroud,Thorson,Young,"MakingSenseofInformationandJudgingitsCredibility.
"20.
XiaoyanQiuetal,"Limitedindividualattentionandonlineviralityoflow-qualityinformation,"NatureHumanBehavior,26June2017,https://www.
nature.
com/articles/s41562-017-0132.
21.
Qiuetal,"Limitedindividualattentionandonlineviralityoflow-qualityinformation,"NatureHumanBehavior.
22.
DonnaM.
WebsterandArieW.
Kruglanski,"CognitiveandSocialConsequencesoftheNeedforCognitiveClosure,"EuropeanReviewofSocialPsychology,15April2011,https://www.
tandfonline.
com/doi/abs/10.
1080/14792779643000100journalCode=pers20.
23.
MariaKonnikova,"WhyWeNeedAnswers,"TheNewYorker,30April2013,https://www.
newyorker.
com/tech/annals-of-technology/why-we-need-answers.
Thesheervolumeofcontentisanotherimportantfactorenablingdisinformation.
SixthousandtweetsaresenteverysecondandFacebookhosts300millionnewphotoseveryday.
8ArieKruglanski,apsychologyprofessorattheUniversityofMaryland,definesthisphenomenonastheneedforcognitiveclosure,orcertaintyinanuncertainworld.
24Thoughseekinganswersinthefaceofuncertaintyisanaturalhumanmotivation,furtherresearchindicatesthattheanswersuponwhichpeopleseizecanbesoclear-cutastobedetrimental,specificallywhensuchanswerspromoteblackandwhitethinkinganddisallowotherviewpoints.
25Inparticularcontexts,thiscancreatetheconditionsconducivetotheextremismandpolarizationthatallowsdisinformationtoflourish.
26Similartocognitiveclosure,theliteraturehasidentifiedothercognitivebiasesthatdictatehowpeopletakeinandinterpretinformationtohelpthemmakesenseoftheworld.
Forexample,selectiveexposureleadspeopletopreferinformationthatconfirmstheirpreexistingbeliefs,whileconfirmationbiasmakesinformationconsistentwithone'spreexistingbeliefsmorepersuasive.
27Thesebiasesinteractwith,andcomplement,twoothertypesofbias:motivatedreasoningandnaverealism.
28Whileconfirmationbiasleadsindividualstoseekinformationthatfitstheircurrentbeliefs,motivatedreasoningisthetendencytoapplyhigherscrutinytounwelcomeideasthatareinconsistentwithone'sideasorbeliefs.
29Inthisway,peopleusemotivatedreasoningtofurthertheirquestforsocialidentityandbelonging.
Furtherentrenchingtheeffectsofthesebiases,theresearchshowsthatnaverealismplaysanimportantroleduringtheintakeandassessmentofinformation.
Naverealismleadsindividualstobelievethattheirperceptionofrealityistheonlyaccurateview,andthatthosewhodisagreearesimplyuninformedorirrational.
30Thesecognitivebiasesshowthatalthoughindividualsmaybelievetheirconvictionsarebasedonhigherprinciples,inrealitypeopleprocessmoral-politicalstatementsaspreferencesasopposedtohardfacts.
31Giventhatpreferenceschangethroughoutone'slife,particularconvictionsmayalsochangeintherightcontextofpersuasion,attitudechange,orsocialnetwork.
Thisisespeciallytrueofconvictionsthataremorecontroversialorsubjecttomixedpublicconsensus,likepolitics.
32CognitivelimitationsinanonlinejungleSohowdothesecognitivebiasesplayoutinthesocialmediasphereA2016studyofnewsconsumptiononFacebookexamined376millionusersand920newsoutletstoanswerthisquestion.
Theyfoundthat24.
ArieW.
KruglanskiandDonnaM.
Webster,"MotivatedClosingoftheMind:"Seizing"and"Freezing","NationalCenterforBiotechnologyInformation,April1996,https://www.
ncbi.
nlm.
nih.
gov/pubmed/8637961.
25.
D.
Webberetal,"TheRoadtoExtremism:FieldandExperimentalEvidencethatSignificanceLoss-InducedNeedforClosureFostersRadicalization,"USNationalLibraryofMedicine,04September2017,https://www.
ncbi.
nlm.
nih.
gov/pubmed/28872332.
26.
Webberetal,"TheRoadtoExtremism:FieldandExperimentalEvidencethatSignificanceLoss-InducedNeedforClosureFostersRadicalization.
"27.
DavidM.
J.
Lazeretal,"TheScienceofFakeNews,"Science,09March2018,http://science.
sciencemag.
org/content/359/6380/1094/tab-pdf.
28.
DrewCalvert,"ThePsychologyBehindFakeNews,"KelloggInsight,06March2017,https://insight.
kellogg.
northwestern.
edu/article/the-psychology-behind-fake-news.
29.
GaryMarcus,"HowDoestheMindWorkInsightsfromBiology,"TopicsinCognitiveScience,17November2008,http://www.
psych.
nyu.
edu/gary/marcusArticles/Marcus%202009%20topics.
pdf.
30.
Calvert,"ThePsychologyBehindFakeNews.
"31.
Calvert,"ThePsychologyBehindFakeNews.
"32.
Calvert,"ThePsychologyBehindFakeNews.
"9userstendtoconfinetheirattentiontoalimitedsetofpages,seekingoutinformationthatalignswiththeirviewsandcreatingpolarizedclustersofinformationsharing.
33Inanotherstudy,researchersassessed330rumorthreadsonTwitterassociatedwithninenewsworthyevents,suchastheDecember2014SydneyhostagesiegeandtheJanuary2015CharlieHebdoshootinginParis,tounderstandhowpeopleinteractwithrumorsonsocialmedia.
Theiranalysisdeterminedthatusersaremoreactiveinsharingunverifiedrumorsthantheyareinlatersharingthattheserumorswereeitherdebunkedorverified.
34Theveracityofinformationthereforeappearstomatterlittle.
Arelatedstudyfoundthatevenafterindividualswereinformedthatastoryhadbeenmisrepresented,morethanathirdstillsharedthestory.
35Inadditiontohighlightingthelimitationsofhumancognition,theresearchalsopointstodecliningtrustinthepublicsphere.
RichardFletcherandRasmusNielsenfromtheUniversityofOxfordarguethatdisinformationmustbeanalyzedinthecontextofotherfactors,includingdecliningtrustinnewsmediaandincreasingskepticismofonlineinformation,whichhasbeenexacerbatedbyclickbaitandadvertisementsthatmasqueradeasnews.
36Inacomplementarystudy,researchersfoundthatparticipantswhoperceivedthemediaandtheword"news"negativelywerelesslikelythanotherstoidentifyafakeheadlineandlessabletodistinguishnewsfromopinionoradvertising.
37Thevaryinglevelsoftrustinthemediahaveimplicationsforeffortstovalidatetheveracityofnews.
Forexample,taggingsocialmediapostsas"verified"mayworkwellinenvironmentswheretrustinnewsmediaisrelativelyhigh(suchasSpainorGermany),butthisapproachmaybecounterproductiveincountrieswheretrustinnewsmediaismuchlower(likeGreece).
38DoublingdownonlineGiventhehumanmotivationsthatdriveonlinebehavior,researcherscontendthatitismorelikelythatpolarizationexacerbatesfakenews,ratherthanfakenewsexacerbatingpolarization.
3933.
AnaLuciaSchmidtaetal,"AnatomyofnewsconsumptiononFacebook,"ProceedingsoftheNationalAcademyofSciences,21March2017,http://www.
pnas.
org/content/pnas/114/12/3035.
full.
pdf.
34.
ArkaitzZubiagaetal,"AnalysingHowPeopleOrienttoandSpreadRumorsinSocialMediabyLookingatConversationalThreads,"PLOSONE11(3):e0150989,04March2016,https://doi.
org/10.
1371/journal.
pone.
0150989.
35.
LauraHazardOwen,"Americansmayappreciateknowingwhenanewsstoryissuspect,butmorethanathirdwillsharethatstoryanyway,"NiemanLab,29June2018,http://www.
niemanlab.
org/2018/06/americans-may-appreciate-knowing-when-a-news-story-is-suspect-but-more-than-a-third-will-share-that-story-anyway/.
36.
RichardFletcherandRasmusNielsen,"PeopleDon'tTrustNewsMedia–andthisisKeytotheGlobalMisinformationDebate"inUnderstandingandAddressingtheDisinformationEcosystem,AnnenbergSchoolforCommunicationsworkshop,15-16December2017,pp.
13-17,https://firstdraftnews.
org/wp-content/uploads/2018/03/The-Disinformation-Ecosystem-20180207-v2.
pdf.

37.
"HowthePublic,NewsSources,andJournalistsThinkaboutNewsinThreeCommunities,"NewsCo/LabatArizonaStateUniversityincollaborationwiththeCenterforMediaEngagementatTheUniversityofTexasatAustin,2018,https://mediaengagement.
org/wp-content/uploads/2018/11/How-the-Public-News-Sources-and-Journalists.
pdf.
38.
FletcherandNielsen,"PeopleDon'tTrustNewsMedia–andthisisKeytotheGlobalMisinformationDebate.
"39.
Calvert,"ThePsychologyofFakeNews.
"Usersaremoreactiveinsharingunverifiedrumorsthantheyareinlatersharingthattheserumorswereeitherdebunkedorverified.
Theveracityofinformationthereforeappearstomatterlittle.
10People'spropensitytoward"usversusthem"tribalismappliesjustasmuchtotheinformationtheyconsume.
What,then,canbedonetoreducepolarizationonlineTheliteraturehighlightsanumberofchallenges.
Inanefforttoavoidechochambers,somehaveadvocatedforincreasingonlinecommunities'exposuretodifferentviewpoints.
However,onestudythatattemptedthisapproachfoundittobenotjustineffective,butcounterproductive.
40ThestudyidentifiedalargesampleofDemocratsandRepublicansonTwitterandofferedfinancialincentivestofollowabotthatexposedtheparticipantstomessagesofopposingpoliticalideologies.
Theresultsweresurprising:Republicanswhofollowedtheliberalbotbecamesubstantiallymoreconservative,whileDemocratswhofollowedtheconservativebotbecameslightlymoreliberal.
Thestudyoffersacautionarytaleforfutureeffortstoreducepolarizationonline.
Theobservedbackfireeffectmaybeexplainedbycomplementaryresearch,whichfoundthatacknowledgingunwelcomefactsaboutcontroversialissuescanbethreatening.
41Humansreactthesamewaytoundesirableinformationastheydowhenfacingadangerousanimal–fightorflight.
42Todealwiththethreat,peopledoubledowntodefendtheirpreviouslyheldbeliefsorshunthenewinformationratherthanamendtheirviews.
43Giventhisingrainedresistancetonewideas,canpeoplechangetheirmindsThejuryisstillout.
Theabilityofindividualstoadjusttheirperceptionsafterbeingshowncorrectedinformationmayvarybasedontheircognitiveability.
44Onestudy,inwhichindividualswereshowncorrectionstomisinformation,foundthatindividualswithlowcognitiveabilitylessfrequentlyadjustedtheirviewpointsthanthosewithhighcognitiveability.
45Asimilarstudyshowedthatanaudience'slevelofcognitiveactivityislikelytopredictthepersistenceofmisinformationandeffectivenessofacorrection.
46TheresonanceofdisinformationandwhyitisdifficulttodebunkWhilesomehavearguedforanincreaseinfact-checkingordebunkingeffortstocounterdisinformation,theliteratureisagainmixedontheeffectivenessofsuchapproaches.
40.
ChristopherA.
Bailetal,"ExposuretoOpposingViewsonSocialMediaCanIncreasePoliticalPolarization,"ProceedingsoftheNationalAcademyofSciences,September2018,http://www.
pnas.
org/content/115/37/9216.
41.
BrendanNyhanandJasonReifler,"MisinformationandFact-Checking:ResearchFindingsfromSocialScience,"NewAmericaFoundation,February2012,https://www.
dartmouth.
edu/~nyhan/Misinformation_and_Fact-checking.
pdf.
42.
ArthurLupia,"CommunicatingScienceinPoliticizedEnvironments,"ProceedingsoftheNationalAcademyofSciences,20August2013,http://media.
wix.
com/ugd/fa8393_6973c3639e3c4bdfa2908cab10587cf4.
pdf.
43.
NyhanandReifler,"MisinformationandFact-Checking:ResearchFindingsfromSocialScience.
"44.
JonasDeKeersmaeckerandArneRoets,"'Fakenews':Incorrect,buthardtocorrect.
Theroleofcognitiveabilityontheimpactoffalseinformationonsocialimpressions,"Intelligence,Volume65,November2017,pp.
107-110,https://doi.
org/10.
1016/j.
intell.
2017.
10.
005.
45.
DeKeersmaeckerandRoets,"'Fakenews':Incorrect,buthardtocorrect.
"46.
Man-puiSallyChanetal,"Debunking:AMeta-AnalysisofthePsychologicalEfficacyofMessagesCounteringMisinformation,"PsychologicalScience,12September2017,https://journals.
sagepub.
com/doi/full/10.
1177/0956797617714579.
Humansreactthesamewaytoundesirableinformationastheydowhenfacingadangerousanimal–fightorflight.
11Disinformationisdifficulttocorrectbecauseofhowitresonatescognitivelyandemotionallywithitsintendedaudienceandhowhumansformbeliefsbasedontheinformationtheyhaveathand.
Thisphenomenonisreferredtoasbeliefperseverance,whichistheinabilityofpeopletochangetheirmindsevenafterbeingshownnewinformation.
47Factscanmatterlittleinthefaceofstrongsocialandemotionaldynamicsthatrelatetoone'spersonalidentityandsenseofcommunity.
48Othershaveadvocatedforincreasingmedialiteracyandmakingsocialmediausersmorecriticalconsumersofcontent.
However,researchershavefoundthateffortstoboostcriticalthinkingmaybeoflimiteduseormayhavethecounterproductiveeffectofleadingindividualstodoubtthecredibilityofnewsingeneral.
49Researchevenindicatesthatmanyconsumersofdisinformationalreadyperceivethemselvesascriticalthinkerswhoarechallengingthestatusquo.
50Theliteraturemakesexplicitthatconsumersarenotwell-equippedtoidentifycertainmessagesasfalseormisleading,norshouldtheonusbeonthemtodoso.
Tounderscorethesechallenges,onestudyanalyzedtheeffectivenessofmessagesmeanttoreducemisinformationaboutthelinksbetweenvaccinesandautism.
Itfoundthatwhilethemessagesoccasionallyreducedthebeliefthatvaccinescauseautism,theydidnotaffectthebehaviorortheintenttovaccinateamongparentswhohadthemostnegativeattitudesonvaccines.
51Asimilarstudyfoundthesamephenomenonamongskepticsofclimate-changeresearch.
Evidencethatcounteredtheiremotion-basedbeliefsdidnotfundamentallychangethosebeliefs.
52Consideringthesechallenges,arethereanyapproachestofact-checkingthatmightworkAccordingtotheresearchersbehindthevaccinestudy,onewaytocorrectmisinformationisbyprovidinganalternatecausalexplanationtodisplaceinferencesmadefromthefalseinformation.
However,otherresearchcastsdoubtonhowwellevenathoroughdebunkingstatementwillfare.
Astudyfoundthatdebunkingwasineffectivewhentheconsumercouldgeneratecompetingexplanationssupportingthemisinformation.
Furthermore,adebunkingmessagethatsimplyidentifiedmisinformationasincorrectwithoutofferingcorrectiveinformationwassimilarlyineffective.
Evenwhenadetaileddebunkingmessageincludedcorrectiveinformation,thedebunkingdidnotalwaysreduceparticipants'beliefinthemisinformation.
5347.
BrendanNyhanandJasonReifler,"DisplacingMisinformationaboutEvents:AnExperimentalTestofCausalCorrections,"JournalofExperimentalPoliticalScience,01April2015,https://doi.
org/10.
1017/XPS.
2014.
22.
48.
Calvert,ThePsychologyBehindFakeNews,"https://insight.
kellogg.
northwestern.
edu/article/the-psychology-behind-fake-news;andBrendanNyhanetal,"EffectiveMessagesinVaccinePromotion:ARandomizedTrial,"Pediatrics,April2014,http://pediatrics.
aappublications.
org/content/133/4/e835.
49.
Calvert,ThePsychologyBehindFakeNews.
"50.
DeenFreelon,"PersonalizedInformationEnvironmentsandTheirPotentialConsequencesforDisinformation"inUnderstandingandAddressingtheDisinformationEcosystem,AnnenbergSchoolforCommunicationsworkshop,15-16December2017,pp.
38-44,https://firstdraftnews.
org/wp-content/uploads/2018/03/The-Disinformation-Ecosystem-20180207-v2.
pdf.

51.
BrendanNyhanetal,"EffectiveMessagesinVaccinePromotion:ARandomizedTrial,"Pediatrics,April2014,http://pediatrics.
aappublications.
org/content/133/4/e835.
52.
PaulThagardandScottFindlay,"ChangingMindsAboutClimateChange:BeliefRevision,Coherence,andEmotion,"inBeliefRevisionMeetsPhilosophyofScience,eds.
EricJ.
OlssonandSebastianEnqvis[Netherlands:Springer,03November2010],http://cogsci.
uwaterloo.
ca/Articles/thagard.
climate.
2011.
pdf.
53.
Chanetal,"Debunking:AMeta-AnalysisofthePsychologicalEfficacyofMessagesCounteringMisinformation.
"12Compoundingmattersisthefactthatrepeatingafalseclaimcanincreaseitsbelievability.
54Instudieswhereparticipantsratedstatementsontheirtruthfulness,resultsshowedthatrepetitionincreasedparticipants'perceptionsofthetruthfulnessoffalsestatementsevenwhenparticipantsknewthesestatementswerefalse.
55Giventhatindividualsaresusceptibletofamiliarityofinformation,therepetitionofverifiablyfalsestatementscandecreasethepowerofindividualknowledgeandreasoning.
56Thisfindinghasfurtherimplicationsforfact-checking:afact-checkermustrepeatafalsestatementtodebunkit,therebyinadvertentlyincreasingtherepetitionandpotentialbelievabilityofthefalseclaim.
57Otherliteraturerevealsthatthenatureofhowonerelatestomisperceptionsimpactsthelikelihoodofacceptingorrejectingcorrections.
Forexample,misperceptionstiedtosalientorcontroversialissues,particularlythosethatmatterhighlytoanindividual'sconceptofself,arehighlypersistentandhardtocorrect.
Conversely,itmaybeeasiertocountermisperceptionsontopicswherepeopledonothavestronglyheldbeliefs.
58Thereis,however,asocialelementtofact-checkingthatcanencouragemoreaccurateinformation.
Forexample,iftherearestrongsocialconnectionsbetweenindividualswhogeneratefalsecontentandindividualswhofact-checkthiscontent,theformeraremorelikelytocorrecttheirfalsestatements.
59Unfortunately,becausecorrectedstatementsareoftenlessreadthanthemisleadingoriginalstatement,itisunclearhoweffectivesuchcorrectionsare.
60Anotherstudyshowedthatwhenconsumersoffakenewswerepresentedwithafact-check,theyalmostneverreadit.
61FightingfirewithfireSo,whatstrategiesmightworktocounterdisinformationRecentresearchismorepositiveregardingpotentialapproaches.
54.
AdamJ.
Berinsky,"RumorsandHealthcareReform:ExperimentsinPoliticalMisinformation,"BritishJournalofPoliticalScience,April2017,https://doi.
org/10.
1017/S0007123415000186.
55.
LisaK.
Fazioetal,"Knowledgedoesnotprotectagainstillusorytruth,"JournalofExperimentalPsychology,2015,https://apa.
org/pubs/journals/features/xge-0000098.
pdf.
56.
Fazioetal,"Knowledgedoesnotprotectagainstillusorytruth;"andLynnHasherandDavidGoldstein,"FrequencyandtheConferenceofReferentialValidity,"JournalofVerbalLearningandVerbalBehavior,1977,http://www.
psych.
utoronto.
ca/users/hasherlab/PDF/Frequency%20and%20the%20conference%20Hasher%20et%20al%201977.
pdf.
57.
DavidM.
J.
Lazeretal,"TheScienceofFakeNews,"Science,09March2018,http://science.
sciencemag.
org/content/359/6380/1094/tab-pdf.
58.
BrendanNyhanandJasonReifler,"MisinformationandFact-Checking:ResearchFindingsfromSocialScience,"NewAmericaFoundation,February2012,https://www.
dartmouth.
edu/~nyhan/Misinformation_and_Fact-checking.
pdf.
59.
DrewB.
Margolin,AnikoHannak,andIngmarWebber,"PoliticalFact-CheckingonTwitter:WhenDoCorrectionsHaveanEffect,"PoliticalCommunication,2018,https://www.
tandfonline.
com/doi/full/10.
1080/10584609.
2017.
1334018scroll=top&needAccess=true;andAnisaSubedar,"TheGodfatherofFakeNews,"BBC,27November2018,https://www.
bbc.
co.
uk/news/resources/idt-sh/the_godfather_of_fake_news.
60.
AliceMarwick,"WhyDoPeopleShareFakeNewsASociotechnicalModelofMediaEffects,"GeorgetownLawTechnologyReview,2018,https://www.
georgetownlawtechreview.
org/wp-content/uploads/2018/07/2.
2-Marwick-pp-474-512.
pdf.

61.
AndrewGuess,BrendanNyhan,andJasonReifler,"SelectiveExposuretoMisinformation:Evidencefromtheconsumptionoffakenewsduringthe2016USpresidentialcampaign,"EuropeanResearchCouncil,January2018,https://www.
dartmouth.
edu/~nyhan/fake-news-2016.
pdf.
Anotherstudyshowedthatwhenconsumersoffakenewswerepresentedwithafact-check,theyalmostneverreadit.
13Onestudyof7,200participantsfoundthatcounterfactualinformationcanchangepartisanopinionswhentheinformationispresentedwithstrongevidence.
62Ofnote,thisstudyalsofoundthatparticipantsgenerallymaintainedanopennesstoopposinginformationunlesstheywereprimedtofeeladversarial,oriftheopposingargumentswereladenwithinsultinglanguage.
Combiningtheseinsightswiththeaforementionedresearchonfightorflightreactionsoffersapotentialroadmapforcounteringmisleadinginformationonpolarizingtopics:presentcorrectiveinformationinatactfulandwell-supportedmannerthatavoidsdisparagingthosewithopposingviews.
Researchhasalsorevealeddifferentpsychologicalinterventionsthatmaybuildresilienceagainstdisinformation.
OnestudyfromtheCambridgeSocialDecision-MakingLabapproachedthetopicofmisinformationasametaphorical"contagion.
"Thestudycontendedthatmisinformationspreadsfromperson-to-personlikeadiseaseandthereforeexploredapotentialimmunizationintheformofattitudinalinoculation.
63Similartohowavaccinebuildsresistancetoavirus,attitudinalinoculationwarnspeoplethattheymaybeexposedtoinformationthatchallengestheirbeliefs,beforepresentingaweakenedexampleofthe(mis)informationandrefutingit.
Thisstrategycanbetterinform,andevenimmunize,participantstosimilarmisleadingargumentsinthefuture.
64Whenappliedtopublicattitudesaboutclimatechange,anexperimentthatusedattitudinalinoculationwithapolarizedaudiencefoundthatclimatemisinformationwaslesseffectivewhenparticipantswereinoculatedtosimilarmisinformationinadvance.
65Otherresearchoncognitiveabilityexaminesintegrativecomplexity(IC),whichisameasureofaperson'sabilitytoacceptandintegratemultipleviewpoints.
LowICindicatesapropensityforbinarythinkingandresistancetoopposingperspectives,whichhasdirectimplicationsfortheresonanceofdisinformationinpolarizedcontexts.
66TocounterlowIC,researchershavedevelopedinterventionstoexploretopicsthroughthelensofdifferentperspectives,whichallowspeopletounderstandandovercomethecognitivebiasesthatmayrenderthemadversarialtowardopposingideas.
Theseinterventionsfocuslessonthecontentofone'sthoughtsandmoreonthestructureofone'sthoughts,thereforeofferinganapproachthatcanbeappliedinmanydifferentcontexts.
67Asthesecombinedstrategiessuggest,manyofthesamepsychologicalfactorsthatmakehumanssusceptibletodisinformationcanalsobeusedtodefendagainstit.
Repeatingfacts,offeringsolidevidence,preemptivelywarningaboutanddebunkingdisinformationthemes,andencouragingopennesstodifferingviewpointsareallpotentialapproachesforreducingvulnerabilitiestodisinformation.
62.
JinWooKim"EvidenceCanChangePartisanMinds:RethinkingtheBoundsofMotivatedReasoning,"JobMarketPaper,30September2018,https://jinwookimqssdotcom.
files.
wordpress.
com/2018/10/kim_ws.
pdf.
63.
SandervanderLindenetal,"InoculatingthePublicagainstMisinformationaboutClimateChange,"WileyOnlineLibrary,23January2017,https://onlinelibrary.
wiley.
com/doi/full/10.
1002/gch2.
201600008.
64.
VanderLinden,"InoculatingthePublicagainstMisinformationaboutClimateChange.
"65.
VanderLinden,"InoculatingthePublicagainstMisinformationaboutClimateChange.
"66.
PeterSuedfeld,"TheCognitiveProcessingofPoliticsandPoliticians:ArchivalStudiesofConceptualandIntegrativeComplexity,"JournalofPersonality78,no.
6,October2010,https://doi.
org/10.
1111/j.
1467-6494.
2010.
00666.
x.
67.
EoleneM.
Boyd-MacMillanetal.
,"ISEE!
Scotland:TacklingSectarianismandPromotingCommunityPsychosocialHealth,"JournalofStrategicSecurity9,no.
4,2016,http://dx.
doi.
org/10.
5038/1944-0472.
9.
4.
1556.
14Alookatforeignstate-sponsoreddisinformationandpropagandaAstheadoptionofnewtechnologyandsocialmediaplatformshavespreadglobally,sotoohavegovernmenteffortstoexploittheseplatformsfortheirowninterests,athomeandabroad.
RussianattemptstoinfluencetheUnitedStates2016presidentialelectionandthe2016BrexitvoteintheUnitedKingdomaretworecent,high-profileexamples.
YettheuseofdisinformationextendswellbeyondRussianinterferenceintheUSandtheUK.
AUniversityofOxfordstudyfoundevidenceoforganizeddisinformationcampaignsin48countriesin2018,upfrom28theyearprior.
68Belowisanoverviewofseveralcountriesnotablefortheextentandsophisticationoftheirforeigninfluenceanddisinformationcampaigns.
RussianinfluenceanddisinformationcampaignsRevelationsofRussianinterferenceinthelead-uptothe2016USpresidentialelectionheightenedthepublic'sawarenessofdisinformationattacksagainsttheUnitedStates.
A2017reportbytheUSDirectorofNationalIntelligenceconcludedthatRussianPresidentVladimirPutinorderedaninfluencecampaignthatcombinedcovertcyberoperations(hacking,trollfarms,andbots)withovertactions(disseminationofdisinformationbyRussian-backedmedia)inanefforttounderminepublictrustintheelectoralprocessandinfluenceperceptionsofthecandidates.
69Theextentofthiscampaignwassignificant–thousandsofRussian-backedhumanoperativesandautomatedbotscreatedmorethanonemilliontweetsandhundredsofthousandsofFacebookand68.
SamanthaBradshawandPhilipN.
Howard,"ChallengingTruthandTrust:AGlobalInventoryofOrganizedSocialMediaManipulation,"OxfordInternetInstitute'sComputationalPropagandaResearchProject,July2018,http://comprop.
oii.
ox.
ac.
uk/wp-content/uploads/sites/93/2018/07/ct2018.
pdf.
69.
USOfficeoftheDirectorofNationalIntelligence,"Backgroundto"AssessingRussianActivitiesandIntentionsinRecentUSElections,"TheAnalyticProcessandCyberIncidentAttribution,"06January2017,https://www.
dni.
gov/files/documents/ICA_2017_01.
pdf.
15Instagramposts,whileuploadingmorethan1,000YouTubevideos.
70Thetweetsgarnered288millionviewsandtheFacebookpostsreached126millionUSaccounts.
71Russia'seffortsin2016mayhavecomeasasurprisetomany,yetRussiandisinformationcampaignsagainsttheUnitedStatesarenotarecentphenomenon.
TheSovietUniondeployedsimilarstrategiescalled"activemeasures"duringtheColdWarera,whichwerecoordinatedeffortsbySovietorganizationsandintelligenceagenciestoinfluenceopinionsinforeigncountries.
72IntheSovietUnion,propagandawasakeyelementofstatecraft,asimportantastheworkofthemilitaryorintelligenceagencies.
Duringthelate1980s,forexample,theSovietUnioncoordinatedaglobaldisinformationcampaigntoconvincetheworld'spublicthattheUnitedStateshadcreatedtheAIDSvirusasabiologicalweapon.
73Thiscampaignintendedtostirupanti-AmericanisminthedevelopingworldanddistractpublicattentionfromAmericanchargesthattheSovietUnionwasitselfdevelopingbiologicalweapons.
Therumor,firstpublishedin1983viaaSoviet-backednewspaperinIndia,appearedinRussianmediain1985andwaslaterpublisheddozensoftimesinRussianmediaoutlets.
BroadcastsbyRadioMoscowtargetedatAfricancountriesclaimedthatAmericawasdeliberatelyspreadingAIDSinZaire,echoingpreviousclaimsbytheoutletthattheUnitedStateswasdevelopingbiologicalweaponscapableofkillingonlynon-whites.
AsimilarcampaignwasmountedbytheSovietsaroundthesametimetospreadunfoundedallegationsoverchildorgantraffickingintheUnitedStates.
TherumorwasfirstreportedbyjournalistsduringaninterviewofaHondurangovernmentofficialin1987.
74Althoughthestatementwasquicklycorrectedbyhigh-levelHonduranofficials,Russianmediarepeatedlyranthesensationalistrumorsbetween1987and1988withoutmentioningthecorrections.
Theallegationsgainedmomentumoversubsequentyears,resultinginTurkeysuspendingitsadoptionprogramwiththeUnitedStatesintheearly1990s.
Notlongafter,aGuatemalanmobattackedUScitizensin1994onthebasisofthisstory.
75TheriseofmajorsocialmediaplatformshasofferedtheKremlinanopportunitytoinnovateonthisdecades-oldplaybook.
UnderformerPresidentDmitryMedvedev,theRussiangovernmentengagedinits70.
BenPopkin,"Russiantrollswentonattackduringkeyelectionmoments,"NBCNews,20December2017,https://www.
nbcnews.
com/tech/social-media/russian-trolls-went-attack-during-key-election-moments-n827176;and:MikeIsaacandDaisukeWakabayashi,"Russianinfluencereached126millionAmericansthroughFacebookalone,"NewYorkTimes,30October2017,https://www.
nytimes.
com/2017/10/30/technology/facebook-google-russia.
html.
71.
IsaacandWakabayashi,"Russianinfluencereached126millionAmericansthroughFacebookalone.
"72.
SteveAbrams,"Beyondpropaganda:SovietactivemeasuresinPutin'sRussia.
"Connections:TheQuarterlyJournal,15(1),31May2016,doi:http://dx.
doi.
org.
ezproxy.
cul.
columbia.
edu/10.
11610/Connections.
15.
1-0173.
USDepartmentofState,"SovietInfluenceActivities:AReportonActiveMeasuresandPropaganda,1986-87,"August1987,https://www.
globalsecurity.
org/intell/library/reports/1987/soviet-influence-activities-1987.
pdf.

74.
USInformationAgency,"TheChildOrganTraffickingRumor:AModern'UrbanLegend',"AReportSubmittedtotheUNSpecialRapporteurOnTheSaleOfChildren,ChildProstitution,AndChildPornography,December1994,http://pascalfroissart.
online.
fr/3-cache/1994-leventhal.
pdf.
75.
USInformationAgency,"TheChildOrganTraffickingRumor:AModern'UrbanLegend',"AReportSubmittedtotheUNSpecialRapporteurOnTheSaleOfChildren,ChildProstitution,AndChildPornography,December1994,http://pascalfroissart.
online.
fr/3-cache/1994-leventhal.
pdf.
Russia'seffortsin2016mayhavecomeasasurprisetomany,yetRussiandisinformationcampaignsagainsttheUnitedStatesarenotarecentphenomenon.
16firstwidespreaddeploymentofbotstospreadpoliticaldisinformationdomestically.
76Thisdisinformationcampaignproveddifficult;twoofRussia'smoredevelopedandcompetitiveindustriesaretechnologyandmedia,andtheRussianblogospherewasnoteasilyswayedbygovernmentdisinformation.
ThesechallengesspurredtheRussiangovernmenttodevelophighly-sophisticatedautomatedbotsandtrollingtechniquestobettercontroltheRussianpublic'sconversationsonline.
Thesesametoolsandtheirsuccessorswouldlaterbeunleashedonforeignadversaries.
77Atpresent,Russia'sinformationwarfaremachinefunctionslikeadiverseandinterconnectedecosystemofactors,includingstate-backedmediaoutlets,socialmediaaccounts,intelligenceagencies,andcybercriminals.
78AlthoughmanyofthehallmarksofSovietpropagandaarepresentinRussia'smodern-daypropagandaefforts,whathaschangedisthespeedwithwhichitsnarrativesarecreatedanddisseminated.
Before2016,Russiahoneditsonlinedisinformationeffortsinitsimmediatesphereofinfluence.
Asnotedattheoutsetofthisreport,RussiadeployedacoordinatedonlineinfluencecampaignduringitsannexationofCrimeain2014.
Russianstate-controlledmediaoutletspaintedauniquelyanti-Ukrainian,pro-Russiannarrativesurroundingthen-PresidentViktorYanukovych'sflightfromUkraineandthesubsequentRussianinvasionofCrimea.
79TohelpshoreupdomesticsupportforRussia'sactions,RussiangovernmentbotsdominatedthedomesticpoliticalconversationinRussiaduringthisperiod.
Between2014-2015,asmuchas85percentoftheactiveTwitteraccountsinRussiatweetingaboutpoliticswere,infact,governmentbots.
80Inmid-2016,theKremlinunleashedthesetacticsduringtheUnitedKingdom'ssuccessfulJune2016referendumvotetoleavetheEuropeanUnion.
Oneanalysisoftweetsfoundthatinthe48hoursleadinguptothevote,over150,000Russianaccountstweetedabout#Brexitandpostedmorethan45,000messagesaboutthevote.
81Onthedayofthereferendum,Russianaccountstweeted1,102timeswiththehashtag#ReasonsToLeaveEU.
82Meanwhile,Russiawasdeployingasimilarstrategyduringthe2016USpresidentialcampaign.
TheKremlin-backedInternetResearchAgency(IRA)initiateditseffortstointerfereinUSpoliticsasearlyas2014,spending$1.
25millionpermonthonitscombineddomesticandglobaloperations,whichincluded76.
SergeySanovich,"ComputationalPropagandainRussia:TheOriginsofDigitalDisinformation,"Eds:SamuelWoolleyandPhilipN.
Howard,WorkingPaper[Oxford,UK:ProjectonComputationalPropaganda,March2017]http://comprop.
oii.
ox.
ac.
uk/.
77.
Sanovich,"ComputationalPropagandainRussia:TheOriginsofDigitalMisinformation.
"78.
AlinaPolyakovaandSpencerP.
Boyer,"TheFutureofPoliticalWarfare:Russia,theWest,AndtheComingAgeofGlobalDigitalCompetition,"BrookingsInstitution,March2018,https://www.
brookings.
edu/wp-content/uploads/2018/03/the-future-of-political-warfare.
pdf.
79.
TomilaLankinaandKoheiWatanabe,"RussianSpring'or'SpringBetrayal'TheMediaasaMirrorofPutin'sEvolvingStrategyinUkraine,"Europe-AsiaStudies,March2018,https://doi.
org/10.
1080/09668136.
2017.
1397603.
80.
DenisStukaletal,"DetectingBotsonRussianPoliticalTwitter,"BigData,December2017,https://www.
liebertpub.
com/doi/10.
1089/big.
2017.
0038.
81.
UKParliament,"Russianinfluenceinpoliticalcampaigns,"Disinformationand'fakenews':InterimReport,https://publications.
parliament.
uk/pa/cm201719/cmselect/cmcumeds/363/36308.
htm.
82.
MatthewFieldandMikeWright,"Russiantrollssentthousandsofpro-LeavemessagesondayofBrexitreferendum,Twitterdatareveals,TheTelegraph,17October2018,https://www.
telegraph.
co.
uk/technology/2018/10/17/russian-iranian-twitter-trolls-sent-10-million-tweets-fake-news/.
AlthoughmanyofthehallmarksofSovietpropagandaarepresentinRussia'smodern-daypropagandaefforts,whathaschangedisthespeedwithwhichitsnarrativesarecreatedanddisseminated.
17dedicatedEnglishlanguagestafffocusedonthe2016USpresidentialcampaign.
83Thesecretiveagencywasheadquarteredinaheavily-guardedbuildingindowntownSt.
Petersburg.
84Ononefloor,employeesproducedahighvolumeoffakearticles,usingmostlyoriginaltexttocreateaveneerofauthenticity,andonanotherflooraseparategroupofemployeescreatedfakesocialmediaaccountstodistributethesearticlesandthenpostcommentsaboutthem.
85AnNBCreportidentified2,752Russian"troll"accountsthatpostedmorethan200,000tweets;thesetweetsearned2.
1millionretweetsand1.
9millionlikes.
86TwitterreportedanevenmoreexpansivecampaignthatlikelyextendedbeyondtheIRA,with36,000automatedaccountsposting1.
4milliontweetsthatearned288millionviewsleadinguptotheelection.
87OnFacebook,Russianpostsreached126millionUSFacebookaccounts.
OnInstagram,whichiswhollyownedbyFacebook,170Russianaccountscreatedmorethan120,000piecesofcontent,whichreachedmorethan20millionUSaccounts.
88TheactivitiesoftheIRAwerenotlimitedtoFacebook,Instagram,andTwitter;italsotargetedYouTube,Google+,Vine,Meetup,Pinterest,Tumblr,Gab,Medium,Reddit,andevenPayPal,whichhelpedsellitsmerchandise.
89TheIRA'sactivitiesonInstagramwereparticularlyeffectiveatgeneratingimpressions.
Instagram'splatformisconduciveforpostingthemostviralcontent–jokesandmemes–andRussianaccountsleveragedthisplatformtomaximizetheirreach.
Between2014and2017,IRAcontentonInstagramreached187millionengagements(likesandshares),farexceedingtheircontent's76.
5millionengagementsonFacebook.
90TheNewKnowledgeReportontheInternetResearchAgency'sdisinformationtacticspredictsthat"Instagramislikelytobeakeybattlegroundonanongoingbasis.
"91ItisclearthattherewasasignificantvolumeofRussianpostsandimpressionsgeneratedduringthe2016USpresidentialcampaign.
However,somehavecautionedagainstexaggeratingtheimpactofRussiandisinformationontheoutcomeoftheelection.
9283.
USDepartmentofJustice,"UnitedStatesofAmericavs.
InternetResearchAgency,"filed16February2018,https://www.
justice.
gov/file/1035477/download.
84.
BenPopkenandKellyCobiella,"Russiantrolldescribesworkintheinfamousmisinformationfactory,"NBCNews,16November2017,https://www.
nbcnews.
com/news/all/russian-troll-describes-work-infamous-misinformation-factory-n821486.

85.
PopkenandCobiella,"Russiantrolldescribesworkintheinfamousmisinformationfactory.
"86.
BenPopken,"Russiantrollswentonattackduringkeyelectionmoments"NBCNews,20December2017,https://www.
nbcnews.
com/tech/social-media/russian-trolls-went-attack-during-key-election-moments-n827176.
87.
MikeIsaacandDaisukeWakabayashi,"Russianinfluencereached126millionAmericansthroughFacebookalone,"NewYorkTimes,30October2017,https://www.
nytimes.
com/2017/10/30/technology/facebook-google-russia.
html.
88.
ReneeDiRestaetal,"TheTacticsandTropesoftheInternetResearchAgency,"NewKnowledge,December2018,https://disinformationreport.
blob.
core.
windows.
net/disinformation-report/NewKnowledge-Disinformation-Report-Whitepaper.
pdf;andIsaacandWakabayashi,"Russianinfluencereached126millionAmericansthroughFacebookalone.
"89.
DiRestaetal,"TheTacticsandTropesoftheInternetResearchAgency,"December2018;andPhilipN.
Howardetal,"TheIRA,SocialMediaandPoliticalPolarizationintheUnitedStates,2012-2018"[Oxford,UK:ProjectonComputationalPropaganda,2018],https://comprop.
oii.
ox.
ac.
uk/wp-content/uploads/sites/93/2018/12/IRA-Report-2018.
pdf.
90.
DiRestaetal,"TheTacticsandTropesoftheInternetResearchAgency.
"91.
DiRestaetal,"TheTacticsandTropesoftheInternetResearchAgency.
"92.
PatrickRuffini,"WhyRussia'sFacebookadcampaignwasn'tsuchasuccess,"WashingtonPost,3November2017,https://www.
washingtonpost.
com/outlook/why-russias-facebook-ad-campaign-wasnt-such-a-success/2017/11/03/b8efacca-bffa-11e7-8444-a0d4f04b89eb_story.
htmlutm_term=.
78cb0da3de03.
Duringthe2016presidentialcampaign,Russianpostsreached126millionUSFacebookaccounts.
18MostAmericans,forexample,probablyonlyscrolledpastaverysmallnumberofRussian-backedpoststhroughoutthedurationofthecampaign,whichsaysnothingaboutwhethertheyread,clickedon,orwereinfluencedinanymeaningfulwaybythecontent.
Furthermore,theseveralhundredmillionimpressionsofRussianpropagandaacrossTwitterandFacebookduringthecampaignweredwarfedbythebillionsoftotaldailyimpressionsofallcontentacrossbothplatforms.
Kremlin-generatedimpressionswereadropinthebucketcomparedtototaluseractivity,whichcallsintoquestiontheirabilitytohaveplayedadecisiveroleinswayingpublicopinion.
Russia'sad-targetingalsoappearedtolackanoverarchingelectoralstrategy.
Forexample,lessthan$2,000wasspentonRussianadsinthebattlegroundstateofWisconsin,andevenlessonthebattlegroundstatesofPennsylvaniaandMichigan,suggestingthatRussiancontentdidnotdelivermeaningfulimpactontheelectoralcollegevotesthatdecidedtheelection.
93OthershavearguedthattheIRA'sdisinformationcampaignwasamateurishandcareless,evenfailingtohidetheoriginofitscontent,whichfurtherunderscorestheneedforcautionwhenassessingtheeffectivenessofitspropaganda.
94ItisperhapsmoreplausiblethatRussiancyberhacksintotheClintoncampaign-ratherthantheKremlin'ssocialmediadisinformation-impactedthecourseoftheelection.
KathleenJamieson,thedirectoroftheAnnenbergPublicPolicyCenterattheUniversityofPennsylvania,hasarguedthatthedisclosuresfromWikiLeaks'releaseofRussian-hackedClintoncampaignemailscausedthedeclineinpolledvoters'trustinClintoninOctober2016.
95Inthewakeofthe2016election,theKremlinappearsintentoncontinuingtoleveragedisinformationtoinfluencepoliticaldiscourseintheUnitedStatesandelsewhere.
Indeed,USsanctionsandcondemnationsseemtohavedonelittletodissuadetheRussiansfrommaintainingtheseefforts.
WhiletheIRAspent$12millionduringthe2016electioncampaign,itsbudgettotaled$12.
2millionin2017and$10millionforthefirsthalfof2018leadinguptheUSmidterms.
96Russianpostershavealsoadaptedtheirtactics,shiftingawayfromproducingfictionalcontentwhichcanbecensoredbyplatformmoderators,towardsamplifyingexistingpoliticalmemespromotedbyfar-rightandfar-leftsources.
97Russia-backedSyrianinfluenceanddisinformationcampaignsPro-SyriangovernmentdisinformationhasproliferatedthroughsocialmediasincetheSyrianconflictbeganin2011.
MuchofthedisinformationappearstobeproducedanddisseminatedbyKremlin-linked93.
AlexisC.
Madrigal,"Russia'sTrollOperationWasNotThatSophisticated,"TheAtlantic,19February2018,https://www.
theatlantic.
com/technology/archive/2018/02/the-russian-conspiracy-to-commit-audience-development/553685/.

94.
NeilMacFarquhar,"RussianTrollsWereSloppy,butIndictmentStill'PointsattheKremlin',"NewYorkTimes,17February2018,https://www.
nytimes.
com/2018/02/17/world/europe/russia-indictment-trolls-putin.
html.
95.
JaneMayer,"HowRussiaHelpedSwingtheElectionforTrump,"TheNewYorker,1October2018,https://www.
newyorker.
com/magazine/2018/10/01/how-russia-helped-to-swing-the-election-for-trump.
96.
USDepartmentofJustice,"UnitedStatesofAmericav.
ElenaAlekseevnaKhusyaynova,"filed28September2018,https://www.
justice.
gov/usao-edva/press-release/file/1102591/download.
97.
JosephMenn,"RussiaseenadoptingnewtacticsinUSelectioninterferenceefforts,"Reuters,06November2018,https://www.
reuters.
com/article/us-usa-election-russia/russia-seen-adopting-new-tactics-in-u-s-election-interference-efforts-idUSKCN1NB0PIfeedType=RSS&feedName=worldNews.
Inthewakeofthe2016election,theKremlinappearsintentoncontinuingtoleveragedisinformationtoinfluencepoliticaldiscourseintheUnitedStatesandelsewhere.
19accountsasanextensionofRussia'smilitarysupportfortheAssadregime.
Thefollowingareafewnotableexamplesofthedisinformationtiedtotheongoingconflict.
InApril2018,inthewakeofasaringasattackinIdlibProvince,therewasawaveofalt-rightonlinepostingsintheUnitedStatesclaimingtheattackwasa"falseflag"operationintendedtounderminetheAssadregime.
TheAtlanticCounciltracedthesealt-rightpostingsbacktoapro-Assadoutlet,Al-MasdarNews,whichhadfirstpublishedtheallegationintheattack'simmediateaftermath.
98Inretaliationforthesaringasattack,theUnitedStateslaunchedastrikeagainsttheSyrianregime.
ThePentagonreportedanobserved2,000percentincreaseinRussianbotactivityspreadingdisinformationonTwitterinthe24hoursaftertheUSstrike.
99Severalmonthslater,thispatternofonlinemediadisinformationrepeated.
InSeptember2018,Assad'sregimeannounceditsintentiontolaunchamilitaryassaultontherebel-heldIdlibprovince.
AsTrumpwarnedSyriaagainstananticipatedchemicalweaponsattack,TwittersawasurgeinfakeRussianaccountcreation,ostensiblypreparingtospreaddisinformationaroundafutureattack.
100Concurrently,theRussianMinistryofDefensereleasedastatementclaimingtheSyrianrebelswerepreparingafalseflagoperationtogastheirownpeopleandhaveitfilmedbytheWhiteHelmets,avolunteerorganizationcreatedtoconductsearchandrescueoperationsinrebel-heldSyria.
101TheMinistryofDefense'sallegationswerelateramplifiedbyRussiaToday,theKremlin'sexternalmediaarm.
102A2018inquirybytheBellingcatinvestigationteamfoundthattheWhiteHelmetsbecameatargetofa"brutalandunrelenting"disinformationcampaignbecausetheyprovidedevidenceofgrosshumanrightsviolationsbyboththeSyriangovernmentandtheRussianmilitary.
103Thecampaign,whichwasplannedandunleashedbytheRussianandSyriangovernments,portrayedtheWhiteHelmetsasterrorists,agentsoftheWest,and"legitimatetargets.
"104Insupportoftheirpropagandacampaign,RussianagenciesandmediaoutletspublishednumerousreportsstartinginFebruary2018thatmadeunsubstantiatedclaimsabouttheWhiteHelmetstransportingchemicalweaponstoIdlibprovince.
105InDecember2018,theUSDepartmentofStateissuedastatement98DigitalForensicResearchLab,"Howthealt-rightbrought#SyriaHoaxtoAmerica,"MediumBlog,07April2017,https://medium.
com/dfrlab/how-the-alt-right-brought-syriahoax-to-america-47745118d1c9.
99.
JessicaKwong,"RussianTrollsIncreased'2,000Percent'AfterSyriaAttack,PentagonSays,"Newsweek,14April2018,https://www.
newsweek.
com/russian-trolls-increased-2000-percent-after-syria-attack-pentagon-says-886248.

100.
JackNassettaandEthanFecht,"RussiaisgearinguptomisinformtheUSpublicaboutSyria.
Here'sourcheatsheettoidentifyTwittertrolls,"WashingtonPost,17September2018,https://www.
washingtonpost.
com/news/monkey-cage/wp/2018/09/17/russia-is-gearing-up-to-misinform-the-u-s-public-about-syria-heres-our-cheat-sheet-to-identify-twitter-trolls/noredirect=on&utm_term=.
24ff76b55894.

101.
JosieEnsor,"Russianmisinformationabout'imminent'WhiteHelmetschemicalattackcouldspellstartofIdlibseige,"Telegraph,02September2018,https://www.
telegraph.
co.
uk/news/2018/09/02/russian-disinformation-campaign-syria-threatened-spark-new-war/.

102.
"FilmingofstagedchemicalattackinIdlib,Syriabegins–RussianMoD,"RTNews,11September2018,https://www.
rt.
com/news/438158-staged-chemical-attack-idlib/.
103.
BellingcatInvestigationTeam,"ChemicalWeaponsandAbsurdity:TheDisinformationCampaignAgainsttheWhiteHelmets,"AJointReportinCollaborationwithNewsy,18December2018,https://www.
bellingcat.
com/news/mena/2018/12/18/chemical-weapons-and-absurdity-the-disinformation-campaign-against-the-white-helmets/.
104.
BellingcatInvestigationTeam,"ChemicalWeaponsandAbsurdity.
"105.
LouisaLoveluck,"RussiandisinformationcampaigntargetsSyria'sbeleagueredrescueworkers,"WashingtonPost,18December2018,https://www.
washingtonpost.
com/world/russian-disinformation-campaign-targets-syrias-beleaguered-rescue-workers/2018/12/18/113b03c4-02a9-11e9-8186-4ec26a485713_story.
htmlutm_term=.
8733e7fd7096.
20thata"chemical"attackinAlepporeportedbySyrianandRussianstatemediawasactuallyatear-gasattackconductedbySyriangovernmentforcesto"undermineconfidenceintheceasefireinIdlib.
"106ChineseinfluenceanddisinformationcampaignsInSeptember2018,afour-pageadvertisementsponsoredbythestate-ownedChinaDailyranintheDesMoinesRegister.
Theadvertisement,mirroringanactualnewspaperspreadwithjournalisticarticles,includedaselectionofpiecesthattoutedthebenefitsoffreetradeforUSfarmers,theeconomicrisksofChina-UStradetensions,andPresidentXi'slongtiestothestateofIowa.
107TargetinganIowaaudienceinthemidstofChina'sagriculturaltradedisputewithTrump–andduringthemidtermcampaignseasonnoless–madeclearthatChinawouldnothesitatetotryshapingtheUSpoliticalconversation.
AstheDesMoinesRegisterexampleillustrates,China'spropagandaeffortsaredistinctfromRussia'sinseveralrespects.
WhileRussia'scampaignstendtobefocusedonthecyberdomain,asevidencedbyits2016electioninterferenceefforts,China'sinternationalinfluencecampaignsarelargelycharacterizedbyeconomic,political,andpersonalrelationship-building.
108Chinesecampaignshavebeenwidespread.
Theyrangefromtheproductionandglobaldistributionofpro-Chinesemedia,toattemptstoinfluenceeducationalandpolicyinstitutionsabroad,tothewieldingoffinancialinfluencethroughaggressiveloansandinfrastructureinvestment.
109In2010,Chinalaunchedatelevision,radio,andprintmediacampaigntochangehowconsumersofnewsviewedthecountryanditsplaceintheworld.
ThisincludednearlytriplingthenumberofChinaCentralTelevision(CCTV)bureausglobally,increasingthenumberofforeigncorrespondentsfortheChinaDaily,andbuildingoutanEnglish-languagetabloid,theGlobalTimes.
110China'sEnglish-languageoutletshaveproducedhundredsofarticlestoutingChina'sprosperityandstability,aimedprimarilyataforeignaudience.
111In2018,PresidentXiannouncedthemergerofCCTV,ChinaRadioInternational,andChinaNationalRadiointoasinglenetworknamedVoiceofChina.
Thenewnetwork'smissionincludesstrengtheninginternationaloutreachandinfluencingpublicopinionabroad.
112106.
BureauofPublicAffairs,USDepartmentofState,"TheRussianandAssadRegime'sFalseAllegationsonChemicalWeaponsUseinAleppo,"07December2018,https://www.
state.
gov/r/pa/prs/ps/2018/12/287941.
htm.
107.
DonnelleEller,"Chinese-backednewspaperinserttriestoundermineIowafarmsupportforTrump,tradewar,"DesMoinesRegister,24September2018,https://www.
desmoinesregister.
com/story/money/agriculture/2018/09/24/china-daily-watch-advertisement-tries-sway-iowa-farm-support-trump-trade-war-tariffs/1412954002/.
108.
AbigailGrace,"China'sInfluenceOperationsArePinpointingAmerica'sWeaknesses,"ForeignPolicy,04October2018,https://foreignpolicy.
com/2018/10/04/chinas-influence-operations-are-pinpointing-americas-weaknesses/;andAndreaKendall-TaylorandDavidShullman,"HowRussiaandChinaUndermineDemocracy,"ForeignAffairs,02October2018,https://www.
foreignaffairs.
com/articles/china/2018-10-02/how-russia-and-china-undermine-democracy.
109.
SamanthaCusteretal,"TiesThatBind:QuantifyingChina'spublicdiplomacyandits"goodneighbor"effect,"[Williamsburg,VA:AidDataatWilliam&Mary,2018]http://docs.
aiddata.
org/ad4/pdfs/Ties_that_Bind--Executive_Summary.
pdf.
110.
"TheChineseAreComing,"TheEconomist,04March2010,https://www.
economist.
com/asia/2010/03/04/the-chinese-are-coming.
111.
PaulMazur,"ChinaSpreadsPropagandatoUSonFacebook,aPlatformItBansatHome,"NewYorkTimes,08November2017,https://www.
nytimes.
com/2017/11/08/technology/china-facebook.
htmlmtrref=www.
google.
com&mtrref=undefined&gwh=16B98CC8D9E3ABFA00633192121284B5&gwt=pay.
112.
"Chinastatemediamergertocreatepropagandagiant,"TheGuardian,21March2018,https://www.
theguardian.
com/world/2018/mar/21/china-state-media-merger-to-create-propaganda-giant21ThroughChina's"UnitedWorkFrontDepartment,"theChineseCommunistParty(CCP)hasattemptedtoinfluenceleadingacademicinstitutionsandthinktanks.
113DelegationsofChineseauthoritiesfocusedonTibethavepaidvisitstouniversitieswithinfluentialTibetanacademics,includingColumbia,Harvard,andtheUniversityofVirginia,toexchangeviewsandshareofficialCCPtalkingpoints.
OnesuchexampleincludedChineseconsularofficialsinNewYorkCitytwicepayingvisitstoColumbiaTibetanprofessorRobertBarnett,threateningtocutofffuturecommunicationifhedidnotbetteralignhisviewsonTibetwiththeCCP.
114TheCCP-linkedChina-UnitedStatesExchangeFoundation(CUSEF)haspartneredwith,andinsomecasesfunded,institutionsincludingtheBrookingsInstitution,theCenterforStrategicandInternationalStudies,theAtlanticCouncil,theEast-WestInstitute,theCarterCenter,theCarnegieEndowment,andJohnsHopkinsUniversity.
115Inaddition,Chineseeducationalcentersandeventshavedramaticallyexpandedinrecentyears.
ConfuciusInstitutes,locatedonuniversitycampusesabroad,havegrowntomorethan500globallyandhavebeenusedtoapplypressureonprofessorsandcensorwhatChineseprofessorscanteachabroad.
116Certainuniversitieshavepushedback,includingtheUniversityofChicago,whichremoveditsConfuciusInstitutefromcampusin2014after100professorssignedapetitionprotestingitspresence.
117China'seffortstospreadinfluenceextendstosocialmediaplatformsthathavebeenbannedwithinitsownborders.
China'sdomesticinternetcontrolshavelongbeenrobust,couplingcensorshipofsensitivetopicswithanoutrightbanofmanywesternsocialmediaandtechnologyplatforms(amongthemGoogle,Facebook,andYouTube).
This"GreatFirewall"hascontinuedtogrowin2018,withthegovernmentrecentlyexpandingcybersecuritylawsandadvancingitssurveillancecapabilities,whilemakingitincreasinglydifficulttousevirtualprivatenetworkservicestoavoidthefirewall.
118WhileChinesesocialmediaisallowed,itisbothtightlycontrolledbygovernmentcensorsandfloodedwithpropaganda.
Sometwomilliongovernment-paidindividualscontributeroughly450millionpro-governmentpostsannuallytodistractanddrownoutanydomesticcriticismsoftheCCP.
119China'sbanonwesternsocialmediaisatacitacknowledgmentoftheseplatforms'potentialtoinfluenceChinesecitizens.
Meanwhile,theCCPusesforeignplatforms'networkstospreadstate-sponsoredadvertisementsinforeigncountries,includingtheUnitedStates.
ChineseentitiesareFacebook'slargestad-buyersinAsia,eventhoughChinesecitizenscannotusetheplatform.
120SomeestimateChinesebuyers113.
AlexanderBowe,"China'sOverseasUnitedFrontWork:BackgroundandImplicationsfortheUnitedStates,"US-ChinaEconomicandSecurityReviewCommission,24August2018,https://www.
uscc.
gov/sites/default/files/Research/China%27s%20Overseas%20United%20Front%20Work%20-%20Background%20and%20Implications%20for%20US_final_0.
pdf.
114.
AnastasyaLloyd-Damnjanovic,"APreliminaryStudyofPRCPoliticalInfluenceandInterferenceActivitiesinAmericanHigherEducation,"WilsonCenter,August2018,https://www.
wilsoncenter.
org/sites/default/files/prc_political_influence_full_report.
pdf.
115.
Bowe,"China'sOverseasUnitedFrontWork:BackgroundandImplicationsfortheUnitedStates.
"116.
RachellePeterson,"AmericanUniversitiesAreWelcomingChina'sTrojanHorse,"ForeignPolicy,09May2017,https://foreignpolicy.
com/2017/05/09/american-universities-are-welcoming-chinas-trojan-horse-confucius-institutes/.

117.
Te-PingChen,"Thanks,ButNoThanks,UniversityofChicagoTellsConfuciusInstitute,"WallStreetJournal,26September2014,https://blogs.
wsj.
com/chinarealtime/2014/09/26/thanks-but-no-thanks-university-of-chicago-tells-confucius-institute/.

118.
AdrianShahbaz,"Fakenews,datacollection,andthechallengetodemocracy"inFreedomontheNet2018:TheRiseofDigitalAuthoritarianism,FreedomHouse,2018,https://freedomhouse.
org/report/freedom-net/freedom-net-2018/rise-digital-authoritarianism.

119.
GaryKing,JenniferPan,andMargaretE.
Roberts,"HowtheChineseGovernmentFabricatesSocialMediaPostsforStrategicDistraction,notEngagedArgument,AmericanPoliticalScienceReview,2017,pp.
484-501,https://gking.
harvard.
edu/50C.
120.
Mazur,"ChinaSpreadsPropagandatoUSonFacebook,aPlatformItBansatHome.
"22spentfivebilliondollarsonFacebookadsin2018,makingthemthesecondlargestmarketaftertheUnitedStates.
121State-sponsoredmediaads,asmallfractionofthattotal,mirrortheCCP'sofflineeffortstopaintpositiveportrayalsoftheChinesegovernmentandbroaderChinesesociety.
Furthermore,theCCPhasestablishedFacebookpagesforitsvariousstate-runmediaoutlets,wherecontenthighlightingChinesesuccessesisdistributedandpromotedthroughpagefollowersandpaidadvertising.
Increasingly,ChinahasinterferedinforeignstatesinamannersimilartoRussia.
OneofthemorerecentandaggressiveexamplesistheNovember2018electioncampaigninTaiwan.
TheChinesegovernmentundertookasubstantialpropagandaefforttoundermineTaiwanesePresidentTsaiIng-wenandherDemocraticProgressiveParty(DPP)inhopesofoustingthemfromofficeinfavoroftheKuomintangoppositionparty,whoareseenasmorecomplianttoBeijing'swill.
122TheChineseengagedinamonths-longcampaignofanti-Tsai,anti-DPPsocialmediamessagingusingplatformsincludingFacebook,Twitter,andchatgroupsinamannersimilartoRussia'sinfluencecampaigns.
123Althoughdifficulttoascertainthefullimpactofthiscampaign,Tsai'spartydidsufferasignificantdefeatduringtheelection,promptinghertostepdownasleaderoftheDPPandimprovingKuomintang'schancestoretakethegovernment.
CovertsocialmediainfluencetacticshavealsobeenusedbytheCCP.
ChinesespieshavecreatedfakeLinkedInaccountstoconnectandbuildrelationshipswithwesternbusinessleadersandacademics.
Afterorganizingin-personmeetingsunderfalsepretenses,theseChineseofficialshavemadefinancialoffersinexchangeforestablishingintelligence-sharingrelationships.
124China'sinfluencecampaignisvastandmultifaceted,butitsabilitytochangemindshasbeenquestioned.
MuchofChinesemediacontentissoclearlybiasedinfavoroftheCCPthatinternationalaudiencesmayhavelittletroubleunderstandingtheseoutlets'trueintentionsandarethereforeunlikelytobeswayedinanymeaningfulway.
125Similarly,alackofobjectivityonthepartofChinesemediaemployees,suchasthe2018exampleofaChinaGlobalTelevisionNetworkreporterverballyandphysicallyharassingattendeesataUKpressconferenceforbeing"traitors"and"fakeChinese,"hasledsometoconcludethatChinesejournalists'effortsareaimedmoreatimpressingtheirsuperiorsthantrulyswayingheartsandmindsoverseas.
126121.
PreetiVarathan,"China,whereFacebookisbanned,couldmake$5billionforthecompanythisyear,"Quartz,16May2018,https://qz.
com/1279700/china-is-facebooks-second-largest-ad-spender-after-the-us/.
122.
ChrisHorton,"SpecterofMeddlingbyBeijingLoomsOverTaiwan'sElections,"NewYorkTimes,22November2018,https://www.
nytimes.
com/2018/11/22/world/asia/taiwan-elections-meddling.
html.
123.
JoshRogin,"China'sinterferenceinthe2018electionssucceeded—inTaiwan,"WashingtonPost,18December2018,https://www.
washingtonpost.
com/opinions/2018/12/18/chinas-interference-elections-succeeded-taiwan/utm_term=.
f0bf442eab64.

124.
JonasParello-Plesner,"China'sLinkedInHoneyTraps,"TheAmericanInterest,23October2018,https://www.
the-american-interest.
com/2018/10/23/chinas-linkedin-honey-traps/.
125.
HiltonYip,"China's$6BillionPropagandaBlitzisaSnooze,"ForeignPolicy,23April2018,https://foreignpolicy.
com/2018/04/23/the-voice-of-china-will-be-a-squeak/.
126.
JamesPalmer,"China'sGlobalPropagandaIsAimedatBosses,NotForeigners,"ForeignPolicy,01October2018,https://foreignpolicy.
com/2018/10/01/chinas-global-propaganda-is-aimed-at-bosses-not-foreigners/.
TheCCPusesforeignplatforms'networkstospreadstate-sponsoredadvertisementsinforeigncountries,includingtheUnitedStates.
23Nonetheless,asignificantportionofChinese-languagemediaoutletsabroadhavedirectorindirecttiestomembersoftheCCP,andconsequentlytheseoutletspumpoutadisproportionatelylargeamountofpro-Chinacontent.
ThisleavesinternationalChinese-languagecommunitiesparticularlyexposedtothisinformation.
127IranianinfluenceanddisinformationcampaignsBetweenAugustandOctober2018,FacebookcrackeddownontwoseparateIranianpropagandacampaigns,removinghundredsofFacebookandInstagramaccounts,pages,andgroups,someofwhichdatedbackto2011.
ThepagesalonewerefollowedbymorethanonemillionFacebookaccounts.
128FollowingtheAugustcrackdown,Twitterinturnremovedhundredsofaccountsthathadbeenengagedin"coordinatedmanipulation.
"129OnReddit,usersbegannoticingapatternofsuspiciouspostsbeginninginJuly2017whichtargetedtheWorldNewssubreddit's19millionfollowers.
ThepostsincludedpoliticalcontentlinkedtoobscurewebsitesthatRedditmoderatorswereabletotracebacktoIranafterinvestigating.
130ThecybersecuritycompanyFireEyereleaseditsownreportthatconcludedIranwasbehindacoordinateddisinformationcampaign,targetingUSandUKaudiencesanddatingbacktoatleast2017.
131IraniandisinformationhasdisplayedsimilaritiestobothRussianandChinesetactics.
Forexample,theyleveragedfakeTwitter,Facebook,Instagram,andRedditaccountstocoordinatedisinformationduringthe2018midtermsinamannersimilartotheKremlin'seffortsduringthe2016and2018USelectioncampaigns.
LikeChina,IranianpropagandaeffortshavelargelyfocusedonpromotingparticularpolicyintereststhatarealignedwiththeIraniangovernment.
Iranianinterestspromotedbytheseaccountsincludeanti-Israeliandpro-Palestiniannarratives,aswellascondemnationsofIran'sadversarySaudiArabia.
Likewise,therehasbeenanoverwhelminganti-Trumpbiastotheircontent,whichsomehaveperceivedasaresponsetoPresidentTrump'shardlinerhetorictowardsIranandhisdecisiontowithdrawtheUnitedStatesfromthe2015nuclearagreementwithIran.
132AReutersinvestigationinNovember2018foundthatIranusedmorethan70propagandawebsitestospreaddisinformationto15countries,includingtheUSandUK.
Thesewebsites,whichhadmorethan127.
HiltonYip,"China's$6BillionPropagandaBlitzisaSnooze.
"128.
"TakingDownCoordinatedInauthenticBehaviorfromIran,"FacebookNewsroom,26October2018,https://newsroom.
fb.
com/news/2018/10/coordinated-inauthentic-behavior-takedown/;andCraigTimbergetal,"SprawlingIranianinfluenceoperationglobalizestech'swarondisinformation,"WashingtonPost,21August2018,https://www.
washingtonpost.
com/technology/2018/08/21/russian-iran-created-facebook-pages-groups-accounts-mislead-users-around-world-company-says/noredirect=on&utm_term=.
aa3fbb707c54.

129.
Timberg,etal,"Iranianinfluenceoperationglobalizestech'swarondisinformation.
"130.
BenCollins,"VolunteersfoundIran'spropagandaeffortonReddit—buttheirwarningswereignored,"NBCNews,24August2018,https://www.
nbcnews.
com/tech/tech-news/volunteers-found-iran-s-propaganda-effort-reddit-their-warnings-were-n903486.

131"SuspectedIranianInfluenceOperation:LeveragingInauthenticNewsSitesandSocialMediaAimedatUS,UK,OtherAudiences,"Fireeye,21August2018,https://www.
fireeye.
com/content/dam/fireeye-www/current-threats/pdfs/rpt-FireEye-Iranian-IO.
pdf.

132.
Collins,"VolunteersfoundIran'spropagandaeffortonReddit—buttheirwarningswereignored;"andIssieLapowsky,"WhatWeNowKnowAboutIran'sGlobalPropagandaCampaign,"Wired,24August2018,https://www.
wired.
com/story/iran-global-propaganda-fireeye/.
24500,000monthlyvisitors,werepromotedonsocialmediabyIranianaccountswithoveronemillionfollowers.
133Onesuchsite,AWDnews,concoctedafakepieceinlate2016allegingthattheIsraeligovernmentthreatenedanuclearattackifPakistansenttroopstoSyria.
Failingtorealizethestorywasfake,Pakistan'sthen-DefenseMinisterrespondedwithanactualnuclearthreatagainstIsraelviaTwitter.
Coolerheadsprevailedoncetheinitialpiecewasrevealedtobeahoax.
134Iranianpageshavealsofoundsuccessingeneratingfollowersbydoctoringmemes,oftenaroundpolarizingtopics.
Onepopularpagewithmorethan400,000likes,titled"Noracismnowar,"photoshoppedanimageofTomHanksbyaddingaBlackLivesMatterslogantohist-shirt.
Theimagegenerated95,000shares.
135Since2011,well-produced,fakeBBCPersianvideoshavebeencreatedtocoverstoriesandprovideanalysisinconsistentwithBBCPersian'sactualcontent.
ThesevideosarepostedtowebsitesthatareprominentinsearchresultsforBBCPersian.
Thevideoshavealsobeenspreadthroughpro-Iraniansocialmediapages.
136MilitarypropagandaplaysaprominentroleinIranianinfluenceoperationsaswell.
Iran'sMinistryofIntelligenceandNationalSecurity,whichcoordinatesthisdisinformation,hasreleasedreportsexaggeratingIranianmilitarystrengthandtechnologicaldevelopments,hopingtoobscureandcomplicateaccurateassessmentsoftheircapabilities.
BybolsteringtheperceptionofIranianmilitarymightoverseas,Iranbelievesitcanweatherinternationalpressureandhelpdeterfuturemilitarythreats.
However,likeChina,someanalystsviewIran'soverteffortsatpropagandatobetooconspicuoustoeffectivelyswayinternationalaudiences.
137NorthKoreaninfluenceanddisinformationcampaignsTheNorthKoreangovernment(DPRK)hasengagedindisinformationeffortstonotonlyinfluenceinternationalactorsandspreadpro-DPRKpropaganda,butalsotoskirtinternationalsanctions.
OneexampleoftheformerwasNorthKorea'shackingattackagainstSonyPicturesin2014.
ItwasabrazenattempttoblackmailthecompanyintocancellingthereleaseoftheparodyfilmTheInterview,anditillustratedtheDPRK'swillingnesstotrytosilencecriticsthroughcriminalcyberactivity.
138133.
JackStubbsandChristopherBing,"SpecialReport:HowIranSpreadsDisinformationAroundtheWorld,"Reuters,30November2018,https://www.
reuters.
com/article/us-cyber-iran-specialreport-idUSKCN1NZ1FT.
134.
"IrandupedPakistanintoIsraelnukethreatastinypartofhugefakerycampaign,"TheTimesofIsrael,30November2018,https://www.
timesofisrael.
com/iran-duped-pakistan-into-israel-nuke-threat-as-tiny-part-of-huge-fakery-campaign/.

135.
AlexisC.
Madrigal,"IranianPropagandaTargetedAmericansWithTomHanks,"TheAtlantic,26October2018,https://www.
theatlantic.
com/technology/archive/2018/10/irans-facebook-propaganda-targeted-americans-tom-hanks/574129/.

136.
ThomasBrewster,"InsideThe7-Year-OldIranianPropagandaMachineProducingFakeBBCNews,"Forbes,28February2018,https://www.
forbes.
com/sites/thomasbrewster/2018/02/28/bbc-iran-fake-news/#3ec2371b54f1.
137.
FirasElias,"Iran'sMilitaryPropaganda:FailuresandSuccesses,"TheWashingtonInstituteforNearEastPolicy,10September2018,https://www.
washingtoninstitute.
org/fikraforum/view/irans-military-propaganda-failures-and-successes.

138.
TimStarks,"USindictsNorthKoreannationalforSonyhack,massivecyberattacks,"Politico,6September2018,https://www.
politico.
com/story/2018/09/06/justice-department-north-korea-sony-hack-771212.
OneIraniannewssiteconcoctedafakepieceinlate2016allegingthattheIsraeligovernmentthreatenedanuclearattackifPakistansenttroopstoSyria.
Failingtorealizethestorywasfake,Pakistan'sthen-DefenseMinisterrespondedwithanactualnuclearthreatagainstIsraelviaTwitter.
25CoverttacticshavealsobeenusedextensivelybyNorthKorea.
Asof2018,anestimatedseveralhundredDPRKagentsoperatefakecyberaccountstoinfluenceonlinediscourseinfavoroftheregime.
AimedataSouthKoreanaudience,theseagentshavecreatedaccountsmeanttoappearSouthKoreaninordertopostpro-DPRKcomments,blogposts,andvideos.
Pyongyang'sintentistwofold:topaintNorthKoreainafavorablelightandtostokedivisioninSouthKorea.
139TheDPRKhasfurtherleveragedcovertdisinformationtacticsfornontraditionalmeans-tofundtheregimeinthefaceofinternationalsanctionspressure.
NorthKoreansbasedinplaceslikeChinahavecreatedaweboffakeprofilesandbusinessesonprofessionalnetworkingandfreelancingplatformstodeceiveviewersandearnITcontractingbusinessfromclientsaroundtheglobe.
ThismoneyissentbacktotheNorthKoreanregimeatatimewhentheDPRKisdesperateforfunds.
Bymisrepresentingtheiridentities,theseillicitgovernment-backedbusinesseshaveskirtedinternationalsanctionsandfunneledpotentiallymillionsofdollarstotheDPRK.
140139.
Tae-junKang,"NorthKorea'sInfluenceOperations,Revealed,"TheDiplomat,25July2018,https://thediplomat.
com/2018/07/north-koreas-influence-operations-revealed/.
140.
WenxinFan,TomWright,andAlastairGale,"Tech'sNewProblem:NorthKorea,"WallStreetJournal,14September2018,https://www.
wsj.
com/articles/north-koreans-exploit-social-medias-vulnerabilities-to-dodge-sanctions-1536944018.

Asof2018,anestimatedseveralhundredDPRKagentsoperatefakecyberaccountstoinfluenceonlinediscourseinfavoroftheregime.
26Platform-specificchallengesandeffortstocounterdisinformationHavingexaminedhowforeignstatesareweaponizingdisinformation,thisreviewnowturnstothemajortechnologyplatformswheredisinformationandpropagandaaredisseminated.
WhatarecompanieslikeGoogle,FacebookandTwitterdoingtocountertheproblemThisisanessentialquestionforthesimplereasonthattoneutralizeonlinedisinformation,theplatformsthemselvesmustplayacentralrole.
Theirabilitytosolvethisproblem,atthemoment,farexceedsthatofanyotherorganization–includingnationalgovernments.
CompanieslikeGoogle,FacebookandTwitterpossesstheoverwhelmingmajorityofthedatapertainingtothisissue.
Theyhavedevelopedtheproprietaryalgorithmsthatidentifyhow,when,andwhereinformationisviewedbyusers.
Theyhavethemostexperienceworkingontheseissuesandarebest-equippedtoimprovehowdisinformationistrackedandcountered.
Andtheypossessunparalleledlevelsoftechnicalandmonetaryresourcestoaddresstheseissues;asofFebruary2019,Google,Facebook,andTwitterhaveacombinedmarketvaluationwellaboveonetrilliondollars.
Techgiantshavedemonstrated,tosomeextent,thattheyarewillingtoaddressdisinformation.
Google,Facebook,andTwitterareeachbuildingtheirowninternalandexternalteamsofsecuritypersonnelandfact-checkerstocounterpropagandaandillegalcontent.
Theyarealsodevelopingproprietarytools,someusingartificialintelligence(AI)andmachinelearning,tolimitdisinformationontheirplatforms.
Facebook,forinstance,announcedplansin2018toopentwonewAIlabsthatwillfocus,inpart,onhowtocounterdisinformation.
141Yet,socialmediaplatforms'incentivesarenotalwaysprioritizedtolimitdisinformation.
Insomerespects,theirincentivesarealignedwithspreadingmoreofit.
Techgiants'revenuesaregeneratedalmostentirelythroughadvertising,whichdependsonmaximizinguserengagementwiththeplatform.
Asoutlinedearlier,usersaremorelikelytoclickonorsharesensationalandinaccuratecontent;increasingclicksandsharestranslatesintogreateradvertisingrevenue.
Theshort-termincentives,therefore,arefortheplatformstoincrease,ratherthandecrease,theamountofdisinformationtheiruserssee.
141.
CadeMetz,"FacebookAddsA.
I.
LabsinSeattleandPittsburgh,PressuringLocalUniversities,"NewYorkTimes,04May2018,https://www.
nytimes.
com/2018/05/04/technology/facebook-artificial-intelligence-researchers.
html.

27Inthelongterm,however,thespreadofdisinformationandthegrowingpublicoutcriesagainstitmaydecreasepublictrustinsocialmediabrandsandtriggermoredecisiveactionbythesecompanies.
Asoutlinedbelow,theheightenedpublicawarenessandscrutinyleveledatsocialmediaplatformsfollowingthe2016USpresidentialelectionspurredthemtoenactandoverhaulanumberofpoliciesdesignedtocounteronlinedisinformation.
Thissectiondetailshowthemajorsocialmediaplatformshaverespondedtodisinformationandoutlinessomeofthechallengesthatremain.
Althoughthisreportattemptstocapturethemostrecenteffortsofeachplatform,thesecompaniesfrequentlyupdateuserpolicies,algorithms,andstrategiesforcounteringdisinformation.
Thisassessment,therefore,shouldbeviewedasarepresentativesnapshotoftheplatformlandscapeinlate2018andearly2019.
CounteringdisinformationatFacebookAsRussianandChinesepropagandaeffortshavemadeclear,Facebookisvulnerabletothespreadofdisinformationandinfluencethroughbothcovertandovertstrategies.
Botsandtrollscancreatefakeaccountstospreadfaketext,image,andvideocontentthroughposts,pages,andpaidadvertising.
Atthesametime,state-backedmediacompaniesoftenspreadovertinfluencethroughlegitimateaccountsandpagesthatusepaidadvertisingtodisseminatetheirmessages.
InthewakeoftherevelationthatdisinformationwassharedwidelyoverFacebookduringthe2016presidentialcampaign,Facebookannouncedtherolloutofseveralfeaturestohelpcombatdisinformation.
142First,thecompanymadeiteasiertoflagfalsenewstoplatformadministratorsbyallowinguserstoclickonthenewspostinquestionandselectfrompre-setreportingoptions.
Facebookalsobeganenlistingthehelpofthird-partyfact-checkerstoreviewreportsfromthecommunity.
Afterreviewing,thesethird-partyfact-checkersprovidearatingonthetrustworthinessofanarticle.
Newsdeemedinaccurateautomaticallyappearsloweronauser'snewsfeedandpreviouslydisplayeda"disputed"labeltowarnuserswhomayreadorsharethestoryonFacebook.
143Facebookadditionallybegantestingchangestoitsnewsfeedrankings,weighingitsalgorithmagainstarticlesthatpresentdisinformationwarningsigns.
Concurrently,Facebookbeganreducingthefinancialincentivesforfalsenews.
Thatincludedpreventingfakenewssitesfrom"spoofing"thedomainsofrealones.
Amaliciousactorcannolongerimitatealegitimatenewssite'sdomaininanattempttodeceivereaders.
Facebook'sactionswerenotlimitedtotheimmediatewakeofthe2016race.
In2018,theytookadditionalstepstofurtherrestrictdisinformation.
FollowingFacebook'sdebutofthe"disputed"label142.
AdamMosseri,"AddressingHoaxesandFakeNews,"FacebookNewsroom,15December2016,https://newsroom.
fb.
com/news/2016/12/news-feed-fyi-addressing-hoaxes-and-fake-news/.
143.
"HowisFacebookaddressingfalsenewsthroughthird-partyfact-checkers"FacebookHelpCenter,https://www.
facebook.
com/help/1952307158131536.
28ayearprior,thecompanyfoundthatmanyoftheuserswhoreadthedisputedinformationandwarninglabelactuallybecamemoreinclinedtosharethenews.
144Inresponse,Facebookchangeditsapproachandbeganshrinkingthelinkinthenewsfeedandincludinganumberofrelatedarticlesthatdebunkedthenewsinquestion.
145Theplatformalsoincreaseditsadvertisingcontrols,announcingthatallpoliticalandissue-specificadsandadvertiserswouldbevetted.
AdvertisersnowneedtohavetheiridentityandlocationconfirmedbyFacebookbeforereceivingauthorizationtorunadsontheirplatform.
Furthermore,allpoliticalandissue-basedadvertisementsareclearlylabeledwith"politicalad"inthetopcorner,alongwithinformationaboutwhofundedthead.
146ByNovember2018,Facebookhadalsodramaticallyscaledupitsfact-checkingoperation.
Ithadexpandeditsnetworkofthird-partyfact-checkingprovidersto23countries.
147Thecompanyalsobegantoimplementmachinelearningtoolstohelpthesefact-checkers.
Thetoolscanspotwarningsignsinarticlesandhelptheirhumancounterpartsprioritizetheirefforts.
148Toprovidemorecontextaboutnewssources,theplatformrolledoutanew"information"featureinthebottomcornerofnewsarticlesbeingshared.
Userscanclickontheinformationiconandseeadditionaldetailsaboutthepublicationandauthorofthepiece.
149Finally,Facebookannounceditsintentiontoexpanditsfact-checkingservicetophotoandvideocontentinordertowarnusersaboutdoctoredmediathatdepictinaccurateeventsormediathatisfalselyattributedtopeopleorevents.
150InOctober2018,Facebooklaunchedits"HuntforFalseNews"blogtodetailcasestudiesandoffersometransparencyabouthowitsplatformistacklingdisinformation.
151Duringthe2018USmidtermelections,FacebookprovideddirectaccesstoUSstategovernmentofficialstoreportpostscontainingfalseinformationaboutvoting.
152144.
JeffSmith,GraceJackson,andSeethaRaj,"DesigningAgainstMisinformation,"MediumBlog,20December2017,https://medium.
com/facebook-design/designing-against-misinformation-e5846b3aa1e2.
145.
JoshConstine,"Facebookshrinksfakenewsafterwarningsbackfire,"TechCrunch,May2018,https://techcrunch.
com/2018/04/27/facebook-false-news/.
146.
AlexHern,"NewFacebookcontrolsaimtoregulatepoliticaladsandfightfakenews,"TheGuardian,06April2018,https://www.
theguardian.
com/technology/2018/apr/06/facebook-launches-controls-regulate-ads-publishers.
147.
JonathanVanian,"FacebookExpandingFact-CheckingProjecttoCombatFakeNews,"Fortune,21June2018,http://fortune.
com/2018/06/21/facebook-fake-news-fact-checking/;and"Third-PartyFact-CheckingonFacebook,"FacebookBusiness,https://www.
facebook.
com/help/publisher/182222309230722helpref=faq_content.
148.
Constine,"Facebookshrinksfakenewsafterwarningsbackfire.
"149.
KifLeswing,"Facebookisfinallylaunchinganewfeaturetocombatfakenews,aftersixmonthsoftesting–andhere'showitworks,"BusinessInsider,03April2016,https://www.
businessinsider.
com/facebook-fake-news-about-this-article-feature-2018-4.
150.
LuluChangandEricBrackett,"Facebookoutlinesplansforcombatingfalsenews,"DigitalTrends,21June2018,https://www.
digitaltrends.
com/social-media/facebook-fight-fake-news/.
151.
JuliaAlexander,"Facebookintroduces'HuntforFalseNews'seriesinattempttobetransparentaboutmisinformation,"TheVerge,19October2018,https://www.
theverge.
com/2018/10/19/18001288/facebook-fake-news-hunt-spread-blog-election-misinformation.

152.
KevinCollier,"FacebookAndTwitterAreTakingStepsToHelpStatesKeepBogusElectionInformationOffTheirSites,"BuzzfeedNews,23October2018,https://www.
buzzfeednews.
com/article/kevincollier/facebook-twitter-false-information-election.

29However,theimpactoftheseactionsisstillunclear.
Facebook'sactionsmaynowmakeitharderforRussiantrollstocovertlyinfluencepoliticsabroad,butthecompany'snewpolicyvettingadvertisersstilldoesnotpreventforeigngovernment-backedmediaoutletsfromovertlyrunninginfluencecampaigns,asevidencedbytheongoingproliferationofChineseinfluenceadvertisementsonFacebook.
Likewise,Facebook'sunsuccessfulexperimentwithflaggingdisputednewssourcessuggeststhatmanypeopleareundeterredfromsharingarticlesevenwhenthefactshavebeenquestionedbyathirdparty.
Removingcontestedarticlescouldmitigatethisrisk,butpermanentlyremovingsuchcontestednews,ratherthanminimizingitsappearanceandnewsfeedposition,wouldopenFacebooktocriticismthatitiscensoringfreespeechandunfairlytargetingpoliticalviews.
InMarch2018,MarkZuckerbergconveyedhisstrongreluctancetoFacebookmakingthosekindsofeditorialdecisions.
153Facebook'scurrentapproachtowardsdisinformationleavessomevulnerabilities.
Oneisontheplatform'smessagingapps.
Itisespeciallydifficultforuserstodiscernthevalidityandsourceoftextcontentsharedviaprivatemessage.
Furtherchallengingmatters,privatemessagesaresharedbetweenuserswithoutthesameleveloffact-checkingsafeguardsthattakesplaceonapublicnewsfeed.
OnFacebook-ownedWhatsApp,whichfeaturesanencryptedmessagingserviceandhasapproximately1.
5billionmonthlyusersacrosstheglobe,disinformationandmisinformationhasbeenwidespread.
154InIndia,forinstance,adoctoredvideoappearingtoshowachildkidnappingwentviralonWhatsApp,leadingtomorethan30deathsfromdozensofmobincidentsbetween2017and2018.
155DuringBrazil's2018presidentialelectioncampaign,millionsofdollarswerespentbygroupsspreadingdisinformationonWhatsAppabouttheleftistcandidateFernandoHaddad,whoultimatelylosttohisopponentJairBolsonaro.
156A2018surveybyNiemanLabindicatesthatmorethanathirdofKenyansandSouthAfricans,andalmost30percentofNigerians,havesharedonlinefakenewsonWhatsApp,whichisthedominantplatformformobilemessaginginthosecountries.
157Facebookhasrecentlytakenstepstocounterdisinformationonmessagingapps,buttheopportunityforabuseremainssignificant.
MarkZuckerbergclaimedina2018Voxinterviewthatthecompany'salgorithmsscanprivatemessagecontenttopreventsharingofharmfulcontent,butthewell-documentedandongoingproliferationofinflammatoryanti-Rohingyacontenton153.
KurtWagner,"MarkZuckerbergsayshe's'fundamentallyuncomfortable'makingcontentdecisionsforFacebook,"Recode,22March2018,https://www.
recode.
net/2018/3/22/17150772/mark-zuckerberg-facebook-content-policy-guidelines-hate-free-speech.

154.
JoshConstine,"WhatsApphits1.
5billionmonthlyusers.
$19BNotsobad,"TechCrunch,31January2018,https://techcrunch.
com/2018/01/31/whatsapp-hits-1-5-billion-monthly-users-19b-not-so-bad/.
155.
TimothyMclaughlin,"DisinformationisSpreadingonWhatsAppinIndia-AndIt'sGettingDangerous,"TheAtlantic,05September2018,https://www.
theatlantic.
com/international/archive/2018/09/fighting-whatsapp-disinformation-india-kerala-floods/569332/.

156.
TaiNalon,"DidWhatsAppHelpBolsonaroWintheBrazilianPresidency"WashingtonPost,01November2018,https://www.
washingtonpost.
com/news/theworldpost/wp/2018/11/01/whatsapp-2/utm_term=.
7a9e914f926d.
157.
HermanWassermanandDaniMadrid-Morales"NewDataSuggestsAfricanAudiencesSeeSignificantlyMoreMisinformationthanAmericansdo,"NiemanLab,November2018,http://www.
niemanlab.
org/2018/11/new-data-suggests-african-audiences-see-significantly-more-misinformation-than-americans-do/;andDanielFunke,"NigeriaistheNextBattlegroundforElectionMisinformation,"ThePoynterInstitute,30November2018,https://www.
poynter.
org/fact-checking/2018/nigeria-is-the-next-battleground-for-election-misinformation/.

OnFacebook-ownedWhatsapp,disinformationhasbeenwidespread.
InIndia,forinstance,adoctoredvideoappearingtoshowachildkidnappingwentviralonWhatsApp,leadingtomorethan30deathsfromdozensofmobincidentsbetween2017and2018.
30FacebookMessengerinMyanmarsuggeststhatitsdetectionsystemsarenotfail-safe.
158Inmid-2018,FacebookbegantestingnewWhatsAppfeatures,includinglabelingforwardedmessagesandlimitingthenumberofforwardsperuser.
159Labelingamessageas"forwarded"stillleavestheresponsibilityofidentifyingtheauthenticityofamessage–andpreventingitsdistributionifdeemedfalse–squarelyontheuser.
AsecondvulnerabilityinFacebook'soperationsisitsheavyrelianceonoutsourced,third-partyfact-checkingservicesstaffedbyhumanoperators.
Manyofthesegroupsareunder-resourcedandoverwhelmedbythesheervolumeoffalseorunverifiedcontentontheplatform.
Forexample,Facebook'sfact-checkingproviderinthePhilippines,Rappler,hasbeeninundatedwithfalsenewsandstrugglingtokeepupwiththevolumeofdisinformationcreateddaily.
160Evenwhenfact-checkerscanhandlethevolumeofcontent,fakenewscaneasilygoviralinthetimebetweenitscreationandwhenfact-checkersareabletomanuallydisputethecontentandadjustitsnewsfeedranking.
Ifdisinformationreachesabroadaudiencebeforeitsremoval,itwillhaveaccomplisheditsintendedpurpose.
Nonetheless,Facebook'srecentstepsofferpromisethattheplatformwillbeabletorestrictthespreadofdisinformation.
Someearlyresearchsuggeststhattheireffortsmaybesucceeding.
AcademicsatNewYorkUniversityandStanfordUniversityhaveanalyzedtheproliferationofmorethan500fakenewswebsitesandmorethan10,000fakenewsstoriesonplatformsincludingFacebookbetweenJanuary2015andJuly2018.
TheyfoundthatthespreadoffakenewsonFacebookrosefrom70millionengagementspermonthinearly2015to200millionengagementsaroundthe2016USpresidentialelection.
Then,thedatapeaked.
Thespreadoffakenewsfellthroughout2017andreturnedtoapproximately70millionmonthlyengagementsbyearly2018.
161ThereductioninFacebook'sfakenewsengagementin2017and2018coincideswiththecompany'seffortstocounterdisinformation,indicatingthattheirtacticsmaybeeffective.
Yetotherfactorsmightalsoexplainthisdecline.
Forinstance,neither2017nor2018hadUSelectionsofthesameprominenceasthe2016presidentialelection,whichmayhavelessenedforeignincentivestomeddleinthefirstplace.
Maliciousactorsmayhavealsochangedtheirtacticsandsourcestobetterevadedetectioninthelatterstagesofthestudy.
Thesekindsofconfoundingfactors,alongwiththeproprietarynatureofFacebook'sdata,complicatestheabilitytoperformacomprehensiveassessmentanddrawdefinitiveconclusions.
158.
EzraKlein,"MarkZuckerbergonFacebook'sHardestYear,andWhatComesNext,"Vox,02April2018,https://www.
vox.
com/2018/4/2/17185052/mark-zuckerberg-facebook-interview-fake-news-bots-cambridge;andSteveStecklow,"WhyFacebookisLosingtheWaronHateSpeechinMyanmar,"Reuters,15August2018,https://www.
reuters.
com/investigates/special-report/myanmar-facebook-hate/.
159.
"LabelingForwardedMessages,"WhatsAppBlog,10July2018,https://blog.
whatsapp.
com/10000645/Labeling-Forwarded-Messages;and"MoreChangestoForwarding,"WhatsAppBlog,19July2018,https://blog.
whatsapp.
com/10000647/More-changes-to-forwarding.
160.
AlexandraStevenson,"SoldiersinFacebook'sWarOnFakeNewsAreFeelingOverrun,"NewYorkTimes,09October2018,https://www.
nytimes.
com/2018/10/09/business/facebook-philippines-rappler-fake-news.
html.
161.
HuntAllcott,MatthewGentzkow,andChuanYu,"TrendsintheDiffusionofMisinformationonSocialMedia,"StanfordInstituteforEconomicPolicyResearch,October2018,http://web.
stanford.
edu/~gentzkow/research/fake-news-trends.
pdf.
31CounteringdisinformationatTwitterLikeFacebook,Twitterhasgrappledwiththespreadofdisinformationthroughbots,fakeaccounts,andadvertisements.
Insomerespects,Twitterisuniquelyvulnerabletocoordinatedandviraldisinformationcampaigns.
WhilemostFacebookusers'newsfeedactivityistiedtotheirpersonalnetworks,Twitterisaplatformwherenewsandcommentaryisoftensharedwidelythroughissue-specifichashtags,publiccommentthreads,andinfluenceraccountswithlargefollowings.
ResearchersfromtheMassachusettsInstituteofTechnologyanalyzedthousandsofstoriesonTwitterthatweretweetedmillionsoftimesbetween2016and2017andfoundthatdisinformationoftenreachedabroaderaudiencethanthetruth.
Forexample,thetoponepercentoffakenewsstoriestypicallyreachedbetween1,000and100,000people,whereastruestoriesrarelyexceededanaudienceofonethousand.
Inlinewithsimilarstudies,theresearchersconcludedthatpeopleweremorelikelytosharefakenewsbecausethesestoriesgeneratedmorereactionsofsurpriseanddisgust.
162Twitterbotsplayanimportantroleindisseminatingdisinformation.
Researchersanalyzingthespreadof400,000articlesvia14millionmessagesonTwitterbetween2016and2017foundthatbotswereanimportantvehicleforsharingcontentthathad"low-credibilitysources.
"Botsregularlysharelow-credibilitycontentinthefirstfewsecondsofitspublication,therebyincreasingthenumberofimpressionsandconsequentlyitschancesofgoingviral.
Sixpercentoftheidentifiedbotsaccountedfor31percentofthespreadoflow-credibilitycontent,typicallywithinthefirst10secondsafteranarticleispublished.
163Furthermore,aKnightFoundationstudyof700,000Twitteraccountslinkedtodisinformationduringthe2016USelectionalsofoundthatthemajorityoftheseaccountswereeitherfullyorsemi-automatedbots.
164Recentestimatessuggestthatthenumberofbotsontheplatformincreasedsignificantlybetween2014and2017.
Twitterestimatedin2014thatbetweenfiveand8.
5percentofitsuserbaseconsistedofbots.
Severalyearslater,inMarch2017,researchbyacademicsfromIndianaUniversityandtheUniversityofSouthernCaliforniaindicatedthattheprevalenceofbotswassignificantlyhigher:15percentormoreoftheplatform'stotalaccounts.
165TwitterbegantakingstrongeractiontocounterthisthreatinOctober2017,whenthecompanyannouncedanewsetofpoliciestoincreaseadvertisingtransparency.
Theinitiative,whichlabelspoliticaladswiththeorganizationthatpurchasedthecontent,alsoallowsuserstoreportinappropriate162.
SoroushVosoughi,DebRoy,SinanAral,"Thespreadoftrueandfalsenewsonline,"Science,09March2018,http://science.
sciencemag.
org/content/359/6380/1146.
163.
ChengchengShaoetal,"Thespreadoflow-credibilitycontentbysocialbots,"NatureCommunications,20November2018,https://www.
nature.
com/articles/s41467-018-06930-7.
164.
KnightFoundation,"Disinformation,'FakeNews'andInfluenceCampaignsonTwitter,"04October2018,https://knightfoundation.
org/reports/disinformation-fake-news-and-influence-campaigns-on-twitter.
165.
OnurVaroletal,"OnlineHuman-BotInteractions:Detection,Estimation,andCharacterization,"IndianaUniversityandUniversityofSouthernCalifornia,27March2017,https://arxiv.
org/pdf/1703.
03107.
pdf.
OnTwitter,thetoponepercentoffakenewsstoriestypicallyreachedbetween1,000and100,000people,whereastruestoriesrarelyexceededanaudienceofonethousand.
32adsandtoinformTwitterwhentheyseeadstheydonotlike.
Twitteralsolauncheda"TransparencyCenter,"whichallowsuserstosearchpoliticaladcampaignsandviewadditionaldetailsincludingadcreative,adcampaignhistory,targeteddemographicsanddollaramountsspentbytheadvertiser.
166InMay2018Twitterannouncedthatitwasusingmachinelearningandartificialintelligencetoidentifytraitsassociatedwithaccountsengagedintrollingorcoordinateddisinformationattacks.
Individuals,forinstance,whosignupformultipleTwitteraccountssimultaneouslywouldbeflagged.
Whilecontentfromtheseaccountsisnotremoved,itisdeprioritizedinsearchandconversationresults.
Twitterfoundinearlytestingthatreportsofabuseinconversationsdroppedeightpercent,andsimilarreportsfromsearchcontentdroppedfourpercent.
167AmonthafterTwitterannounceditsuseoftheseAItools,italsocompletedtheacquisitionofSmyte,atechnologycompanyfocusedoncounteringspam,fraud,andabuseonline.
TwitterincorporatedSmyte'sexistingtoolsintoitsplatform,includingSmyte'slabelingsystemthatautomaticallyidentifiespotentiallymaliciousactivity.
Itdoesthisbasedonanumberofsignalsandaccountrelationships,andthenflagstheactivityforinternalreview.
168InMayandJuneof2018,Twitteralsobeganacceleratingthepacewithwhichitsuspendedsuspiciousaccounts.
Inthosetwomonthsalone,Twittersuspended70millionaccounts,twicetherateofsuspensionscomparedtofall2017.
169ThiswasfarfromatrivialmoveforTwitter.
Removingsuchalargevolumeofaccountsimpacteditsactiveusermetrics,whichplayanimportantroleinthefinancialvaluationofsocialmediacompanies.
Twitter'sJuly2018quarterlyearningsannouncednegativeusergrowthasaresultoftheiraccountpurge,andthecompany'sstockpromptlyfell21%.
170(Thiscleartradeoffbetweensuspendingfakeaccountsandthecompany'svaluationmayhelptoexplainTwitter'sinitialreluctancetomoreproactivelyremovesuspiciousaccounts.
)Nevertheless,Twitter'soverallrecordoncounteringdisinformationappearsmixedatbest.
Inmid-2016,TwitterreportedlyofferedtheKremlin-backedRussiaTelevisionNetworka15percentshareofitsUSelectionadvertisinginexchangeforthreemilliondollars.
171166.
@brucefalck,"NewTransparencyforAdsonTwitter,"TwitterBlog,24October2017,https://blog.
twitter.
com/official/en_us/topics/product/2017/New-Transparency-For-Ads-on-Twitter.
html.
167.
@delbiusand@gasca,"ServingHealthyConversation,"TwitterBlog,15May2018,https://blog.
twitter.
com/official/en_us/topics/product/2018/Serving_Healthy_Conversation.
html.
168.
@twittersafety,"Continuingourcommitmenttohealth,"TwitterBlog,21June2018,https://blog.
twitter.
com/official/en_us/topics/company/2018/CommitmentToHealth.
html.
169.
CraigTimbergandElizabethDwoskin,"Twitterissweepingoutfakeaccountslikeneverbefore,puttingusergrowthatrisk,"WashingtonPost,06July2018,https://www.
washingtonpost.
com/technology/2018/07/06/twitter-is-sweeping-out-fake-accounts-like-never-before-putting-user-growth-risk/utm_term=.
ccd2163a4107.
170.
"Twitterstockplunges21%afterearningsshoweffectsoffake-accountpurge,"Marketwatch,28July2018,https://www.
marketwatch.
com/story/twitter-shares-slide-16-after-fake-account-purge-new-rules-in-europe-2018-07-27.
171.
AlexKantrowitz,"TwitterOfferedRussianTelevisionNetworkRT15%OfItsTotalShareOfUSElectionsAdvertising,"BuzzfeedNews,01November2017,https://www.
buzzfeednews.
com/article/alexkantrowitz/twitter-offered-rt-15-of-its-total-share-of-us-elections.

33Furthermore,theKnightFoundationstudyfoundthatmorethan80percentofthe700,000Twitteraccountslinkedtodisinformationinthe2016electionwerestillactiveinspring2018,andmanywerecontinuingtoproducemillionsoftweets.
Theyalsofoundthatnearly90percentofTwitterdisinformationwastracedbacktothesame50fakenewssites,manyofwhichremainedleadingsourcesofdisinformationonTwitterformanymonthsafterthe2016election.
Conspiracynewssitesreceivedapproximately13percenttheamountofTwitterlinksasacomparativesetofnationalnewssites,indicatingthatthespreadandconsumptionoffakenewswasextensive.
172TheNYUandStanfordresearchersconfirmedthetrend.
WhiletheyfoundthatfakenewswasdecreasingonFacebookafter2016,theiranalysisconcludeditwasrisingonTwitter:Fakenewsaccountedfortwomillionsharespermonthin2015,about4.
5millionsharespermonthattheendof2016,andaboutsixmillionsharespermonthinmid-2018.
173InAugust2018,TwitterCEOJackDorseyexpressedhesitationabouttakingforcefulactionagainstperceivedfakenews.
Inapublicinterview,heraisedconcernsaboutTwitterbecomingthearbiteroftruthbyprojectingthecompany'sbiasesontotheconversationsthatareallowedon,andbannedfrom,itsplatform.
DorseyfurtherhighlightedthelogisticalchallengeofreviewingallcontentonTwitterforsignsoffakenews–notingthelargevolumeofpersonnelresourcesneededforsuchanundertaking–tojustifyreviewingcontentonlyonceithasbeenflaggedbyusers.
174Dorsey'sstatements,togetherwithhiscompany'smixedrecordontacklingknownsourcesofdisinformation,raisesomequestionsabouthowproactiveTwitterwillbeincounteringthisthreatgoingforward.
CounteringdisinformationatGoogleMaliciousactorshavespreaddisinformationonGoogle'splatforminanumberofways.
WhileGoogle-ownedYouTubehashostedfakevideocontent,therehavealsobeendisinformationadsonGoogle'sadplatform,andbadactorshavemanipulatedGoogle'ssearchresultstoprominentlyfeaturefakenewssites.
YouTubeisespeciallypronetodisinformation,andtheplatformhasfacedstrongcriticismforitsalgorithmsthatdeterminetrendingcontentandviewers'personalizedrecommendations.
Withmorethan1.
8billionmonthlyusers,YouTubeisapowerfulconveyorofvideonews,withoutmanyofthetraditionalgatekeepersthatcuratecontentontelevisionnewschannels.
175OneconsequenceisthatYouTube'strendinglistisvulnerabletosensationalizedcontent,wherealgorithmsoptimizeforviralityoverqualityoraccuracy.
Theplatform'salgorithmscanalsoinadvertentlypromotedisinformationthroughpersonalized"upnext"videorecommendations.
172.
KnightFoundation,"Disinformation,'FakeNews'andInfluenceCampaignsonTwitter.
"173.
HuntAllcott,MatthewGentzkow,andChuanYu,"TrendsintheDiffusionofMisinformationonSocialMedia,"StanfordInstituteforEconomicPolicyResearch,October2018,http://web.
stanford.
edu/~gentzkow/research/fake-news-trends.
pdf.
174.
JonFingas,"JackDorseyExplainsWhyTwitterIsReluctanttoFightFakeNews,"Engadget,19August2018,https://www.
engadget.
com/2018/08/19/jack-dorsey-explains-twitter-reluctance-to-fight-fake-news/.
175.
BenGilbert,"YouTubenowhasover1.
8billionuserseverymonth,inspittingdistanceofFacebook's2billion,"BusinessInsider,04May2018,https://www.
businessinsider.
com/youtube-user-statistics-2018-5.
Withmorethan1.
8billionmonthlyusers,YouTubeisapowerfulconveyorofvideonews,withoutmanyofthetraditionalgatekeepersthatcuratecontentontelevisionnewschannels.
34TheserecommendationsarecriticaltoYouTube'ssuccess;thecompanyclaimsthat70percentofitsmorethanonebillionhoursofdailyvideoconsumptionisdrivenbyitsplaylistrecommendations.
176Algorithmsautomaticallygeneratesuggestedcontentintendedtomaximizeuserengagement,whichmeansservinguserscontentthatmimicsorappealstotheirpastbehavior.
IfaviewerconsumesconspiratorialorothermisleadingcontentonYouTube,itsalgorithmswillrecommendsimilarcontent,whichcanreinforcetheviewers'beliefthattheconspiracyis,infact,real.
Whatismoreconcerning,however,isthatviewingquestionablecontenthasnotbeenaprerequisiteforreceivingquestionablerecommendations.
Onedoesnothavetobeaconspiracytheoristtobefedconspiracytheories.
A2018WallStreetJournalinvestigationfoundthatviewingmainstreampoliticalcontentonYouTube,acrossthepoliticalspectrum,ledtocontentsuggestionswithmoreextremeandfringeviewpointsthandisplayedintheoriginalvideo.
177Perhapswiththeseconcernsinmind,Googlehasamplifieditseffortoverthepasttwoyearstocounterdisinformationonitsplatforms.
Followingthe2016election,GooglechangeditsadvertisingpoliciestobanknowndisinformationwebsitesfromusingGoogle's"AdSense"programwhichenablescompaniestomonetizewebsitesthroughadplacements.
178InJanuary2017,Googlebannedagroupof200publishersfromitsAdSensenetworkwhoviolatedthesepolicies.
179Buildingupontheseefforts,GoogleannouncedapartnershipwiththeInternationalFactCheckingNetworkinOctober2017toexpandaglobalnetworkoffact-checkersandprovidefreetoolstosupportitsfact-checkingefforts.
180Thatsamemonth,however,Googlewasfoundtobeunknowinglyservingfakenewsadsonthefact-checkingwebsitesPolitifactandSnopes,whichservedasanawkwardreminderoftheongoingchallengesfacedbyGoogleandhostwebsitesindealingwithdisinformationinadvertisements.
181Googletookstepsin2017toimprovethetransparencyandqualityofitssearchengineresults.
Itintroduceda"fact-check"featurethatwoulddisplaythird-partypublications'fact-checkingassessmentsunderneathGoogle'sresultsforfrequently-searchedpublicclaims.
182However,inJanuary2018,Googleannounceditwastemporarilysuspendingitsfact-checkingfeatureafterconservativemediaoutletsraisedconcernsthatthefeaturewasbeingselectively–andinsomecases,incorrectly–appliedtoconservativemediareporting.
183176.
JackNicas,"HowYouTubeDrivesPeopletotheInternet'sDarkestCorners,"WallStreetJournal,07February2018,https://www.
wsj.
com/articles/how-youtube-drives-viewers-to-the-internetInternets-darkest-corners-1518020478.

177.
Nicas,"HowYouTubeDrivesPeopletotheInternet'sDarkestCorners.
"178.
JackNicas,"GoogletoBarFake-NewsWebsitesFromUsingItsAd-SellingSoftware,"WallStreetJournal,04November2016,https://www.
wsj.
com/articles/google-to-bar-fake-news-websites-from-using-its-ad-selling-software-1479164646.

179.
TessTownsend,"Googlehasbanned200publisherssinceitpassedanewpolicyagainstfakenews,"Recode,25January2017,https://www.
recode.
net/2017/1/25/14375750/google-adsense-advertisers-publishers-fake-news.
180.
EricaAnderson,"BuildingtrustonlinebypartneringwiththeInternationalFactCheckingNetwork,"GoogleBlog,26October2017,https://www.
blog.
google/outreach-initiatives/google-news-initiative/building-trust-online-partnering-international-fact-checking-network/.

181.
DaisukeWakabayashiandLindaQiu,"GoogleServesFakeNewsAdsinanUnlikelyPlace:Fact-CheckingSites,",NewYorkTimes,17October2017,https://www.
nytimes.
com/2017/10/17/technology/google-fake-ads-fact-check.
html.
182.
JustinKosslynandCongYu,"FactChecknowavailableinGoogleSearchandNewsaroundtheworld,"GoogleBlog,07April2017,https://blog.
google/products/search/fact-check-now-available-google-search-and-news-around-world/.

183.
DanielFunke,"Googlesuspendsfact-checkingfeatureoverqualityconcerns,"PoynterInstitute,19January2018,https://www.
poynter.
org/news/google-suspends-fact-checking-feature-over-quality-concerns.
35Forsearchesaboutmediapublications,Googleaddeda"KnowledgePanel"sectiondedicatedtoprovidingadditionalinformationaboutthepublication,includingitsmostcommonly-coveredtopics,aswellaspreviousawardswonforitsreporting.
184Googlealsoannouncedthatnewssitesthathidormisrepresentedtheircountryoforiginwouldnolongerappearinitssearchresults.
185Aspartofa$300million"GoogleNewsInitiative"announcedinMarch2018,Googlecommitted$10milliontoananti-disinformationmedialiteracyeffort,whichwouldproduceeducationalcontentinpartnershipwithYouTubestarstohelpeducatethepubliconhowtoidentifyfakenewsonline.
186GooglealsoannouncedinJuly2018thatYouTubewouldmoreheavilypromote"authoritative"newssourcesinthewakeofbreakingnews.
Italsobegansharingshorttextpreviewssummarizingthesebreakingnewsstoriesinthesearchresults,particularlyduringthetimegapbetweenastoryfirstbreakingandwhentraditionalmediasourcesareabletoreleaseverified,high-qualityvideocontent.
187Thisstrategyisintendedtocurbthespreadofdisinformationandmisinformationclipsthatcanoftengoviralintheimmediateaftermathofabreakingnewsstory.
Furthermore,inresponsetoincreasingpublicscrutinyoveritsrecommendationalgorithms,YouTubeannouncedinearly2019thatitwouldbeginadjustingitsalgorithmstoreduceitsrecommendationsofharmfulorpurposefullymisleadingcontent.
188ThestructuralbarrierstotacklingdisinformationontechplatformsThepastseveralyearshaveshownthattherearetimeswhensocialmediacompanieswillnot–orcannot–adequatelypolicetheirownplatformsagainstallformsofdisinformation.
Eliminatingeverymanifestationofdisinformationisanextraordinarilydifficultchallenge,consideringthesheernumberofbadactors,hugevolumeofdailycontentfromhundredsofmillionsofusers,andquickly-changingtechnologylandscape.
Inmanycases,theseplatforms'mostmeaningfuleffortsagainstdisinformationhaveoccurredinresponsetopublicoutcries.
Yet,theirreactionaryeffortshavenotsucceededinpreventingthesubsequent,moreevolveddisinformationstrategiesbyforeignstates.
Therearealwaysnewgapstobeexploited.
184.
RannaZhou,"LearnmoreaboutpublishersonGoogle,"GoogleBlog,07November2017,https://blog.
google/products/search/learn-more-about-publishers-google/.
185.
JonFingas,"Googlewon'tshownewsfromsitesthathidetheircountryoforigin,"Engadget,16December2017,https://www.
engadget.
com/2017/12/16/google-bans-news-sites-which-hide-country-of-origin/.
186.
ShanWang,"Googleannouncesa$300M'GoogleNewsInitiative',"NiemanLab,20March2018,http://www.
niemanlab.
org/2018/03/google-announces-a-300m-google-news-initiative-though-this-isnt-about-giving-out-grants-directly-to-newsrooms-like-it-does-in-europe/.

187.
BarbaraOrtutay,"YouTubeiscrackingdownon'fakenews'withnewtextpreviews,"AssociatedPress,09July2018,https://www.
usatoday.
com/story/tech/2018/07/09/youtube-cracks-down-fake-news/769861002/.
188.
"ContinuingourworktoimproverecommendationsonYouTube,"Officialblog,YouTube,January25,2019,https://youtube.
googleblog.
com/2019/01/continuing-our-work-to-improve.
html.
Socialmedia'sbusinessmodel,bypromotingclicks,views,andshares,canbeeasilyweaponizedbybadactorstospreaddisinformation–andmanymightarguethatitisoptimizedtodoso.
36Socialmedia'sbusinessmodel,bypromotingclicks,views,andshares,canbeeasilyweaponizedbybadactorstospreaddisinformation–andmanymightarguethatitisoptimizedtodoso.
189ResearchfromNewAmericareiteratesthatthecentralproblemofdisinformationisnotadversarialstatemeddlingorevenjustoneplatform'svulnerabilities,butratherthattheentiresocialmediaindustryhasbeenbuilttoleveragesophisticatedtechnologytoaggregateuserattentionandselladvertising.
190Assuch,strategiestocounterdisinformationmustconsideranumberofdigitalmarketingfeatures,including:behavioraldatatrackingthatenablesprecisiontargeting;onlineadbuyingtoreachandimpactcertainaudiences;searchengineoptimizationthattricksalgorithmsanddominatessearchresults;socialmediamanagementservicesthatpreconfiguremessagesforselectaudiencesacrossmultiplemediachannels;andmarketingAIthatimprovesbehavioraldatatracking,audiencesegmentation,messagetargeting,andcampaignmanagement.
191Furthermore,althoughitistemptingtosimplyimplicatetheplatforms'businessmodelsastheprimaryconveyorofonlinedisinformation,therearealsothesocialandpsychologicalfactorsthatindependentlyandconcurrentlyenableitsspread.
ConsiderthatWhatsApp,Facebook'smessagingplatform,hadyettoimplementaneffectiveadvertisingmodelforits1.
5billionusersthroughtheendof2018,buthasnonethelessgrappledwithsomeofthemostpervasivefakenewscampaignsglobally.
192Onmessagingappslargelydevoidofadvertising,badactorshavebeenabletospreaddisinformationbyleveraginginfluencersandtrustednetworkstowidelyshareinflammatorycontentovermessagegroups.
Effortstocounterdisinformationcanalsohaveunintendedconsequences,suchasFacebook'sexperimentaddingwarninglabelstodisputedcontent-whichactuallyincreasedthespreadofdisinformation.
Othereffortshaveinvitedcritiquesthattheplatformsarestiflingfreespeechorarepoliticallybiased.
Recognizingthatthechallengeposedbydisinformationismassiveandtranscendsanysingletechnicalvulnerability,otherorganizationslikeacademia,non-governmentalorganizations,andtechnologystartupshavesteppedintodevelopsolutionstocountercertainaspectsofonlinedisinformation.
193Theseorganizations,though,facetheproblemofinformationasymmetry.
Theysimplydonothavethesamelevelofaccesstorelevantplatform-specificalgorithmsandrawuserdataasthe189.
SeeAnthonyNadler,MatthewCrain,andJoanDonovan,"WeaponizingtheDigitalInfluenceMachine,"Data&Society,17October2018,https://datasociety.
net/output/weaponizing-the-digital-influence-machine/;DouglasGuilbeault,"DigitalMarketingintheDisinformationAge,"JournalofInternationalAffairs,ColumbiaUniversitySchoolofInternationalandPublicAffairs,17September2018,https://jia.
sipa.
columbia.
edu/digital-marketing-disinformation-age;andDipayanGhoshandBenScott,"Russia'sElectionInterferenceisDigitalMarketing101,"TheAtlantic,19February2018,https://www.
theatlantic.
com/international/archive/2018/02/russia-trump-election-facebook-twitter-advertising/553676/.
190.
DipayanGhoshandBenScott,"#DigitalDeceit-TheTechnologiesBehindPrecisionPropagandaontheInternet,"23January2018,NewAmerica,https://www.
newamerica.
org/public-interest-technology/policy-papers/digitaldeceit/.
191.
GhoshandScott,"#DigitalDeceit.
"192.
ParmyOlson,"Facebook'sRevenueDreamsforWhatsAppAreUnderThreat–FromOld-FashionedSMS,"Forbes,21November2018,https://www.
forbes.
com/sites/parmyolson/2018/11/21/facebooks-revenue-dreams-for-whatsapp-are-under-threatfrom-old-fashioned-sms/#372cb358556e.
193.
See,forexample,CenterforAmericanProgressetal,"RecommendedInternetCompanyCorporatePoliciesandTermsofServicetoReduceHatefulActivities,"2018,https://uploads-ssl.
webflow.
com/5bba6f4828dfc3686095bf6b/5bd0e36186e28d35874f0909_Recommended%20InternetInternet%20Company%20Corporate%20Policies%20%20Terms%20of%20Service_final-10-24.
pdf.

37platformsthemselves.
Forthird-partyorganizationstodeveloptrulyeffectiveideasandsolutionstopiecesoftheonlinedisinformationpuzzle,theyneedtocollaboratewithplatformsandworkjointlytowardssolutions.
TakingresponsibilityWhothenbearsresponsibilityforcounteringdisinformationInsomeways,theanswerappearssimple.
WhenstateslikeRussiaorIranspreaddisinformationonFacebookorTwitter,theyarenotdoingsotoattackFacebookorTwitter.
Theyaredoingittounderminegeopoliticaladversaries,includingtheUnitedStates.
Governments,then,seemtobeartheultimateresponsibilityfordefendingtheirnationsagainstthiskindofdisinformation.
However,thatanswerobscuresamajorcomplication:thebattlegroundrestsfirmlyinprivatehands.
Thiswouldsuggestthatthereisagreaterroleforgovernmentstoplayinengagingwithandregulatingsocialmediacompanies.
Afterall,onlineplatforms,whilewell-resourcedbothfinanciallyandtechnicallytowagethisbattle,donotnecessarilyhaveperfectly-alignedincentiveswithgovernmentswhoareseekingtoguardagainstforeignmeddling.
Noraretheynecessarilycapableofdefendingagainsteveryeffortfromsophisticatedhostileactors.
Ontheotherhand,significantgovernmentinvolvementcarriesitsownrisks,includingthepotentialforimpinginguponfreedomofexpressionandoutrightcensorship.
194However,certaintailoredregulationsmayavoidsuchlimitations.
Forexample,GuillaumeChaslot,aformerYouTubesoftwareengineer,hassuggestedholdingtechnologycompanieslegallyliablefortheiralgorithmicrecommendations,asopposedtoeverypieceofcontenttheyhost.
195Suchanapproachcouldprotectfreedomofexpressionwhilestillholdingsocialmediacompaniesaccountable,andincentivized,topreventtheirplatformsfromrecommendingdisinformation-relatedcontent.
Intheabsenceofcleardelineationsofresponsibility,areasonablenextstepcouldinvolvegreatercollaborationbetweentechnologycompaniesandgovernments.
Aproductivepublic-privaterelationshipwouldenabletransparentinformationsharing,fact-finding,andthedevelopmentanddeploymentoftargetedsolutionsmeanttoquicklycounterforeigndisinformationonline.
Anumberofsuccessfulpublic-privatestructuresalreadyexistthatcanserveasamodelforcounteringdisinformation.
OnesuchmodeltoconsideremulatingistheGlobalInternetForumtoCounterTerrorism(GIFCT),ledbyFacebook,Google,Microsoft,andTwitter.
Formedin2017,theGIFCTisaninitiativeworkingwithgovernments,multilateralorganizationsincludingtheUN,NGOs,andacademiatocurbtheonlinespreadofterroristcontentinamannerthatprotectshumanrights,freedomofexpression,194.
DarrellM.
West,"Howtocombatfakenewsanddisinformation,"BrookingsInstitution,18December2017,https://www.
brookings.
edu/research/how-to-combat-fake-news-and-disinformation/.
195.
KevinRoose,"YouTubeunleashedaconspiracytheoryboom.
Canitbecontained"NewYorkTimes,19February2019,https://www.
nytimes.
com/2019/02/19/technology/youtube-conspiracy-stars.
html.
Intheabsenceofcleardelineationsofresponsibility,areasonablenextstepcouldinvolvegreatercollaborationbetweentechnologycompaniesandgovernments.
38userprivacy,andtheroleofjournalism.
196Asoneexampleoftheircollaborativework,bymid-2018GIFCTmembercompanieshadaddednearly100,000linksofterrorism-relatedcontenttoashareddatabasethatallowsparticipatingcompaniestocollectivelyblockthematerialbeforeitisposted.
197Anotherpotentialmodelcomesfromtheworldoffinancialcrimesenforcement,whereseveralframeworkspromotecooperationbetweengovernmentsandthefinancialsectortobetteridentifyanddisruptmoneylaunderingandterroristfinancing.
Oneprominentexampleistheinter-governmentalFinancialActionTaskForce(FATF),whichencouragesinformationsharingbetweenfinancialinstitutions,lawenforcementauthorities,andgovernments.
TheFATFworkstoidentifycountry-levelvulnerabilities,promoteregulatoryreform,andleveragenewtechnologiestoconfrontmoneylaunderingandterroristfinancingacrossits37memberstates.
198Giventhenumerousactorsthatshapeandareshapedbytheinformationanddigitallandscape,addressingdisinformationwillrequireongoingandopencooperation.
199Thereisnosinglesolutionorsilverbullettoaddressthiscomplexproblem.
However,socialmediaandtechnologycompaniesarewell-placedtoleadtheseefforts,incollaborationwithgovernmentsandotherpartners.
196.
"GlobalInternetForumtoCounterTerrorism,"https://www.
gifct.
org/about/.
197.
DaveLee,"Techfirmshail'progress'onblockingterror,"BBCNews,8June2018,https://www.
bbc.
com/news/technology-44408463.
198.
FATFGuidance–PrivateSectorInformationSharing,http://www.
fatf-gafi.
org/fr/publications/recommandationsgafi/documents/guidance-information-sharing.
htmlhf=10&b=0&s=desc(fatf_releasedate)199.
"Amulti-dimensionalapproachtodisinformation:ReportoftheindependentHighlevelGrouponfakenewsandonlinedisinformation,"EuropeanCommission,12March2018,https://ec.
europa.
eu/digital-single-market/en/news/final-report-high-level-expert-group-fake-news-and-online-disinformation.
39Thechallengeofconfrontingdisinformationwillcontinuetochangeasoldtechnologiesevolveandnewonesdevelop.
Thesechangesmaywidenexistinggapsbetweenthethreatofdisinformationandtheabilitytocounterit.
Tounderstandthesechallenges,thissectionoutlinesthreebroadcategoriesthatimpacteffortsagainstdisinformation.
Firstaretechnologygaps.
Manyobserversclassifythemoderndisinformationenvironmentasanarmsraceinwhichresearchers,technologists,andgovernmentsscrambletodeveloptoolstodetect,counter,andkeeppacewithnefariousactors'methodsandactivities.
Thisenvironmentischaracterizedbyawideavailabilityofsophisticatedtechnologiesthat,untilrecently,wereconcentratedinleadingtechcompaniesorresearchlabs.
Thesecondsetofchallengesisstructural.
Theserelatetotheeconomicincentivesofdevelopingcounter-disinformationtechnology,thedearthofavailabledatasetstotrainmachine-learningtools,andtheslowrateofadoptionofexistingtools.
Thethirdandfinalcategoryofchallengesrelatetothegapinunderstandingexactlyhowtechnologies–suchasAI–areevolving,andwithit,thethreatfromdisinformation.
Whatfollowsisasnapshotofareaswheretechnology,structural,andknowledgegapscurrentlyhinderthedetectionandpreventionofonlinedisinformation.
Itisbynomeansexhaustive,butaimstohighlightseveralimportantcategoriesthatwarrantattentionoverthecomingyears.
GapsintechnologyBotsBotsarebecomingmoresophisticated,thanksinparttotechnologydevelopedtoexploitthelucrativeonlinemarketingandadvertisingmarkets.
200DetectingspambotsonTwitterbasedonsyntax,semantics,ornetworkfeaturesiseffective.
However,detectingnextgenerationpoliticalbotsthatdo200.
SamuelWoolleyandMarinaGorbis,"SocialMediaBotsThreatenDemocracy.
ButWeAreNotHelpless,"TheGuardian,16October2017,https://www.
theguardian.
com/commentisfree/2017/oct/16/bots-social-media-threaten-democracy-technologyCMP=twt_gu.

Knowledgegapsandfuturetechnologychallenges40notjustrepeatorretweetinformationbutactuallybecomeintelligentandcapableofproducingcontentontheirownisachallenge.
Mostoftheworkonpoliticalbotshasfocusedonshortperiodsoftime,inaspecificpoliticalcontext,andonTwitter.
Questionsremainabouttheaveragelifespanofabot,thetransferabilityfromonecountryorpoliticalcontexttoanother,andtheefficacyofbuildingalgorithmstodetectbotsonplatformsotherthanTwitter.
201PhotosandvideosDetectingalteredphotosandvideosatscaleisdifficult,andrapidlyadvancingAIanddeeplearningtechnologyismakingsyntheticmedia(manipulatedorartificially-createdvideoandaudiocontent)easiertoproduce.
Imagesareeasytomanipulatebutmoredifficulttodetectwiththecurrentlyavailableimageanalysisandforensictools.
TheDefenseAdvancedResearchProjectsAgency's"MediaForensics"programisundertakinganefforttodevelopanddeploysuchtoolsonaplatformthatwillautomatetheassessmentofanimage'sintegrity.
Commercialsolutionsarealsocomingtomarket.
OnecompanycalledTruepicisreleasinganimageandvideoforensicstoolinmid-2019thatauthenticatesmediabyscanningforanyabnormalitiesthatwouldindicatemanipulation.
202Furtherworkwillbeneededtostayabreastofadvancementsindoctoringvideos.
203AsAItechnologyprogresses,syntheticvideoandaudiowillappearincreasinglyauthentictothepublicandwillbecomesignificantlyeasiertomanufacture.
Thiswillleadtothemigrationofdisinformationcontentfrombeinglargely"static"(memes,fakearticles)to"dynamic"(videoandaudio).
204Forexample,thevideomappingofoneperson'sfaceontoanother,termeda"deepfake,"isalreadywidelyavailablethroughpublicapps.
Videotovideosynthesistechnologycancreaterealisticlookingartificialvideocontentbasedonasetofinputs.
205Whilereverseimagesearchtoolsexist,robustreversevideosearchtoolsaresimilarlyneededtodetectsyntheticvideocontent.
Reverseimagesearchcanbeusedtoidentifythesourceofanimageonlinebypointingtowhereelseithasappeared,whichhelpspeopleverifytheoriginofanimagequickly.
GoogleReverseImageSearchisonesuchtool.
However,reversevideosearchtoolsarelimitedintheirfunctionality,relyingonthumbnailsordissectedportionsofvideos.
201.
JoshuaA.
Tuckeretal,"SocialMedia,PoliticalPolarization,andPoliticalDisinformation:AReviewoftheScientificLiterature,"William+FloraHewlettFoundation,19March2018,https://hewlett.
org/library/social-media-political-polarization-political-disinformation-review-scientific-literature/.
202.
J.
J.
McCorvey,"Thisimage-authenticationstartupiscombatingfauxsocialmediaaccounts,doctoredphotos,deepfakes,andmore,"FastCompany,19February2019,https://www.
fastcompany.
com/90299000/truepic-most-innovative-companies-2019.
203.
ElizabethGibney,"Thescientistwhospotsfakevideos,"Nature,06October2017,https://www.
nature.
com/news/the-scientist-who-spots-fake-videos-1.
22784.
204.
AlinaPolyakova,"Weaponsoftheweak:RussiaandAI-drivenasymmetricwarfare,"Brookings'ArtificialIntelligenceandEmergingTechnologiesInitiative,15November2018,https://www.
brookings.
edu/research/weapons-of-the-weak-russia-and-ai-driven-asymmetric-warfare/.
205.
Polyakova,"Weaponsoftheweak:RussiaandAI-drivenasymmetricwarfare.
"41Goingforward,asAIalgorithmslearntobetterimitaterealitythroughdeepfakeandsyntheticvideos,itwillbecomeincreasinglydifficultforalgorithmstodetectfakes.
ThegapbetweenAI'sabilitytocreateratherthancountersyntheticmediaisexpectedtowiden.
AGartnerreportpredictsthatby2020,theabilitiesofAItogeneratecounterfeitmediawillsurpassthoseofAItoidentifysuchmedia.
206Relyingonalgorithmsalonetosolvealgorithm-drivendisinformationmaythereforebeinsufficient.
207Fact-checkingFact-checkinginitiativeshaveproliferatedinrecentyears,butautomatedfact-checkingisstillanascentareaofdevelopment.
Thepossibilitiesrangefromaugmentinghumanfact-checkerswithdatabasesofsourcematerialtoannotatingcontentwithfact-checksinnewsarticles.
Ongoinghurdlesincludehowtoteachcomputerstodiscernthepartsofasentencethatshouldbefact-checked,andhowtobuilddatabaseswithenoughcontentto,forexample,checkapolitician'spronouncementsagainsttheirpriorstatements.
Thebiggerchallenge,however,ishowtoreducetheresonanceandreachofcontestedinformation.
Facebook'sfailedexperimentlabelingfalseinformationas"disputed,"whichperverselycausedmoresharingoftheflaggedcontent,suggeststhatmerelylabelingdisputedinformationassuchdoesnotleadtoareductioninitspublicappealororganicspread.
208StructuralchallengesVolumeofcontentIncaseswheresolutionstodisinformationareavailable,existingstructuralchallengescanimpedetheireffectiveness.
Thesheermagnitudeofcontentandplatformsisperhapsoneofthebiggestobstacleshinderingmonitoringanddetection.
Evenifplatformshadthecapabilitytodetectsyntheticimagesoralteredvideosandregulationsforcedthemtouseit,theenormousandsteadilygrowingvolumeofcontentbeinguploadedraisesquestionsabouttheabilityoftheseplatformstoeffectivelymonitorallofit.
209HundredsofhoursofvideoareuploadedtoYouTubeeveryminute.
210Additionally,therapidproliferationofplatformsamplifiesthisproblem.
Forexample,aUKHomeOfficepressreleasenotedthatin2017,ISISusedmore206.
KaseyPanetta,"GartnerTopStrategicPredictionsfor2018andBeyond,"Gartner,Inc.
,03October2017,https://www.
gartner.
com/smarterwithgartner/gartner-top-strategic-predictions-for-2018-and-beyond/.
207.
ChrisMeseroleandAlinaPolyakova"DisinformationWars,"ForeignPolicy,25May2018,https://foreignpolicy.
com/2018/05/25/disinformation-wars/.
208.
JeffSmith,GraceJackson,andSeethaRaj,"DesigningAgainstMisinformation,"MediumBlog,20September2017,https://medium.
com/facebook-design/designing-against-misinformation-e5846b3aa1e2.
209.
"FightingFakeNews:WorkshopReport,"TheInformationSocietyProjectatYaleLawSchoolandtheFloydAbramsInstituteforFreedomofExpression,07March2017,https://law.
yale.
edu/system/files/area/center/isp/documents/fighting_fake_news_-_workshop_report.
pdf.

210.
"HoursofvideouploadedtoYouTubeeveryminuteasofJuly2015,"TheStatisticsPortal,https://www.
statista.
com/statistics/259477/hours-of-video-uploaded-to-youtube-every-minute/.
AGartnerreportpredictsthatby2020,theabilitiesofAItogeneratecounterfeitmediawillsurpassthoseofAItoidentifysuchmedia.
Relyingonalgorithmsalonetosolvealgorithm-drivendisinformationmaythereforebeinsufficient.
42than400uniqueplatformstodistributecontent.
Inthelatterhalfof2017alone,ISISused145newplatformsforthispurpose.
211EncryptionThegrowingadoptionofencryptedandprivatemessagingappsposesvariouschallengesforcounteringdisinformation.
Rumorsanddisinformationthatspreadthroughawidenetworkofprivategroupchatscannotbeeasilydetectedorfact-checkedinrealtime,andcertainlynotbyhumans.
Compoundingmatters,theprivateandencryptednatureoftheseappspreventstheplatformsfrompubliclyflaggingcontentasfalse,widelydisseminatingcorrections,orremovingtheobjectionablecontentfrommessagegroups.
Someexpertssuggest,however,thatthegrowinguseofencryptedmessagingappscouldalsodecreasetheeffectivenessofcertaintypesofdisinformation,asitwillbecomehardertotargetselectaudienceswhoseidentifyingdataisnolongerfreelyavailable.
212VerificationtoolsAlimitednumberofsocialmediaverificationtoolsexist,suchasreverseimagesearch,butadoptionofthesetoolsremainsachallenge.
TheInternationalCenterforJournalistsconductedastudythatfoundthat71percentofjournalistsusesocialmediatofindstoriesbutonly11percentuseanysocialmediaverificationtools.
213Platforms,too,willrequireconvincingtoadoptapproacheslikeauniformstandardforimprintingorwatermarkingvideoswiththeirdigitalorigin.
Asonereportnoted,"Evenifaneffectivedetectionmethodemerges,itwillstruggletohavebroadimpactunlessthemajorcontentdistributionplatforms,includingtraditionalandsocialmedia,adoptitasascreeningorfilteringmechanism.
"214AccesstodataLargedatasetsarecriticaltotrainthemachinelearningtoolsusedincounter-disinformationefforts,butresearcherslackaccesstosuchdata.
Companiesrarelysharetheirdata,andvalidated,empiricaldataonbotsandtrollsisdifficulttofindastheircreatorstypicallyremainanonymous.
Thisisonereasonwhymachinelearningaloneisofteninsufficientandmustbeaugmentedwithhumanreview.
215Thereareavailabledatasetsthatcanbeusedtotraingenerativeadversarialnetworksondetectingsyntheticimagesoffaces,butmoredatasetsareneededforotherareasofforensicinquiry.
216Asone211.
GovernmentoftheUnitedKingdomHomeOfficeandTheRtHonAmberRudd,PressRelease,"Newtechnologyrevealedtohelpfightterroristcontentonline,"13February2018,https://www.
gov.
uk/government/news/new-technology-revealed-to-help-fight-terrorist-content-online.
212.
InterviewwithsubjectmatterexpertSeanMurphy,12January2019.
213.
NicDias,"TheBigQuestion:HowWill'Deepfakes'andEmergingTechnologyTransformDisinformation"NationalEndowmentforDemocracy,01October2018,https://www.
ned.
org/the-big-question-how-will-deepfakes-and-emerging-technology-transform-disinformation/.
214.
"DisinformationonSteroids,"CouncilonForeignRelationsDigitalandCyberspacePolicyProgram,16October2018,https://www.
cfr.
org/report/deep-fake-disinformation-steroids.
215.
V.
S.
Subrahmanianetal,"TheDARPATwitterBotChallenge,"(OnfilewithCornellUniversity),20January2016,https://arxiv.
org/pdf/1601.
05140.
pdf.
216.
AndreasRssleretal,"FaceForensics:ALarge-scaleVideoDatasetforForgeryDetectioninHumanFaces,"(OnfilewithCornellUniversity),24March2018,https://arxiv.
org/abs/1803.
09179.
43organizationnoted,detectioniscurrentlyeasierthanforgerysolongastrainingdatashowsnewtypesoffakedimages,audio,andvideo.
217Thepaceoftechnologicaladvancementsalsolimitshowquicklynewtoolscanbedevelopedtoconfrontthechallengesofdisinformation.
Forexample,newproductsmayrequirespecializedAIchipsthatcanhandlecomplextasksbutrequirenewmaterialsandproductiontechniques.
218Furthermore,mandatoryrequirementsfordisinformationmonitoringthatrequireexpensiveorcomplexsolutionscanraisethebarriersofentrytoyoung,innovativecompaniesandentrenchthepositionofincumbents.
Itisworthnoting,however,thatsomepartieshavedevelopedsolutionswiththeintentofsharingthemwithsmallercompanies–theUKgovernmentandASIDataSciencearedoingsowithtechnologytheydevelopedincollaborationtodetectterroristcontentonline.
219GapsinknowledgeTobetterunderstanddisinformation'simpactonpeople,additionalresearchisneededtounderstandif,how,andwhenexposuretodisinformationinfluencesorchangestherecipient'sbeliefs.
Toseparatesignalfromnoise,itisimportanttodifferentiatebetween1)disinformationthatisinauthenticandthusineffective;2)disinformationthatsimplyappealstopreexistingprejudiceswithoutchangingitstargetaudience'sviews;and3)disinformationthatsuccessfullyaltersviewpoints,discourse,anddecisions.
However,makingthesedistinctionsisnotintendedtodismisstherisksofdisinformation"noise.
"Agreaterunderstandingofwhether,andtowhatextent,seeminglyinauthenticdisinformation"noise"stillunderminesthepublic'strustinonlinemediawillhelpresearchersbettergaugeandanalyzetheriskfromallformsofdisinformation,bothsophisticatedandunsophisticated.
Onthetechnologysideoftheequation,expertscontinuetolackanunderstandingofwhycomplexAIproductsbehaveastheydo.
Neuralnetworksunderpinmanyofthetoolspeopleuseeveryday.
Yettheirinnerworkingsaresometimesbeyondcomprehension.
220WithoutunderstandingtheAIthatadversariesusetocreateanddisseminatedisinformation,methodstocountersuchtechniquesmaybedifficulttogenerate.
ResearchmustalsoconsiderthepotentiallynefarioususesforostensiblyinnocuousAI.
Forexample,Microsoft'sXiaoIcehasbeenapopularchatbotinChinasince2014andisdesignedtofulfillhumanneedsforcommunication,affection,andsocialbelonging.
221AIlikeXiaoIcecouldconceivablybetrainedbyauthoritariangovernmentstocircumventcertainconversationsorpropagatecertainideasthatfallwithintherealmofdisinformation.
217.
SamGregory,"DeepfakesandSynthethicMedia:WhatShouldWeFearWhatCanWeDo"WitnessBlog,July2018,https://blog.
witness.
org/2018/07/deepfakes/.
218.
CadeMetz,"ChipsOfftheOldBlock:ComputersAreTakingDesignCuesFromHumanBrains,"NewYorkTimes,16September2017,https://www.
nytimes.
com/2017/09/16/technology/chips-off-the-old-block-computers-are-taking-design-cues-from-human-brains.
html.

219.
GovernmentoftheUnitedKingdomHomeOfficeandTheRtHonAmberRudd,PressRelease,"Newtechnologyrevealedtohelpfightterroristcontentonline,"13February2018,https://www.
gov.
uk/government/news/new-technology-revealed-to-help-fight-terrorist-content-online.
220.
WillKnight,"TheDarkSecretattheHeartofAI,"MITTechnologyReview,11April2017,https://www.
technologyreview.
com/s/604087/the-dark-secret-at-the-heart-of-ai/.
221.
LiZhouetal,"TheDesignandImplementationofXiaoIce,anEmpatheticSocialChatbot,"(OnfilewithCornellUniversity),21December2018,https://arxiv.
org/abs/1812.
08989.
44Whilenotyetareality,theanticipationofworst-casescenariosliketheseandothersarewhatpromptedresearcherswithOpenAItoscalebacktheirreleaseofamachinelearningsystemthatgeneratedtextbasedonbriefwritingprompts.
Thenarrativesproducedbythefullytrainedmodelwerealarmingintheirabilitytomimichumanwriting,leadingitsdeveloperstoreleaseonlyasmallmodelforresearchpurposes.
222Othereffortstodevelopvirtualdigitalinfluencersthatarecompletelycomputer-generatedalsoposeinterestingquestionsabouthowsuchinfluencersmightbesubvertedtopushmaliciousstoriesandideastotheironlinefollowers.
223OneneedonlyconsidertheexampleofMicrosoft's"Tay"chatbotonTwitterthatwasactiveforonlyadaybeforeitbeganmimickingthevitriolicracismandsexismitencounteredontheplatform.
224LivingwithdisinformationTheuseofdisinformationasatoolforinfluenceandobfuscationwillnevercease.
Similarly,theunderlyingpsychologicalfactorsthatmakehumansvulnerabletodisinformationareenduring.
Whatwillchange,however,arethetechnologiesbywhichdisinformationiscreatedandspread.
Indeed,thisconstantlychangingarsenaloftoolshasledamajorityofsurveyedtechexpertstoconcludethattheproblemwillnotbesolvedwithinthenextdecade.
225Stakeholdersfacethedistinctchallengeofdevelopingpolicysolutionstoprotecttheinformationenvironmentinawaythatdoesnotunderminepublictrust,whilecurbingadisinformationproblemthatwillonlycontinueevolving.
Whatstakeholdersshouldaimfor,then,arestrategiestomitigatedisinformationanditspotentiallydisastrousconsequenceswhilemaintainingarobustcommitmenttocivilliberties,freedomofexpression,andprivacy.
Theproposedpublic-privatepartnershipmodelincludedinthisreportisonepossibleapproachforharnessingdiverseexpertisetosolvethisproblem.
Aligningindustryandtechnicalexpertswiththelawmakerswhoshapepublicpolicywillhelpproduceaninformedandmeasuredresponsetoacomplex,rapidlytransformingthreat.
Itistobeexpectedthatcompetinginterestsandincentiveswillhindercoordination,butacollaborativepublic-privateframeworkisaprudentfoundationonwhichtobuildconsensusandcoordinateaction.
222.
"BetterLanguageModelsandTheirImplications,"OpenAI,February14,2019,https://blog.
openai.
com/better-language-models/#sample2.
223.
JuliaAlexander,"Virtualcreatorsaren'tAI-butAIiscomingforthem,"TheVerge,30January2019,https://www.
theverge.
com/2019/1/30/18200509/ai-virtual-creators-lil-miquela-instagram-artificial-intelligence.
224.
IngridAngulo,"FacebookandYouTubeshouldhavelearnedfromMicrosoft'sracistchatbot,"CNBC,17March2018,https://www.
cnbc.
com/2018/03/17/facebook-and-youtube-should-learn-from-microsoft-tay-racist-chatbot.
html.

225JannaAndersonandLeeRainee,"TheFutureofTruthandMisinformationOnline,"PewResearchCenter,19October2017,http://www.
pewinternetInternet.
org/2017/10/19/the-future-of-truth-and-misinformation-online/.