mailings贝尔金路由器设置

GroupLens:AnOpenArchitectureforCollaborativeFilteringofNetnewsPaulResnick*,NeophytosIacovou**,MiteshSuchak*,PeterBergstrom**,JohnRiedl***MITCenterforCoordinationScienceRoomE53-32550MemorialDriveCambridge,MA02139617-253-8694Email:presnick@mit.
edu**UniversityofMinnesotaDepartmentofComputerScienceMinneapolis,Minnesota55455(612)624-7372Email:riedl@cs.
umn.
eduFromProceedingsofACM1994ConferenceonComputerSupportedCooperativeWork,ChapelHill,NC:Pages175-186Copyright1994,AssociationforComputingMachineryABSTRACTCollaborativefiltershelppeoplemakechoicesbasedontheopinionsofotherpeople.
GroupLensisasystemforcollaborativefilteringofnetnews,tohelppeoplefindarticlestheywilllikeinthehugestreamofavailablearticles.
Newsreaderclientsdisplaypredictedscoresandmakeiteasyforuserstoratearticlesaftertheyreadthem.
Ratingservers,calledBetterBitBureaus,gatheranddisseminatetheratings.
Theratingserverspredictscoresbasedontheheuristicthatpeoplewhoagreedinthepastwillprobablyagreeagain.
Userscanprotecttheirprivacybyenteringratingsunderpseudonyms,withoutreducingtheeffectivenessofthescoreprediction.
Theentirearchitectureisopen:alternativesoftwarefornewsclientsandBetterBitBureauscanbedevelopedindependentlyandcaninteroperatewiththecomponentswehavedeveloped.
KEYWORDS:Collaborativefiltering,informationfiltering,electronicbulletinboards,socialfiltering,Usenet,netnews,usermodel,selectivedisseminationofinformation.
INTRODUCTIONComputernetworksallowtheformationofinterestgroupsthatcrossgeographicalbarriers.
Bulletinboardshavebeenanimportantmechanismforthat.
Ratherthanaddressinganarticledirectlytoaknownsetofpeople,thewriterpostsitinanewsgroup,apublicplaceavailabletoanyoneinterestedinthetopic.
TheUsenetnetnewssystemcreatestheillusionofasinglebulletinboardavailableanywhereintheworld.
Itpropagatesarticlessothat,withsomedelays,anarticlepostedfromanywhereintheworldisavailabletoeveryoneelse.
Permissiontocopywithoutfeeallorpartofthismaterialisgrantedprovidedthatthecopiesarenotmadeordistributedforcommercialadvantage,theACMcopyrightnoticeandthetitleofthepublicationanditsdateappear,andnoticeisgiventhatcopyingisbypermissionoftheAssociationforComputingMachinery.
Tocopyotherwise,ortorepublish,requiresafeeand/orspecificpermission.
Recentcountsindicatethattherearemorethan8000newsgroups,withanaveragetrafficofmorethan100MBperday[1].
Thenewsgroupscarryannouncements,questions,anddiscussions.
Inadiscussion,oftencalledathread,onearticleinducesrepliesfromseveralothers,eachofwhichmayalsoinducereplies.
TheJanuary24,1994estimatesofnetnewsparticipationindicatethatmorethan140,000peoplepostedarticlesintheprevioustwoweeks.
Therearemanymore"lurkers"whoreadbutdonotpostarticles.
Clearly,alotofpeoplearegettingvaluefromthesebulletinboards.
Infact,netnews'rapidbroadcastnatureandwidespreadreadershiphasreshapedthewaythecomputingcommunityworks.
Systemadministratorsdependonnetnewstokeepintouchwiththelatestdevelopmentwork,thelatestsecurityholes,andthelatestbugfixes.
Researchersdependonnetnewsasawayofkeepingup-to-dateonnewresearchdirectionsandimportantresultsinbetweenconferences.
Manyothersusenetnewsjusttokeepintouchwithotherpeoplearoundtheworld,tolearnaboutnewbooks,newrecipes,newmusic,andwhatlifeinothercitiesislike.
Overtheyearsnetnewshasbecomeaprincipalmediumforsharingamongcomputerusers.
Evenso,theexperienceofusingnetnewsisnotcompletelysatisfying.
Almosteveryonecomplainsthatthesignaltonoiseratioistoolow.
Writerscannoteasilytellwhethertheircommentsarevalued,exceptbythevocalfewwhopostresponses.
Someseemnottocareaboutreaderinterest,onlyabouttheirownrighttowrite.
Moreover,tastesdiffer,sothatnoonearticlewillappealtoallthereadersofanewsgroup.
Eachreaderendsupsiftingthroughmanynewsarticlestofindafewvaluableones.
Often,readersfindtheprocesstoofrustratingandstopreadingnetnewsaltogether.
Netnewsprovidestwomechanismsthathelpreaderslimittheirattentiontoarticleslikelytointerestthem.
First,thedivisionofthebulletinboardintonewsgroupsallowsreaderstofocusonafewtopics.
Whenthenumberofpostingsinanewsgroupgetstoolarge,itisoftensplitintotwoormorenewsgroupswithidentifiablesubtopics.
Second,somenewsgroupsaremoderated.
Attemptedpostingstothesenewsgroupsareautomaticallyforwardedtothemoderator,whodecideswhetherornottheybelonginthenewsgroup.
Usenetpropagatesonlythosearticlesthatreceivethemoderator'sstampofapproval.
Inaddition,softwarepackagesforreadingnetnews(hereafterreferredtoasnewsclients)provideothermechanismsthateasereaders'burdens.
First,mostnewsclientsdisplayasummaryoftheauthorandsubjectlineforeachmessageinanewsgroup.
Theuserthenindicateswhicharticlesshewouldliketoread.
Second,mostnewsclientsdisplayallofthearticlesinaparticulardiscussionthreadtogether.
Someinitiallyshowonlythefirstarticleineachthread,allowinguserstoquicklyperusethecurrentdiscussiontopics.
Third,somenewsclientsprovide"killfiles.
"Akillfileidentifiestextstringsthatarenotinterestingtoaparticularuser.
Ifauserputsthesubjectlineofanarticleintothekillfile,nofurtherarticlesonthatsubjectwillbedisplayed.
Ifauserputstheauthor'snameintoakillfile,nofurtherarticlesfromthatauthorwillbedisplayed.
Finally,somenewsreadersprovidestringsearchfacilities.
Iftheuserisparticularlyinterestedinarticlesthatmention"collaborativefiltering,"thenewsclientcanfindthem.
GroupLensprovidesanewmechanismtohelpfocusattentiononinterestingarticles.
Itdrawsonadeceptivelysimpleidea:peoplewhoagreedintheirsubjectiveevaluationofpastarticlesarelikelytoagreeagaininthefuture.
Afterreadingarticles,usersassignthemnumericratings.
GroupLensusestheratingsintwoways.
First,itcorrelatestheratingsinordertodeterminewhichusers'ratingsaremostsimilartoeachother.
Second,itpredictshowwelluserswilllikenewarticles,basedonratingsfromsimilarusers.
TheheartofGroupLensisanopenarchitecturethatincludesnewsclientsforentryofratingsanddisplayofpredictions,andratingserversfordistributionofratingsanddeliveryofpredictions.
RelatedWorkThegeneralproblemsofinformationoverloadandlowsignaltonoiseratiohavereceivedconsiderableattentionintheresearchliterature.
Weusetheterminformationfilteringgenericallytoreferbothtofindingdesiredinformation(filteringin)andeliminatingthatwhichisundesirable(filteringout),butrelatedworkalsoappearsunderthelabelsofinformationretrievalandselectivedisseminationofinformation[2].
Inaddition,researchonagents[12,13],usermodeling[1,9],knowbots[8],andmediators[21]hasexploredsemi-autonomouscomputerprogramsthatperforminformationfilteringonbehalfofauser.
Maloneetal.
[13]describethreecategoriesoffilteringtechniques,cognitive,social,andeconomic,basedontheinformationsourcesthetechniquesdrawoninordertopredictauser'sreactiontoanarticle.
Thethreecategoriesprovideausefulroadmaptotheliterature.
Cognitive,orcontent-basedfilteringtechniquesselectdocumentsbasedonthetextinthem.
Forexample,thekillfilesandstringsearchfeaturesprovidedbynewsclientsperformcontentfiltering.
Eventhedivisionofnetnewsintonewsgroupsisaprimitiveexample,sinceareaderrestrictshisattentiontothosearticleswithaparticulartextstringintheir"newsgroup:"field.
Othercontent-basedfilteringtechniquescouldpotentiallybeusedaswell.
Theprofileofwhichtextstoincludeorkillcouldbemorecomplexthanacollectionofcharacterstrings.
Forexample,stringscouldbecombinedwiththeBooleanoperatorsAND,OR,andNOT.
Alternatively,theprofilecouldconsistofweightvectors,withtheweightsexpressingtherelativeimportanceofeachofasetofterms[4,5,16].
Somecontentfilteringtechniquesupdatetheprofilesautomaticallybasedonfeedbackaboutwhethertheuserlikesthearticlesthatthecurrentprofileselects.
Informationretrievalresearchreferstothisprocessasrelevancefeedback[17].
ThetechniquesforupdatingcandrawonBayesianprobability[2],geneticalgorithms[18],orothermachinelearningtechniques.
Socialfilteringtechniquesselectarticlesbasedonrelationshipsbetweenpeopleandontheirsubjectivejudgments.
Placinganauthor'snameinakillfileisacrudeexample.
Moresophisticatedtechniquesmightalsofilteroutarticlesfrompeoplewhopreviouslyco-authoredpaperswiththeobjectionableperson.
Collaborativefiltering,basedonthesubjectiveevaluationsofotherreaders,isanevenmorepromisingformofsocialfiltering.
Humanreadersdonotsharecomputers'difficultieswithsynonymy,polysemy,andcontextwhenjudgingtherelevanceoftext.
Moreover,peoplecanjudgetextsonotherdimensionssuchasquality,authoritativeness,orrespectfulness.
Amoderatednewsgroupemploysaprimitiveformofcollaborativefiltering,choosingarticlesforallpotentialreadersbasedonevaluationsbyasingleperson,themoderator.
TheTapestrysystem[6]makesmoresophisticateduseofsubjectiveevaluations.
Thoughitwasnotdesignedtoworkspecificallywithnetnews,itallowsfilteringofallincominginformationstreams,includingnetnews.
Manypeoplecanpostevaluations,notjustasinglemoderator,andreaderscanchoosewhichevaluatorstopayattentionto.
Theevaluationscancontaintext,notjustbinaryaccept/rejectrecommendations.
Moreover,filterscancombinecontent-basedcriteriaandsubjectiveevaluations.
Forexample,areadercouldrequestarticlescontainingtheword"CSCW"thatJoehasevaluatedandwheretheevaluationcontainstheword,"excellent".
OurworkissimilarinspirittoTapestrybutextendsitintwoways.
First,Tapestryisamonolithicsystemdesignedtoshareevaluationswithinasinglesite.
Weshareratingsbetweensitesandourarchitectureisopentothecreationofnewnewsclientsandratingserversthatwouldusetheevaluationsindifferentways.
Second,Tapestrydoesnotincludeanyaggregatequeries.
Theratingserverswehaveimplementedaggregateratingsfromseveralevaluators,basedoncorrelationoftheirpastratings.
Areaderneednotknowinadvancewhoseevaluationstouseandinfactneednotevenknowwhoseevaluationsareactuallyused.
InGroupLens,ratingsenteredunderapseudonymarejustasusefulasthosethataresigned.
Maltzhasdevelopedasystemthataggregatesallratingsofeachnetnewsarticle,determiningasinglescoreforeach[14].
Bycontrast,GroupLenscustomizesscorepredictiontoeachuser,thusaccommodatingdifferinginterestsandtastes.
Inreturnforitsreducedfunctionality,Maltz'sschemescalesbetterthanours,becauseratingserverscanexchangesummariesofseveralusers'ratingsofanarticle,ratherthanindividualratings.
Thesubjectiveevaluationsusedincollaborativefilteringmaybeimplicitratherthanexplicit.
ReadWearandEditWear[7]guideusersbasedonotherusers'interactionswithanartifact.
TheGroupLensnewsclientsmonitorhowlongusersspendreadingeacharticlebutourratingserversdonotyetusethatinformationwhenpredictingscores.
Economicfilteringtechniquesselectarticlesbasedonthecostsandbenefitsofproducingandreadingthem.
Forexample,Malonearguesthatmassmailingshavealowproductioncostperaddresseeandshouldthereforebegivenlowerpriority.
Applyingthisideatonetnews,anewsclientmightfilteroutarticlesthathadbeencross-postedtoseveralnewsgroups.
Moreradicalschemescouldprovidepayments(inrealmoneyorreputationpoints)toreaderstoconsiderarticlesandpaymentstoproducersbasedonhowmuchthereaderslikedthearticles.
Stodolskyhasproposedaschemethatcombinessocialandeconomicfilteringtechniques[19].
Heproposeson-linepublicationswherethepublicationdecisionultimatelyrestswiththeauthor.
Duringapreliminarypublicationperiod,otherreadersmaypostratingsofthearticle.
Theauthormaythenwithdrawthearticle,toavoidthecosttohisreputationofpublishinganarticlethatisdisliked.
OutlineTheGROUPLENSsectionofthepaperdescribestheGroupLensarchitectureanditsevolution.
TheONGOINGEXPERIMENTATIONsectiondescribesalargerscaletestofthearchitecturethatisinpreparation.
TheSOCIALIMPLICATIONSsectionaddressessocialchangesintheuseofNetnewsthatmaybeprecipitatedbyGroupLens.
GROUPLENSGroupLensisadistributedsystemforgathering,disseminating,andusingratingsfromsomeuserstopredictotherusers'interestinarticles.
ItincludesnewsreadingclientsforbothMacintoshandUnixcomputers,aswellas"BetterBitBureaus,"serversthatgatherratingsandmakepredictions.
Boththeoverallarchitectureandparticularcomponentshaveevolvedthroughiterativedesignandpilottestingtomeetthefollowinggoals:Openness:Therearecurrentlydozensofnewsclientsincommonuse,eachwithastrongfollowingamongitsusercommunity.
AnyoralloftheseclientscanbeadaptedtoparticipateinGroupLens.
GroupLensalsoallowsforthecreationofalternativeBetterBitBureausthatuseratingsindifferentwaystopredictuserinterestinnewsarticles.
EaseofUse:Ratingsareeasytoformandcommunicate,andpredictionsareeasytorecognizeandinterpret.
Thisminimizestheadditionalburdenthatcollaborativefilteringplacesonusers.
Compatibility:Thearchitectureiscompatiblewithexistingnewsmechanisms.
Compatibilityreducesuseroverheadintakingadvantageofthenewtool,andsimplifiesitsintroductionintonetnews.
Scalability:Asthenumberofusersgrows,thequalityofpredictionsshouldimproveandthespeednotdeteriorate.
Onepotentiallimittogrowthwillbetransportandstorageoftheratings,ifGroupLensgrowsverylarge.
Privacy:Someuserswouldprefernottohaveothersknowwhatkindsofarticlestheyreadandwhatkindstheylike.
TheBetterBitBureausinGroupLenscanmakeeffectiveuseofratingseveniftheyareprovidedunderapseudonym.
OverviewUsenetconsistsofInternetsitesaswellasUUCPsites.
Typicallyasitewilldeclareamachinetoactasitsnewsserver.
Usersateachsiteinvokenewsclientsontheircomputersandconnecttothenewsserverinordertoretrievenewsarticles.
Userscanalsowritenewarticlesandpostthemtothenewsserverthroughtheirnewsclients.
Whenauserpostsanarticle,ittravelsfromthenewsclientwherethearticleiscomposedtothelocalnewsserverandfromtheretonewsserversatnearbysites.
Afterleavingtheoriginatingsite,anarticlepropagatesthroughoutUsenet,hoppingfromsitetosite.
Sincethereisnocentralizedcoordinationofthedistributionprocess,anarticlemayarriveatasiteviamorethanoneroute.
Becausearticleshavegloballyuniqueidentifiers,however,andareneveralteredoncetheyareposted,anysitecanrecognizeaduplicatecopyofanarticleandavoidpassingiton.
LotusNotesusesasimilardistributionprocess[10].
ThenetnewsarchitectureissummarizedinFigure1.
GroupLensaddsonenewtypeofentitytothenetnewsarchitecture,BetterBitBureaus,asshowninFigure2.
TheBetterBitBureausprovidescoresthatpredicthowmuchtheuserwilllikearticles,andgatherratingsfromnewsclientsaftertheuserreadsthearticles.
TheBetterBitBureausalsousespecialnewsgroupstoshareratingswitheachother,toallowcollaborativefilteringamongusersatdifferentsites.
Theremainderofthissectiontracestheprocessesofratingcreation,distribution,anduseanddescribeshowtheymeetFigure1:Thenetnewsarchitecture.
Newsarticleshopfromnewsservertonewsserver.
Anewsclientconnectstothenewsserveratitssiteandpresentsarticlestousers.
Figure2:TheGroupLensarchitecture.
BetterBitBureauscollectratingsfromclients,communicatethembywayofnewsservers,andusethemtogeneratenumericscorepredictionsthattheysendtoclients.
Clientsconnecttoalocalnewsserver,andcanconnecttoaBetterBitBureauthatusesthesameoradifferentnewsserver.
thedesigngoalsofopenness,easeofuse,compatibility,scalability,andprivacy.
EnteringRatingsInGroupLens,aratingisanumberfrom1to5,optionallysupplementedbythenumberofsecondswhichtheuserspentreadingthearticle.
Usersareencouragedtoassignratingsbasedonhowmuchtheylikedthearticle,with5highestand1lowest.
Theuserchoosesapseudonymtoassociatewithherratingsthatmaybedifferentfromthenamesheusesforpostingnewsarticles.
Thispreservestheabilitytodetectthattworatingscamefromthesameperson,whilepreventingdetectionofexactlywhothatpersonis.
TheGroupLenschoiceoftheformandmeaningofratingsisonlyonepossibilityinarichdesignspace.
Therearemanypossibledimensionsalongwhichtoratearticles:interestinsubject,qualityofwriting,authoritativenessoftheauthor,etc.
Ratherthanasinglecompositerating,separateratingsonseveraldimensionscouldbesolicitedfromreaders.
Freetextratingscouldbeenteredratherthannumbers.
Readerscouldbeaskedtopredicthowwelltheythinkotherreaderswilllikeanarticleratherthanreporthowmuchtheythemselveslikedit.
Ratingscouldberestrictedonlytopositive,oronlytonegativeevaluations.
Thedegreeofprivacycouldalsobevaried,fromcompletelyanonymoustoauthenticatedsignatures.
Infact,anearlierimplementationofaMacintoshnewsclient[20]employedratingswithquiteadifferentformthanthecurrentGroupLensarchitecture.
Usersenteredonlyendorsements,positiveratings,ontheassumptionthatsincethesignaltonoiseratioinnetnewsissolowitisonlyimportanttopointoutthegoodarticles.
Readersendorsedarticlesthattheythoughtothersinaknownsmallgroupwouldlike.
Finally,readerssignedendorsementswiththeirrealnames,allowingotherpeopletoselectallthearticlesendorsedbyaparticularfriend.
ApilottestofthatearlierendorsementmechanismataSchlumbergerresearchlabindicatedthatagroupofsevenpeoplemaynotbelargeenoughtogetthefullavailablebenefitofcollaborativefiltering.
Aswecontemplatedamuchlargergroupsize,webelievedthatsomeuserswouldbelesswillingtosigntheirratingsandthatitwouldbecomeincreasinglydifficultforuserstoknowwhatarticlesothersinthegroupwouldlike.
Thepilottestalsoreinforcedtheimportanceofmakingitaseasyaspossibletoenterendorsements.
Tomakeanendorsement,auserhadtoselectfromapull-downmenu,waitforawindowtoopenup,optionallyentertextinthewindow,andthencloseit.
Whilethewholeprocesstookonlyamatterofsecondsiftheuserenterednotext,itwasstillsignificantlylongerthanitnormallytakestogoontothenextarticle.
WehavetakencareintheGroupLenssystemtomakeentryofratingsaseasyaspossible.
Wehavemodifiedthreenewsclients,EmacsGnusandNNforUNIXmachinesandNewsWatcherforMacintoshes.
Ineachcase,entryofaFigure3.
ReadinganarticlewiththemodifiedNewsWatcherclient.
Theusercanclickononeofthefiveratingsbuttonswiththemouse,ortypeanumberfrom1to5onthekeyboard.
ratingfitsintotheoverallparadigmofthenewsclient.
Forexample,inthemodifiedNewsWatcher,thenumbers1to5appearasselectablebuttonsanytimeauserreadsanarticle(Figure3),andtheusercanalsotypeanumberasakeyboardshortcutforthosebuttons.
InGnus,nobuttonsaredisplayed,butreadersstilltypetheratingsdirectly.
WithNN,readersfirsttypetheletter'v'(toenterinto"ratingmode")andthentherating.
TheGroupLensarchitecturerequiresonlythatratingsbereportedona1to5scale,notthattheybedisplayedbynewsclientsonthatscale.
Tomaketheratingscaleeasyforstudentstounderstand,theNNandGnusclientsacceptlettergradesratherthannumbers.
WhenreportingtheratingstotheBetterBitBureau,theytranslate'a'to5,'b'to4andsoon.
Othernewsclientscouldallowmoregradationsofratings(e.
g.
,1to100)andreportthemasfractionsbetween1and5.
DistributingRatingsGroupLensdoesnotinterferewiththeUsenetpropagationschemeatall.
Onthecontrary,itreliesuponitheavily.
TheBetterBitBureaupackagesoneormoreratingsintoanewsarticle,followingtheformatinFigure4,andpostsittoanewsserver.
ThisallowsGroupLenstotakeadvantageoftheUsenetpropagationscheme.
OvertheyearsUsenethasdemonstrateditsabilitytopropagatearticlestoeveryotherUsenetsite,evenasthenumberofnewsservershasgrowndramatically.
Ratingserverscouldexchangeratingsdirectly,throughinternetorUUCPlinks,buttheywouldhavetoreimplementmanyofthepropagationfeaturesalreadyfoundinUsenet.
Themessageformatwehavedefinedallowsseveralratingstobebatchedinasinglearticle.
Eachratingisjustonelineoftext,whileeachUsenetnetnewsarticlerequiresseverallinesofheaders.
Thus,packagingseveralratingsinonearticlecansaveaconsiderableamountofoverhead.
OurBetterBitBureaus(BBBs)batchatthesessionlevel(i.
e.
,allratingsenteredbyauserduringareadingsessiongointooneratingsarticle).
Otherbatchingpolicies,suchasallratingsfromasiteoverthelasthour,couldbeimplemented.
Ratingsarepostedinnewsgroupsdedicatedsolelytoratingsarticles.
Onenaturalconfigurationistosetupaparallel"ratingstransport"newsgroupforeach"normal"Usenetgroup.
Onedeficiencyofthisapproachisthatifaratingarticlecontainsseveralratings,itmayhavetobecross-postedtomanyratingsnewsgroups.
Anotherdeficiencyisthatitrequiresnewsserverstocarryalargenumberofnewnewsgroupsdevotedsolelytoratings,whichmayincreaseadministrativeoverhead.
Currently,ourBBBspostallratingsinasinglenewsgroup.
TofacilitatetheinitialspreadofGroupLens,userscanparticipateeveniftheirlocalnewsserversdonotcarrytheratingsnewsgroupandeveniftheirlocalsiteadministratorshavenotsetupBetterBitBureaus.
TheGroupLensarchitecturepermitsthisbyallowinguserstoconnecttoaremoteBBB.
TheleftsideofFigure2illustratesalocalBBBthatpostsratingsarticlestothesamenewsserverthattheclientsconnectto.
TherightsideofFigure2illustratesaclientconnectingtoaremoteBBBthatpropagatesratingsarticlesthroughadifferentnewsserver.
PredictingScoresTheBetterBitBureaus(BBBs)predicthowmuchreaderswilllikearticles.
Whilecontentfilterswouldmakepredictionsbasedonthepresenceorabsenceofwordsinthearticles,theBBBsinGroupLensusetheopinionsofotherpeoplewhohavealreadyratedthearticles.
Ifnoonehasreadanarticle,theBBBsareunabletomakepredictionsaboutit.
Whenratingsforanarticleareavailable,theyareunlikelytobeuniform,duetodifferencesofopinionandgoalsamongtheraters.
ABBBcombinesthedifferentratingstoproduceapredictedscore.
Moreover,additionalreadersarelikelytohavedifferentopinionsaboutthearticle.
ABBBthusmightusethesameratingstopredictdifferentscoresfordifferentreaders,bychangingtherelativeweightgiventotheratings.
Whenpredictionsareonthesamescaleasratings,predictioncanbemodeledasmatrixfilling,wherethecolumnsarepeople,therowsarearticles,andthecellscontaintheratingsthatpeoplehaveposted,asshowninFigure5.
Manyofthecellsofthematrixareempty,becausereadershavenotyetexaminedthosearticlesorhaveelectednottoratethem.
ABBBpredictsscoresformissingcellsbeforethereadersexaminethecorrespondingarticles.
From:MITGroupLensBetterBitBureauSubject:Ratings;pleaseignoreMessage-ID:Groups_Rated:news.
adin.
policy,news.
groupsRaters:[Pseudo1][Pseudo1]112news.
adin.
policy[Pseudo1]27news.
groupsFigure4:Asampleratingsarticle.
Eachlineinthebodyofthearticlecontainsaratingofonearticlebyoneperson.
Thefivefieldsoneachlinearetheidofthearticle,thepseudonymoftherater,arating,thenumberofsecondsthereaderspentexaminingthearticlebeforeratingit,andthenewsgroupsthearticleisin.
Thetimecountisoptional.
Additionalkeywordidentifiedfieldscanalsobeincludedattheendofline.
Figure5:asamplematrixofratings.
Allthescoringmethodswehaveimplementedarebasedontheheuristicthatpeoplewhoagreedinthepastarelikelytoagreeagain,atleastonarticlesinthesamenewsgroup.
Thisheuristicwillmisleadonoccasion,butpreferencesformostkindsofarticlesarelikelytobefairlystableovertime.
Toimplementthisheuristic,ourBBBsfirstcorrelateratingsonpreviousarticlestodetermineweightstoassigntoeachoftheotherpeoplewhenmakingpredictionsforoneofthem.
Then,theyusetheweightstocombinetheratingsthatareavailableforthecurrentarticle.
Wehaveinvestigatedseveraltechniquesforcorrelatingpastbehaviorandusingtheresultantweights,basedonreinforcementlearning[12],multivariateregression,andpairwisecorrelationcoefficientsthatminimizelinearerrororsquarederror.
WeillustrateoneofthecorrelationandpredictiontechniquesbycomputingKen'spredictedscoreonarticle6,thelastrowofthematrix.
First,wecomputecorrelationcoefficients[15],weightsbetween-1and1thatindicatehowmuchKentendedtoagreewitheachoftheothersonthosearticlesthattheybothrated.
Forexample,Ken'scorrelationcoefficientwithLeeiscomputedas:Intheformulaabove,istheaverageofKen'sratings.
AllthesummationsandaveragesintheformulaarecomputedonlyoverthosearticlesthatKenandLeebothrated.
Wehaveconvenientlyarrangedforandtobe3inthisexample,butthatneednotbetrueinpractice.
Similarly,Ken'scorrelationcoefficientwithMegis+1andwithNanis0.
Thatis,KentendstodisagreewithLee()andagreewithMeg().
HisratingsarenotcorrelatedwithNan's.
TopredictKen'sscoreonthelastarticleinthematrix,takeaweightedaverageofalltheratingsonarticle6accordingtothefollowingformula:ThisisareasonablepredictionforKen,sincethearticlereceivedahighratingfromsomeonewhoagreedwithhiminthepastandalowratingfromsomeonewhodisagreed.
CarryingthroughsimilarcalculationsforNanyieldsalowerpredictionof3.
75.
SinceNanhadpartialagreementwithLeeinthepast,Lee'slowratingforthearticlepartiallycancelsoutthehighratingsthatMeggaveit.
Thescorepredictionsystemisrobustwithrespecttocertaindifferencesofinterpretationoftheratingscale.
Iftwousersareperfectlycorrelated,butoneusergivesonlyscoresbetween3and5andtheotheronlyscoresbetween1and3,a5scorefromthefirstuserwillresultinapredictionof3forthesecond.
Iftwouserswouldbeperfectlycorrelated,butthefirstmistakenlythinks1isagoodscoreand5isbad,thetwowillbenegativelycorrelatedanda1scorefromthefirstwillresultinapredictionof5forthesecond.
Thisleadstoaclearexplanationtotheuserofhowtoassignratings:assigntheratingyouwishGroupLenshadpredictedforthisarticle.
Allen'sstudyoffivesubjects'preferencesfornewswirearticles[1]foundverysmallcorrelationsbetweensubjects,thuscallingintoquestionourbasicassumptionthatpeoplewhoagreedinthepastarelikelytoagreeagain.
Itmaybe,however,thatalargersampleofsubjectswouldhaveyieldedsomepairswithlargeroverlapsintheirratings.
Moreimportantly,itmaybethatpairsofpeoplewillshareinterestsinsometopicsbutnotothers.
Twopeoplemayagreeintheirevaluationsoftechnicalarticles,butnotjokes.
OurBBBskeepseparateratingmatricesforeachnewsgroup.
OnehopesthattheaccuracyofthepredictionsimproveastheBBBhasmorepastratingstouseincomputingcorrelations.
FourpeopleattheUniversityofMinnesotaparticipatedinapilottestofanearlierversion,usingaslightlydifferentscoringfunction.
Whileallfourparticipantsreportedthatthepredictedscoreseventuallymatchedtheirinterestsfairlyclosely,theydidobservethattherewasastart-upintervalbeforethepredictionswereveryuseful.
Furtherexperimentsandanalysisarenecessarytodeterminejusthowlongthestart-upintervalislikelytobeforeachnewuser.
Itseemslikelythatbetterscoringmechanismscanbedeveloped.
Inadditiontobettermatrixfillingtechniques,itmaybehelpfultousebothothers'ratingsandthecontentsofarticlesinmakingpredictions.
Itmayalsobehelpfultotakeintoaccountthetimepeoplespentreadingarticlesbeforeratingthem,informationcollectedbutnotusedbyourBBBs.
Fortunately,theGroupLensarchitectureisopen:anyonecanimplementanalternativeBBBsolongasitpostsratingsarticlesintheformatdescribedaboveandcommunicateswithclientsthesamewaythatourBBBsdo.
WehopethatthedevelopmentofalternativeBBBswillbecomeanactiveareaforfutureresearch.
Aswedescribebelow,ournextpilottestshouldyieldratingsetsthatwewillmakeavailabletootherswhowishtoevaluatealternativescoringalgorithms.
UsingRatingsItisuptothenewsclienthowbesttousethescoresgeneratedbyaBBB.
Somemayfilteroutthosearticleswithscoresbelowathreshold.
Somemaysortthearticlesbasedonthescores.
Othersmaysimplydisplaythescores,numericallyorgraphically.
Inkeepingwiththeeaseofusedesigngoal,developersshouldmodifyeachnewsclientinamannerconsistentwiththatclient'soveralldesign.
Onetrendinnewsclientsistodisplayasummaryoftheunreadarticlesinanewsgroup.
Eachlineofthesummarycontainsinformationaboutonearticle,typicallytheauthor,thesubjectlineandthelength.
Auserbrowsesthesummaryandrequestsdisplayofthefulltextofthosearticlesthatseeminteresting.
Allthreeofthenewsclientswemodifiedusethisdisplaytechnique.
Thethreemodifiedclientsweimplementedmakeslightlydifferentusesofthescoresinthesummarydisplay.
ThemodifiedNNclientdisplaysarticlesinthesameorderaregularNNclientdoes,namelytheorderinwhichthearticlesarrivedatthenewsserver.
Itmerelyaddsanadditionalcolumncontainingthepredictedscores.
Inthefirstversionofthisclient,thescoresweredisplayednumerically.
ThemodifiedGnusclientusesthepredictedscorestoaltertheorderofpresentationofarticlesinthesummary.
Gnusclustersarticlesbythread.
ThemodifiedGnusclientsortsthethreadsbasedonthemaximumpredictedscoreoverthearticlesinthethread.
Withineachthread,however,articlesarestilldisplayedinchronologicalorder,topreservetheflowofdiscussion.
AsinthemodifiedNN,thescoresaredisplayedinanadditionalcolumninthesummary.
TheMinnesotapilottestincludedusersofboththeGnusandNNclients.
Asexpected,participantstendedtobelievethatthesortinganddisplaymechanismsoftheirownnewsreaderwerebest,butallweregladtoseethescorepredictionsincorporatedintothatstandardformat.
Severalusers,however,noticedthatitwassomewhatdifficulttovisuallyscanthepredictionstofindthehighones.
ArevisedversionoftheNNclient(Figure6)roundsofftothenearestintegerandreportsthatasalettergrade(A-E),ascalefamiliartostudentsatU.
S.
Universities.
ThemodifiedNewsWatcherclientdisplaysthepredictedscoresasbargraphsratherthannumbers(Figure7),makingiteasiertovisuallyscanforarticleswithhighscores(longerbars).
Otherwise,itfollowstheconventionsoftheoriginalNewsWatcherclient.
Articlesaregroupedintothreadsandthesummarydisplayinitiallyshowsheaderlinesonlyforthefirstarticleineachthread.
Userscantwistdownthetriangleassociatedwithathreadtoseetheheaderlinesfortherestofthearticles.
Figure6:ThemodifiedNNclient.
Thethirdcolumndisplaysthenumberoflinesinthearticle.
Thefourthcolumndisplaysthescorepredictionsaslettergrades,translatedfromthenumericpredictionsthattheBetterBitBureaumakes(5=A,4=B,etc.
).
Whennoonehasevaluatedanarticle,nopredictionismade.
Figure7:ThemodifiedNewsWatcherclientdisplayspredictedscoresasbargraphs.
Disclaimer:thescoreswererandomlygeneratedfordemonstrationpurposes.
Inpractice,wewouldexpectarticlesbyPeteBergstrom(oneoftheauthorsofthispaper)tohavemuchhigherpredictedscores.
ScaleIssuesFurtherresearchisneededtounderstandhowperformancewillchangeasthescaleincreases.
InthecaseofGroupLens,thereareseveralrelevantperformancemeasures:predictionquality,usertime,BetterBitBureaucomputetimeanddiskstorage,andnetworktraffic.
Thefirstmeasureisthequalityofscorepredictions.
Weexpectpredictionqualitytoincreaseasthenumberofusersincreases,sincemoredatawillbeavailabletothepredictionalgorithm.
Anothermeasureishowlongusershavetowaittopostratingsandreceivepredictions.
InanearlierversionofGroupLens,thefunctionsoftheBBBwereincorporatedinthenewsclientitself.
OnemajoradvantageoftheseparateBBBisthatitcanpre-fetchratingsandpre-computepredictionsratherthancomputingthemwhentheuserstartsthenewsclient.
Thus,usertimeshouldremainroughlyconstantasGroupLensgrows,evenifittakesmoreCPUtimetocomputescores.
FormanypossiblepredictionformulasCPUtimewillgrowevenfasterthanlinearlywithincreasesinthenumberofusers.
ToreduceCPUtime,BBBscoulduseonlyapartoftheratingsmatrix,tradingoffcomputetimeagainstqualityofpredictions.
Eventhougheachratingisshort,eachnewsarticlemightbereadandratedbymanyraters,sothetotalvolumeofratingscouldexceedthevolumeofnews.
Tominimizestoragerequirements,BBBsmayemployalgorithmsthatuseanddiscardratingsastheyarrive,ratherthanstoringthem.
Threebasictechniquescouldreducenetworktraffic:reducethesizeoftheratings,reducethenumberofratings,andreducethenumberofplaceswhereeachratingissent.
OurBBBsbatchseveralratingsinasinglearticle,afirststeptowardreducingtheamountofstorageperrating,butfurthercompressionispossible.
Thenumberofratingscouldbereducedbylimitingthetotalnumberofratingsperarticleorthenumberofratingsfromuserswithsimilarprofiles.
TheseparationoftheBBBsfromthenewsclientsintheGroupLensarchitecturereducesthenumberofdestinationsforeachrating:eachnewsclientreceivesonlyscorepredictionsratherthanalltheindividualratingsthatcontributetothosepredictions.
ThenumberofdestinationsforeachratingcouldbefurtherreducedbysendingratingstosomeBBBsbutnotothers.
Forexample,BBBscouldbeclustered,basedongeographyorinterest,andexchangeratingsonlywithinclusters.
Thesizeofeachclustermustbesmallenoughtolimittheamountofratingsinformationdistributed,butlargeenoughtoprovideaneffectivepeergroup.
Thetablebelowestimatesdailynetworktrafficforvariousclustersizesassumingeachuserrates100articlesperdayandeachratingrequiresapproximately100bytes.
Forcomparisonpurposes,thecurrentnetnewstrafficisaround100MBperday.
ClustersizeDailyratingstraffic100users1MB10,000users100MB1,000,000users10GBSummaryofGroupLensArchitectureTheheartofGroupLensisanopenarchitecturefordistributingratings.
ThearchitecturespecifiestheformatofratingsproducedinbatchesbyBBBs,thepropagationoftheratingsbyUsenet,andtheinterfacefordeliveringpredictionsandratingsbetweennewsclientsandBBBs.
Otherwise,thearchitectureiscompletelyopen.
BBBsandnewsclientscanbefreelysubstituted,providinganenvironmentforexperimentationinpredictingratingsandinuserinterfacesforcollectingratingsandpresentingpredictions.
ONGOINGEXPERIMENTATIONBothofthepreviouspilottests,atSchlumbergerandtheUniversityofMinnesota,involvedonlylocalsharingofratings.
Thesetestsledtoimprovementsinboththeoverallarchitectureandtheuserinterfacesofnewsclients,asdiscussedalready.
Thenextstepisalargerscale,distributedtest,thatweplantocarryoutthissummer.
WehaveestablishedanewsgrouponthenewsserversatMITandMinnesotaandtwo(slightlydifferent)BetterBitBureausthatcommunicateratingsthroughthatnewsgroup.
Thetestisnotdesignedtodemonstratethatpeopleprefertoreadnetnewswithourcollaborativefiltersthanwithoutthem.
Webelievethatsuchanevaluationshouldwaitforatleastonemoreiterativedesigncycle.
Rather,thegoalsaretoidentifyanyunexpectedscalingissuesthatmayariseandtogatheradatasetthatwillbeusefulinevaluatingalternativescorepredictionalgorithms.
Theprimarybenchmarkofanyalgorithm'seffectivenesswillbeitsabilitytopredictvaluesthathavebeendeletedfromaratingmatrix.
Atfirstglance,itmightseemthatanylargesetofratingswouldbeusefulincreatingsuchabenchmark.
Uponcloserinspection,however,completeratingsmatricesaremuchmorevaluablethansparseones.
Forexample,supposethatusersreadandrateonlyasmallnumberofarticles,basedonscorepredictionstheyreceivefromBBBX.
Ifusersreaddifferentarticles,thisgeneratesasparsematrixofratings.
NowsupposethatwewishtocompareXtoanalternative,Y,thatpredictsdifferentscoresfortheusers.
WecancompareY'sandX'spredictionsonthosearticlesthatusersread,butthesampleisbiased.
PerhapswithY'sscores,theuserswouldhavereadotherarticlesandlikedthem.
Toallowunbiasedcomparisons,weareaskingeachoftheparticipantsinthenextpilottesttoreadandrateallthearticlesinatrainingset.
Thetrainingsetwillcontainanumberofarticlesfromeachofthenewsgroupsthatwillbeincludedinthetest.
Sinceuserswillcontributeratingsunderapseudonym,wewillbeabletosharetheratingsinthistrainingsetwithotherresearchers.
Inaddition,wewillretainthefulltextsofthearticlesinthetrainingset.
ThatwillenableevaluationofBBBsthatperformcontentfiltering,oracombinationofcontentfilteringandcollaborativefiltering,aswellasthosethatuseonlyotherusers'ratings.
SOCIALIMPLICATIONSCollaborativefilteringmayintroducemanysocialchangesinthealreadyrapidlyevolvingNetnewscommunity.
Forexample,theutilityofmoderatednewsgroupsmaydecline.
Newsocialpatternswillhavetodeveloptoencouragesociallybeneficialbehaviors,suchasreviewingarticlesthathavealreadyreceivedafewlowratings.
Finally,ifGroupLensiseffectiveatcreatingpeergroupswithsharedinterests,willthosepeergroupsbepermeableorwilltheglobalvillagefractureintotribesChangestoNetnewsBehaviorsGroupLenshasthepotentialtochangeNetnewsaswenowknowit.
Foronethingthequalityofarticlesindividualuserschoosetoreadshouldincrease.
Moresignificantly,asmoreandmoreusersrelyonGroupLensthetotalnumberoflow-qualityarticlesonUsenetmaydecreasesignificantly.
Sincefewpeoplewillreadsucharticles,theincentivetopostthemwilldecrease.
GroupLensmayalsosupplantorsupplementotherestablishedNetnewsbehaviors.
ModeratedNewsgroupsGroupLensmayreducetheneedformoderatednewsgroups.
TheadvantagesofGroupLensovertheexistingapproacharethat"moderators"canbegroupsofpeopleaswellasindividuals,andthateachusercanrelyonadifferentmoderatorratherthanhavingasinglemoderatorfortheentiregroup.
SomenewsgroupsmightchoosetousebothamoderatorandGroupLens.
Themoderatorofanewsgroupwillmaketheinitialpassthroughthearticlesubmissions.
Peerratingswouldthenallowfurtherfiltering.
NewsgroupSplitsCurrently,newsgroupsstartoffwithbroadtopicsandsplitintonarrowertopicsastrafficincreases.
Forexample,thenewsgrouprec.
sport.
footballeventuallysplitintothesubgroupsaustralian,canadian,rugby,pro,college,fantasy,misc,andoneforeachteamintheNFL.
Thesesplitsareaformofcontentfiltering,initiatedandmanagedbytheusers.
GroupLensusersmayfindthatmanysuchsplitsarelessimportant,andinsomecasesundesirable.
Overthecourseoftimeuserswillfindthemselvesreadingonlythesubsetofthenewsgrouptheyaremostinterestedin,astheycorrelatewithapeergroupwithsimilarinterests.
Splitsofinterestbetweengroupsofuserswillappearnaturally,withnoadditionaluseroradministrativeeffort.
AllowingthesplitstohappenthroughGroupLensratherthanthroughexplicitcontentfilteringallowsmorecross-pollinationofgeneralinterestarticles.
Forinstance,interestingarticlespostedbyBillsfansaboutanupcomingfootballgameagainsttheCowboyswouldalsoreachCowboysfanswithGroupLens,butwouldnotifthearticleswerepostedinthemorespecializednewsgrouprec.
sport.
football.
bills.
Kill-FilesKillfilesareacontentfilteringmechanismimplementedinsomenewsclients.
Manyuserswhostronglydislikeparticularsubjectsorparticularauthors,however,donotusekillfilesbecausetheyfindthemechanismcomplicatedandcumbersome.
GroupLensmightbeaneasiermeanstothesameend.
Auser'speergroupwillgivesucharticleslowratings,soonlyafewuserswillhavetoreadthem.
IncentivesIndividualsputadditionaleffort,albeitamodestamount,intoprovidingratingsthroughGroupLens.
Theseratingsprovidebenefittootheruserswhocanusethemtoselectinterestingarticles.
It'satwo-waystreet:everyonecanbebothaproducerandaconsumerofratings.
Whensomeonereadsandratesanarticle,thereisanincentivetoprovidehonestratings,becausedishonestratingswillcausetheBBBtomakepoorfuturepredictionsforthatuser.
Ontheotherhand,thereisnoincentivetoratearticlesatall.
Onthecontrary,thereisanincentivetowaitforothers'ratingsratherthanreadandrateanarticleoneself.
Acertainamountofaltruismorguiltmaycausemostpeopleto"dotheirshare"ofrating,butfewerthanthesociallyoptimalnumberofratingsarelikelytobeproduced.
Thefour-personMinnesotapilottestincludedahigh-volumenewsgroup,rec.
arts.
movies.
Thevolumeofarticleswassohighthateachparticipantwasunwillingtoreadaone-quartershareofthetotaldailyvolume.
Thenewsgroupwasquicklydroppedfromthetest.
Itmaybethatalargeruserpopulationwouldgenerateratingsevenforahigh-volumelistsuchasrec.
arts.
movies,butitishardertodrawona"do-your-share"mentalitywhencollaboratingwithlargergroupsofpeople.
Thereareother,moresubtleincentiveproblemsthatcanariseaswell.
Forexample,thereisanasymmetrybetweentheeffectsofpositiveandnegativeratings.
Ifthefirstfewreadersrateanarticletoohighly,otherswillreadthearticleandgiveitlowerratings.
Ontheotherhand,ifthefirstfewratingsofanarticlearenegative,otherswhowouldhaveratedithighlymayneverlookatitbecauseoftheinitialnegativerating.
Toavoidthis,itmaybenecessarytoprovideexternalincentivestosomepeopletoreadandratearticlesthathaveinitiallylowratings.
Theexternalincentivescouldbemoney,fame,orsimplyaccesstoothers'ratings:thosewhodidnotcontributetheirshareofratingsmightbedeniedaccesstotheBetterBitBureau'spredictions.
GlobalVillagesPresentnewsgroups,likenewspapersandlocaltelevisionshowsbeforethem,provideasharedhistoryfortheircommunityofreaders.
WithGroupLens,usersmaychoosetoreadarticlesonlyfromasmallgroupwithwhomtheysharemanycommoninterests.
Overtimethiscouldleadtoafractureoftheglobalvillageintomanysmalltribes,eachformingavirtualcommunitybutnonethelessisolatedfromeachother.
Somekindoffractureisinevitableandevendesirable,becausenousercankeepupwiththeoverwhelmingvolumeofnewsproducedeachday.
Thequestioniswhetherthesubgroupswillbeclosedorpermeable.
Oneargumentforprognosticatingpermeabilityisthatmanygroupswillformforashorttimeandthendisband[3].
Anotheristhatmanyuserswillparticipateinseveralsubgroups,providingamechanismforthebestideastocrossboundariesofinterestgroups.
CONCLUSIONSharedevaluationsareusefulinallsortsofactivities.
Weaskfriends,colleagues,andprofessionalreviewersfortheiropinionsaboutbooks,movies,journalarticles,cars,schools,andneighborhoods.
Clearly,someformofsharedevaluationsshouldalsohelpinfilteringelectronicinformationstreamssuchasnetnews.
Itisnotyetclearexactlywhatformthoseevaluationsshouldtake,howtheyshouldbecollectedanddisseminated,andhowtheyshouldbeusedinselectingarticlestoread.
GroupLensisonepromisingapproach.
Asinglenumbergivesacompositeratingofanarticleonalldimensionsrelevanttoaparticularreader.
Wehavemodifiedthreenewsreadingclientstoenableeasyentryofsuchnumericratings.
Wehavealsomodifiedthewaythattheclientsdisplaysubjectlinestoincludepredictedscoresbasedonothers'ratings.
Naturally,therewillbedifferencesofopinionamongreadersaboutparticulararticles,duetovaryinginterestsorqualityassessments.
Toaccommodatedifferencesofopinion,notallreaderswillplaceequaltrustinparticularevaluators.
Thealgorithmswehaveimplementedautomaticallydeterminehowmuchweighttoplaceoneachevaluation,basedonthedegreeofcorrelationbetweenpastopinionsofthereaderandevaluator.
Thishasthebeneficialsideeffectsthatreadersneednotknowinitiallywhoseevaluationstotrustandtheevaluators'opinionscanbecometrustedeveniftheevaluatorschoosetoremainanonymous.
TheGroupLensarchitectureallowsnewuserstoconnectandnewratingserverstocomeonline,withoutglobalcoordination.
Anewuserneedonlyuseamodifiednewsclientandhaveaconnectiontoaratingserver.
Theuserneednotconvincetheadministratorofhernetnewsservertomodifythenewsserver,runanyadditionalsoftware,oreventocarryanyadditionalnewsgroups.
Anewratingserverneedsonlytogetaccesstoanewsserverthatcarriestheratingsnewsgroups.
Moreover,thearchitectureisopen.
Anyonewhowishestocanmodifyanewsclienttoallowentryofevaluationsortousepredictedscores,solongastheclientfollowstheprotocolwehaveestablishedforcommunicatingwiththeratingserver.
Anyonewhowishestoimproveonthescorepredictionsthatourratingserversmakecandoso.
Theremaybebetterwaystocorrelatepastevaluations.
Theremayalsobewaystousetheevaluationsinconjunctionwithcontentfiltering.
Forexample,whencorrelatingpastevaluations,thescoringalgorithmmightconsiderevaluationsonlyofpastarticlesthataresomehowsimilartothecurrentone.
Ournextpilottestshouldyieldadatasetthatcanbeusedforevaluatingalternativepredictionmethods.
OnlyfurthertestingcanrevealwhetherGroupLensgatherstherightkindofevaluationsandusestheminwaysthatpeoplelike.
Ifthesimplenumericevaluationsturnouttobesufficient,thearchitecturewillscaleuptolargenumbersofratingserversandusers.
Ifnot,thendatafromourtestswillhelpdevelopandevaluateothermechanismsforsharingandusingevaluations.
Rightnow,peoplereadnewsarticlesandreacttothem,butthosereactionsarewasted.
GroupLensisafirststeptowardminingthishiddenresource.
ACKNOWLEDGMENTSShumpeiKumon'skeynoteaddressatCSCW92[11]inspiredourinvestigationofthepracticalapplicationofreputationstosocialfiltering.
ThankstoLorinHittandCarlFeynmanforhelpfuldiscussionsabouthowtopredictscoresbasedonpastcorrelations.
PeterFoltzandSueDumaisgenerouslyprovidedatestratingsetgeneratedfromoneoftheirexperimentsoncontentfiltering[5].
ThanksalsotoChrisAvery,JoeAdler,YannisBakos,ErikBrynjolfsson,DavidGoldberg,BillMacGregor,TomMalone,DavidMaltz,VahidMashayekhi,LisaSpears,DougTerry,MarkUhrmacher,andZbigniewWieckowski.
REFERENCES1.
Allen,R.
B.
UserModels:Theory,Method,andPractice.
InternationalJournalofMan-MachineStudies,32,(1990),pp.
511-543.
2.
Belkin,N.
J.
andCroft,B.
W.
InformationFilteringandInformationRetrieval:TwoSidesoftheSameCoinCACM,35,12(1992),pp.
29-38.
3.
Brothers,L.
,Hollan,J.
,Nielsen,J.
,Stornetta,S.
,Abney,S.
,Furnas,G.
andLittman,M.
SupportingInformalCommunicationviaEphemeralInterestGroups.
InProceedingsofCSCW92(1992,NewYork:ACM),pp.
84-90.
4.
Deerwester,S.
,Dumais,S.
T.
,Furnas,G.
W.
,Landauer,T.
K.
andHarshman,R.
IndexingbyLatentSemanticAnalysis.
JournaloftheAmericanSocietyforInformationScience,41,6(1990),pp.
391-407.
5.
Foltz,P.
W.
andDumais,S.
T.
PersonalizedInformationDelivery:AnAnalysisofInformationFilteringMethods.
CommunicationsoftheACM,35,12(1992),pp.
51-60.
6.
Goldberg,D.
,Nichols,D.
,Oki,B.
M.
andTerry,D.
UsingCollaborativeFilteringtoWeaveanInformationTapestry.
CommunicationsoftheACM,35,12(1992),pp.
61-70.
7.
Hill,W.
C.
,Hollan,J.
D.
,Wroblewski,D.
andMcCandless,T.
EditWearandReadWear.
InProceedingsofCHI92ConferenceonHumanFactorsinComputingSystems(1992,NewYork:ACM),pp.
3-9.
8.
Kahn,R.
E.
andCerf,V.
G.
TheDigitalLibraryProject,Volume1:TheWoldofKnowbots.
AnOpenArchitectureforaDigitalLibrarySystemandaPlanforItsDevelopment.
CNRI,1895PrestonWhiteDrive,Suite100,Reston,VA22091TechReport(March,1988).
9.
Karlgren,J.
NewsgroupClusteringBasedonUserBehavior--ARecommendationAlgebra.
SwedishInstituteofComputerScience#SICS-T--94/04-SE(March,1994).
10.
Kawell,L.
J.
,Beckhardt,S.
,Halvorsen,T.
andOzzie,R.
ReplicatedDocumentManagementinaGroupCommunicationSystem.
InProceedingsofCSCW88(1988,NewYork:ACM).
11.
Kumon,S.
FromWealthtoWisdom:AChangeintheSocialParadigm.
InProceedingsofCSCW92(1992,NewYork:ACM),pp.
3.
12.
Maes,P.
andKozierok,R.
LearningInterfaceAgents.
InProceedingsofAAAI93(1993,SanMateo,CA:AmericanAssociationforArtificalIntelligence).
13.
Malone,T.
W.
,Grant,K.
R.
,Turbak,F.
A.
,Brobst,S.
A.
andCohen,M.
D.
IntelligentInformationSharingSystems.
CommunicationsoftheACM,30,5(1987),pp.
390-402.
14.
Maltz,D.
A.
DistributingInformationforCollaborativeFilteringonUsenetNetNews.
MITDepartmentofEECSMSThesis(May,1994).
15.
Pindyck,R.
S.
andRubinfeld,D.
L.
EconometricModelsandEconomicForecasts.
MacGraw-Hill,NewYork,1991.
16.
Salton,G.
andBuckley,C.
Term-WeightingApproachesinAutomaticTextRetrieval.
InformationProcessingandManagement,24,5(1988),pp.
513-523.
17.
Salton,G.
andBuckley,C.
ImprovingRetrievalPerformancebyRelevanceFeedback.
JournaloftheAmericanSocietyforInformationScience,41,4(1990),pp.
288-297.
18.
Sheth,B.
ALearningApproachtoPersonalizedInformationFiltering.
MITDepartmentofEECSMSThesis(February,1994).
19.
Stodolsky,D.
S.
InvitationalJournalsBasedUponPeerConsensus.
RoskildeUniversityCentre,InstituteofGeography,SocioeconomicAnalysis,andComputerScience.
ISSN0109-9779-29#No.
29/1990(,1990).
20.
Suchak,M.
A.
GoodNews:ACollaborativeFilterforNetworkNews.
MITDepartmentofEECSMSThesis(February,1994).
21.
Wiederhold,G.
MediatorsintheArchitectureofFutureInformationSystems.
IEEEComputer,March,(1992),pp.
38-49.