Python实现协同过滤

协同过滤的 Python 实现及其实例 Jcr 2010-3-4

什么是协同过滤 ,[object Object],[object Object]

算法步骤 ,[object Object],[object Object],[object Object],[object Object]

1 、搜集偏好 fav={ ‘ p1’:{‘A’:3,‘B’:4,‘C’:3,‘D’:5,‘E’:1,’F’:4}, 'p2':{'A':2,'B':4,'C':4,'D':5,'E':3,’F’:2}, 'p3':{'A':3,'B':5,'C':4,'D':5,'E':2,’F’:5}, 'p4':{'A':2,'B':2,'C':3,'D':4,'E':3,’F’:1}, 'p5':{'A':4,'B':4,'C':4,'D':5} } A B C D E F P1 3 4 3 5 1 4 P2 2 4 4 5 3 2 P3 3 5 4 5 2 5 P4 2 2 3 4 3 1 p5 4 4 4 5 0 0

2 、寻找相似用户 ,[object Object],[object Object]

方法 1 ：欧几里德距离 A B P1 3 4 P2 2 4 P3 3 5 P4 2 2 p5 4 4

方法 2 ：皮尔逊相关度 A B C D E F P1 3 4 3 5 1 4 P2 2 4 4 5 3 2

2 、寻找相似用户 2.1 ,[object Object],from math import sqrt def sim_distance(prefs,person1,person2): si={} for item in prefs[person1]: if item in prefs[person2]: si[item]=1 if len(si)==0: return 0 sum_of_squares=sum([pow(prefs[person1][item]-prefs[person2][item],2) for item in prefs[person1] if item in prefs[person2]]) return 1/(1+sqrt(sum_of_squares)) >>> bookrec.sim_distance(bookrec.fav,'p5','p1') 0.41421356237309509 >>> bookrec.sim_distance(bookrec.fav,'p5','p2') 0.33333333333333331 >>> bookrec.sim_distance(bookrec.fav,'p5','p3') 0.41421356237309509 >>> bookrec.sim_distance(bookrec.fav,'p5','p4') 0.2402530733520421

2 、寻找相似用户 2.2 ,[object Object],def sim_pearson(prefs,p1,p2): si={} for item in prefs[p1]: if item in prefs[p2]:si[item]=1 n=len(si) if n==0: return 1 sum1=sum([prefs[p1][it] for it in si]) sum2=sum([prefs[p2][it] for it in si]) sum1Sq=sum([pow(prefs[p1][it],2) for it in si]) sum2Sq=sum([pow(prefs[p2][it],2) for it in si]) pSum=sum([prefs[p1][it]*prefs[p2][it] for it in si]) num=pSum-(sum1*sum2/n) #return n,sum1,sum2,sum1Sq,sum2Sq,pSum,num den=sqrt((sum1Sq-pow(sum1,2)/n)*(sum2Sq-pow(sum2,2)/n)) if den==0: return 0 r=num/den return r 依次获得 p5 与 p1/p2/p3/p4 之间的相关度： >>> bookrec.sim_pearson(bookrec.fav,'p5','p1') 1.1547005383792517 >>> bookrec.sim_pearson(bookrec.fav,'p5','p2') 0.89442719099991586 >>> bookrec.sim_pearson(bookrec.fav,'p5','p3') 0.57735026918962584 >>> bookrec.sim_pearson(bookrec.fav,'p5','p4') 1.1547005383792517

2 、寻找相似用户 2.3 ,[object Object],def topMatches(prefs,person,n=5,similarity=sim_pearson): scores=[(similarity(prefs,person,other),other) for other in prefs if other!=person] scores.sort() scores.reverse() return scores[0:n] 获得与 p5 在喜好方面最相似三个用户： 1- 使用默认的 sim_pearson （皮尔逊相关度） >>> bookrec.topMatches(bookrec.fav,'p5',3) 2- 使用 sim_distance （欧几里德距离） >>> bookrec.topMatches(bookrec.fav,'p5',3,similarity=bookrec.sim_distance) [(0.41421356237309509, 'p3'), (0.41421356237309509, 'p1'), (0.33333333333333331, 'p2')]

3. 推荐物品 3.1 ,[object Object],这个示例图，可以说明这个计算过程：左侧第 1 列是人名，第 2 列是它们跟 Toby 的相似度，第 3 、 5 、 7 列分别是对三部电影的评分，第 4 、 6 、 8 列分别是“评分”和“相似度”的乘积，权重用“相似度”（第 2 列）来表示。

3. 推荐物品 3.1 ,[object Object],def getRecommendations(prefs,person,similarity=sim_pearson): totals={} simSums={} for other in prefs: if other==person: continue sim=similarity(prefs,person,other) if sim<=0: continue for item in prefs[other]: if item not in prefs[person] or prefs[person][item]==0: totals.setdefault(item,0) totals[item]+=prefs[other][item]*sim simSums.setdefault(item,0) simSums[item]+=sim rankings=[(total/simSums[item],item) for item,total in totals.items()] rankings.sort() rankings.reverse() return rankings

3 、推荐物品 3.2 ,[object Object],1- 使用 sim_distance （欧几里德距离） >>> bookrec.getRecommendations(bookrec.fav,'p5',similarity=bookrec.sim_distance) [(3.3058466965322566, 'F'), (2.1136742554471271, 'E')] 2- 使用 sim_pearson （皮尔逊相关度） >>> bookrec.getRecommendations(bookrec.fav,'p5',similarity=bookrec.sim_pearson) [(2.7634528012205828, 'F'), (2.2365471987794177, 'E')]

[object Object],[object Object]

实例 1 ：使用 delicious API 对某用户推荐书签 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

实例 1 ：使用 delicious API 对某用户推荐书签获取某用户发布的书签 >>> pydelicious.get_userposts('jingstory') 为某用户发布新的书签 >>> jcr=pydelicious.apiNew('jingstory','158jing159') >>> jcr.posts_add(url="http://www.jcr.com",description="jcr t@sina",extended="no no no ",tags="blog"){'result': (True, 'done')} 试用 API >>> import pydelicious 获取某标签下的热门书签 >>> pydelicious.get_popular(tag='programing')

实例 1 ：使用 delicious API 对某用户推荐书签 ,[object Object],[object Object],[object Object]

实例 1 ：使用 delicious API 对某用户推荐书签 ,[object Object],from pydelicious import get_popular,get_userposts,get_urlposts def initializeUserDict(tag,count=5): user_dict={} for p1 in get_popular(tag=tag)[0:count]: for p2 in get_urlposts(p1['href']): user=p2['user'] user_dict[user]={} return user_dict 获取某一标签下热点链接的所有发布者： >>> delusers=initializeUserDict('oracle',6) >>> delusers {u'audionerd': {}, u'jpsbarros': {}, u'dbourguignon': {}, u'EdenR': {}, u'roborative': {}, u'tagpe3': {}, u'tiutmcclain': {}, u'sinclair.bain': {}, u'julio.santos': {}, u'davidrupp': {}, u'gimi.liang': {}, u'alex.dilley': {}, u'krzysiek_herod': {}, u'nbeyer': {}, u'paulo.machado': {}, u'jlimasa': {}, u'ron_smits': {}, u'ruis': {}, u'imsaar': {}, u'shjejurkar': {}, u'l33trich': {}, u't.bak': {}, u'kanazawa': {}, u'atog': {}, u'argotechnica': {}, u'glitch666_98': {}, u'rryanatdelicious': {}, u'csmall': {}, u'petermarriott': {}, u'sevarleth': {}}

实例 1 ：使用 delicious API 对某用户推荐书签 ,[object Object],def fillItems(user_dict): all_items={} for user in user_dict: for i in range(3): try: posts=get_userposts(user) break except: print "Failed user "+user+",retrying" time.sleep(4) for post in posts: url=post['href'] user_dict[user][url]=1.0 all_items[url]=1 for ratings in user_dict.values(): for item in all_items: if item not in ratings: ratings[item]=0.0 >>> fillItems(delusers) >>> delusers {u'audionerd': {u'http://www.tuaw.com/2010/02/23/5-tips-for-switchers/': 1.0, u'http://www.2008m.com/Speech/jingxuan/200901/20090121231559.html': 0.0, u'http://terrbear.org/?p=253': 0.0, u'http://dresdencodak.com/': 1.0, u'http://www.techmixer.com/free-bootable-antivirus-rescue-cds-download-list/': 0.0, u'http://ingraminternet.com/posts/17-easiest-postgres-install-ever-mac-edition': 0.0 …………… …………… }

实例 1 ：使用 delicious API 对某用户推荐书签 ,[object Object],-- 获取最近邻 >>> recommendations.topMatches(delusers, u'glitch666_98') [(0.028828828828828829, u't.bak'), (0.028828828828828829, u'sinclair.bain'), (0.028828828828828829, u'nbeyer'), (0.028828828828828829, u'l33trich'), (0.028828828828828829, u'krzysiek_herod')] >>> recommendations.topMatches(delusers,'jingstory') [(0, u'kanazawa'), (0, u'dbourguignon'), (0, u'audionerd'), (0, u'argotechnica'), (-0.040540540540540536, u'tiutmcclain')] - - 推荐书签 >>> recommendations.getRecommendations(delusers,u'glitch666_98')[0:5] [(0.16666666666666669, u'http://www.twilio.com/'), (0.16666666666666669, u'http://border-radius.com/'), (0.16666666666666669, u'http://amerine.net/2010/02/24/rvm-rails3-ruby-1-9-2-setup.html'), (0.083333333333333343, u'https://wiki.ubuntu.com/VirtualBoxNetworking'), (0.083333333333333343, u'https://opensso.dev.java.net/source/browse/opensso/extensions/saml2ruby/source/README?view=markup')] >>> recommendations.getRecommendations(delusers,'jingstory')[0:10] []

实例 2 ：使用 MovieLens 中真实数据集对用户进行推荐 ,[object Object],[object Object],[object Object],@ 测试环境—普通 PC 机（ 2G ） Download Links ： http://grouplens.org/ http://grouplens.org/node/73 用户电影评分数量基于用户 / 获得推荐的效率基于物品 / 计算物品相似度的效率 Data set 1 943 1,682 100,000 <1 秒 3-4 秒 /100 个 Data set 2 71,567 10,681 10,000,054 15 秒 8 分钟 /100 个

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Python实现协同过滤

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Python实现协同过滤

Ähnlich wie Python实现协同过滤 (20)

Python实现协同过滤