使用BeautifulSoup 检索 www.163.com 主页面所有超链接

#coding:utf-8import urllib2from bs4 import BeautifulSoup

response=urllib2.urlopen("http://www.163.com")

HtmlDoc=response.read()

soup = BeautifulSoup(HtmlDoc,‘html.parser‘,from_encoding=‘utf-8‘)

links =soup.find_all("a")print(‘打印所有链接‘)for link in links:    print link.name,link[‘href‘]print len(links)

执行结果

打印所有链接a http://www.163.com/#f=topnava http://m.163.com/newsapp/#f=topnava http://music.163.com/#f=topnava http://yuedu.163.com/#f=topnava http://note.youdao.com/#f=topnava http://y.163.com/?from=wsdha http://open.163.com/#f=topnava http://caipiao.163.com/mobile/client_cp.jsp#from=yingyonga http://cidian.youdao.com/?vendor=topnava http://mail.163.com/client/dl.html?from=mail46a http://www.lofter.com/?act=qb163rk_20141031_01a http://study.163.com/client/download.htm?from=163app&utm_source=163.com&utm_medium=web_app&utm_campaign=businessa http://www.163.com/a http://reg.163.com/a http://reg.163.com/RecoverPassword.shtml?f=wwwa http://mail.163.com/client/dl.html?from=mail46a http://reg.email.163.com/mailregAll/reg0.jsp?from=163navi&regPage=163a http://reg.vip.163.com/register.m?from=topnava http://reg.163.com/Logout.jspa http://rd.da.netease.com/redirect?t=I4iYc8&p=EA7B9E&target=http%3A%2F%2Fwww.kaola.com%2Fa http://www.kaola.com/outter/promote/myzq.htmla http://www.kaola.com/outter/promote/mrcz.htmla http://www.kaola.com/outter/promote/jjry.htmla http://www.kaola.com/outter/promote/jkms.htmla http://www.kaola.com/outter/promote/yybj.htmla http://www.kaola.com/outter/promote/hwzy.htmla http://rd.da.netease.com/redirect?t=W1rULs&p=pESsw1&proId=1024&target=http%3A%2F%2Fwww.kaola.com%2Factivity%2Fdetail%2F5288.html%3Ftag%3Dbe3d8d027a530881037ef01d304eb505a http://www.kaola.com/outter/promote/khd.htmla http://email.163.com/#from=163nav_icona http://email.163.com/#f=topnava http://vipmail.163.com/#f=topnava http://qiye.163.com/#f=topnava http://reg.email.163.com/mailregAll/reg0.jsp?from=ntes_nav&regPage=163a http://reg.email.163.com/unireg/call.do?cmd=register.entrance&flow=mobile&from=ntes_nava http://mail.163.com/dashi/dlpro.html?from=mail46a http://pay.163.com/